Anti-vibration control device and method, and imaging device

Abstract

To more appropriately correct parallel shake according to the state of a subject or an imaging apparatus.SOLUTION: A vibration-proof control device has: motion vector detection means that detects motion vectors from images repeatedly output from an image pick-up device; first arithmetic means that calculates a first correction coefficient for calculating a parallel shake amount based on the motion vectors; second arithmetic means that calculates a second correction coefficient for calculating the parallel shake amount based on the acceleration of a shake of an imaging apparatus; determination means that selects one of the first correction coefficient and the second correction coefficient based on information on the accuracy of detection of the motion vectors in the motion vector detection means; and calculation means that determines the parallel shake amount by using the first correction coefficient or the second correction coefficient selected by the determination means. When the information indicates that the accuracy is low, the determination means selects the second correction coefficient, and in cases other than that, selects the first correction coefficient.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image stabilization control device and method, and an imaging device, and more particularly to an image stabilization control device and method for correcting translational shake, and an imaging device. [Background technology]

[0002] In recent years, cameras equipped with anti-shake control devices have been increasing on the market. In order to capture a blur-free image even if camera shake occurs during exposure, it is necessary to detect the angular and translational shake of the camera caused by camera shake and to operate the shake correction unit according to the detected value to offset the shake.

[0003] In recent years, the performance of angular velocity sensors has improved, making it possible to detect angular shake in a wider band than before, especially in the low frequency band. This has improved the correction performance of angular shake of a camera caused by hand shake. On the other hand, the improvement in the correction performance of angular shake has made translational shake more noticeable. As a method of correcting translational shake, Patent Document 1 discloses a technology in which the ratio of the output of an accelerometer to the output of an angular velocity meter is calculated as a radius of rotation, the translational shake is calculated from the radius of rotation and the output of the angular velocity meter, and a shake correction unit is driven to correct the translational shake. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 4789614 Summary of the Invention [Problem to be solved by the invention]

[0005] However, as image stabilization performance has improved, it has become possible to shoot handheld even with long exposure times, which has created a need to compensate for camera shake at even lower frequencies.To compensate for camera shake at even lower frequencies, image signals are used in addition to accelerometer signals, but the question remains as to how to utilize the image signals.

[0006] The present invention has been made in consideration of the above problems, and has an object to more accurately correct translational shake in accordance with the state of the subject and the imaging device. [Means for solving the problem]

[0007] In order to achieve the above object, an image stabilization control device of the present invention includes a motion vector detection means for detecting a motion vector from an image repeatedly output from an image sensor, a first calculation means for calculating a first correction coefficient for calculating a translational shake amount based on the motion vector, a second calculation means for calculating a second correction coefficient for calculating the translational shake amount based on the acceleration of shake of the image pickup device, a determination means for selecting either the first correction coefficient or the second correction coefficient based on information regarding accuracy of detection of the motion vector by the motion vector detection means, and a calculation means for determining the translational shake amount using the first correction coefficient or the second correction coefficient selected by the determination means, wherein the determination means: A ratio of a translational shake image plane velocity on the image plane of the image sensor, which was calculated based on the motion vector in the past to obtain the first correction coefficient, to an angular shake image plane velocity on the image plane of the image sensor, which is calculated based on the angular velocity of the shake of the imaging device, is lower than a predetermined ratio. If so, the second correction coefficient is selected, and otherwise the first correction coefficient is selected. Effect of the Invention

[0008] According to the present invention, it is possible to more accurately correct translational shake in accordance with the state of the subject and the imaging device. [Brief description of the drawings]

[0009] [Figure 1] 1 is a central cross-sectional view of an imaging device according to a first embodiment of the present invention and a block diagram showing an outline of a functional configuration thereof. [Diagram 2] FIG. 2 is a block diagram showing a configuration for calculating a translational shake amount according to the first embodiment. [Diagram 3] 5A to 5C are diagrams for explaining calculations performed by a translational shake extraction unit in the first embodiment. [Figure 4] 5 is a flowchart showing a calculation process of a translational shake amount in the first embodiment. [Diagram 5]FIG. 11 is a diagram showing an example of a table for determining a gain value in the second embodiment. [Figure 6] 10 is a flowchart showing a calculation process of a translational shake amount in the second embodiment. [Figure 7] FIG. 13 is a block diagram showing a configuration for calculating a translational shake amount according to a third embodiment. [Figure 8] 13 is a flowchart showing a calculation process of a first correction coefficient in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0011] <First embodiment> 1A and 1B are diagrams showing an image pickup apparatus 1000 equipped with an anti-shake control apparatus in a first embodiment, in which FIG. 1A is a central cross-sectional view of the image pickup apparatus 1000, and FIG. 1B is a block diagram showing the functional configuration of the image pickup apparatus 1000.

[0012] 1(a), the imaging device 1000 of this embodiment mainly comprises a camera body 1 and a lens unit 2 that is detachable from the camera body 1. The camera body 1 and the lens unit 2 are electrically connected via an electrical contact 11. Note that the imaging device of the present invention is not limited to this configuration, and may be an imaging device in which the camera body and the lens unit are integrally configured.

[0013] The lens unit 2 includes a photographing optical system 3 arranged on an optical axis 4 and consisting of multiple lenses, including a focus lens, a zoom lens, an anti-vibration lens 9, and an aperture, and a lens system control unit 12. The camera body 1 also includes an image sensor 6, a rear display device 10a, an EVF 10b, a shake correction unit 14, an angular velocity detection unit 15, an acceleration detection unit 16, and a shutter mechanism 17.

[0014] In Fig. 1(a), the z-axis is parallel to the optical axis 4. The x-axis and y-axis are perpendicular to the z-axis and parallel to each side of the image sensor 6. In order to make the drawing easier to see, the origin is written outside the image sensor 1000 in Fig. 1(a), but in reality, the origin is located at the center of the image sensor 1000.

[0015] 1(b), the lens unit 2 further includes a lens driver 13 that drives the focus lens, zoom lens, anti-vibration lens 9, aperture, etc., included in the photographing optical system 3. The camera body 1 further includes a camera system control unit 5, a release detection unit 7, an operation detection unit 8, a display unit 10, and a shutter driver 18. The display unit 10 includes a rear display device 10a provided on the rear surface of the camera body 1 shown in FIG. 1(a) and an EVF (electronic viewfinder) 10b provided in the viewfinder of the camera body 1.

[0016] The release detection unit 7 detects an open / close signal of a release switch (not shown) that opens / closes by pressing a release button (not shown), and sends the detected open / close signal to the camera system control unit 5. There are usually two types of open / close signals detected by the release detection unit 7, and it detects the open / close signal of a two-stage switch (hereinafter referred to as "SW1") that turns ON when the release button is pressed halfway, and a switch (hereinafter referred to as "SW2") that turns ON when the release button is pressed all the way. An operation detection unit 8 detects operations by the photographer, such as the shutter speed, F-number, and mode settings.

[0017] Light from a subject incident via the photographing optical system 3 of the lens unit 2 is imaged on the imaging surface of the image sensor 6 while the shutter mechanism 17 is open. The lens driving unit 13 receives a control signal from the lens system control unit 12 and drives the photographing optical system 3 so that a good image can be formed on the image sensor 6. The shutter mechanism 17 is also driven and controlled by the shutter driving unit 18 so that exposure is performed at a shooting time set by the photographer or determined by the camera system control unit 5. The image sensor 6 photoelectrically converts the incident light and outputs an electrical signal (image signal) according to the amount of light.

[0018] The camera system control unit 5 calculates the amount of shake compensation to reduce the effects of shake based on signals output from the release detection unit 7, the operation detection unit 8, the angular velocity detection unit 15, and the acceleration detection unit 16, and outputs the amount of shake compensation to the shake compensation unit 14 and the lens system control unit 12.

[0019] The lens system control unit 12 outputs a command based on the shake correction amount received from the camera system control unit 5 to the lens driving unit 13. The lens driving unit 13 drives the vibration-proof lens 9 in the x and y directions in FIG. 1(a) to perform shake correction that takes into account both angular shake and translational shake.

[0020] Meanwhile, the shake correction unit 14 drives the image sensor 6 based on the shake correction amount received from the camera system control unit 5. The shake correction unit 14 drives the image sensor 6 in the x and y directions in Fig. 1(a) to perform shake correction that takes into account both angular shake and translational shake. The shake correction unit 14 also drives the image sensor 6 to rotate around the z axis to perform shake correction that takes into account both angular shake and translational shake caused by rotational motion around the z axis.

[0021] It should be noted that the correction method based on the shake correction amount received from the camera system control unit 5 is not limited to this method, and other forms may be used. For example, so-called electronic shake reduction may be used, in which the cut-out position of the image output from the imaging element 6 is shifted to correct the shake, or electronic shake reduction may be appropriately combined with the above-mentioned shake reduction that drives the imaging element 6.

[0022] When shake is applied to the imaging device 1000, image plane shake occurs in the imaging element 6. Hereinafter, the image plane shake of the imaging element 6 caused by parallel shake is referred to as "image plane parallel shake", and the image plane shake of the imaging element 6 caused by angular shake is referred to as "image plane angular shake".

[0023] FIG. 2 is a block diagram showing a configuration for calculating parallel shake in the camera system control unit 5 and the angular velocity detection unit 15 and the acceleration detection unit 16.

[0024] The motion vector detection unit 201 detects the speed of image plane shake of the imaging element 6 in the x-axis direction, y-axis direction, and z-axis direction of FIG. 1 by detecting the moving speed of the feature points from the amount of movement of the feature points of the image between frames. When the subject is stationary, the moving speed of the feature points indicates the speed of image plane shake caused by the shake of the imaging device. When the subject is moving, the moving speed of the feature points indicates the sum of the speed of image plane shake caused by the shake of the imaging device and the moving speed of the subject.

[0025] In addition, the speed of image plane shake detected by the motion vector detection unit 201 includes a mixture of an angular shake image plane speed and a parallel shake image plane speed. The angular shake image plane speed is the speed of the shake generated on the image plane due to image plane angular shake, and the parallel shake image plane speed is the speed of the shake generated on the image plane due to image plane parallel shake.

[0026] The shake correction amount detection unit 208 detects and outputs the corrected angular shake image plane speed that is corrected by the lens drive unit 13 and the shake correction unit 14.

[0027] The angular velocity detection unit 15 detects the angular velocities around the x-axis, y-axis, and z-axis of the coordinate axes shown in FIG. 1(a) and outputs an angular velocity signal. The angular shake calculator 204 calculates and outputs the angular shake image plane speed by multiplying the angular velocity signal output from the angular velocity detection unit 15 by the focal length.

[0028] The translational shake extractor 207 calculates a translational shake image plane velocity and an uncorrected rotational shake image plane velocity using the output of the motion vector detection unit 201, the output of the shake correction amount detection unit 208, and the output of the rotational shake calculator 204. Here, a method of calculating the translational shake image plane velocity and the uncorrected rotational shake image plane velocity will be described with reference to FIG.

[0029] Fig. 3 is a conceptual diagram of the calculation performed by the translational shake extractor 207. The sum of the rotational shake image plane velocity (corrected) 301, the rotational shake image plane velocity (uncorrected) 302, and the translational shake image plane velocity (uncorrected) 303 shown in Fig. 3 becomes the velocity of the image plane shake that appears on the image plane of the image sensor 6 when no vibration reduction is performed.

[0030] Angular shake image plane velocity (corrected) 301 is the output of the shake compensation amount detection unit 208. In addition, the output of the angular shake calculator 204 corresponds to the sum of the angular shake image plane velocity (corrected) 301 and the angular shake image plane velocity (uncorrected) 302. Therefore, the angular shake image plane velocity (uncorrected) 302 can be obtained by subtracting the output of the angular shake calculator 204 from the output of the shake compensation amount detection unit 208.

[0031] Moreover, the output of the motion vector detection unit 201 corresponds to the sum of the rotational shake image plane velocity (uncorrected) 302 and the translational shake image plane velocity (uncorrected) 303. Therefore, the translational shake image plane velocity (uncorrected) 303 can be obtained by subtracting the rotational shake image plane velocity (uncorrected) 302 from the output of the motion vector detection unit 201.

[0032] The LPF 211 receives the translational shake image plane velocity (uncorrected) 303 from the translational shake extractor 207 as an input, applies a low-pass filter, extracts a specific band, and outputs it. The band to be extracted is a band of 10 Hz or less that includes camera shake, but since the dominant band becomes closer to low frequencies when shooting with a long exposure, the band of the filter is changed according to the conditions, such as extracting a band of 1 Hz or less.

[0033] The LPF 212 extracts and outputs a specific band by applying a low-pass filter to the angular velocity signal output from the angular velocity detection unit 15. The band extracted by the LPF 212 is basically the same as that of the LPF 211.

[0034] The comparator 216 calculates a first correction coefficient from the ratio of the output signals of the LPF 211 and the LPF 212, and outputs the first correction coefficient.

[0035] 1A, the acceleration detection unit 16 detects acceleration in the x-axis direction, y-axis direction, and z-axis direction, and outputs an acceleration signal. The BPF 213 applies a band pass filter to the acceleration signal output from the acceleration detection unit 16 to extract and output a signal of a specific band. The integrator 205 integrates the output signal of the BPF 213 to calculate and output a velocity signal. The translational shake calculator 401 multiplies the velocity signal output from the integrator 205 by an image magnification (focal length / shooting distance) to calculate and output a translational shake image plane velocity.

[0036] The BPF 209 applies a band pass filter to the angular velocity signal output from the angular velocity detection unit 15 to extract and output a specific band. The filter band of the BPF 209 is basically the same as that of the BPF 213. The HPF 210 applies a high pass filter to the output signal of the BPF 209 to output the signal in phase with the output signal of the integrator 205. However, when the integrator 205 performs perfect integration, the processing of the HPF 210 is not necessary.

[0037] The comparator 217 calculates a second correction coefficient from the ratio between the output signals of the translational shake calculator 401 and the HPF 210, and outputs the second correction coefficient.

[0038] The subject information detection unit 214 detects the subject to be photographed by the imaging device 1000 and the imaging device 1000 The subject information includes the brightness of the subject, the moving speed of the subject, the contrast of the subject, the pattern of the subject, the movement of the subject's background, the movement of the subject other than the background, and the like.

[0039] The shake information detection unit 215 receives the angular shake image plane velocity (uncorrected) 302 and the translational shake image plane velocity (uncorrected) 303 as inputs from the translational shake extractor 207, and outputs them as they are.

[0040] The determination unit 218 detects the subject and the image capturing device from the subject information detection unit 214. 1000 The output signal indicating the state of the image pickup unit 212 is input to the judgment unit 218, as well as the angular shake image plane velocity (uncorrected) 302 and the translational shake image plane velocity (uncorrected) 303 from the shake information detection unit 215, the first correction coefficient from the comparator 216, and the second correction coefficient from the comparator 217. The judgment unit 218 then judges whether to use the first correction coefficient or the second correction coefficient. The judgment method used by the judgment unit 218 will be described below.

[0041] Basically, the determination unit 218 makes a determination on the premise that a first correction coefficient calculated based on the speed of image plane shake from the motion vector detection unit 201 is used. This is because the signal detected by the motion vector detection unit 201 has a wider measurable band than the acceleration signal detected by the acceleration detection unit 16. The band here refers to the camera shake band, which is generally a band of 10 Hz or less.

[0042] However, there are situations in which the detection accuracy of the motion vector detection unit 201 drops. The motion vector detection unit 201 searches for feature points in an image signal and detects their movement speed. Therefore, if the subject included in the image signal has a repeating pattern, low contrast, or low brightness, the feature points may not be captured accurately, and accurate detection may not be possible. Also, if completely different subjects are captured in images between frames, the amount of movement of the feature points cannot be calculated, and accurate detection cannot be possible. Therefore, in such cases, a second correction coefficient calculated based on the acceleration signal from the acceleration detection unit 16 is used.

[0043] Based on the output signal of the subject information detection unit 214, the determination unit 218 determines that the situation is one in which the detection accuracy of the motion vector detection unit 201 will decrease if the subject brightness is low, the subject contrast is low, the subject has a repeating pattern, the subject is moving fast, the movement of things other than the background is fast, etc. When the determination unit 218 determines that the situation is one in which the detection accuracy of the motion vector detection unit 201 will decrease, it outputs the second correction coefficient without using the first correction coefficient.

[0044] The determination unit 218 also makes a determination according to the ratio of the angular shake image plane velocity (uncorrected) and the translational shake image plane velocity (uncorrected). As shown in FIG. 3, the translational shake image plane velocity (uncorrected) 303 is obtained by subtracting the angular shake image plane velocity (uncorrected) 302 from the output of the motion vector detection unit 201. Therefore, when the translational shake image plane velocity (uncorrected) 303 is small, it may be buried in the detection errors of the motion vector detection unit 201, the angular velocity detection unit 15, and the shake correction amount detection unit 208. Therefore, the determination unit 218 determines whether the translational shake image plane velocity (uncorrected) 303 is greater than the angular shake image plane velocity (uncorrected). 302 If there is a possibility that the detected error may be hidden by the detection error, the first correction coefficient is not used and the second correction coefficient is output.

[0045] When comparing the angular shake image plane velocity (uncorrected) 302 and the translational shake image plane velocity (uncorrected) 303, it is also possible to use an estimated value of the translational shake image plane velocity (uncorrected) 303 instead of the detected values. For example, first, an average ratio of the total of the angular shake image plane velocity (corrected) 301 and the angular shake image plane velocity (uncorrected) 302 to the translational shake image plane velocity (uncorrected) 303 is acquired. Then, using the acquired ratio, the translational shake image plane velocity (uncorrected) 303 is estimated from the obtained total of the angular shake image plane velocity (corrected) 301 and the angular shake image plane velocity (uncorrected) 302. Then, the estimated translational shake image plane velocity (uncorrected) 303 and the angular shake image plane velocity (uncorrected) 302 are compared to determine whether there is a possibility that they may be buried in a detection error. In addition, since the average ratio between the sum of the angular shake image plane velocity (corrected) 301 and the angular shake image plane velocity (uncorrected) 302 and the translational shake image plane velocity (uncorrected) 303 varies depending on the shooting time, the ratio is stored for each shooting time.

[0046] Moreover, by estimating the translational shake image plane velocity (uncorrected) 303 from the sum of the rotational shake image plane velocity (corrected) 301 and the rotational shake image plane velocity (uncorrected) 302, it is possible to estimate whether the subject is moving. If the output of the motion vector detection unit 201 is greater than the sum of the rotational shake image plane velocity (corrected) 301 and the rotational shake image plane velocity (uncorrected) 302 and the estimated translational shake image plane velocity (uncorrected) 303, there is a possibility that the subject is moving. Estimation of whether the subject is moving is effective when the image of the moving subject is dominant in the image of each frame output by the image sensor 6. When the subject is moving, the determination unit 218 uses the second correction coefficient instead of the first correction coefficient. The second correction coefficient calculated based on the acceleration signal from the acceleration detection unit 16 can also be used to check whether the value of the first correction coefficient is significantly different, except in a situation where there is a large vibration due to shutter drive such as during continuous shooting.

[0047] As a result of the determination by the determination unit 218, the selected first correction coefficient or second correction coefficient is integrated in an integrator 219, and is multiplied by the angular velocity signal from the angular velocity detection unit 15 in a translational shake amount calculation unit 220, thereby being converted into an amount of translational shake on the imaging plane.

[0048] Next, referring to the flow chart of FIG. Amount The flow of the calculation process will be described. 4 starts when an aiming signal is input. The aiming signal is input based on a signal from a viewfinder (not shown), a signal from the angular velocity detector 15, and a signal from the switch SW1. When this process starts, the shake correction unit 14 and the lens driver 13 start driving the image sensor 6 and the vibration-proof lens 9, respectively.

[0049] In S401, the motion vector detection unit 201 detects a motion vector, the angular velocity detection unit 15 detects an angular velocity, and the acceleration detection unit 16 detects an acceleration. At the same time, the image plane velocity of the corrected angular shake, which is shake correction information, is detected from the shake correction amount detection unit 208. Take You will benefit.

[0050] Next, in S402, the first correction coefficient and the second correction coefficient are calculated as described above based on the information acquired in S401. In S403, the subject information detection unit 214 detects information about the subject. In S404, the determination unit 218 determines whether to select the first correction coefficient or the second correction coefficient as described above, based on the subject information obtained in S403 and the state of the image capturing apparatus 1000 regarding shake.

[0051] In S405, it is determined whether the switch SW2 is ON. If the switch SW2 is OFF, the process returns to S401. If the switch SW2 is ON, the process proceeds to S406, where the translational shake amount calculation unit 220 calculates the image plane translational shake amount using the correction coefficient selected in S404. When S406 starts, the shake correction unit 14 and the lens driving unit 13 drive the image sensor 6 and the vibration-proof lens 9 with the correction amount including the translational shake amount. When the exposure ends, the process ends.

[0052] As described above, according to the first embodiment, translational shake is detected using a plurality of methods, and the state of the subject and the imaging device is To By selecting the translational shake detected by the method selected accordingly, it becomes possible to correct the translational shake more accurately.

[0053] <Second embodiment> Next, a second embodiment of the present invention will be described. The configuration of an image pickup apparatus in the second embodiment is similar to that of the image pickup apparatus 1000 described with reference to Figs. 1 and 2, and therefore a description thereof will be omitted here.

[0054] If the accuracy of translational shake compensation is not very high, performing translational shake compensation when the translational shake amount is small may result in over-compensation of the translational shake, which may in turn worsen image quality. Therefore, in the second embodiment, a method of preventing deterioration of image quality by changing the gain of the translational shake compensation amount according to the shooting distance and shooting time will be described. Note that the shooting distance referred to here is non-dimensionalized by dividing the shooting distance by the focal length, and is detected by the camera system control unit 5.

[0055] If you want to keep the shooting range constant regardless of the focal length, you can achieve this by making this non-dimensional shooting distance the same value. Also, when converting the amount of shake of the image pickup device in a translational shake to a value on the imaging plane of the image pickup device, the conversion can be done by dividing by this non-dimensional shooting distance. Due to these advantages, the non-dimensional shooting distance is used.

[0056] The shooting time is calculated by the photographer's operation or by the photometry function of the imaging device, and is input to the shutter drive unit 18, which drives the shutter mechanism 17. If the imaging element 6 has an electronic shutter function, the charge reset and charge readout timing of the imaging element 6 may be controlled according to the shooting time.

[0057] While the amount of angular shake does not depend on the shooting distance, the amount of translational shake differs depending on the shooting distance. Also, the closer the shooting distance, the greater the amount of translational shake, and the farther the shooting distance, the smaller the amount of translational shake. Also, both the amount of angular shake and the amount of translational shake become greater the longer the shooting time, and become smaller the shorter the shooting time. In this way, since the amount of angular shake does not change depending on the shooting distance, it is necessary to change the amount of correction only depending on the shooting time.

[0058] On the other hand, since the amount of translational shake varies depending on the shooting distance and shooting time, it is necessary to change the correction amount taking into account the shooting distance and shooting time. However, since the correction amount of translational shake varies depending on the shooting distance, when there are multiple subjects and the shooting distances of each are significantly different, the amount of translational shake to be corrected cannot be determined, and translational shake correction is not performed. In such a case, translational shake is corrected by a method such as performing correction for each area after shooting.

[0059] Furthermore, even if the shooting distances of multiple subjects are significantly different from one another, if the AF state of the imaging device is set to a central single point, for example, it is assumed that the photographer intends to shoot a single point, so translational shake correction is performed.

[0060] It is desirable to calculate the amount of translational shake compensation based on the translational shake that actually occurs during exposure to obtain the gain, but this is difficult to achieve because the total amount of shake during exposure is unknown until the exposure is completed. Therefore, a gain corresponding to the shooting distance and shooting time is calculated in advance based on the average amount of translational shake, and is stored as a table (hereinafter referred to as a "gain table"). When performing translational shake compensation, once the shooting distance and shooting time are determined, the gain of the translational shake compensation is determined by referring to this gain table. Since the amount of translational shake differs depending on the photographer, the gain table may be rewritten according to the characteristics of the photographer. Furthermore, since the amount of translational shake differs depending on the posture of the photographer, gain tables corresponding to several amounts of translational shake may be stored in advance and switched. The gain table may be switched according to the mode set by the photographer. The mode may be, for example, the strength or weakness of translational shake compensation, a macro shooting mode, etc.

[0061] In addition, the translational shake amount during exposure may be estimated based on the translational shake amount before exposure, and the gain table to be referred to may be switched.The table to be referred to may basically be rewritten by determining whether the translational shake amount is large or small based on the tendency of the translational shake amount before exposure.

[0062] The amount of translational shake during exposure may be estimated based on the amount of translational shake before exposure, taking into account the shooting time and shooting distance, and a gain value may be calculated and used by comparing the amount of translational shake with a predetermined shake reference.

[0063] Here, the features of the table created from the shooting time and shooting distance will be described with reference to FIG. 5, the horizontal axis indicates the shooting distance, and the vertical axis indicates the shooting time. The hatched area 501 is an area where the shake amount is larger than a predetermined shake standard, that is, an area where translational shake should be corrected. The translational shake amount varies even within the area 501, and the shorter the shooting distance and the longer the shooting time, the larger the translational shake amount becomes. Therefore, even within the area 501, the gain is increased in areas where the translational shake amount is large.

[0064] On the other hand, the white area 502 is an area where the amount of translational shake is equal to or less than a predetermined standard. This area 502 is an area where it cannot be said that correction of translational shake is necessarily required. However, since a certain amount of translational shake is included particularly near the boundary with the area 501, correction may be performed, and the gain is determined according to the accuracy of the translational shake. For example, the gain is set high in the area 502 close to the area 501, and low in the other area.

[0065] In this way, when the detection accuracy of translational shake is high, the effect of translational shake compensation can be maximized by inputting a large gain value even in an area where translational shake compensation is not essential.

[0066] The gain value may be set in any way, but for example, a value of 1 may be entered in an area where translational shake compensation is essential, and a value close to 0 may be entered in an area where translational shake compensation is not essential. When translational shake compensation is performed, the detected translational shake compensation amount is multiplied by the gain value.

[0067] Next, referring to the flowchart of FIG. 6, the translational shake correction in the second embodiment will be described. Amount The flow of the calculation process will be described below. The process shown in Fig. 6 starts when the power supply of the imaging device is turned ON or the switch SW1 is turned ON.

[0068] The shooting distance is detected in S501, and the setting value of the shooting time is confirmed in S502. In S503, a gain table is referenced according to the photographer's characteristics, shooting mode, etc., and in S504, a gain value is determined according to the shooting distance and shooting time. After that, the processes of S401 to S404 in Fig. 4 are performed to select a correction coefficient, but a detailed description will be omitted here.

[0069] In S405, it is determined whether switch SW2 is ON. If it is ON, In S506, The gain value determined in S504 and the correction coefficient selected in S404 are used to perform translational shake correction. 4 If it is determined in step S505 that the gain is OFF, the process returns to step S501 to determine a new gain value.

[0070] As described above, according to the second embodiment, in addition to the effects of the first embodiment, it is possible to further suppress deterioration of image quality.

[0071] In the second embodiment, the motion vector detection unit 100 is configured as shown in FIG. 201 A first correction coefficient based on the motion vector detected by the acceleration sensor 15 and a second correction coefficient based on the acceleration detected by the acceleration sensor 16 are obtained, and one of them is selected and a gain is applied. However, the present invention is not limited to this, and there is no particular restriction on the method of obtaining the translational shake amount. For example, it is also possible to apply the present invention to an imaging device having a configuration capable of obtaining only one of the first and second correction coefficients described in the first embodiment. In addition, a method of performing a second-order integration of the acceleration detected by the acceleration sensor, inertial navigation, etc. can also be used.

[0072] <Third embodiment> Next, a third embodiment of the present invention will be described. In the above-described first embodiment, a method for calculating the first correction coefficient has been described using the motion vector detection unit 201 and the angular velocity detection unit 15. However, when the motion vector detection unit 201 cannot detect a motion vector, when the photographer is not holding a camera, or when the photographer is performing a panning motion, an error may occur and the calculation of the translational shake may not be stable.

[0073] In the third embodiment, a method of calculating the first correction coefficient will be described for cases where the motion vector detection unit 201 cannot detect a motion vector, where the photographer is not ready, or where the photographer is performing a panning operation.

[0074] Comparator 216 compares the output of LPF 211 with the output of LPF 212. In the first embodiment, a method of performing calculations using all of the outputs of LPF 211 and LPF 212 was described. In contrast, in the third embodiment, a calculation method in the case where a timing at which data cannot be acquired is included as described above will be described with reference to Fig. 7. Note that the same reference numerals are used to designate the same components as those in Fig. 2, and description thereof will be omitted.

[0075] In the third embodiment, the outputs of LPF 211 and LPF 212 are input to a data determination unit 703. The data determination unit 703 determines whether or not at least one of the following applies: the detection accuracy of the motion vector of the motion vector detection unit 201 has decreased, the motion vector cannot be detected, the photographer is not holding a camera, or the photographer is performing a panning operation. If any of the following applies, the data determination unit 703 outputs a flag together with the signals of LPF 211 and LPF 212, and otherwise outputs the signals of LPF 211 and LPF 212.

[0076] The data storage unit 704 stores the output of the data judgment unit 703, including the flag, as time-series data. The comparator 216 calculates a first correction coefficient from the ratio of the output signals of the LPF 211 and the LPF 212 output from the data storage unit 704. At this time, the comparator 216 inputs the output signals of the LPF 211 and the LPF 212, which are obtained within the same length of time as the shooting time, and which are the total number of data pieces including data pieces with and without flags. Then, the comparator 216 calculates the first correction coefficient using only the data pieces with and without flags. This is because the correction effect is higher when the first correction coefficient is calculated using data pieces obtained within the same length of time as the shooting time. However, depending on the data sampling period, the number of data pieces may be small when the shooting time is short. In such a case, the first correction coefficient is calculated using data pieces obtained in a time longer than the shooting time.

[0077] The reliability calculation unit 707 receives the output signal of the data storage unit 704 and calculates the reliability of the first correction coefficient calculated by the comparator 216. The reliability is calculated by comparing the maximum number of data usable for the calculation of the first correction coefficient with the number of data usable for the current calculation of the first correction coefficient. For example, when the number of data usable for the first correction coefficient becomes half or less, the reliability of the first correction coefficient is significantly low, so the first correction coefficient calculated in the past is also used for the calculation of the final first correction coefficient. Also, for example, the reliability may be calculated from the number of data currently usable with respect to the number of data for which the first correction coefficient is stably calculated. In other words, the reliability can be calculated by any method as long as it can be calculated whether the number of data usable for the calculation of the current correction coefficient is reliable.

[0078] The correction coefficient storage unit 705 receives the output signal from the data storage unit 704, the first correction coefficient calculated by the comparator 216, and the reliability calculated by the reliability calculation unit 707. Then, the flag of the data immediately preceding the latest data is set. I , the latest data is flagged Not Furthermore, if the reliability calculated by the reliability calculation unit 707 is equal to or greater than a certain value, the first correction coefficient is stored.

[0079] The stored correction coefficient reliability calculation unit 706 updates the reliability of the first correction coefficient stored in the correction coefficient storage unit 705. This is because it is considered that the reliability of the first correction coefficient stored in the correction coefficient storage unit 705 decreases with the passage of time. Therefore, the stored correction coefficient reliability calculation unit 706 calculates the elapsed time from the time stored in the correction coefficient storage unit 705, and performs a calculation to lower the reliability according to the elapsed time. The calculation of the reliability coefficient at this time may be a linear calculation or a nonlinear calculation.

[0080] The correction coefficient calculation unit 708 calculates a final first correction coefficient from the outputs of the reliability calculation unit 707, the comparator 216, the correction coefficient storage unit 705, and the stored correction coefficient reliability calculation unit 706. Specifically, in the correction coefficient calculation unit 708, when the reliability output by the reliability calculation unit 707 is equal to or higher than a certain threshold, the correction coefficient calculation unit 708 outputs the first correction coefficient output by the comparator 216 as the final first correction coefficient. On the other hand, when the reliability output by the reliability calculation unit 707 is lower than a certain threshold and the reliability output by the stored correction coefficient reliability calculation unit 706 is equal to or higher than a certain threshold, the correction coefficient calculation unit 708 performs a weighted average of the first correction coefficient output by the comparator 216 and the first correction coefficient stored in the correction coefficient storage unit 705 according to their respective reliability, and outputs the weighted average as the final first correction coefficient. In addition, when the reliability output by the reliability calculation unit 707 is lower than a certain threshold and the reliability output by the stored correction coefficient reliability calculation unit 706 is lower than a certain threshold, the correction coefficient calculation unit 708 outputs 0 as the first correction coefficient.

[0081] Thereafter, the determination unit 218 determines whether to use the first correction coefficient or the second correction coefficient, as described in the first embodiment.

[0082] Next, a flow of processing performed by the correction coefficient calculation unit 708 in the third embodiment will be described with reference to the flowchart of Fig. 8. Note that this processing is implemented as the calculation processing of the first correction coefficient performed in S402 of Fig. 4.

[0083] In S601, the data determination unit 703 determines whether at least one of the following conditions is met: the detection accuracy of the motion vector detection unit 201 has decreased or is unable to detect the motion vector, the photographer is not holding a camera, or the photographer is performing a panning operation.

[0084] Then, in S602, if any of the conditions is met according to the determination result of S601, the data storage unit 704 stores the flags output together with the output signals of the LPFs 211 and 212. If none of the conditions is met, the data storage unit 704 stores the output signals of the LPFs 211 and 212.

[0085] In S603, the comparator 216 calculates a first correction coefficient from the output signals of the LPFs 211 and 212 for which no flags are set. In S604, the reliability calculation unit 707 calculates the reliability of the first correction coefficient calculated by the comparator 216.

[0086] Next, in S605, the correction coefficient storage unit 705 detects that the data immediately before the latest data is flagged. I , the latest data is flagged Not It is also determined whether the reliability calculated by reliability calculation unit 707 is equal to or greater than a certain value. If the above conditions are met, the process proceeds to S608, and if not, the process proceeds to S606.

[0087] In S608, the correction coefficient storage unit 705 stores the first correction coefficient, and the process proceeds to S609. In S609, the stored correction coefficient reliability calculation unit 706 adjusts the reliability in accordance with the passage of time.

[0088] In S606, the correction coefficient calculation unit 708 judges whether the reliability is equal to or greater than a threshold value. If it is equal to or greater than the threshold value, the process proceeds to S607, where the first correction coefficient calculated in S603 is used as the final first correction coefficient.

[0089] On the other hand, if it is less than the threshold, the process proceeds to S610, where the correction coefficient calculation unit 708 judges whether the reliability adjusted in S609 is equal to or greater than the threshold. If it is equal to or greater than the threshold, the process proceeds to S611, where the first correction coefficient calculated in S603 and S 608 The final first correction coefficient is determined by taking a weighted average of the first correction coefficient stored in the correction coefficient storage unit 705 in step 100 and the first correction coefficient stored in the correction coefficient storage unit 705 in step 100 based on their respective reliability.

[0090] In S610, if the adjusted reliability is less than the threshold, the process proceeds to S612, where the final first correction coefficient is set to 0. Here, the coefficient may be set to 0 so that no correction is performed, or a signal indicating that correction cannot be performed by this method may be output. When the first correction coefficient is determined, the process ends.

[0091] As described above, according to the third embodiment, in addition to the effects of the first embodiment, it is possible to further suppress detection errors of translational shake based on a motion vector.

[0092] <Other embodiments> The present invention may be applied to a system consisting of multiple devices (such as a host computer, interface device, scanner, video camera, etc.), or to an apparatus consisting of a single device (such as a copier, facsimile machine, etc.).

[0093] The present invention can also be realized by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0094] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0095] 1: camera body, 2: lens unit, 5: camera system control unit, 6: image sensor, 9: anti-vibration lens, 13: lens driving unit, 14: shake correction unit, 15: angular velocity detection unit, 16: acceleration detection unit, 201: motion vector detection unit, 204: angular shake calculator, 207: translational shake extractor, 214: subject information detection unit, 215: shake information detection unit, 216, 217: comparator, 218: judgment unit, 220: translational shake amount calculation unit, 401: translational shake calculator, 704: data storage unit, 705: correction coefficient storage unit, 706: stored correction coefficient reliability calculation unit, 707: reliability calculation unit, 708: correction coefficient calculation unit

Claims

1. a motion vector detection means for detecting a motion vector from images repeatedly output from the imaging element; a first calculation means for calculating a first correction coefficient for calculating a translational shake amount based on the motion vector; a second calculation means for calculating a second correction coefficient for calculating the translational shake amount based on the acceleration of the shake of the imaging device; a determination means for selecting either the first correction coefficient or the second correction coefficient based on information regarding accuracy of detection of the motion vector by the motion vector detection means; a calculation means for calculating the translational shake amount by using the first correction coefficient or the second correction coefficient selected by the determination means, the determination means selects the second correction coefficient when a ratio of a translational shake image plane velocity on the image plane of the image sensor that was previously calculated based on the motion vector in order to obtain the first correction coefficient to an angular shake image plane velocity on the image plane of the image sensor that is calculated based on an angular velocity of shake of the imaging device is lower than a predetermined ratio, and selects the first correction coefficient in other cases.

2. A motion vector detection means for detecting a motion vector from images repeatedly output from an imaging element; a first calculation means for calculating a first correction coefficient for calculating a translational shake amount based on the motion vector; a second calculation means for calculating a second correction coefficient for calculating the translational shake amount based on the acceleration of the shake of the imaging device; a determination means for selecting either the first correction coefficient or the second correction coefficient based on information regarding accuracy of detection of the motion vector by the motion vector detection means; a calculation means for calculating the translational shake amount by using the first correction coefficient or the second correction coefficient selected by the determination means, the determination means estimates the translational shake image plane velocity from the rotational shake image plane velocity calculated based on the newly detected angular velocity based on a ratio between a rotational shake image plane velocity on the image plane of the image sensor previously calculated based on the angular velocity of shake of the imaging device and a translational shake image plane velocity on the image plane of the image sensor previously calculated based on the motion vector in order to obtain the first correction coefficient, and selects the second correction coefficient when a ratio of the estimated translational shake image plane velocity to the rotational shake image plane velocity used in the estimation is lower than a predetermined ratio, and selects the first correction coefficient in other cases.

3. The image stabilization control device according to claim 1 or 2, characterized in that the information includes whether or not the image contains a repeating pattern, the contrast of the image, the brightness of the image, the movement of the subject contained in the image, and the state of the background of the image, and the judgment means judges that the accuracy is low when the information indicates at least any of the following states: a state in which the image contains the repeating pattern, a state in which the contrast is lower than a predetermined contrast, a state in which the brightness is lower than a predetermined brightness, a state in which the movement of the subject is greater than a predetermined movement, and a state in which the movement of the background is greater than a predetermined movement.

4. a gain unit that determines a gain based on a shooting time and a shooting distance when the image is shot, and multiplies the gain by the first correction coefficient or the second correction coefficient selected by the determination unit, 4. The image stabilization control device according to claim 1, wherein the calculation means determines the translational shake amount by using the first correction coefficient or the second correction coefficient multiplied by the gain.

5. 5. The image stabilization control device according to claim 4, wherein the gain means sets a larger gain when the photographing time is a first photographing time than when the photographing time is a second photographing time that is shorter than the first photographing time.

6. 6. The image stabilization control device according to claim 4, wherein the gain means sets a larger gain when the shooting distance is a first shooting distance than when the shooting distance is a second shooting distance that is longer than the first shooting distance.

7. A gain table is provided which holds gains corresponding to a plurality of different shooting times and a plurality of different shooting distances, 7. The image stabilization control device according to claim 4, wherein the gain means determines the gain by referring to the gain table.

8. a plurality of gain tables corresponding to the mode of the imaging device and the characteristics of a photographer; 8. The image stabilization control device according to claim 7, wherein said gain means refers to a gain table among said plurality of gain tables according to a mode of said image pickup device and characteristics of a photographer.

9. A motion vector detection means for detecting a motion vector from images repeatedly output from an imaging element; a first calculation means for calculating a first correction coefficient for calculating a translational shake amount based on the motion vector; a second calculation means for calculating a second correction coefficient for calculating the translational shake amount based on the acceleration of the shake of the imaging device; a determination means for selecting either the first correction coefficient or the second correction coefficient based on information regarding accuracy of detection of the motion vector by the motion vector detection means; a calculation means for calculating the translational shake amount by using the first correction coefficient or the second correction coefficient selected by the determination means; a reliability calculation means for calculating a reliability of the first correction coefficient; a storage means for storing the first correction coefficient and the reliability of the first correction coefficient; an adjustment means for adjusting the reliability stored in the storage means in accordance with the elapsed time since the reliability was stored; a third calculation means for determining the first correction coefficient to be input to the determination means, from the newly calculated first correction coefficient and the first correction coefficient held in the holding means, based on the reliability of the first correction coefficient newly calculated by the first calculation means and the reliability adjusted by the adjustment means; An anti-vibration control device comprising:

10. The third calculation means includes: selecting the newly calculated first correction coefficient when the reliability of the newly calculated first correction coefficient is equal to or greater than a first threshold; when the reliability of the newly calculated first correction coefficient is less than a threshold and the adjusted reliability is equal to or greater than a second threshold, a weighted average of the newly calculated first correction coefficient and the held first correction coefficient is calculated based on the reliability of the newly calculated first correction coefficient and the adjusted reliability; When the reliability of the newly calculated first correction coefficient is less than a threshold and the adjusted reliability is less than a second threshold, the first correction coefficient is not output.

10. The vibration isolation control device according to claim 9.

11. 11. The vibration isolation control device according to claim 9, wherein the adjustment means reduces the reliability in accordance with the time that has elapsed since the reliability was held by the holding means.

12. The image stabilization control device according to any one of claims 9 to 11, characterized in that the first calculation means calculates the first correction coefficient based on a plurality of the motion vectors output during a period of time equal to the shooting time, excluding the motion vectors for which the information indicates low accuracy.

13. An image stabilization control device according to any one of claims 9 to 12, characterized in that when a shooting time is shorter than a predetermined shooting time, the first calculation means calculates the first correction coefficient based on a plurality of motion vectors output for a time longer than the shooting time, excluding the motion vectors whose information indicates low accuracy.

14. An anti-vibration control device according to any one of claims 1 to 13, The imaging element; An imaging device comprising:

15. a motion vector detection step in which a motion vector detection means detects a motion vector from an image repeatedly output from an image pickup device; a first calculation step in which a first calculation means calculates a first correction coefficient for calculating a translational shake amount based on the motion vector; a second calculation step in which a second calculation means calculates a second correction coefficient for calculating the translational shake amount based on the acceleration of the shake of the imaging device; a determination step of determining whether the first correction coefficient or the second correction coefficient is selected based on information regarding accuracy of detection of the motion vector in the motion vector detection step; a calculation step of calculating the translational shake amount by using the first correction coefficient or the second correction coefficient selected in the determination step, a vibration reduction control method, in which, in the determination step, when a ratio of a translational shake image plane velocity on the image plane of the image sensor that was previously calculated based on the motion vector in order to obtain the first correction coefficient to an angular shake image plane velocity on the image plane of the image sensor that is calculated based on an angular velocity of shake of the imaging device is lower than a predetermined ratio, the second correction coefficient is selected, and the first correction coefficient is selected in other cases.

16. A motion vector detection step in which a motion vector detection means detects a motion vector from images repeatedly output from an image pickup device; a first calculation step in which a first calculation means calculates a first correction coefficient for calculating a translational shake amount based on the motion vector; a second calculation step in which a second calculation means calculates a second correction coefficient for calculating the translational shake amount based on the acceleration of the shake of the imaging device; a determination step of determining whether the first correction coefficient or the second correction coefficient is selected based on information regarding accuracy of detection of the motion vector in the motion vector detection step; a calculation step of calculating the translational shake amount by using the first correction coefficient or the second correction coefficient selected in the determination step, a vibration reduction control method, in which, in the determination step, based on a ratio between an angular shake image plane velocity on the image plane of the image sensor previously calculated based on an angular velocity of shake of the imaging device and a translational shake image plane velocity on the image plane of the image sensor previously calculated based on the motion vector in order to obtain the first correction coefficient, the translational shake image plane velocity is estimated from the angular shake image plane velocity calculated based on the newly detected angular velocity, and when a ratio of the estimated translational shake image plane velocity to the angular shake image plane velocity used in the estimation is lower than a predetermined ratio, the second correction coefficient is selected, and the first correction coefficient is selected in other cases.

17. A motion vector detection step in which a motion vector detection means detects a motion vector from images repeatedly output from an image pickup device; a first calculation step in which a first calculation means calculates a first correction coefficient for calculating a translational shake amount based on the motion vector; a second calculation step in which a second calculation means calculates a second correction coefficient for calculating the translational shake amount based on the acceleration of the shake of the imaging device; a reliability calculation step in which a reliability calculation means calculates the reliability of the first correction coefficient; a storing step of storing the first correction coefficient and the reliability of the first correction coefficient in a storing means; an adjustment step in which an adjustment means adjusts the reliability held in the holding means in accordance with the elapsed time since the reliability was held; a third calculation step in which a third calculation means determines a third correction coefficient from the newly calculated first correction coefficient and the first correction coefficient held in the holding step, based on the reliability of the first correction coefficient newly calculated in the first calculation step and the reliability adjusted in the adjustment step; a determination step of determining whether the third correction coefficient or the second correction coefficient is selected based on information regarding accuracy of detection of the motion vector in the motion vector detection step; a calculation step of calculating the translational shake amount by using the third correction coefficient or the second correction coefficient selected in the determination step; 13. A vibration isolation control method comprising:

18. A program for causing a computer to function as each of the means of the vibration isolation control device according to any one of claims 1 to 13.

19. A computer-readable storage medium storing the program according to claim 18.

Images