Image shake correction device, optical instrument, method for controlling image shake correction device, and program

Abstract

To provide an image shake correction device that can accurately perform image shake correction.SOLUTION: An image shake correction device (100) has: first acquisition means (101) that acquires a first shake signal; second acquisition means (102) that acquires a second shake signal having a larger amount of noise in a high-frequency band and a smaller amount of noise in a low-frequency band than the first shake signal; generation means (111) that generates a composite shake correction signal on the basis of a first signal included in the first shake signal and including a higher frequency than a divided band, and a second signal included in the second shake signal and including a lower frequency than the divided band; and setting means (110) that sets the divided band. The setting means changes the divided band according to photographing condition information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image stabilization device, an optical device, a control method for an image stabilization device, and a program. [Background technology]

[0002] In imaging devices (optical instruments) such as digital cameras, a technique is known for correcting image blur caused by shake applied to the device by moving an imaging element such as a CMOS sensor or some optical elements of the imaging optical system in a direction perpendicular to the optical axis. Such imaging devices with an image blur correction function generally detect shake applied to the imaging device using a gyro sensor.

[0003] Patent Document 1 discloses a method of calculating an offset component of a gyro sensor detection signal based on a composite signal obtained by a complementary filter of the detection signals of an acceleration sensor or a geomagnetic sensor and the gyro sensor detection signal, and subtracting the offset component from the gyro sensor detection signal. Patent Document 2 discloses a method of changing a weight of the composite based on the reliability of a motion vector when combining a motion vector and a gyro sensor using a complementary filter or the like. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2018-116134 [Patent Document 2] Japanese Patent Application Publication No. 2018-205551 Summary of the Invention [Problem to be solved by the invention]

[0005] When calculating a composite signal using a complementary filter or similar from the low-frequency components of an acceleration sensor or geomagnetic sensor and the high-frequency components of a gyro sensor, the noise characteristics of the composite signal change depending on the subbands of the low-frequency and high-frequency components. For this reason, it is important to set the subbands appropriately.

[0006] However, Patent Document 1 does not disclose how to set the sub-bands. Patent Document 2 discloses how to set the sub-bands, but does not consider how to set the sub-bands when combining with sensors other than the motion vector.

[0007] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an image stabilization device that can perform image stabilization with high accuracy. [Means for solving the problem]

[0008] An image stabilization device according to one aspect of the present invention includes a first acquisition means for acquiring a first blur signal, a second acquisition means for acquiring a second blur signal having a higher amount of noise in a high frequency band and a lower amount of noise in a low frequency band than the first blur signal, a generation means for generating a composite blur stabilization signal based on a first signal of the first blur signal that includes frequencies higher than the divided bands and a second signal of the second blur signal that includes frequencies lower than the divided bands, and a setting means for setting the divided bands, wherein the setting means changes the divided bands in accordance with shooting condition information.

[0009] Other objects and features of the present invention will be described in the following embodiments. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide an image stabilization device that can perform image stabilization with high accuracy. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram of an image stabilization device according to a first embodiment. [Figure 2] 3A and 3B are diagrams illustrating the relationship between the imaging device and the axial directions in each embodiment. [Figure 3] 4 is a flowchart showing the processing of the image stabilization device in the first embodiment. [Figure 4]FIG. 4 is a diagram showing the relationship between target subbands and exposure time in the first embodiment. [Figure 5] FIG. 4 is a diagram showing the time transition of divided bands in the first embodiment. [Figure 6] FIG. 10 is a block diagram of an image stabilization device according to a second embodiment. [Figure 7] 10 is a flowchart showing the processing of an image stabilization device according to a second embodiment. [Figure 8] FIG. 10 is a diagram showing the time transition of divided bands in the second embodiment. [Figure 9] 5A and 5B are schematic diagrams illustrating frequency characteristics of complementary filters in each embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe multiple features, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0013] (First embodiment) 1 is a diagram showing the configuration of an image stabilization device 100 according to a first embodiment of the present invention. The image stabilization device 100 is provided in an optical device such as an imaging device or a lens device.

[0014] The first shake detection means (first acquisition means) 101 is a gyro sensor that detects (acquires) shake in the rotation axis directions (pitch / yaw / roll) around three mutually orthogonal axes (X / Y / Z). FIG. 2 is a diagram showing the relationship between the imaging device and the axial directions. The horizontal direction of the imaging device is the X axis, the vertical direction of the imaging device is the Y axis, and the optical axis direction is the Z axis, with the direction around the X axis being the Pitch axis, the direction around the Y axis being the Yaw axis, and the direction around the Z axis being the Roll axis. The first shake detection means 101 detects a first shake signal (angular velocity signal) applied to the imaging device and outputs the first shake signal to the first shake correction signal calculation means (first calculation means) 107 and the composite shake correction signal calculation means (generation means) 111.

[0015] The second shake detection means (second acquisition means) 102 is at least one of an acceleration sensor or a geomagnetic sensor that detects (acquires) shake in three mutually orthogonal axis directions (X / Y / Z). The second shake detection means 102 detects a second shake signal (at least one of an acceleration signal or a geomagnetic signal) applied to the imaging device, and outputs the second shake signal to the angle attitude calculation means 103, the LPF 105, and the shooting condition information acquisition means 108. In this embodiment, the second shake signal has a characteristic that it has more noise in the high frequency band and less noise in the low frequency band than the first shake signal.

[0016] The gyro sensor serving as the first shake detection means 101 is a sensor capable of detecting rotational shake, which is one of the shakes applied to the device, in terms of angular velocity, and improving the detection accuracy of the gyro sensor is extremely important. One of the important challenges in improving the detection accuracy of the gyro sensor is how to remove the low-frequency noise (offset component) that the gyro sensor possesses. If the gyro sensor signal is used without removing the offset component, when the output signal of the gyro sensor is integrated and treated as an angle, so-called drift occurs, which gradually accumulates as an integration error, making it difficult to perform accurate image blur correction.

[0017] Therefore, it is preferable that the second shake detection means 102 detects shake by combining detection information from another sensor with different noise characteristics, such as an acceleration sensor or a geomagnetic sensor, with detection information from a gyro sensor. Generally, acceleration sensors and geomagnetic sensors tend to have less low-frequency noise and more high-frequency noise than gyro sensors. Therefore, by combining the low-frequency components of the acceleration sensor or geomagnetic sensor with the high-frequency components of the gyro sensor using a complementary filter or the like, shake detection can be performed with higher accuracy.

[0018] When a composite signal is calculated using a complementary filter or the like from the low-frequency components of an acceleration sensor or geomagnetic sensor and the high-frequency components of a gyro sensor, the noise characteristics of the composite signal change depending on the split band between the low-frequency and high-frequency components. Setting the split band low makes it difficult for high-frequency noise from the acceleration sensor or geomagnetic sensor to be superimposed, but the low-frequency noise from the gyro sensor cannot be sufficiently removed, resulting in a composite signal with low high-frequency noise and high low-frequency noise. On the other hand, setting the split band high makes it easy for high-frequency noise from the acceleration sensor or geomagnetic sensor to be superimposed, but the low-frequency noise from the gyro sensor can be removed, resulting in a composite signal with high high-frequency noise and low low-frequency noise. In other words, the lower the split band, the lower the high-frequency noise in the composite signal, and the higher the split band, the higher the high-frequency noise. However, the lower the split band, the higher the low-frequency noise, and the higher the split band, the lower the low-frequency noise. Thus, because there is a trade-off between low-frequency noise reduction and high-frequency noise superposition depending on the split band, it is important to appropriately set the split band.

[0019] The angular attitude calculation means 103 calculates the angular attitude (angular attitude information) of the imaging device by finding Euler angles using a known method using the second shake signal. Note that the angular attitude may be calculated using an output signal of the LPF 105, which will be described later. The calculated angular attitude information is output to the angle calculation axis determination means 104.

[0020] Based on the angular attitude information, the angle calculation axis determination means 104 determines whether to use the detection signal of the acceleration sensor or the geomagnetic sensor to calculate the angles of the rotational axes of pitch, yaw, and roll. For example, when the imaging device is in a substantially normal position and attitude (a position in which the Y axis is parallel to the direction of gravity and the X and Z axes are perpendicular), the angle of the pitch / roll axis direction is calculated based on the detection signal of the acceleration sensor, and the angle of the yaw axis direction is calculated based on the detection signal of the geomagnetic sensor. This is because rotation in the yaw axis direction does not involve changes in gravitational acceleration along the X, Y, and Z axes, and therefore the angle of the yaw axis direction cannot be calculated based on the detection signal of the acceleration sensor. Alternatively, when the imaging device is in a substantially vertical position and attitude (a position in which the X axis is parallel to the direction of gravity and the Y and Z axes are perpendicular), the angle of the yaw / roll axis direction is calculated based on the detection signal of the acceleration sensor, and the angle of the pitch axis direction is calculated based on the detection signal of the geomagnetic sensor. This is for the same reasons as described above.

[0021] LPF 105 removes high frequency components from the second shake signal and outputs the resulting signal (second signal) to angle calculation means 106. The cutoff frequency of LPF 105 is preferably about 1 to 10 Hz, but is not limited to this.

[0022] The angle calculation means 106 calculates angles (angle signals) in the pitch / yaw / roll axis directions according to the combination of each sensor detection signal and the rotation axis directions determined by the angle calculation axis determination means 104. The calculated angle signals are output to the composite shake correction signal calculation means 111.

[0023] The first shake correction signal calculation means 107 converts the angular velocity signal, which is the first shake signal, into an angle by integration processing to calculate the first shake correction signal, and outputs the first shake correction signal to the shake correction signal selection means (selection means) 112.

[0024] Shooting condition information acquisition means 108 acquires shooting condition information of the image capture device and outputs it to division band setting means (setting means) 110. The shooting condition information includes at least one of the exposure time, the shutter type of the image capture device, and the norm of the second blur signal in three mutually orthogonal axis directions described below. The shutter type of the image capture device generally includes three types: a mechanical shutter type (both the first and second curtains are mechanical shutters), an electronic first curtain shutter type (an electronic first curtain shutter and a mechanical second curtain shutter), and an electronic shutter (both the first and second curtains are electronic shutters). However, the types are not limited to these.

[0025] Counter 109 counts at least one of the elapsed time from the start of calculation by LPF 105 and the elapsed time after the mechanical shutter is driven, and notifies sub-band setting means 110 of the count value. Sub-band setting means 110 sets sub-bands based on the shooting condition information and the count value, and notifies composite shake correction signal calculation means 111 of the sub-bands. In other words, sub-band setting means 110 changes the sub-bands in accordance with the shooting condition information.

[0026] The composite shake correction signal calculation means 111 uses a complementary filter to combine the high-frequency component signal (first signal) obtained from the first shake detection means 101 and the low-frequency component signal (second signal) obtained from the second shake detection means 102 to calculate a composite shake correction signal. In this embodiment, the composite shake correction signal calculation means 111 combines the first signal and the second signal based on the divided bands notified by the divided band setting means 110.

[0027] The composite blur correction signal calculation means 111 constitutes at least a part of a complementary filter that combines the first signal and the second signal. The divided band corresponds to the cutoff frequency of the complementary filter. The complementary filter has an HPF that passes the first signal out of the first blur signal detected by the first blur detection means 101, and an LPF that passes the second signal out of the second blur signal detected by the second blur detection means 102.

[0028] Fig. 9 is a schematic diagram showing the frequency characteristics of the complementary filter in this embodiment. In Fig. 9, the vertical axis represents the gain (dB) of the complementary filter, and the horizontal axis represents the frequency (Hz). The complementary filter has a characteristic of providing an appropriate gain to each of the first and second blur signals according to the frequency (frequency band), and the sum of the gains of the HPF and LPF is 1 in any frequency band. In Fig. 9, the frequency at which the gains of the first and second signals are equal (the frequency at which the first and second signals intersect) is the cutoff frequency, i.e., the subband Fc.

[0029] The first signal is a signal of the first blur signal having a higher frequency than the divided band, and the second signal is a signal of the second blur signal having a lower frequency than the divided band. However, this embodiment is not limited to this, and it is sufficient that the first signal is a signal of the first blur signal having a higher frequency than the divided band, and the second signal is a signal of the second blur signal having a lower frequency than the divided band.

[0030] The image stabilization signal selection means 112 selects either the first image stabilization signal or the composite image stabilization signal based on the divided bands, and outputs the selected signal (final image stabilization signal) to the image stabilization drive signal calculation means (second calculation means) 113.

[0031] The image blur correction drive signal calculation means 113 performs known calculations, such as multiplying the blur correction signal output from the blur correction signal selection means 112 by the focal length or the drive resolution of the image blur correction member (correction member) 114. The image blur correction drive signal calculation means 113 then calculates an image blur correction drive signal for driving the image blur correction member 114, and outputs the image blur correction drive signal to the image blur correction member 114. The image blur correction member 114 performs image blur correction by being driven in accordance with the image blur correction drive signal.

[0032] Note that some of the above-mentioned means are realized by a processor such as one or more CPUs (Central Processing Units) reading and executing a program. For example, the angle attitude calculation means 103, the angle calculation axis determination means 104, the angle calculation means 106, and the first image stabilization signal calculation means 107 may be realized by a processor. In addition, the shooting condition information acquisition means 108, the divided band setting means 110, the composite image stabilization signal calculation means 111, the image stabilization signal selection means 112, and the image stabilization drive signal calculation means 113 may also be realized by a processor.

[0033] Next, the processing of the image stabilization device 100 (image stabilization operation and control method of the image stabilization device 100) will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the processing of the image stabilization device 100.

[0034] "START" in FIG. 3 indicates a start due to an ON state, such as when the imaging device is powered on or the image stabilization mode is on. In the ON state, the process returns to START without transitioning to the final END, and the series of processes in FIG. 3 is repeated. The cycle of the series of processes follows the detection cycle of the first shake detection means 101 and the second shake detection means 102 and the calculation cycle of the complementary filter. "END" in FIG. 3 indicates an end due to when the imaging device is powered off or the image stabilization mode is turned off, etc.

[0035] First, in step S301, the first shake detection means 101 detects a first shake signal applied to the imaging device. Then, in step S302, the first shake correction signal calculation means 107 converts the first shake signal into an angle through integration processing to calculate a first shake correction signal. Then, in step S303, the second shake detection means 102 detects a second shake signal applied to the imaging device. Then, in step S304, the angular attitude calculation means 103 calculates the angular attitude of the imaging device from the second shake signal. Then, in step S305, the angle calculation axis determination means 104 determines a combination of each rotation axis direction and sensor type based on the angular attitude of the imaging device. Then, in step S306, the LPF 105 removes high-frequency components from the second shake signal.

[0036] Next, in step S307, the angle calculation means 106 calculates the angle from the low-frequency component of the second shake signal. As an example, the angle calculation when the image capture device is in an approximately normal position and orientation will be described. The angle θ of the roll axis a_Roll and the angle of the pitch axis θ a_Pitch is the low frequency component (a x , a y , a z ) are calculated using the following formulas (1) and (2), respectively.

[0037]

number

[0038]

number

[0039] Also, the angle of the Yaw axis is θ m_Yaw is the angle θ of the roll axis a_Roll and the angle θ of the pitch axis a_Pitch and the low-frequency component (m x , m y , m z ) is used to calculate the following equation (3):

[0040]

number

[0041] In addition, the angle obtained by integrating the gyro signal is a relative angle from the start time of the integration calculation, whereas the angle calculated above (θ a_Roll , θ a_Pitch , θ m_Yaw) must be converted into a relative angle because the absolute angle with respect to the direction of gravity or orientation is calculated. These absolute angles can be converted into relative angles by subtracting a fixed absolute angle value at an arbitrary time from the absolute angle calculated in time series. The arbitrary time may be, for example, the time when the imaging device or the image stabilization device is turned on, or the time when the processing of step S310 or the processing of step S320 is performed.

[0042] Next, in step S308, the shooting condition information acquisition means 108 calculates the norm of the second shake signal from the second shake signal. The norm of the second shake signal is calculated using the second shake signals in the three axial directions (X, Y, Z) according to the following equation (4).

[0043]

number

[0044] The second shake signal has a large amount of noise in the high frequency band, so the norm of the second shake signal may be calculated using the low frequency components of the second shake signal output from LPF 105.

[0045] Next, in step S309, the sub-band setting means 110 determines whether the elapsed time since the start of calculation by the LPF 105 and the elapsed time since the mechanical shutter was driven, counted by the counter 109, have each exceeded a predetermined time. The sub-band setting means 110 also determines whether the norm of the second shake signal exceeds the threshold value TH. Based on these determination results, the sub-band setting means 110 sets the sub-bands in step S310 and thereafter.

[0046] If either the elapsed time from the start of calculation of the LPF 105 or the elapsed time after driving the mechanical shutter is less than a predetermined time, or if the norm of the second shake signal exceeds the threshold value TH, the process proceeds to step S310. On the other hand, if not, the process proceeds to step S313.

[0047] The predetermined time corresponding to the elapsed time from the start of calculation by LPF 105 is preferably set to approximately the time constant of LPF 105. This is because the output signal from LPF 105 contains a transient response and does not provide an accurate output until approximately the time constant has elapsed. The predetermined time corresponding to the elapsed time after driving the mechanical shutter is preferably set to the time until vibrations of the imaging device are sufficiently attenuated by driving the mechanical shutter.

[0048] Regarding the threshold for the norm of the second shake signal, the norm of the acceleration signal is determined based on whether the absolute value of the difference from the gravitational acceleration of 1 G is equal to or greater than the threshold. For example, if the threshold is 0.2 G, the norm of the acceleration signal is determined to not exceed the threshold if it is between 0.8 and 1.2 G. If the norm of the acceleration signal significantly deviates from 1 G, this may occur when the user pans the imaging device significantly or when the imaging device is mounted on a moving object such as a car. In this case, the angle cannot be accurately calculated from the acceleration signal. Similarly, the norm of the geomagnetic sensor can be determined based on whether the absolute value of the difference from the norm obtained when the geomagnetic sensor is calibrated using a known method is equal to or greater than the threshold. Generally, when a magnetic object such as iron approaches the geomagnetic sensor, the threshold may be exceeded. Because the causes of changes in the acceleration signal norm and the geomagnetic signal norm are different, it is preferable to determine them independently for each sensor.

[0049] In step S310, the sub-band setting means 110 sets the sub-band (Hz) of the complementary filter to a high band. Meanwhile, in step S311, the sub-band setting means 110 sets (determines) a target sub-band based on the exposure time obtained from the photographing condition information acquisition means 108.

[0050] For example, as shown in FIG. 4, the exposure time and the target division band may be associated with each other. FIG. 4 is a diagram showing the relationship between the target division band and the exposure time. In FIG. 4, the horizontal axis represents the exposure time, and the vertical axis represents the target division band. In this example, a lower limit Fc1 and an upper limit Fc2 of the target division band are provided. Even when the exposure time is short, the target division band is set so as not to be lower than the lower limit Fc1. On the other hand, even when the exposure time is long, the target division band is set so as not to be higher than the upper limit Fc2. However, it is only necessary that the exposure times Tv1 and Tv2 satisfy the relationship Tv1 <= Tv2, and it is only necessary that the lower limit Fc1 and the upper limit Fc2 satisfy the relationship Fc1 < Fc2. Alternatively, Fc1 = 0 (Hz) may be used. Fc2 may be determined according to the respective noise characteristics of the first shake detection means 101 and the second shake detection means 102. For example, Fc2 is about 0.005 to 0.5 (Hz).

[0051] Thus, in the present embodiment, the division band setting means 110 sets a higher division band as the exposure time is longer, and sets a lower division band as the exposure time is shorter. That is, when the exposure time is the first exposure time (Tv1), the division band setting means 110 sets the division band to the first band (Fc1), and when the exposure time is the second exposure time (Tv2) longer than the first exposure time, the division band setting means 110 sets the division band to the second band (Fc2) higher than the first band. Note that the change in the target division band when the exposure time is between the first exposure time and the second exposure time may be a linear change or a non-linear change as shown in FIG. 4.

[0052] Subsequently, in step S312, the division band setting means 110 compares the current division band actually used in the operation of the complementary filter with the target division band. If the current division band is larger than the target division band, the process proceeds to step S313. On the other hand, if the current division band is smaller than the target division band, the process proceeds to step S315.

[0053] In step S313, the sub-band setting means 110 sets the sub-band of the complementary filter lower than that of the previous sample. In steps S314 and S315, the composite shake correction signal calculation means 111 uses the complementary filter to combine the high-frequency components of the first shake signal and the low-frequency components of the second shake signal according to the sub-bands using the following equations (5) and (6), thereby calculating a composite shake correction signal.

[0054]

number

[0055]

number

[0056] where θ is the composite image stabilization signal, F C are the subbands, K are the complementary filter coefficients, and ω g is the first vibration signal, θ am is the second blur signal, T S indicates the sampling time, n indicates the current sample, and (n-1) indicates the sample immediately before the current sample.

[0057] In step S316, the blur correction signal selection means 112 selects the first blur correction signal, and the image blur correction drive signal calculation means 113 calculates the image blur correction drive signal using the first blur correction signal. In step S317, the blur correction signal selection means 112 selects the composite blur correction signal, and the image blur correction drive signal calculation means 113 calculates the image blur correction drive signal using the composite blur correction signal.

[0058] Next, in step S318, the image stabilization device 100 drives the image stabilization member 114 in accordance with the image stabilization drive signal. Next, in step S319, the image stabilization device 100 determines whether the mechanical shutter has been driven by shooting. If the mechanical shutter has been driven, the process proceeds to step S320. On the other hand, if the mechanical shutter has not been driven, the process returns to START or proceeds to END to end this flow.

[0059] In step S320, the image stabilization device 100 initializes the LPF 105 and resets the counter 109. The count value of the counter 109 is used for conditional branching in step S309. Then, the process returns to START or transitions to END to end this flow.

[0060] Next, with reference to Fig. 5, a method for setting the sub-bands by the sub-band setting means 110 and a method for selecting the shake correction signal by the shake correction signal selection means 112 will be described. Fig. 5 is a diagram showing the transition of the sub-bands over time. In Fig. 5, the vertical axis represents the sub-bands, and the horizontal axis represents time. Here, it is assumed that the norm of the second shake signal does not exceed the threshold value except immediately after the mechanical rear-curtain shutter is driven by the electronic front-curtain shutter method and at time T510.

[0061] Time T500 is the time when the image stabilization mode is turned on and the series of calculations in the flowchart of FIG. 3 begin. At the same time, counter 109 begins counting. In the conditional branch of step S309, it is determined that the elapsed time since the start of calculations by LPF 105 is less than a predetermined time. Therefore, the process proceeds to step S310, where sub-band Fc is set to a high sub-band Fc_high. Furthermore, in step S314, a composite image stabilization signal is calculated by a complementary filter based on sub-band Fc_high.

[0062] The period from time T500 to T501 is a waiting period for stabilization of the transient response of the LPF 105, and during this period the sub-band is set to Fc_high. However, the sub-band does not need to be fixed, and may be gradually lowered from this period, for example.

[0063] At time T501, the time elapsed since the start of calculation of the LPF 105 exceeds a predetermined time, so the conditional branch in step S309 causes the process to transition to step S311, where the target sub-band is determined to be low Fc_low based on the set exposure time Tv_long (not shown).

[0064] From time T501 to T502, the current sub-band Fc is greater than the target sub-band Fc_low according to the conditional branch in step S312, so the process proceeds to step S313, where the sub-band Fc is lowered for each sample compared to the previous sample. The sub-band Fc may be lowered linearly as shown in FIG. 5 or nonlinearly. Then, in step S314, a composite image stabilization signal is calculated using a complementary filter based on the current sub-band Fc. By providing a period for gradually lowering the sub-band Fc, the transient response period of the complementary filter can be shortened. That is, immediately after the composite image stabilization signal calculation means 111 starts generating the composite image stabilization signal, the sub-band setting means 110 sets the sub-band to a predetermined high sub-band. Then, the sub-band setting means 110 lowers the sub-band Fc over time to a predetermined low sub-band.

[0065] At time T502, the sub-band Fc becomes the target sub-band Fc_low, and the process transitions to step S315 due to the conditional branch in step S312, where a composite image stabilization signal is calculated using a complementary filter based on the sub-band Fc_low. At time T503, the electronic front-curtain shutter is actuated to start exposure. Because this is not a mechanical shutter, the sub-band is not increased. Furthermore, if the mechanical front-curtain shutter had been actuated to start exposure at time T503, the sub-band Fc would be set to Fc_high from time T503.

[0066] At time T504, the mechanical rear curtain shutter is driven, so the divided band Fc is set to Fc_high. At the same time, in accordance with the conditional branch of step S319, the process proceeds to step S320, where the LPF 105 is initialized and the counter 109 is reset. Furthermore, if the electronic shutter method had been set and exposure had started at time T504 by driving the electronic rear curtain shutter instead of the mechanical rear curtain shutter, the divided band at time T504 would not have been increased but would have remained at Fc_low.

[0067] At time T505, the time elapsed since the start of calculation by the LPF 105 exceeds the predetermined time, but the time elapsed since the mechanical shutter was driven is still less than the predetermined time, so the sub-band Fc remains at Fc_high.

[0068] At time T506, the elapsed time since the mechanical shutter was driven exceeds the predetermined time, so the process transitions to step S311 due to the conditional branch in step S309, and the target subband is determined to be Fc_Low based on the set exposure time Tv_long. In this example, the predetermined time for determining the elapsed time since the mechanical shutter was driven is set to be longer than the predetermined time for determining the elapsed time from the start of calculations by the LPF 105, but this is not the only case. If the image capture device takes time for vibration damping by the mechanical shutter or if the time constant of the LPF is set long, the relationship between the lengths of the predetermined times will be reversed.

[0069] That is, when the shutter method is a mechanical shutter method, the divided band setting means 110 sets the divided bands higher than in the case of an electronic shutter method until a predetermined period has elapsed after the mechanical shutter is driven. Also, when the shutter method is an electronic front-curtain shutter method, the divided band setting means 110 sets the divided bands higher than in the case of an electronic shutter method or after the predetermined period has elapsed until a predetermined period has elapsed after the mechanical rear-curtain shutter is driven. Also, after the predetermined period has elapsed, the divided band setting means 110 lowers the divided bands over time to a predetermined low divided band.

[0070] At time T507, the exposure time is set to Tv_short, which is shorter than Tv_long, and in step S311 for that sample, the target sub-band is determined to be Fc_middle, which is higher than Fc_Low.

[0071] From time T506 to T508, similarly to time T501 to T502, the current sub-band Fc is greater than the target sub-band Fc_middle according to the conditional branch of step S312, and so the process proceeds to step S313. Then, the sub-band Fc of each sample is set lower than the sub-band of the sample immediately preceding that sample. Then, in step S314, a composite shake correction signal is calculated using a complementary filter based on the current sub-band Fc.

[0072] At time T508, the sub-band Fc becomes the target sub-band Fc_middle, and the process proceeds to step S315 due to the conditional branch in step S312, where a composite shake correction signal is calculated by a complementary filter based on the sub-band Fc_middle.

[0073] At time T509, the set exposure time is returned from Tv_short to Tv_long, and in step S311 the target sub-band is determined to be Fc_low. Then, in step S312, a conditional branch is made, and the process proceeds to step S313, where the sub-band is lowered to Fc_low. Then, in the next sample, a conditional branch is made in step S312, and the process proceeds to step S315, where a composite image stabilization signal is calculated using a complementary filter based on the sub-band Fc_low.

[0074] At time T510, when the norm of the second blur signal momentarily exceeds a threshold (predetermined value) due to, for example, an object hitting the imaging device, the process proceeds to step S310 according to the conditional branch of step S309, and the sub-band is set to Fc_high. By temporarily increasing the sub-band, the transient response of the complementary filter is shortened. That is, when the norm of the second blur signal is equal to or greater than a predetermined value, the sub-band setting means 110 sets the sub-band higher than when the norm is less than the predetermined value. Furthermore, when the norm transitions from equal to or greater than the predetermined value to less than the predetermined value, the sub-band setting means 110 lowers the sub-band over time to a predetermined low sub-band.

[0075] The period from time T510 to time T511 is the same as the period from time T500 to time T501, and the period from time T511 to time T512 is the same as the period from time T501 to time T502, so a description thereof will be omitted.

[0076] The motion compensation signal selection means 112 selects the composite motion compensation signal when the sub-band Fc is the target sub-band, that is, when the conditional branch in step S312 causes a transition to step S315. On the other hand, in other cases, the motion compensation signal selection means 112 selects the first motion compensation signal. Therefore, in FIG. 5, the first motion compensation signal is selected during the periods from time T500 to T502, from time T504 to T508, and from time T510 to T512. Furthermore, the composite motion compensation signal is selected during the periods from time T502 to T504, from time T508 to T510, and from time T512 onward.

[0077] In this embodiment, when combining the acceleration sensor or geomagnetic sensor with the gyro sensor using a complementary filter, an appropriate sub-band can be set based on the exposure time, shutter method, and norms of the acceleration sensor or geomagnetic sensor. Also, accurate image blur correction can be performed by appropriately selecting and switching between the blur correction signal calculated only by the gyro sensor and the combined blur correction signal calculated using the complementary filter.

[0078] (Second embodiment) Next, a second embodiment of the present invention will be described. In the first embodiment, the image stabilization drive signal was calculated by selecting either the first image stabilization signal or the composite image stabilization signal using the image stabilization signal selection means 112. However, if the composite image stabilization signal calculation means 111 generates a composite image stabilization signal using a complementary filter with a division band of 0 Hz, the composite image stabilization signal will be equivalent to the first image stabilization signal. For this reason, as shown in FIG. 6, the image stabilization device 100a of this embodiment has a configuration in which the first image stabilization signal calculation means 107 and the image stabilization signal selection means 112 are removed from the image stabilization device 100 of FIG. 1. FIG. 6 is a block diagram of the image stabilization device 100a of this embodiment. Note that the components shown in FIG. 6 are the same as those in the first embodiment, and therefore their description will be omitted.

[0079] Next, the processing of the image stabilization device 100a (image stabilization operation and control method of the image stabilization device 100a) will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the processing of the image stabilization device 100a. In Fig. 7, the same processes as those in Fig. 3 are assigned the same numbers, and their description will be omitted.

[0080] In step S710, the sub-band setting means 110 sets the sub-band of the complementary filter to a low band. At this time, the sub-band may be set to approximately 0 (Hz). In step S711, the sub-band setting means 110 sets a target sub-band based on the exposure time obtained from the shooting condition information acquisition means 108.

[0081] Next, a method for setting sub-bands by the sub-band setting means 110 will be described with reference to Fig. 8. Fig. 8 is a diagram showing the time transition of sub-bands in this embodiment. In Fig. 8, the vertical axis represents sub-bands, and the horizontal axis represents time.

[0082] At time T800, the image stabilization mode is turned on and the series of calculations in the flowchart of FIG. 3 begin. At the same time, the counter 109 begins counting. At the conditional branch in step S309, the time elapsed since the start of calculations by the LPF 105 is less than a predetermined time. Therefore, the process proceeds to step S710, where the sub-band is set to an extremely low sub-band Fc_zero. Then, in step S314, a complementary filter based on the sub-band Fc_zero calculates a composite image stabilization signal.

[0083] The period from time T800 to time T801 is a waiting period for the transient response of the LPF 105 to stabilize, and this period is set to the sub-band Fc_zero.

[0084] At time T801, the elapsed time from the start of calculation of the LPF 105 exceeds a predetermined time, so the conditional branch in step S309 causes the process to proceed to step S711. Here, the sub-band is set to the low sub-band Fc_low based on the set exposure time Tv_long (not shown).

[0085] Assume that at time T802, the electronic front-curtain shutter is actuated to start exposure. Because it is not a mechanical shutter, the divided bands are not lowered. If the mechanical front-curtain shutter were actuated to start exposure at time T802, the divided band Fc would be set to Fc_zero from time T802. That is, when the shutter method is a mechanical shutter, the divided band setting means 110 sets the divided bands lower than in the case of an electronic shutter until a predetermined period has elapsed since the mechanical shutter was actuated. Furthermore, when the shutter method is an electronic front-curtain shutter, the divided band setting means 110 sets the divided bands lower than in the case of an electronic shutter until a predetermined period has elapsed since the mechanical rear-curtain shutter was actuated, or after the predetermined period has elapsed.

[0086] At time T803, the mechanical rear curtain shutter is driven, so the divided band Fc is set to Fc_zero. At the same time, in accordance with the conditional branch of step S319, the process proceeds to step S320 to initialize the LPF 105 and reset the counter 109. Furthermore, if the electronic shutter method had been set and exposure had started at time T803 by driving the electronic rear curtain shutter instead of the mechanical rear curtain shutter, the divided band would remain low at Fc_low at time T803 rather than being set to Fc_zero.

[0087] At time T804, the elapsed time from the start of calculation of the LPF 105 exceeds a predetermined time, but the elapsed time since the mechanical shutter was driven is still less than the predetermined time, so the divided band Fc remains at Fc_zero. At time T805, the elapsed time since the mechanical shutter was driven exceeds the predetermined time, so the conditional branch in step S309 causes a transition to step S711, where the divided band is set to Fc_Low based on the set exposure time Tv_long.

[0088] At time T806, the exposure time is set to Tv_short, which is shorter than Tv_long, and in step S711 for that sample, the sub-band is set to Fc_middle, which is higher than Fc_Low. At time T807, the set exposure time is returned from Tv_short to Tv_long, and in step S711, the sub-band is set to Fc_low.

[0089] At time T808, when the norm of the second blur signal momentarily exceeds a threshold (predetermined value) due to, for example, an object hitting the imaging device, the process proceeds to step S310 according to the conditional branch of step S309, and the sub-band is set to Fc_zero. That is, when the norm of the second blur signal is equal to or greater than the predetermined value, the sub-band setting means 110 sets the sub-band lower than when the norm is smaller than the predetermined value.

[0090] At time T809, the time elapsed since the start of calculation of the LPF 105 exceeds a predetermined time, so the conditional branch in step S309 causes the process to transition to step S711, where the divided band is set to a low Fc_low based on the set exposure time Tv_long.

[0091] According to this embodiment, in combining an acceleration sensor, a geomagnetic sensor, and a gyro sensor using complementary filters, an appropriate divided band can be set based on the exposure time / shutter method / norm of the acceleration sensor or the geomagnetic sensor.

[0092] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0093] According to each embodiment, by setting appropriate split bands based on shooting condition information in the synthesis of the acceleration sensor or geomagnetic sensor and the gyro sensor using a complementary filter, it is possible to remove the offset component of the gyro sensor. As a result, each embodiment can provide an image stabilization device, optical equipment, and a control method and program for an image stabilization device that are capable of performing image stabilization with high accuracy.

[0094] The disclosure of each embodiment includes the following configurations and methods. (Configuration 1) a first acquisition means for acquiring a first shake signal; a second acquiring means for acquiring a second shake signal having a larger amount of noise in a high frequency band and a smaller amount of noise in a low frequency band than the first shake signal; a generating means for generating a composite shake correction signal based on a first signal of the first shake signal that includes a frequency higher than the divided bands and a second signal of the second shake signal that includes a frequency lower than the divided bands; a setting means for setting the divided bands, The image stabilization device according to claim 1, wherein the setting means changes the divided bands in accordance with photographing condition information. (Configuration 2) 2. The image stabilization device according to configuration 1, wherein the photographing condition information includes information on at least one of an exposure time, a shutter method of the image capturing device, and a norm of the second shake signal. (Configuration 3) the photographing condition information is information regarding an exposure time, The setting means If the exposure time is a first exposure time, the divided band is set to a first band; 3. The image stabilization device according to configuration 1 or 2, wherein when the exposure time is a second exposure time that is longer than the first exposure time, the divided band is set to a second band that is higher than the first band. (Configuration 4) the photographing condition information is information regarding the norm of the second blur signal, 3. The image stabilization device according to claim 1, wherein the setting means sets the sub-bands lower when the norm is equal to or greater than a predetermined value than when the norm is smaller than the predetermined value. (Configuration 5) the photographing condition information is information about a shutter method of an image capturing device, 3. The image stabilization device according to claim 1, wherein, when the shutter method is a mechanical shutter method, the setting unit sets the division band lower than in the case of an electronic shutter method until a predetermined period has elapsed after the mechanical shutter is driven. (Configuration 6) the photographing condition information is information about a shutter method of an image capturing device, 3. The image stabilization device according to configuration 1 or 2, wherein, when the shutter method is an electronic front-curtain shutter method, the setting unit sets the division band lower than in the case of an electronic shutter method, until a predetermined period has elapsed after a mechanical rear-curtain shutter is driven, or after the predetermined period has elapsed. (Configuration 7) a first calculation means for calculating a first blur correction signal based on the first blur signal; a selection means for selecting one of the first motion compensation signal and the composite motion compensation signal based on the divided bands, and outputting the selected motion compensation signal; 3. The image stabilization device according to configuration 1 or 2, further comprising: second calculation means for calculating a drive signal for a correction member that corrects image shake based on the shake correction signal. (Configuration 8) the photographing condition information is information regarding the norm of the second blur signal, The setting means When the norm is equal to or greater than a predetermined value, the subbands are set higher than when the norm is smaller than the predetermined value; 8. The image stabilization device according to configuration 7, wherein, when the norm transitions from being equal to or greater than the predetermined value to being smaller than the predetermined value, the sub-bands are lowered over time to be set to predetermined low sub-bands. (Configuration 9) the photographing condition information is information about a shutter method of an image capturing device, The setting means When the shutter method is a mechanical shutter method, the divided bands are set higher than in the case of an electronic shutter method until a predetermined period of time has elapsed after the mechanical shutter is driven; 8. The image stabilization device according to configuration 7, wherein after the predetermined period has elapsed, the divided bands are lowered over time to a predetermined low divided band. (Configuration 10) the photographing condition information is information about a shutter method of an image capturing device, The setting means When the shutter method is an electronic front-curtain shutter method, the divided bands are set higher until a predetermined period has elapsed after the mechanical rear-curtain shutter is driven than when the predetermined period has elapsed or when the shutter method is an electronic shutter method; 8. The image stabilization device according to configuration 7, wherein after the predetermined period has elapsed, the divided bands are lowered over time to a predetermined low divided band. (Configuration 11) The setting means Immediately after the generating means starts generating the composite image stabilization signal, the sub-band is set to a predetermined high sub-band; 8. The image stabilization device according to configuration 7, wherein the divided bands are lowered as time passes to set them to predetermined lower divided bands. (Configuration 12) 12. The image stabilization device according to any one of configurations 1 to 11, wherein the first acquisition means is a gyro sensor. (Configuration 13) 13. The image stabilization device according to any one of configurations 1 to 12, wherein the second acquisition means is at least one of an acceleration sensor and a geomagnetic sensor. (Configuration 14) the generating means constitutes at least a part of a complementary filter that combines the first signal and the second signal; 14. The image stabilization device according to any one of configurations 1 to 13, wherein the divided band is a cutoff frequency of the complementary filter. (Configuration 15) 14. An optical apparatus comprising the image stabilization device according to any one of configurations 1 to 13. (Method 1) a first acquisition step of acquiring a first blur signal; a second acquisition step of acquiring a second shake signal having a larger amount of noise in a high frequency band and a smaller amount of noise in a low frequency band than the first shake signal; a generating step of generating a composite shake correction signal based on a first signal of the first shake signal including a frequency higher than the divided bands and a second signal of the second shake signal including a frequency lower than the divided bands; a setting step of setting the divided bands, A method for controlling an image stabilization device, wherein the setting step changes the divided bands in accordance with shooting condition information. (Configuration 16) A program causing a computer to execute the method for controlling an image stabilization device according to Method 1.

[0095] While the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments and various modifications and changes are possible within the scope of the present invention. In addition, the respective embodiments may be combined as appropriate. [Explanation of symbols]

[0096] 100 Image stabilization device 101 First shake detection means (first acquisition means) 102 second shake detection means (second acquisition means) 110 Division band setting means (setting means) 111 Composite shake correction signal calculation means (generation means)

Claims

1. a first acquisition means for acquiring a first shake signal; a second acquiring means for acquiring a second shake signal having a larger amount of noise in a high frequency band and a smaller amount of noise in a low frequency band than the first shake signal; a generating means for generating a composite shake correction signal based on a first signal of the first shake signal that includes a frequency higher than the divided bands and a second signal of the second shake signal that includes a frequency lower than the divided bands; a setting means for setting the divided bands, The image stabilization device according to claim 1, wherein the setting means changes the divided bands in accordance with photographing condition information.

2. 2. The image stabilization device according to claim 1, wherein the photographing condition information includes information on at least one of an exposure time, a shutter method of the image capturing device, and a norm of the second shake signal.

3. the photographing condition information is information regarding an exposure time, The setting means If the exposure time is a first exposure time, the divided band is set to a first band; 2. The image stabilization device according to claim 1, wherein when the exposure time is a second exposure time that is longer than the first exposure time, the divided band is set to a second band that is higher than the first band.

4. the photographing condition information is information regarding the norm of the second blur signal, 2. The image stabilization device according to claim 1, wherein the setting means sets the divided bands lower when the norm is equal to or greater than a predetermined value than when the norm is smaller than the predetermined value.

5. the photographing condition information is information about a shutter method of an image capturing device, 2. The image stabilization device according to claim 1, wherein, when the shutter method is a mechanical shutter method, the setting unit sets the division band lower than when the shutter method is an electronic shutter method until a predetermined period of time has elapsed after the mechanical shutter is driven.

6. the photographing condition information is information about a shutter method of an image capturing device, 2. The image stabilization device according to claim 1, wherein, when the shutter method is an electronic front-curtain shutter method, the setting unit sets the division band lower than in the case of an electronic shutter method, until a predetermined period has elapsed after a mechanical rear-curtain shutter is driven, or after the predetermined period has elapsed.

7. a first calculation means for calculating a first motion compensation signal based on the first motion compensation signal; a selection means for selecting one of the first motion compensation signal and the composite motion compensation signal based on the divided bands, and outputting the selected motion compensation signal; 2. The image blur correction device according to claim 1, further comprising: second calculation means for calculating a drive signal for a correction member for correcting image blur based on the image blur correction signal.

8. the photographing condition information is information regarding the norm of the second blur signal, The setting means When the norm is equal to or greater than a predetermined value, the subbands are set higher than when the norm is smaller than the predetermined value; 8. The image stabilization device according to claim 7, wherein, when the norm transitions from being equal to or greater than the predetermined value to being smaller than the predetermined value, the sub-band is lowered over time to be set to a predetermined lower sub-band.

9. the photographing condition information is information about a shutter method of an image capturing device, The setting means When the shutter method is a mechanical shutter method, the divided bands are set higher than in the case of an electronic shutter method until a predetermined period of time has elapsed after the mechanical shutter is driven; 8. The image stabilization device according to claim 7, wherein after the predetermined period has elapsed, the divided band is lowered with the passage of time to a predetermined lower divided band.

10. the photographing condition information is information about a shutter method of an image capturing device, The setting means When the shutter method is an electronic front-curtain shutter method, the divided bands are set higher until a predetermined period has elapsed after the mechanical rear-curtain shutter is driven than when the predetermined period has elapsed or when the shutter method is an electronic shutter method; 8. The image stabilization device according to claim 7, wherein after the predetermined period has elapsed, the divided band is lowered with the passage of time to a predetermined lower divided band.

11. The setting means Immediately after the generating means starts generating the composite image stabilization signal, the sub-band is set to a predetermined high sub-band; 8. The image stabilization device according to claim 7, wherein the divided band is lowered as time passes to a predetermined lower divided band.

12. 12. The image stabilization device according to claim 1, wherein the first acquisition means is a gyro sensor.

13. 12. The image stabilization device according to claim 1, wherein the second acquisition means is at least one of an acceleration sensor and a geomagnetic sensor.

14. the generating means constitutes at least a part of a complementary filter that combines the first signal and the second signal; 12. The image stabilization device according to claim 1, wherein the divided band is a cutoff frequency of the complementary filter.

15. An optical instrument comprising the image stabilization device according to any one of claims 1 to 11.

16. a first acquisition step of acquiring a first shake signal; a second acquisition step of acquiring a second shake signal having a larger amount of noise in a high frequency band and a smaller amount of noise in a low frequency band than the first shake signal; a generating step of generating a composite shake correction signal based on a first signal of the first shake signal including a frequency higher than the divided bands and a second signal of the second shake signal including a frequency lower than the divided bands; a setting step of setting the divided bands, A method for controlling an image stabilization device, wherein the setting step changes the divided bands in accordance with shooting condition information.

17. A program causing a computer to execute the method for controlling an image stabilization device according to claim 16.

Images