Image stabilization device, image stabilization method, and program

Abstract

To correct the blur of an image caused by roll revolution, in an imaging apparatus.SOLUTION: A blur correction device (10) includes an image formation optical system (20) for forming a subject image on the imaging surface of an imaging element (70), a blur correction part (30) for correcting the blur of the subject image caused by the roll revolution, an acceleration sensor (40) for acquiring the acceleration information of the image formation optical system (20), an angular velocity sensor (50) for acquiring the angular velocity information of the image formation optical system (20), and an arithmetic part (60) for deriving information indicating a distance from the optical axis of the image formation optical system (20) to the roll revolution axis of the image formation optical system (20), from the acceleration information and the angular velocity information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a motion compensation device, a motion compensation method, and a program. [Background technology]

[0002] Devices for correcting camera shake and other issues in imaging devices such as cameras are known in the prior art. For example, the technology disclosed in Patent Document 1 is said to be able to accurately detect translational motion of the camera and accurately correct image shake. Also, the technology disclosed in Patent Document 2 is said to be able to accurately correct shake in captured image signals caused by pitch and yaw rotations of the imaging device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-164290 [Patent Document 2] WO2020 / 003885 specification Summary of the Invention [Problem to be solved by the invention]

[0004] However, the prior art disclosed in Patent Document 1 can calculate the displacement of translational motion by removing the gravitational acceleration component, but does not correct image blur caused by roll rotation. Furthermore, the technology disclosed in Patent Document 2 can correct blur caused by pitch rotation and yaw rotation, but does not correct image blur caused by roll rotation. Furthermore, prior art does not know any technology for correcting blur caused by roll rotation, which is rotation around an axis away from and parallel to the optical axis.

[0005] An object of one aspect of the present invention is to correct image blur caused by roll rotation in an imaging device. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, a blur correction device according to one aspect of the present invention is a blur correction device that corrects blur in an image of an imaging device, and includes: an imaging optical system that focuses an image of a subject on an imaging surface of an imaging element; a blur correction unit that corrects blur of the subject image due to roll rotation; at least one acceleration sensor that acquires acceleration information of the imaging optical system; at least one angular velocity sensor that acquires angular velocity information of the imaging optical system; and a calculation unit that derives information indicating the distance from the optical axis of the imaging optical system to the roll rotation axis of the imaging optical system from the acceleration information and the angular velocity information.

[0007] In order to solve the above problem, a blur correction method according to one aspect of the present invention includes the steps of acquiring acceleration information of an imaging optical system, acquiring angular velocity information of the imaging optical system, and deriving information indicating the distance from the optical axis of the imaging optical system to the roll rotation axis of the imaging optical system from the acceleration information and the angular velocity information.

[0008] The image stabilization device according to each aspect of the present invention may be realized by a computer. In this case, the image stabilization program for the image stabilization device, which causes the computer to operate as each unit (software element) of the image stabilization device, thereby realizing the image stabilization device on the computer, and the computer-readable non-transitory recording medium on which the program is recorded, also fall within the scope of the present invention. [Effects of the Invention]

[0009] According to one aspect of the present invention, in an imaging device, image blur caused by roll rotation can be corrected. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of an imaging device 1 according to a first embodiment of the present invention. [Figure 2] 4 is a flowchart showing the flow of a shake correction method according to the first embodiment. [Figure 3] FIG. 10 is a block diagram showing the configuration of an imaging device 2 according to a modified example of the first embodiment. [Figure 4] FIG. 10 is a block diagram showing the configuration of an imaging device 1A according to a second embodiment. [Figure 5] FIG. 10 is a diagram showing the arrangement of an acceleration sensor and a second angular velocity sensor according to the second embodiment. [Figure 6] 10 is a diagram showing the relationship between the image pickup element, the sensor AO, and the intersection point WS between the plane including the image pickup surface of the image pickup element and the roll rotation axis, as viewed from the z-axis direction. [Figure 7] FIG. 10 is a diagram showing the distances from sensors AO, AX, and AY to intersection point WS. [Figure 8] FIG. 10 is a diagram showing the movement direction of the imaging element as viewed from the intersection point WS. [Figure 9] FIG. 10 is a diagram showing the direction in which the correction lens is moved for blur correction. [Figure 10] FIG. 2 is a diagram showing a direction in which an image sensor is moved for image stabilization. DETAILED DESCRIPTION OF THE INVENTION

[0011] [Embodiment 1] An embodiment of the present invention will be described in detail below. FIG. 1 is a block diagram showing the configuration of an imaging device 1 according to the first embodiment of the present invention. The imaging device 1 includes a vibration reduction device 10 and an imaging element 70. The vibration reduction device 10 is a device that corrects blur in an image captured by the imaging element 70 of the imaging device 1. The vibration reduction device 10 particularly corrects image blur caused by roll rotation. In this embodiment, roll rotation refers to the movement of an imaging optical system 20 (imaging device 1), which will be described later, rotating (pivoting) around a rotation axis that is away from the optical axis of the imaging optical system 20 and parallel to the optical axis. Hereinafter, the rotation axis about which the imaging optical system 20 rotates in a roll rotation is referred to as the roll rotation axis, and image blur caused by roll rotation is also referred to as roll rotation blur. Roll rotation blur can also be said to be image blur caused by the image sensor 70 moving in a pivotal manner together with the imaging optical system 20 around the roll rotation axis.

[0012] (Image stabilization device) As shown in FIG. 1, image stabilization device 10 includes an imaging optical system 20, a calculation unit 60, and a control unit 80. Image stabilization optical system 20 includes a image stabilization unit 30, an acceleration sensor 40, and an angular velocity sensor (gyro sensor) 50. Image stabilization unit 30 includes a correction lens 31 arranged in image stabilization optical system 20, and a lens drive unit 32 that drives correction lens 31. Calculation unit 60 performs calculations necessary for image stabilization. Control unit 80 provides overall control of image stabilization device 10 as a whole.

[0013] The imaging optical system 20 forms an image of a subject on the imaging plane of the image sensor 70. The imaging optical system 20 includes various optical elements such as an objective lens, a plurality of lens groups such as imaging lenses, an aperture, and a deflection mirror, which are arranged on the optical axis, but these are omitted from FIG. 1. The imaging optical system 20 may be an interchangeable lens configured to be detachable from the main body of the imaging device 1, or may be an imaging optical system that is permanently attached to the imaging device 1. When the imaging optical system 20 is configured as an interchangeable lens, the image blur correction unit 30 that performs correction using a correction lens 31 is also included in the imaging optical system 20 (interchangeable lens).

[0014] Light from the subject that passes through the lens group and other components of the imaging optical system 20 is imaged on the imaging surface of the image sensor 70. The image of the subject that has been imaged is converted into an electrical signal by the image sensor 70, and after undergoing various signal processing, is displayed as an image of the subject on a display and is also recorded in a memory (not shown). Any image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor can be used as the image sensor 70.

[0015] The acceleration sensor 40 acquires acceleration information of the imaging optical system 20 (image capture device 1). The angular velocity sensor 50 acquires angular velocity information of the imaging optical system 20. Specifically, the acceleration sensor 40 detects (hereinafter also referred to as "acquiring") the translational acceleration of the imaging optical system 20 in a predetermined direction. The angular velocity sensor 50 acquires the angular velocity of the imaging optical system 20 around a predetermined axis. In this embodiment, the optical axis of the imaging optical system 20 is defined as the z-axis, and the x-axis and y-axis are two axes that are perpendicular to the optical axis (z-axis) and perpendicular to each other. In this case, the acceleration sensor 40 acquires acceleration information in at least one of the x-axis, y-axis, and z-axis directions at a position on at least one of the x-axis, y-axis, and z-axis. Furthermore, the angular velocity sensor 50 acquires angular velocity information around at least one of the x-axis, y-axis, and z-axis. The acceleration sensor 40 and the angular velocity sensor 50 transmit the acquired acceleration information and angular velocity information to the calculation unit 60 via the control unit 80 .

[0016] The calculation unit 60 derives information indicating the distance from the optical axis of the imaging optical system 20 to the roll rotation axis of the imaging optical system 20 from the acceleration information and angular velocity information received from the acceleration sensor 40 and the angular velocity sensor 50. The information indicating the distance from the optical axis of the imaging optical system 20 to the roll rotation axis is, for example, the coordinate of the roll rotation axis in a Cartesian xyz coordinate system. The calculation unit 60 also derives the amount of rotation of the imaging optical system 20 about the roll rotation axis. A specific method of derivation will be described later. Based on the derived coordinates and amount of rotation, the calculation unit 60 calculates information regarding the drive operation to be performed by the image stabilization unit 30 so as to reduce the amount of blur in the image formed on the imaging surface of the image sensor 70. The information regarding the drive operation to be performed by the image stabilization unit 30 includes the drive direction and drive amount in which the lens drive unit 32 drives the correction lens 31. The calculation unit 60 transmits the calculated drive amount to the lens drive unit 32 via the control unit 80.

[0017] The image stabilization unit 30 corrects blurring of the subject image caused by roll rotation. The lens driver 32 drives the correction lens 31 based on information about the drive operation received from the calculation unit 60. Specifically, the lens driver 32 moves the correction lens 31 two-dimensionally in a plane perpendicular to the optical axis. The lens driver 32 moves the position of the correction lens and changes the direction of light so that the position of the subject imaged on the imaging plane returns to the position before the blurring occurred. Through the above processing, the image stabilization device 10 can correct blurring caused by roll rotation.

[0018] Roll rotational shake occurs when the imaging device 1 rolls. In other words, roll rotational shake occurs when the image sensor 70 moves by a certain amount of rotation around the roll rotation axis. The position of the roll rotation axis and the amount of rotation around the roll rotation axis are derived by the calculation unit 60. The position of the roll rotation axis is represented by the coordinates of the intersection of the roll rotation axis and a plane including the imaging surface of the image sensor 70. The amount of rotation around the roll rotation axis is represented by the amount of rotation of the image sensor 70 around the roll rotation axis. The coordinates of the intersection and the amount of rotation of the image sensor 70 may be derived as a vector having a direction and an amount. Then, based on the derived coordinates of the intersection and the amount of rotation of the image sensor 70, the lens driver 32 moves the correction lens 31 so as to cancel out the amount of rotation (amount of movement) of the image sensor 70. As a result, the image position of the target light moves to the image position before blurring, and roll rotational shake is corrected.

[0019] (Shake correction method) Next, a description will be given of a shake correction method S1 according to this embodiment. FIG. 2 is a flowchart showing the flow of the shake correction method S1. As shown in FIG. 2, the shake correction method S1 includes steps S11 to S13. In step S11, the acceleration sensor 40 acquires acceleration information of the imaging optical system 20. In step S12, the angular velocity sensor 50 acquires angular velocity information of the imaging optical system 20. In step S13, the calculation unit 60 derives information indicating the distance from the optical axis of the imaging optical system 20 to the roll rotation axis of the imaging optical system 20 from the acceleration information and angular velocity information. Once the information indicating the distance from the optical axis of the imaging optical system 20 to the roll rotation axis of the imaging optical system 20 has been derived, the calculation unit 60 calculates the direction and amount of movement of the correction lens 31. The lens driver 32 then drives the correction lens 31 to move by the derived amount in the derived direction. In this manner, shake is corrected. In the image stabilization method S1, steps S11 and S12 are executed in parallel, and step S13 is executed using the results of the steps. This process is constantly repeated, allowing roll rotation shake in captured images to be constantly corrected.

[0020] (Variation 1) In the example described above, the shake correction unit 30 is configured to correct roll rotation shake by placing the correction lens 31 in the imaging optical system 20 and changing the position of the correction lens 31. However, the shake correction unit 30 is not limited to this configuration. For example, the shake correction unit 30 may be configured to move the image sensor 70.

[0021] 3 is a block diagram showing the configuration of imaging device 2 according to Modification 1. As shown in the figure, image stabilization device 10 of imaging device 2 includes an imaging optical system 20, a shake correction unit 30, an acceleration sensor 40, an angular velocity sensor 50, a calculation unit 60, and a control unit 80. The shake correction unit 30 includes an image sensor 70 and an image sensor drive unit 71.

[0022] The acceleration sensor 40, angular velocity sensor 50, calculation unit 60, and control unit 80 are as described in the imaging device 1. The imaging optical system 20 does not have a correction lens. Instead, the shake correction unit 30 includes an image sensor driver 71. The image sensor driver 71 moves the image sensor 70 two-dimensionally in a direction perpendicular to the optical axis. The image sensor driver 71 may also move the image sensor 70 so as to rotate it. The shake correction unit 30 according to this first modification corrects roll rotation shake by moving the image sensor 70 to an imaging position of the subject that has moved due to shake.

[0023] The image stabilization device 10 having the above configuration derives information indicating the distance from the optical axis of the imaging optical system 20 to the roll rotation axis of the imaging optical system 20 from acceleration information and angular velocity information of the imaging optical system 20. Then, based on the distance from the optical axis of the imaging optical system 20 to the roll rotation axis and the amount of rotation about the roll rotation axis, the image stabilization unit 30 corrects roll rotation shake. Through this processing, image shake caused by roll rotation in the imaging device can be corrected.

[0024] [Embodiment 2] Other embodiments of the present invention will be described below with reference to the drawings. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the first embodiment, and the description thereof will not be repeated.

[0025] 4 is a block diagram showing the configuration of an imaging device 1A according to embodiment 2. The imaging device 1A includes a shake correction device 10 and an imaging element 70. The shake correction device 10 includes an imaging optical system 20, a calculation unit 60, and a control unit 80. The imaging optical system 20 includes an acceleration sensor 40, a first angular velocity sensor 51, and a shake correction unit 30, and the acceleration sensor 40 includes a first acceleration sensor 41 and a second acceleration sensor 42. The first angular velocity sensor 51 is a gyro sensor that detects Coriolis force. The gyro sensor can detect angular velocity around a roll rotation axis.

[0026] The image stabilization unit 30 includes a correction lens 31 and a lens driver 32. The calculation unit 60 includes a first calculator 61 and a second calculator 62. The first calculator 61 uses acceleration information and angular velocity information to calculate the coordinates of the roll rotation axis and the amount of rotation around the roll rotation axis, which are information indicating distance. As described above, the distance is the distance from the optical axis of the imaging optical system 20 to the roll rotation axis of the imaging optical system 20. The second calculator 62 calculates information related to the drive operation performed by the image stabilization unit 30 based on the coordinates and the amount of rotation. The information related to the drive operation performed by the image stabilization unit 30 is, for example, the direction and amount of movement of the correction lens 31 to reduce the amount of blur in the image formed on the imaging surface. The image sensor 70 and the control unit 80 are similar to the image sensor 70 and the control unit 80 described in the first embodiment.

[0027] The acceleration sensor 40 and the first angular velocity sensor 51 will be described in detail with reference to FIG. 5. FIG. 5 is a diagram showing the arrangement of the acceleration sensor 40 and the first angular velocity sensor 51 according to the second embodiment. As shown in the figure, consider an optical coordinate system OS, which is the coordinate of the imaging optical system 20 of the imaging device 1A relative to a fixed reference coordinate system CS. The optical coordinate system OS is an orthogonal xyz coordinate system, with its origin O'. The acceleration sensor 40 is composed of three sensors: a sensor AO, a sensor AX, and a sensor AY.

[0028] Sensor AO is disposed at origin O' and acquires at least two accelerations: acceleration aorgx in the x-axis direction and acceleration aorgy in the y-axis direction. Sensor AO is one form of the first acceleration sensor described in the claims. Sensor AX is disposed on the x-axis and acquires at least two accelerations: acceleration axx in the x-axis direction and acceleration axy in the y-axis direction. Sensor AY is disposed on the y-axis and acquires at least two accelerations: acceleration ayy in the y-axis direction and acceleration ayx in the x-axis direction. Sensors AX and AY are one form of the second acceleration sensor described in the claims. FIG. 5 also shows the acceleration in the z-axis direction acquired by sensors AO, AX, and AY, but in this embodiment, the acceleration in the z-axis direction does not need to be acquired.

[0029] The distance from the origin O' to the sensor AX is defined as rx, and the distance from the origin O' to the sensor AY is defined as ry. The distances rx and ry represent the distances to predetermined positions on the x-axis and y-axis, respectively, as defined in the claims. In this embodiment, the acceleration sensor 40 is configured with three sensors, but the number of sensors is not limited to this, and any number of sensors may be used as long as they can acquire the six accelerations described above.

[0030] The first angular velocity sensor 51 acquires an angular velocity ωx about the x-axis, an angular velocity ωy about the y-axis, and an angular velocity ωx about the z-axis. The angular velocity ωx about the x-axis may be acquired by the sensor AX, or may be acquired by a separate angular velocity sensor (not shown) arranged on the x-axis. The angular velocity ωy about the y-axis may be acquired by the sensor AY, or may be acquired by a separate angular velocity sensor (not shown) arranged on the y-axis. The angular velocity ωz about the z-axis may be acquired by the sensor AO, or may be acquired by a separate angular velocity sensor (not shown) arranged on the z-axis.

[0031] Next, the coordinate of the roll rotation axis and the amount of rotation of the imaging optical system 20 about the roll rotation axis, which are calculated by the first calculator 61 using the acceleration information and angular velocity information, will be described. The coordinate of the roll rotation axis is, more specifically, the coordinate of the intersection WS between the roll rotation axis and a plane including the imaging surface of the image sensor 70. The amount of rotation of the imaging optical system 20 is the roll rotation angle of the imaging optical system 20 about the intersection WS.

[0032] Fig. 6 is a diagram showing the relationship between the image sensor 70, sensor AO, and the intersection point WS between the roll rotation axis and a plane including the imaging surface of the image sensor 70, as viewed from the z-axis direction. In Fig. 6, the optical axis of the imaging optical system 20 is set as the z-axis of the optical coordinate system OS, and sensors AO, AX, AY, the image sensor 70, and the intersection point WS are shown.

[0033] At time t, the acceleration aorg(t) acquired by sensor AO, the acceleration ax(t) acquired by sensor AX, and the acceleration ay(t) acquired by sensor AY are respectively expressed by the following equations (1), (2), and (3). Furthermore, the angular velocity ωx(t), angular velocity ωy(t), and angular velocity ωz(t) acquired by second angular velocity sensor 52 are expressed by the following equation (4). Note that in equations (1), (2), and (3), for convenience of calculation, aorg(t) includes an acceleration component aorgz(t) that does not need to be acquired by sensor AO, ax(t) includes an acceleration component axz(t) that does not need to be acquired by sensor AX, and ay(t) includes an acceleration component ayz(t) that does not need to be acquired by sensor AY.

[0034]

number

[0035] As shown in FIG. 6, the mounting position vector Rsensx(t) of sensor AX on the x-axis, with the mounting position O' of sensor AO as the reference, is expressed by the following equation (5). The mounting position vector Rsensy(t) of sensor AY on the y-axis, with the mounting position O' of sensor AO as the reference, is expressed by the following equation (6). The mounting position vector Rimg(t) of sensor AO, with the center position 701 of the image sensor 70 as the reference, is expressed by the following equation (7). The position vector Rroll(t) from the mounting position O' of sensor AO to the intersection point WS is expressed by the following equation (8). Furthermore, the position vector RrollO(t) from the center position 701 of the image sensor 70 as the reference, to the intersection point WS is expressed by the following equation (9).

[0036]

number

[0037] The position vector we want to find is the position vector Rroll0(t). The position vector Rroll0(t) can be found by combining the position vector Rimg(t) and the position vector Rroll(t). Since the position vector Rimg(t) is known from the design information, if the position vector Rroll(t) is found, the position vector Rroll0(t) can be found.

[0038] FIG. 7 is a diagram showing the distances from each of the three sensors, sensors AO, AX, and AY, to the intersection point WS. As shown in FIG. 7, the distance from sensor AO to intersection point WS is defined as Lorg(t), the distance from sensor AX to intersection point WS as Lx(t), and the distance from sensor AY to intersection point WS as Ly(t). The intersection of these three distances is calculated to determine the distance from sensor AO to intersection point WS. In other words, the distance from sensor AO to intersection point WS is derived based on the distance from the first acceleration sensor 41 (sensor AO) to intersection point WS and the distance from the second acceleration sensor 42 (sensors AX and AY) to intersection point WS. The reason for performing such calculations is that, since the values ​​obtained from each sensor contain noise, errors, etc., combining the outputs of multiple sensors allows for a more accurate distance determination.

[0039] Specifically, the magnitude (distance) L of the vector from the sensor AO to the intersection point WS is calculated using the following equation (10). Equation (10) indicates that the distance L(t) is calculated as the intersection point of the vectors Lorg(t), Lx(t), and Ly(t). Here, from equation (11), the components of equation (10) can be expressed as the following equations (12) to (14).

[0040]

number

[0041]

number

[0042] Next, the distance L(t) from the sensor AO to the intersection point WS is calculated by solving the following three simultaneous equations (15), (16), and (17).

number

[0043] Since the values ​​(outputs) acquired by the acceleration sensor contain errors, there are cases where each simultaneous equation does not have a solution. As an example, if there is a solution for two or more simultaneous equations, the average value (x, y) of the two coordinate pairs with the closest coordinates is substituted into the right-hand side of equation (18) below. If there is no solution for two or more equations, it is determined that the accuracy of the acquired acceleration and angular velocity is low, and this calculation result is not adopted. Instead, the calculation result adopted immediately before is retained and the next calculation process is performed. As mentioned above, Rroll(t) on the left-hand side of equation (18) is the position vector from sensor AO to intersection point WS.

[0044]

number

[0045] The condition for determining Rroll(t) is not limited to the existence of solutions for two or more simultaneous equations. For example, it may be the existence of solutions for three simultaneous equations within a certain error range. Alternatively, it may be the existence of solutions for one or more simultaneous equations. Furthermore, how to deal with the case where this calculation result is not adopted is not limited to holding the calculation result that was adopted immediately before. For example, it may be possible to adopt the average of several calculation results that have been adopted up to that point.

[0046] Once Rroll(t) is determined by equation (18), a position vector Rroll0 from the center position 701 of the image sensor 70 to the intersection point WS is then calculated by the following equation (19).

number

[0047] Note that, since equation (19) is an equation for calculating a position vector, it does not include information about the roll rotation angle around the intersection point WS of the imaging optical system 20. However, the roll rotation angle information can be obtained by integrating the output of the second angular velocity sensor 52.

[0048] The position vector Rroll0(t) and the roll rotation vector ωroll(t) represent the movement direction and amount of shake of the image sensor 70. FIG. 8 is a diagram showing the movement of the center position of the image sensor 70 as seen from the intersection point WS. The center position of the image sensor 70 moves from center position 701 to center position 702 along arrow 710. The vector indicated by arrow 710 is the roll rotation vector ωroll(t). The position vector from the intersection point WS to the center position 701 is expressed by the following equation (20), which reverses the direction of the vector Rroll0.

[0049]

number

[0050] Once the position vector -Rroll0(t) and the roll rotation vector ωroll(t) have been determined as described above, the second calculation unit 62 calculates information regarding the drive operation to be executed by the image stabilization unit 30. Specifically, the second calculation unit 62 calculates the direction in which to move the correction lens 31 and the amount of movement.

[0051] FIG. 9 is a diagram showing the direction in which the correction lens is moved for blur correction. Arrow 711 is the movement vector that moves correction lens 31 for blur correction. This movement vector 711 can be calculated as follows. First, the magnitude Lroll0 of vector -Rroll0 shown in FIG. 9 is found. Lroll0 is expressed by the following equation (21). Specifically, Lroll0 is calculated using the following equation (22).

number

number

[0052] Next, the second calculation unit 62 determines whether the calculated Lroll0 satisfies the following formula (23).

number

[0053] Here, Limg is the distance from the center position 701 of the image sensor 70 to the vertex position, as shown in Fig. 9. The reason for determining whether or not Expression (23) is satisfied is as follows. That is, if Lroll0 is smaller than Limg, it means that the roll rotation axis is within the image circle. In that case, the amount of roll rotational shake, as referred to in this embodiment, is small, and therefore the correction effect is also small.

[0054] The image circle is the area of ​​a circumscribed circle that circumscribes the image sensor 70. In image blur correction, it is preferable to correct blur so as to optimize the area of ​​the image sensor 70 that is farther from the roll rotation axis (intersection point WS). If the roll rotation axis is outside the image circle, the amount of blur is greater on the side farther from the roll rotation axis, i.e., on the outside than on the center of the image sensor 70. The correction effect for the entire image circle can be evaluated by the product of the correction amount and the area. The area per unit angle is larger on the outside from the intersection point WS than on the inside. Therefore, the correction effect for the entire image is greater if the correction effect is optimized for the area of ​​the image sensor 70 that is farther from the intersection point WS.

[0055] The second calculation unit 62 performs the following correction process if formula (23) is satisfied, and does not perform the correction process if formula (23) is not satisfied. Note that the second calculation unit 62 may always perform the correction process without making a judgment based on formula (23). Alternatively, different criteria may be set for determining whether or not to perform the correction process.

[0056] When performing the correction process, the second calculation unit 62 obtains Cx(t) and Cy(t), which are components of the movement vector 711, from the following equations (24) and (25). In other words, Cx(t) is the amount of movement of the correction lens 31 in the x-axis direction, and Cy(t) is the amount of movement of the correction lens 31 in the y-axis direction. The correction coefficient k in equation (25) VR is the amount of image plane movement (dimensionless constant) on the imaging surface of the imaging element 70 per unit amount of movement of the correction lens 31.

[0057]

number

[0058] The second calculation unit 62 transmits the calculated Cx(t) and Cy(t) to the lens driving unit 32 via the control unit 80. The lens driving unit 32 drives the correction lens 31 so as to move the correction lens 31 in the x-axis direction and the y-axis direction by Cx(t) and Cy(t), respectively. The shake correction unit 30 performs shake correction through this processing.

[0059] As described above, the shake correction unit 30 can correct roll rotation shake by moving the correction lens 31.

[0060] (Variation 2) In the above-described second embodiment, an example has been described in which the shake correction unit 30 corrects roll rotation shake by moving the correction lens 31. However, the shake correction unit 30 may also correct roll rotation shake by moving the image sensor 70. In this case, the second calculation unit 62 calculates the direction in which the image sensor 70 should be moved and the amount of movement. The case in which the image sensor 70 is moved will be described below.

[0061] FIG. 10 is a diagram showing the direction of movement of the image sensor 70 due to shake. The position of the image sensor 70 before shake is indicated by a dotted line, and the position of the image sensor 70 after shake is indicated by a solid line. It is assumed, however, that the image sensor 70 is attached to an anti-shake device 73. As described in FIG. 8, the center position of the image sensor 70 moves from center position 701 to center position 702 along the roll rotation vector ωroll(t) indicated by arrow 710. In this case, the image stabilizer 30 moves the anti-shake device 73 so that the center position of the image sensor 70 returns from 702 to 701.

[0062] The movement vector from the center position 702 to the center position 701 can be calculated using the procedure described with reference to Fig. 9. However, when the image sensor 70 is moved, the correction coefficient k VR As described above, blur correction may be performed by moving the image sensor 70. Note that in Modification 2, the image sensor 70 may be rotated around the intersection point WS. In Modification 2, the same effects as in Embodiment 2 can be obtained.

[0063] [Software implementation example] The functions of the image stabilization device 10 (hereinafter referred to as the "device") can be realized by a program that causes a computer to function as the device, and a program that causes a computer to function as the calculation unit 60 and control unit 80 of the device.

[0064] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing the functions described in each of the above embodiments.

[0065] The program may be non-transitory and may be recorded on one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

[0066] In addition, some or all of the functions of the above-mentioned units can be realized by logic circuits. For example, integrated circuits in which logic circuits functioning as the above-mentioned units are formed are also included in the scope of the present invention. In addition, the functions of the above-mentioned control blocks can also be realized by, for example, a quantum computer.

[0067] 〔summary〕 The image stabilization device according to this embodiment is a image stabilization device that corrects blur in an image of an imaging device, and includes an imaging optical system that forms an image of a subject on an imaging surface of an imaging element, a blur stabilization unit that corrects blur in the subject image due to roll rotation, at least one acceleration sensor that acquires acceleration information of the imaging optical system, at least one angular velocity sensor that acquires angular velocity information of the imaging optical system, and a calculation unit that derives information indicating the distance from the optical axis of the imaging optical system to the roll rotation axis of the imaging optical system from the acceleration information and the angular velocity information.

[0068] With this configuration, it is possible to correct image blur caused by roll rotation in the imaging device.

[0069] Furthermore, in the image stabilization device according to this embodiment, the at least one acceleration sensor may include at least one first acceleration sensor that detects accelerations in the x-axis and y-axis directions at an origin of the three axes, where the optical axis of the imaging optical system is defined as the z-axis and two axes that are orthogonal to the optical axis and orthogonal to each other are defined as the x-axis and y-axis, and at least two second acceleration sensors that detect accelerations in the three axes at predetermined positions on the x-axis and the y-axis, respectively; the at least one angular velocity sensor may be a first angular velocity sensor that detects angular velocities around each of the x-axis, y-axis, and z-axis; and the calculation unit may include a first calculation unit that calculates, using the acceleration information and the angular velocity information, coordinates of the roll rotation axis and an amount of rotation around the roll rotation axis, which are information indicating the distance; and a second calculation unit that calculates information related to a drive operation to be performed by the image stabilization unit, based on the coordinates and the amount of rotation.

[0070] With this configuration, it is possible to correct image blur caused by roll rotation in the imaging device.

[0071] In the image stabilization device according to this embodiment, the distance may be derived based on the distance from the first acceleration sensor and the distance from the second acceleration sensor.

[0072] With this configuration, the distance from the optical axis of the imaging optical system to the roll rotation axis of the imaging optical system can be calculated with higher accuracy, and therefore image blur correction can be performed with higher accuracy.

[0073] In the image stabilization device according to the present embodiment, the drive operation performed by the image stabilization unit may be an operation of moving a correction lens of the imaging optical system or the image sensor.

[0074] With this configuration, the image stabilization device may be provided in the interchangeable lens of the imaging device, or in the main body of the imaging device. If the image stabilization device is provided in the interchangeable lens, image stabilization can be performed regardless of the configuration of the main body of the imaging device. If the image stabilization device is provided in the main body of the imaging device, image stabilization can be performed even if the interchangeable lens does not have a image stabilization device. In addition, image stabilization can be performed by rotating the imaging element.

[0075] In the image stabilization device according to the present embodiment, the image stabilization unit may correct the image stabilization so as to optimize the region of the image sensor that is farther from the roll rotation axis.

[0076] With this configuration, it is possible to optimize image blur correction for the entire image circle.

[0077] The image stabilization method according to this embodiment includes the steps of acquiring acceleration information of an imaging optical system, acquiring angular velocity information of the imaging optical system, and deriving information indicating the distance from the optical axis of the imaging optical system to the roll rotation axis of the imaging optical system from the acceleration information and the angular velocity information.

[0078] The image stabilization program according to this embodiment causes a computer to execute the following processes: a process of acquiring acceleration information of the imaging optical system; a process of acquiring angular velocity information of the imaging optical system; and a process of deriving information indicating the distance from the optical axis of the imaging optical system to the roll rotation axis of the imaging optical system from the acceleration information and the angular velocity information.

[0079] [Additional Notes] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0080] 1, 1A, 2...imaging device 10. Image stabilization device 20. Imaging optical system 30. Image stabilization unit 31 Corrective lenses 32 Lens drive unit 40 Acceleration sensor 41 First acceleration sensor 42 Second acceleration sensor 50···Angular rate sensor 51 First angular velocity sensor 60... Arithmetic section 61 First calculation unit 62 Second calculation unit 70... Image sensor 71 Imaging device drive unit 73. Anti-vibration device 80 Control unit

Claims

1. A blur correction device that corrects blur in an image of an imaging device, an imaging optical system that forms an image of a subject on an imaging surface of an imaging element; a blur correction unit that corrects blur of the subject image due to roll rotation; at least one acceleration sensor for acquiring acceleration information of the imaging optical system; at least one angular velocity sensor for acquiring angular velocity information of the imaging optical system; a calculation unit that derives information indicating a distance from an optical axis of the imaging optical system to a roll rotation axis of the imaging optical system from the acceleration information and the angular velocity information, The at least one acceleration sensor an optical axis of the imaging optical system is defined as a z-axis, and two axes orthogonal to the optical axis and orthogonal to each other are defined as x-axis and y-axis; and at least one first acceleration sensor that detects accelerations in the x-axis and y-axis directions at the origin of the three axes; at least two second acceleration sensors that detect acceleration in three axial directions at predetermined positions on the x-axis and the y-axis, respectively; the at least one angular velocity sensor is a first angular velocity sensor that detects angular velocities around each of the x-axis, y-axis, and z-axis; The calculation unit a first calculation unit that calculates a coordinate of the roll rotation axis, which is information indicating the distance, and an amount of rotation around the roll rotation axis, using the acceleration information and the angular velocity information; a second calculation unit that calculates information about a drive operation to be performed by the image stabilization unit based on the coordinates and the amount of rotation, Image stabilization device.

2. The image stabilization device according to claim 1 , wherein the distance is derived based on the distance from the first acceleration sensor and the distance from the second acceleration sensor.

3. 3. The image stabilization device according to claim 1, wherein the driving operation performed by the image stabilization unit is an operation of moving a correction lens of the imaging optical system or the image sensor.

4. The image stabilization device according to claim 1 , wherein the image stabilization unit corrects the image stabilization so as to optimize the area of ​​the image sensor that is farther from the roll rotation axis.

5. A blur correction method for correcting blur in an image of an imaging device, comprising: acquiring acceleration information of the imaging optical system by at least one acceleration sensor; acquiring angular velocity information of the imaging optical system using at least one angular acceleration sensor; a calculation step of deriving information indicating a distance from an optical axis of the imaging optical system to a roll rotation axis of the imaging optical system from the acceleration information and the angular velocity information; a blur correction step of correcting blur of the subject image due to roll rotation, The acceleration information is an optical axis of the imaging optical system is defined as a z-axis, and two axes orthogonal to the optical axis and orthogonal to each other are defined as x-axis and y-axis; and at least one first acceleration sensor that detects accelerations in the x-axis and y-axis directions at the origin of the three axes; and at least two second acceleration sensors that detect accelerations in three axial directions at predetermined positions on the x-axis and the y-axis, respectively, the angular velocity information is detected by a first angular velocity sensor that detects angular velocities around each of the x-axis, y-axis, and z-axis; The calculation step includes: a first calculation step of calculating, using the acceleration information and the angular velocity information, a coordinate of the roll rotation axis, which is information indicating the distance, and an amount of rotation around the roll rotation axis; a second calculation step of calculating information related to a drive operation to be executed in the shake correction step based on the coordinates and the rotation amount, Image stabilization method.

6. A blur correction program for correcting blur in an image of an imaging device, On the computer, acquiring acceleration information of the imaging optical system using at least one acceleration sensor; acquiring angular velocity information of the imaging optical system using at least one angular acceleration sensor; a calculation process for deriving information indicating a distance from an optical axis of the imaging optical system to a roll rotation axis of the imaging optical system from the acceleration information and the angular velocity information; A blur correction process for correcting blur of a subject image caused by roll rotation, The acceleration information is an optical axis of the imaging optical system is defined as a z-axis, and two axes orthogonal to the optical axis and orthogonal to each other are defined as x-axis and y-axis; and at least one first acceleration sensor that detects accelerations in the x-axis and y-axis directions at the origin of the three axes; and at least two second acceleration sensors that detect accelerations in three axial directions at predetermined positions on the x-axis and the y-axis, respectively, the angular velocity information is detected by a first angular velocity sensor that detects angular velocities around each of the x-axis, y-axis, and z-axis; The arithmetic processing a first calculation process of calculating a coordinate of the roll rotation axis, which is information indicating the distance, and an amount of rotation around the roll rotation axis, using the acceleration information and the angular velocity information; a second calculation process for calculating information about a drive operation to be executed in the shake correction process based on the coordinates and the rotation amount, Image stabilization program.

Images