Image blur correction device, control method thereof, program, and storage medium

Abstract

To achieve a proper image shake correction control on a video taken using an anamorphic lens.SOLUTION: The image blur correction device includes a calculation unit that calculates the image blur correction amount for correcting an image blur by changing the relative position of the subject image and imaging device based on the detection result by a blur detection unit that detects a shake of the imaging apparatus. The calculation unit calculates the image blur correction amount at different magnifications for each multiple axes of the 2D plane of the pickup image based on a piece of information of an imaging lens used when imaging the pickup image.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a technique for correcting image blur caused by shaking of an imaging device. [Background technology]

[0002] In recent years, the filming technique using anamorphic lenses has become widely used in the film industry. Anamorphic lenses compress the horizontal direction of the shot image at a specific magnification, and then restore the compressed magnification in post-processing to obtain an image with a Cinemascope aspect ratio (2.39:1).

[0003] Methods have been proposed for correcting image blurring in images captured using this anamorphic lens.

[0004] Patent Document 1 discloses the following technology: When detecting a motion vector from an image captured using an anamorphic lens and correcting image blur after returning the horizontal compression, the motion vector is detected from an image compressed in the horizontal direction, so the motion vector is detected as smaller than the actual motion vector. Therefore, image blur is corrected using a motion vector that takes compression into account. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 3278206 Summary of the Invention [Problem to be solved by the invention]

[0006] However, Patent Document 1 has the following problems. In Patent Document 1, after the horizontal compression of the image captured using an anamorphic lens is restored, image blur can be corrected using a motion vector that takes the compression into consideration. However, on the other hand, if image blur correction is performed on the image that remains compressed, overcorrection occurs and the image blur cannot be corrected correctly. Similarly, if shake information is obtained from an inertial sensor such as an angular velocity sensor separately from the motion vector for the image that remains compressed, overcorrection occurs and the image blur cannot be corrected correctly.

[0007] The present invention has been made in consideration of the above-mentioned problems, and an object of the present invention is to realize appropriate image blur correction control for images captured using an imaging lens such as an anamorphic lens that has different compression ratios in the horizontal and vertical directions of the captured image. [Means for solving the problem]

[0008] The image blur correction device according to the present invention includes a calculation means for calculating an image blur correction amount for correcting image blur based on a detection result of a shake detection means for detecting a shake of an imaging device, and a correction means for performing image blur correction based on the image blur correction amount calculated by the calculation means, wherein the shake detection means Desqueeze process not performed Multiple photograph The motion vector detection means detects a motion vector between images, and an angular velocity sensor. The calculation means calculates the motion vector of the two-dimensional plane of the captured image based on information of a photographing lens having different image magnifications in the horizontal and vertical directions used when the captured image was captured. horizontal direction Axis and the vertical axis Based on the detection results of the angular velocity sensor calculated using different magnifications Horizontal and vertical angular velocity and 、 Based on the detection result of the motion vector calculated without using the different magnification factor Horizontal and vertical motion vectors and the correction means calculates the image blur correction amount using of The relative position of the subject image formed by the photographing lens and the image sensor is changed to reduce image blur. and a second correction amount used for a second image blur correction that corrects image blur by changing an image cut-out position from the captured image, For a captured image that has not been subjected to desqueeze processing, 1. Based on the correction amount The first image blur correction and the second image blur correction based on the second correction amount are performed. It is characterized by: Effect of the Invention

[0009] According to the present invention, it is possible to realize appropriate image blur correction control for an image captured using a photographing lens such as an anamorphic lens that has different compression ratios in the horizontal and vertical directions of the captured image. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing the configuration of a digital camera according to a first embodiment of the present invention. [Diagram 2] FIG. 1 is a diagram showing the coordinate system of a digital camera. [Diagram 3] 5 is a flowchart showing an operation for calculating an image blur correction amount in the first embodiment. [Figure 4] 5 is a flowchart showing an operation of converting an amount of electronic correction in the first embodiment. [Diagram 5] FIG. 11 is a block diagram showing the configuration of a digital camera according to a second embodiment. [Figure 6] 10 is a flowchart showing calculation of an RS distortion correction range in the second embodiment. [Figure 7] 13A to 13C are diagrams for explaining RS distortion in the second embodiment. [Figure 8] FIG. 11 is a block diagram showing the configuration of a digital camera according to a third embodiment. [Figure 9] 13 is a flowchart showing an operation of converting an amount of electronic correction in the third embodiment. [Figure 10] FIG. 13 is a diagram illustrating a distortion rate in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0012] (First embodiment) 1 is a block diagram showing the configuration of a lens-interchangeable digital camera (imaging device) 100 for taking still images and moving images, which has an image stabilization device according to a first embodiment of the present invention. Note that the present invention is applicable not only to cameras with interchangeable lenses, but also to cameras with fixed lenses. Furthermore, the present invention is not limited to digital cameras, and can be applied to various types of image stabilization devices.

[0013] In the following description of the embodiment, the vibration applied to the imaging device is referred to as "shake", and the shake of the captured image caused by the shake applied to the imaging device is referred to as "image blur". As shown in Fig. 2, the following description will be given with the rotation axis extending vertically as the Yaw axis, the rotation axis extending horizontally as the Pitch axis, and the rotation axis extending in the optical axis direction as the Roll axis, which form detection axes perpendicular to each other on a plane perpendicular to the optical axis.

[0014] In FIG. 1, a digital camera 100 is composed of a photographing lens 101 and a camera body 150, and the photographing lens 101 is attached to the camera body 150 when in use.

[0015] The photographing lens 101 includes a zoom lens 102 for variable magnification, a correction optical system 103 (first image blur correction means) such as a shift lens for image blur correction, and a focus lens 104 for focus adjustment. The correction optical system 103 performs image blur correction by changing the relative position between the subject image formed by the photographing lens 101 and the image sensor 105. These lens elements perform operations such as zooming, focusing, and image blur correction, and form the subject image on the image pickup surface of the image sensor 105. Note that in this embodiment, the photographing lens 101 is assumed to be an anamorphic lens with different image magnifications in the directions of two axes, horizontal and vertical, which are orthogonal to each other on a two-dimensional plane. The anamorphic lens is assumed to be an anamorphic lens that compresses the horizontal direction of a photographed image by a specific magnification and returns (expands) the compressed magnification in a post-processing process to obtain an image with a cinemascope aspect ratio (2.39:1).

[0016] The image sensor 105 is composed of, for example, an XY address type CMOS (Complementary Metal Oxide Semiconductor) image sensor, etc. Then, the image sensor 105 photoelectrically converts an optical image formed by the photographing lens 101 to accumulate electric charges, and generates an image signal (photographed image) from signals of a plurality of pixels by reading out the electric charges, and supplies the image signal to a signal processing unit 106.

[0017] Furthermore, the image sensor 105 can be moved by a motor 127 in a direction perpendicular to the optical axis of the photographing lens 101. The image sensor 105 corrects image blur of the subject image formed by the photographing lens 101 by horizontal and vertical movement in a plane perpendicular to the optical axis and rotational movement about the optical axis (second image blur correction means). The subject image with image blur corrected is photoelectrically converted by the image sensor 105, and the obtained image signal is supplied to the signal processing unit 106.

[0018] The signal processing unit 106 performs signal processing such as white balance adjustment and gamma correction on the image signal output from the image sensor 105 , and the frame image generated as a result is stored in the image memory 107 .

[0019] The image clipping control unit 108 clips out a predetermined area of ​​the frame image stored in the image memory 107 to generate a new frame image, and supplies it to the display control unit 109 and the recording control unit 111. At this time, the clipping position of the predetermined area is moved in accordance with the shake of the digital camera 100 to correct the movement of the subject position between frames (image blur) caused by the shake of the digital camera 100. The image clipping control unit 108 constitutes an electronic image blur correction means (third image blur correction means). Note that a series of operations performed by the signal processing unit 106 and the image clipping control unit 108 is executed at a cycle of 60 Hz in the case of a video signal conforming to the NTSC format, for example, and moving image data is generated.

[0020] In each embodiment described below, the cutout process is performed on a captured image that has been compressed horizontally by an anamorphic lens and has not yet been subjected to a process for returning the compressed image size. The process for returning an image that has been compressed by an anamorphic lens is generally called a desqueezing process, but performing the desqueezing process increases the image size. Therefore, performing the cutout process before the desqueezing process requires less processing load than performing the cutout process after the desqueezing process, and can also be used for shooting at a high frame rate.

[0021] The display control unit 109 performs an electronic viewfinder function by displaying an image (through image) based on the video signal supplied from the image cropping control unit 108 on the display device. The display control unit 109 also displays a setting menu image, a recorded image, and the like on the display device 110 depending on the application. The display device 110 includes a liquid crystal display element (LCD) and the like.

[0022] When a user instructs the start of recording, the recording control unit 111 controls the recording of moving image data, still image data, metadata, etc. supplied from the image memory 107 onto the recording medium 112. The recording medium 112 is composed of an information recording medium such as a semiconductor memory, a magnetic recording medium such as a hard disk, or the like.

[0023] The angular velocity sensor 113 detects the shake applied to the digital camera 100. The detected shake signal (detection result) is supplied to an image shake correction amount calculation unit 117, which will be described later, and is used to control the image shake correction. The angular velocity sensor 113 is arranged to form detection axes perpendicular to each other on a plane perpendicular to the optical axis. More specifically, the angular velocity sensor 113 has three angular velocity sensors so as to detect shake around a Yaw axis that is an axis extending in the vertical direction, shake around a Pitch axis that is an axis extending in the horizontal direction, and shake around a Roll axis that is an axis extending in the optical axis direction, that is, shake around three axes. The angular velocity sensor 113 detects the angular velocity of the shake applied to the digital camera 100, and outputs a voltage according to the angular velocity.

[0024] A first A / D converter 114 converts the voltage output from the angular velocity sensor 113 into digital angular velocity data, and supplies it to an image blur correction amount calculation unit 117, which will be described later.

[0025] The motion vector detection unit 115 detects motion vectors in two directions, the horizontal direction and the vertical direction, which are mutually orthogonal on a plane perpendicular to the optical axis. The motion vector detection method includes a correlation method, a block matching method, etc. Here, as an example, the motion vector detection unit 115 uses the block matching method.

[0026] In this block matching method, the input image signal is first divided into a number of blocks of an appropriate size (for example, 16 x 16 pixels), and the difference between each block and a certain range of pixels in the previous field or frame is calculated.The block in the previous field or frame that has the smallest sum of the absolute values ​​of these differences is then searched for, and the relative displacement between the two blocks is detected as the motion vector of that block.As a result, the amount of movement in both the vertical and horizontal directions (i.e., the motion vector) can be calculated in pixel units.

[0027] This motion vector indicates the amount of movement per unit time of successive captured images, i.e., the amount of movement of the digital camera 100. If the motion vector cannot be detected successfully, a motion vector error determination is performed. One example of a method for determining an error in a motion vector is to determine whether or not there is a motion vector error based on conditions such as a small luminance signal or a peak value. The detected motion vectors are supplied to an image blur correction amount calculation unit 117 (described later) with a horizontal vector H_Vect and a vertical vector V_Vect, and are used to control image blur correction.

[0028] The optical parameter acquisition unit 116 acquires characteristic information of the photographing lens such as focal length, aperture value, focus position, shift lens movement amount, distortion rate, anamorphic lens compression magnification, and effective image circle diameter from the photographing lens 101. This information may not be acquired directly from the photographing lens 101, but may be acquired as values ​​input via a user interface or the like. The acquired information is supplied to an image blur correction amount calculation unit 117 and used to control image blur correction.

[0029] The image blur correction amount calculation unit 117 calculates a correction amount for correcting image blur caused by shaking of the digital camera 100, and supplies the calculated amount to a correction amount division control unit 118. Note that the image blur correction amount calculated by the image blur correction amount calculation unit 117 is not the correction amount of each of a plurality of image blur correction means, but the overall image blur correction amount of the digital camera 100.

[0030] The correction amount division control unit 118 divides the image blur correction amount for the entire digital camera calculated by the image blur correction amount calculation unit 117 into correction amounts for correction by each of the multiple image blur correction means. In this embodiment, as an example, the image blur correction amount is divided into a correction amount for correction by the movement of the image sensor 105 and a correction amount for correction by the image cropping control unit 108.

[0031] The image sensor correction amount converter 119 converts the correction amount output from the correction amount division controller 118 into a movement amount for appropriately correcting image blur at the image sensor 105, and outputs it as a drive target position.

[0032] The electronic correction amount converter 120 converts the correction amount output from the correction amount division controller 118 into a cut-out position for appropriately correcting image blur in the image cut-out controller 108 , and sets the converted correction amount in the image cut-out controller 108 .

[0033] The position detector 121 detects the movement position of the image sensor 105 and outputs a voltage according to the position. The output voltage of the position detector 121 is amplified to a signal in an appropriate voltage range by the amplifier 122. The output of the amplifier 122 is converted into digital position data by the second A / D converter 123.

[0034] Deviation data, which is the difference between the drive target position of the image sensor 105 and the position data, is input to the control filter 124. The control filter 124 performs various signal processing such as amplification and phase compensation on the input data, and outputs the data to the pulse width modulation unit 125. The pulse width modulation unit 125 modulates the output of the control filter 124 into a waveform that changes the duty ratio of the pulse wave (i.e., a PWM waveform), and supplies the modulated output to the motor drive unit 126.

[0035] The motor 127 is, for example, a voice coil type motor, and is driven by the motor driving unit 126 to move the image sensor 105 in a direction perpendicular to the optical axis. The position of the moved image sensor 105 is detected by the position detection unit 121, and the next deviation data is calculated, forming a feedback loop. Then, the difference between the drive target position and the position data is controlled to be small. As a result, the image sensor 105 is driven and controlled to follow the drive target position. The motor 127 may be a vibration type motor that generates a drive force by vibrating a diaphragm with a piezoelectric element.

[0036] Fig. 3 is a flowchart showing the correction amount calculation operation of the image blur correction amount calculation unit 117. Note that the process shown in Fig. 3 is repeatedly executed at a predetermined cycle, such as 60 Hz in the case of a video signal conforming to the NTSC format.

[0037] First, in step S201, optical parameter information of the photographing lens at the time of shooting is acquired, such as focal length, aperture value, focus position, distortion rate, anamorphic lens compression magnification, effective image circle diameter, etc. The information may be acquired via electrical communication with the photographing lens 101, or values ​​manually input using a user interface of the digital camera 100 may be acquired.

[0038] In step S202, the angular velocity sensor 113 detects the shake of the digital camera 100 around the Yaw axis and the Pitch axis. Since the shake information from the angular velocity sensor 113 is angular velocity information, the low frequency components are removed by the HPF, and the angular velocity information is converted into displacement angle information by being integrated by an integrator. In the integration calculation performed here, incomplete integration is used to prevent saturation, and a commonly known first-order LPF is used. The converted displacement angle data are called Yaw_rad and Pitch_rad, respectively.

[0039] In step S203, a motion vector is detected using the previous frame and the current frame, and motion vector data H_Vect, V_Vect are calculated.

[0040] In step S204, the anamorphic lens compression ratio acquired in step S201 is used to determine whether or not an anamorphic lens is attached to the camera body 150. If it is determined that an anamorphic lens is attached, the process proceeds to step S205, and if it is determined that an anamorphic lens is not attached, the process proceeds to step S206.

[0041] In step S205, the compression magnifications of the photographing lens in the horizontal and vertical directions that are mutually orthogonal on a plane perpendicular to the optical axis are set. Specifically, the compression magnifications of the photographing lens in the horizontal and vertical directions are set as Yaw_Mag and Pitch_Mag, respectively, the anamorphic lens compression magnification obtained in step S201 is set to Yaw_Mag, and 1 indicating unity is set to Pitch_Mag.

[0042] In step S206, similarly to step S205, the compression magnifications of the photographing lens in the horizontal and vertical directions perpendicular to each other on a plane perpendicular to the optical axis are set. Here, the compression magnifications of the photographing lens in the horizontal and vertical directions are designated Yaw_Mag and Pitch_Mag, respectively, and both Yaw_Mag and Pitch_Mag are set to 1, which indicates unity magnification.

[0043] In step S207, a virtual focal length for each image blur correction axis is calculated from the compression ratio of the photographing lens set in step S205 or step S206 and the focal length information acquired in step S201. Specifically, if the focal length acquired in step S201 is f, and the virtual focal lengths are Yaw_f and Pitch_f, respectively, then these can be written as (Equation 1) and (Equation 2).

[0044] Yaw_f=f·(1 / Yaw_Mag) …(Formula 1) Pitch_f=f·(1 / Pitch_Mag) …(Formula 2) In step S208, the image blur correction amounts around the Yaw axis and the Pitch axis are calculated from the angular displacement data Yaw_rad and Pitch_rad converted in step S202, the motion vector data H_Vect and V_Vect acquired in step S203, and the virtual focal lengths Yaw_f and Pitch_f calculated in step S207. Specifically, when the image blur correction amounts are H_Total and V_Total, respectively, they can be written as (Equation 3) and (Equation 4).

[0045] H_Total=Yaw_f·tan(Yaw_rad)+H_Vect …(Equation 3) V_Total=Pitch_f·tan(Pitch_rad)+V_Vect …(Formula 4) Next, a correction amount division control unit 118 that divides the image blur correction amount for the entire digital camera 100 calculated as described above will be described.

[0046] The correction amount division control unit 118 divides the image blur correction amounts H_Total and V_Total calculated by the image blur correction amount calculation unit 117 into correction amounts Yaw_Correct and Pith_Correct used for image blur correction due to movement of the image sensor 105 (second image blur correction), and correction amounts H_hom and V_hom used for electronic image blur correction (third image blur correction). Then, the correction amounts are output to the image sensor correction amount conversion unit 119 and the electronic correction amount conversion unit 120. As for the division method, the ratio between the image sensor correction amount and the electronic correction amount may be changed according to the image blur correction amount calculated from the angular velocity sensor 113, or another method may be used.

[0047] Fig. 4 is a flowchart showing the conversion operation in the electronic correction amount converter 120. Note that the process shown in Fig. 4 is repeatedly executed at a predetermined cycle, such as 60 Hz, in the case of a video signal conforming to the NTSC format.

[0048] First, in step S301, the image blur correction amounts H_hom and V_hom calculated by the correction amount division control unit 118 are obtained.

[0049] In step S302, the image blur compensation range is calculated. The image blur compensation range is set so as not to hit the edge based on the effective image circle diameter information acquired by the optical parameter acquisition unit 116, the readout range of the image sensor 105, the movable range of the image cropping control unit 108, etc., and limits are imposed on the image blur compensation amounts H_hom and V_hom. The limited image blur compensation amounts are respectively designated as H_hom_final and V_hom_final.

[0050] In step S303, the image blur correction amounts H_hom_final and V_hom_final calculated in step S302 are determined as the image blur correction amounts to be used in the image cutout control unit .

[0051] As described above, in this embodiment, when an anamorphic lens is attached, the correction amount is changed for each image blur correction axis, erroneous image blur correction due to differences in compression ratios is prevented, and appropriate image blur correction is realized. As a result, even if shake occurs when an anamorphic lens is attached, it is possible to realize appropriate image blur correction control without overcorrection.

[0052] In addition, in this embodiment, the case where the second image blur correction means (image blur correction by moving the image sensor) and the third image blur correction means (electronic image blur correction) are used has been described. However, the present invention may be applied to the case where the first image blur correction means (image blur correction by a shift lens arranged in the photographing lens) is used instead of the second image blur correction means, or all three types of image blur correction means are used.

[0053] Second embodiment 5 is a block diagram showing the configuration of a lens-interchangeable digital camera (imaging device) 500 for taking still images and moving images, which has an image stabilization device according to a second embodiment of the present invention. Note that the same components as those in FIG. 1 are given the same reference numerals and their explanations are omitted.

[0054] Camera body 151 of digital camera 500 shown in Fig. 5 has an RS distortion correction range calculation unit 128 and an RS distortion correction amount calculation unit 129 added to the configuration in Fig. 1. Here, RS is an abbreviation for rolling shutter, which will be described later. Furthermore, electronic correction amount conversion unit 120 and image clipping control unit 108 have been deleted, and an electronic correction amount conversion unit 130, which is controlled differently from electronic correction amount conversion unit 120, and an image transformation control unit 131 have been added.

[0055] In this embodiment, the calculation of the image blur correction amount for the anamorphic lens is performed in the same manner as the correction amount calculation operation of the image blur correction amount calculator 117 in the first embodiment shown in FIG.

[0056] Fig. 6 is a flowchart showing the calculation operation of the correction range by the RS distortion correction range calculation unit 128. Note that the process shown in Fig. 6 is repeatedly executed at a predetermined cycle, such as 60 Hz in the case of a video signal conforming to the NTSC format.

[0057] First, in step S401, optical parameter information of the photographing lens at the time of photographing is acquired, such as focal length, aperture value, focus position, distortion rate, anamorphic lens compression magnification, effective image circle diameter, etc. The information may be acquired via electrical communication with the photographing lens 101, or values ​​manually input using a user interface of the digital camera 500 may be acquired.

[0058] In step S402, the RS distortion correction range for each focal length is provisionally determined. The RS distortion correction range may be determined based on a predetermined correction table for each focal length, or may be changed based on the magnitude of the shake signal detected by the angular velocity sensor 113 in FIG.

[0059] In step S403, the correction range provisionally determined in step S402 is multiplied by the magnification of the anamorphic lens to finally determine the correction range. Images shot with an anamorphic lens need to be restored to the magnification that was compressed in post-processing. If the magnification is restored while the RS distortion remains, the RS distortion will be more noticeable than when the anamorphic lens is not attached. For this reason, control is performed to increase the correction range beyond normal by the magnification of the anamorphic lens. If the provisionally determined correction ranges are H_RS_PreRange and V_RS_PreRange, and the ranges to be finally determined are H_RS_Range and V_RS_Range, respectively, then the following can be written as (Equation 5) and (Equation 6). H_RS_Range=H_RS_PreRange·Yaw_Mag …(Formula 5) V_RS_Range=V_RS_PreRange·Pitch_Mag …(Formula 6) 5 and the correction range determined by RS distortion correction range calculation unit 128, and outputs the RS distortion correction amount to electronic correction amount conversion unit 130. In this embodiment, the rolling shutter distortion that occurs when a stationary subject is photographed by moving digital camera 500 will be described, and only the H direction (horizontal direction) will be described.

[0060] In order to correct rolling shutter distortion caused by shaking or shaking while walking, it is necessary to calculate the amount of blur on the imaging surface that occurs during the time difference in the exposure period of a pixel of interest when the exposure period of a pixel on the imaging element is used as a reference, using an angular velocity sensor or the like. Here, since the time difference in the exposure period between horizontal pixels on the same line is small enough to be ignored, pixels in the same line are treated as having the same exposure period, and the amount of blur caused by the time difference in the exposure period between the lines is calculated. However, if the amount of blur for all lines is calculated and stored, the amount of calculation is large, which places a heavy burden on the system and requires a large memory capacity. Therefore, in this embodiment, the amount of blur corresponding to the discretely thinned lines is calculated, and the amount of blur corresponding to the lines between them is obtained by interpolation.

[0061] FIG. 7 is a graph in which the lines constituting one screen are thinned to nine lines L0 to L8, the vertical axis represents time, the horizontal axis represents the RS distortion correction amount, and the RS distortion correction amounts C0 to C8 at times T0 to T8 are plotted.

[0062] Times T0 to T8 are time differences between the exposure periods of each line when line L0 is used as a reference, and RS distortion correction amounts C0 to C8 are calculated from the amount of blurring that occurs from time T0 to the time corresponding to each line. Furthermore, based on the discrete RS distortion correction amounts C0 to C8, RS distortion correction amounts corresponding to all lines of the captured image are calculated using known methods such as linear interpolation, polynomial approximation, and the least squares method.

[0063] Thereafter, the range of the calculated RS distortion correction amounts C0 to C8 is limited based on the correction ranges H_RS_Range, V_RS_Range determined by the RS distortion correction range calculation section 128, and output to the electronic correction amount conversion section .

[0064] The electronic correction amount conversion unit 130 converts the image blur correction amount supplied from the correction amount division control unit 118 and the RS distortion correction amount supplied from the RS distortion correction amount calculation unit 129 into clipping position and deformation coordinates for appropriately correcting image blur and RS distortion in the image transformation control unit 131. Then, the converted data are set in the image transformation control unit 131.

[0065] Note that the range that can be controlled by the image transformation control unit 131 is the combined range of the image blur correction range required to correct image blur and the RS distortion correction range required to correct RS distortion, so there is a trade-off between the image blur correction range and the RS distortion correction range. Therefore, based on the effective image circle diameter information acquired by the optical parameter acquisition unit 116 in Fig. 5, the readout range of the image sensor 105, the movable range of the image cropping control unit 108, the RS distortion correction range calculated by the RS distortion correction range calculation unit 128, and the like, limitations are imposed on the image blur correction amounts H_hom and V_hom in the electronic image blur correction so as not to hit the ends. The limited image blur correction amounts are set as H_hom_final and V_hom_final, respectively, and the image blur correction amounts are determined.

[0066] The image transformation control unit 131 cuts out a predetermined area of ​​the frame image stored in the image memory 107 and performs geometric transformation to generate a new frame image, which is then supplied to the display control unit 109 and the recording control unit 111. At this time, the cut-out position of the predetermined area is moved in accordance with the shake of the digital camera 500 to correct the movement of the subject position between frames (image blur) caused by the shake of the camera. In addition, an RS distortion correction function that corrects RS distortion caused by the shake of the digital camera 100 can also be realized at the same time. Note that the series of operations performed by the signal processing unit 106 and the image cut-out control unit 108 is executed at a cycle of 60 Hz in the case of a video signal conforming to the NTSC format, for example, and moving image data is generated.

[0067] As described above, according to this embodiment, when an anamorphic lens is attached, the correction amount is changed for each image blur correction axis, thereby preventing erroneous image blur correction due to differences in compression ratios, and enabling appropriate image blur correction.

[0068] In addition, the image captured by the anamorphic lens needs to be returned to the compression ratio in the post-processing process, and when the magnification is returned to the image with the RS distortion remaining, the RS distortion appears more prominently than when the anamorphic lens is not attached. However, as described in the present embodiment, the higher the compression ratio of the anamorphic lens, the more likely it is that the appropriate image blur correction amount in the H direction (horizontal direction) will decrease, so the reduced area can be used as the RS distortion correction range. This allows for appropriate image blur correction control without overcorrection even when shaking occurs when the anamorphic lens is attached, and allows the amount of RS distortion correction to be increased more than when the anamorphic lens is not attached. As a result, it is possible to provide a blur correction device that prevents RS distortion from becoming prominent even when the compression ratio is returned in the post-processing process.

[0069] (Third embodiment) 8 is a block diagram showing the configuration of a lens-interchangeable digital camera (imaging device) 800 for taking still images and moving images, which has an image stabilization device according to a third embodiment of the present invention. Note that the same components as those in FIG. 1 are given the same reference numerals and their explanations are omitted.

[0070] A camera body 152 of a digital camera 800 shown in FIG. 8 has an electronic correction amount converter 132, which is controlled differently from the electronic correction amount converter 120, added to the configuration shown in FIG.

[0071] In this embodiment, the calculation of the image blur correction amount for the anamorphic lens is performed in the same manner as the correction amount calculation operation of the image blur correction amount calculator 117 in the first embodiment shown in FIG.

[0072] Fig. 9 is a flowchart showing the correction amount conversion operation of the electronic correction amount converter 132 in Fig. 8. Note that the process shown in Fig. 9 is repeatedly executed at a predetermined cycle, such as 60 Hz in the case of a video signal conforming to the NTSC format.

[0073] First, in step S501, the image blur correction amounts H_hom and V_hom in the electronic image blur correction supplied from the correction amount division control unit 118 are obtained.

[0074] In step S502, optical parameter information of the photographing lens at the time of shooting is acquired, such as focal length, aperture value, focus position, distortion rate, anamorphic lens compression magnification, effective image circle diameter, etc. The information may be acquired via electrical communication with the photographing lens 101, or may be acquired by manually inputting values ​​using a user interface of the digital camera 100.

[0075] In step S503, a provisional image blur compensation range is calculated. The provisional image blur compensation range is set so as not to hit the edge based on the effective image circle diameter information acquired in step S502, the readout range of the image sensor 105, the movable range of the image cropping control unit 108, etc., and limits are imposed on the image blur compensation amounts H_hom and V_hom. The limited image blur compensation amounts are respectively designated as H_hom_Limit and V_hom_Limit.

[0076] In step S504, a distortion judgment (determination of distortion) is performed based on the distortion rate acquired in step S502. The acquired distortion rate is set as Lens_Dist, and compared with a predetermined distortion rate limit value Dist_Th to judge whether or not it exceeds Dist_Th. If it is determined that the distortion rate Lens_Dist exceeds Dist_Th, the process proceeds to step S505, and if it is determined that the distortion rate Lens_Dist does not exceed Dist_Th, the process proceeds to step S506.

[0077] 10 is a diagram for explaining the distortion rate Lens_Dist. If the image height from the center of an undistorted image is S and the image height difference with respect to a distorted image is ΔS, the distortion rate Lens_Dist can be expressed as in (Equation 7).

[0078] Lens_Dist[%]=100·(ΔS / S) …(Equation 7) In step S505, the image blur correction amounts further limited by the distortion rate are calculated from the limited image blur correction amounts H_hom_Limit and V_hom_Limit calculated in step S503 and Lens_Dist acquired in step S502. The image blur correction amounts limited by the distortion rate are calculated as H_hom_final and V_hom_final, respectively.

[0079] In step S506, the image blur compensation amounts H_hom_final and V_hom_final calculated in step S505 are determined as the image blur compensation amounts to be used by the image clipping control unit 108. If the distortion rate does not exceed the distortion rate limit value Dist_Th in step S504, the image blur compensation amounts H_hom_Limit and V_hom_Limit, which are limited and calculated in step S503, are determined as the image blur compensation amounts to be used by the image clipping control unit 108 in step S506.

[0080] As described above, according to this embodiment, when an anamorphic lens is attached, the correction amount is changed for each image blur correction axis, thereby preventing erroneous image blur correction due to differences in compression ratios, and enabling appropriate image blur correction.

[0081] Furthermore, even if shaking occurs when a photographing lens with a distortion rate of a predetermined level or higher is attached, it is possible to realize maximum image blur correction control within a range in which appropriate image blur correction is possible.

[0082] In addition, in this embodiment, the case where the second image blur correction means (image blur correction by moving the image sensor) and the third image blur correction means (electronic image blur correction) are used has been described. However, the present invention may be applied to the case where the first image blur correction means (image blur correction by a shift lens) is used instead of the second image blur correction means, or all three types of image blur correction means are used.

[0083] In addition, in the above-described embodiments, an angular velocity sensor has been used as an example of a shake detection means, but other shake detection means may be used. For example, an acceleration sensor may be used to calculate the amount of shake from acceleration, or a combination of multiple sensors may be used to detect shake and calculate the amount of shake of the device.

[0084] Furthermore, a shake detection means such as an angular velocity sensor may be provided in the photographing lens 101 instead of in the camera body 150. Alternatively, a shake detection means such as an angular velocity sensor may be provided in both the camera body 150 and the photographing lens 101, and shake applied to the digital camera 100 may be detected using both of the shake detection means.

[0085] In addition, in each of the above-mentioned embodiments, an anamorphic lens has been described as an example of a photographing lens having different compression ratios in the horizontal and vertical directions of a photographed image, but other photographing lenses may be used. Also, the optical parameter information of the photographing lens may be acquired (determined) by the camera body unit 150 acquiring lens type information such as a lens ID from the photographing lens, based on the optical parameter information for each lens type stored in advance in the memory of the camera body unit 150 and the acquired lens type information.

[0086] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0087] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0088] 100: digital camera, 101: photographing lens, 105: imaging element, 113: angular velocity sensor, 115: motion vector detection unit, 117: image blur correction amount calculation unit, 118: correction amount division control unit, 119: imaging element correction amount conversion unit, 120: electronic correction amount conversion unit, 121: position detection unit, 126: motor drive unit, 127: motor

Claims

1. a calculation means for calculating an image blur correction amount for correcting an image blur based on a detection result of a shake detection means for detecting a shake of the imaging device; a correction unit that performs image blur correction based on the image blur correction amount calculated by the calculation unit, the shake detection means includes a motion vector detection means for detecting a motion vector between a plurality of captured images on which desqueeze processing has not been performed, and an angular velocity sensor; the calculation means calculates the image blur correction amount using horizontal and vertical angular velocities based on detection results of the angular velocity sensor calculated using different magnifications in a horizontal axis and a vertical axis of a two-dimensional plane of the captured image based on information about a photographing lens having different image magnifications in a horizontal direction and a vertical direction when the photographed image was captured, and horizontal and vertical motion vectors based on detection results of the motion vector calculated without using the different magnifications; the correction means divides the image blur correction amount into a first correction amount used for a first image blur correction that corrects image blur by changing a relative position between an image sensor and a subject image formed by the photographing lens, and a second correction amount used for a second image blur correction that corrects image blur by changing a position at which an image is cropped from the photographed image, and performs the first image blur correction based on the first correction amount and the second image blur correction based on the second correction amount on a photographed image on which desqueezing processing has not been performed.

2. 2. The image stabilization device according to claim 1, wherein the photographing lens is a lens that photographs a subject image by compressing it in the horizontal direction.

3. 3. The image blur correction device according to claim 2, wherein the calculation means calculates the image blur correction amount in the horizontal direction to be smaller than the image blur correction amount in the vertical direction.

4. 4. The image stabilization device according to claim 3, further comprising a control unit for changing a correction range for correcting image shake based on information about the photographing lens.

5. 5. The image blur correction device according to claim 4, wherein the control means, when the horizontal image magnification and the vertical image magnification of the photographing lens are different, reduces a correction range for correcting the image blur compared to when the horizontal image magnification and the vertical image magnification of the photographing lens are the same.

6. 6. The image stabilization device according to claim 4, wherein the control means changes a distortion correction range for further correcting rolling shutter distortion of the photographed image based on the information about the photographing lens.

7. 7. The image blur correction device according to claim 6, wherein the control means increases the distortion correction range when the horizontal and vertical image magnifications of the photographing lens are different from each other, compared to when the horizontal and vertical image magnifications of the photographing lens are the same.

8. 5. The image blur correction device according to claim 4, wherein the control means changes a correction range for further correcting distortion of the photographed image based on the information about the photographing lens.

9. a calculation step of calculating an amount of image blur correction for correcting image blur based on a detection result of a shake detection means for detecting a shake of the imaging device; a correction step of performing image blur correction based on the image blur correction amount calculated in the calculation step, the shake detection means includes a motion vector detection means for detecting a motion vector between a plurality of captured images on which desqueeze processing has not been performed, and an angular velocity sensor; In the calculation step, the image blur correction amount is calculated using horizontal and vertical angular velocities based on detection results of the angular velocity sensor calculated using different magnifications in a horizontal axis and a vertical axis of a two-dimensional plane of the captured image based on information of a photographing lens having different image magnifications in a horizontal direction and a vertical direction used when the photographed image was captured, and horizontal and vertical motion vectors based on detection results of the motion vector calculated without using the different magnifications; a control method for an image blur correction device, wherein the correction step divides the image blur correction amount into a first correction amount used for a first image blur correction that corrects image blur by changing a relative position between an image sensor and a subject image formed by the photographing lens, and a second correction amount used for a second image blur correction that corrects image blur by changing an image cropping position from the photographed image, and the first image blur correction based on the first correction amount and the second image blur correction based on the second correction amount are performed on a photographed image on which desqueezing processing has not been performed.

10. A program for causing a computer to execute the control method according to claim 9.

11. A computer-readable storage medium storing a program for causing a computer to execute the control method according to claim 9.

Images