Imaging apparatus

Abstract

To provide an imaging apparatus capable of efficiently correcting distortion of an image caused by a camera shake, while suppressing reduction in an angle of view.SOLUTION: The imaging apparatus includes: an imaging element for generating image data by imaging a subject image through an optical system; a detection part for detecting a shake amount of the imaging apparatus; an image processing part for performing image blur correction by adjusting a part for outputting an image in the image data according to the shake amount detected by the detection part; and a control part for controlling the image blur correction performed in the image processing part. In the image processing part, distortion correction corresponding to distortion aberration of the optical system is performed to an image area indicated by the image data. The control part changes a ratio between first image blur correction and second image blur correction performed in a distortion corrected image area by the image processing part according to a focal distance of the optical system, to correct distortion of the image in the distortion corrected image area, in the first image blur correction and to move the part for outputting the image in the image area, in the second image blur correction.SELECTED DRAWING: Figure 15

Description

[Technical Field]

[0001] The present disclosure relates to an imaging device having an image stabilization function. [Background technology]

[0002] Patent Document 1 discloses an imaging device that aims to perform shake correction by efficiently utilizing the imaging area of ​​an imaging element. This imaging device extracts an image of an extraction area that is rotated or moved according to the shake correction amount (correction angle) from an image that has been corrected for distortion aberration caused by the imaging optical system. In this way, the imaging device of Patent Document 1 uses an image area that is stretched outward from the center of the image by correcting distortion aberration, and corrects shake in the rotational direction around the optical axis of the imaging optical system or in the pan and tilt directions by rotating or moving the extraction area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-273245 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides an imaging device that can efficiently correct image distortion caused by camera shake while suppressing a reduction in the angle of view. [Means for solving the problem]

[0005] An imaging device according to one aspect of the present disclosure includes an imaging element having an imaging area where a subject image is formed via an optical system and capturing the subject image to generate image data, a detection unit that detects the amount of blurring of the imaging device, an image processing unit that performs image blur correction by adjusting a portion of the image data that outputs an image in accordance with the amount of blurring detected by the detection unit, and a control unit that controls the image blur correction performed in the image processing unit. The image processing unit performs distortion correction in accordance with distortion aberration of the optical system on the image area indicated by the image data. The control unit changes the ratio between first and second image blur corrections performed by the image processing unit in the distortion-corrected image area in accordance with the focal length of the optical system, and the first image blur correction corrects image distortion in the distortion-corrected image area, and the second image blur correction moves the portion of the image area that outputs the image. [Effects of the Invention]

[0006] According to the imaging device of the present disclosure, image distortion caused by camera shake can be efficiently corrected while suppressing a reduction in the angle of view. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a perspective view of a digital camera according to a first embodiment of the present disclosure; [Figure 2] FIG. 1 is a block diagram showing the configuration of a digital camera according to a first embodiment. [Figure 3] FIG. 1 is a block diagram showing the configuration of a BIS processing unit in a digital camera according to a first embodiment. [Figure 4] FIG. 1 is a diagram for explaining a correction mode using the EIS function of the digital camera according to the first embodiment. [Figure 5] Diagram to explain EIS function with cropping in a digital camera [Figure 6] FIG. 1 is a diagram for explaining correction according to distortion aberration in the optical system of a digital camera. [Figure 7] A diagram to explain camera shake caused by tilting a digital camera [Figure 8]FIG. 1 is a diagram for explaining a cropless EIS function in a digital camera according to a first embodiment. [Figure 9] 1 is a flowchart illustrating an example of the overall operation of image stabilization for a digital camera according to a first embodiment; [Figure 10] FIG. 10 is a diagram for explaining a calculation process of distortion correction parameters in a digital camera. [Figure 11] 1 is a flowchart illustrating a process for calculating tilt correction parameters in a digital camera; [Figure 12] FIG. 10 is a diagram for explaining a process for calculating a surplus area in a process for calculating a perspective correction parameter. [Figure 13] 10 is a flowchart illustrating the operation of a digital camera according to a modification of the first embodiment. [Figure 14] FIG. 10 is a diagram for explaining the operation of a digital camera according to a second embodiment. [Figure 15] 10 is a flowchart illustrating the operation of a digital camera according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present disclosure will be described with appropriate reference to the drawings. However, in the detailed description, unnecessary parts of the description of the prior art and substantially identical configurations may be omitted. This is for the sake of simplicity. Furthermore, the following description and the accompanying drawings are disclosed to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter of the claims.

[0009] (Embodiment 1) In the first embodiment, a digital camera having an image stabilization function will be described as an example of an imaging device.

[0010] 1. Configuration Fig. 1 is a perspective view of a digital camera 1 according to embodiment 1. Fig. 2 is a block diagram showing the configuration of the digital camera 1. The digital camera 1 is made up of a camera body 100 and an interchangeable lens 200 that can be attached to and detached from the camera body 100.

[0011] In the following description, the function of correcting blur by moving the imaging element in camera body 100 is referred to as the "BIS (Body Image Stabilizer) function." Additionally, the function of correcting blur by moving the correction lens in interchangeable lens 200 is referred to as the "OIS (Optical Image Stabilizer) function." Furthermore, the function of correcting blur by adjusting the image output area in image data generated by the imaging element is referred to as the "EIS (Electronic Image Stabilizer) function." In the following description, the BIS function and OIS function are collectively referred to as the optical image stabilization function, and the EIS function is sometimes referred to as the electronic image stabilization function.

[0012] In the following description, the rotation directions corresponding to the horizontal and vertical directions of the image sensor in the digital camera 1 are referred to as the yaw direction and pitch direction, respectively, and the rotation direction about the rotation axis along the optical axis of the digital camera 1 is referred to as the roll direction (see Figure 1).

[0013] 1-1.Camera body Camera body 100 (an example of an imaging device) includes an image sensor 110, an LCD monitor 120, an operation unit 130, a camera control unit 140, a body mount 150, and a card slot 170. Camera body 100 also includes, as a functional component of camera control unit 140, an image correction unit 143 that implements an EIS function, for example.

[0014] The camera control unit 140 controls the overall operation of the digital camera 1 by controlling components such as the image sensor 110 in response to instructions from the release button. The camera control unit 140 transmits a vertical synchronization signal to the timing generator 112. In parallel with this, the camera control unit 140 generates an exposure synchronization signal. The camera control unit 140 periodically transmits the generated exposure synchronization signal to the lens control unit 240 via the body mount 150 and the lens mount 250. The camera control unit 140 uses the DRAM 141 as a working memory during control operations and image processing operations.

[0015] The image sensor 110 is an example of an imaging element that captures an incident subject image via the interchangeable lens 200 and generates image data. The image sensor 110 is, for example, a CCD, a CMOS image sensor, or an NMOS image sensor. The generated image data is digitized by an AD converter 111. The digitized image data is subjected to predetermined image processing by the camera control unit 140. The predetermined image processing includes, for example, gamma correction, white balance correction, blemish correction, YC conversion, electronic zoom, and JPEG compression.

[0016] The image sensor 110 operates at a timing controlled by a timing generator 112. The image sensor generates still images, moving images, or through images for recording. The through images are mainly moving images, and are displayed on the LCD monitor 120 so that the user can decide on a composition for capturing a still image.

[0017] The LCD monitor 120 displays images such as through images and various information such as menu screens. The LCD monitor 120 is an example of a display unit in this embodiment. Instead of the LCD monitor, other types of display devices, such as an organic EL display device, may be used.

[0018] The operation unit 130 includes various operation members such as a release button for issuing an instruction to start shooting, a mode dial for setting the shooting mode, a power switch, etc. The operation unit 130 also includes a touch panel superimposed on the liquid crystal monitor 120.

[0019] The card slot 170 can accommodate a memory card 171, and controls the memory card 171 under control of the camera control unit 140. The digital camera 1 can store image data in the memory card 171 and read image data from the memory card 171.

[0020] The body mount 150 can be mechanically and electrically connected to the lens mount 250 of the interchangeable lens 200. The body mount 150 can transmit and receive data to and from the interchangeable lens 200 via the lens mount 250. The body mount 150 transmits an exposure synchronization signal received from the camera control unit 140 to the lens control unit 240 via the lens mount 250. The body mount 150 also transmits other control signals received from the camera control unit 140 to the lens control unit 240 via the lens mount 250. The body mount 150 also transmits signals received from the lens control unit 240 to the camera control unit 140 via the lens mount 250.

[0021] Furthermore, as components for realizing the BIS function, camera body 100 further includes a gyro sensor 184 (shake detection unit) that detects shake of camera body 100, and a BIS processing unit 183 that controls shake correction processing based on the detection result of gyro sensor 184. Furthermore, camera body 100 includes a sensor driving unit 181 that moves image sensor 110, and a position sensor 182 that detects the position of image sensor 110.

[0022] The sensor driving unit 181 can be realized by, for example, a magnet and a flat coil. The sensor driving unit 181 may also include other motors or actuators. The position sensor 182 is a sensor that detects the position of the image sensor 110 in a plane perpendicular to the optical axis of the optical system. The position sensor 182 can be realized by, for example, a magnet and a Hall element.

[0023] Based on signals from the gyro sensor 184 and the position sensor 182, the BIS processing unit 183 controls the sensor driving unit 181 to shift the image sensor 110 in a plane perpendicular to the optical axis so as to offset the shake of the camera body 100. There are mechanical limitations on the range over which the image sensor 110 can be driven by the sensor driving unit 181. In the BIS function, the range over which the image sensor 110 can be driven by the sensor driving unit 181 is called the "element driving range."

[0024] 1-2.Interchangeable lenses The interchangeable lens 200 includes an optical system, a lens control unit 240, and a lens mount 250. The optical system includes a zoom lens 210, an OIS (Optical Image Stabilizer) lens 220, a focus lens 230, and an aperture 260.

[0025] The zoom lens 210 is a lens for changing the magnification of a subject image formed by the optical system. The zoom lens 210 is composed of one or more lenses. The zoom lens 210 is driven by a zoom driver 211. The zoom driver 211 includes a zoom ring that can be operated by the user. Alternatively, the zoom driver 211 may include a zoom lever and an actuator or a motor. The zoom driver 211 moves the zoom lens 210 along the optical axis direction of the optical system in response to an operation by the user.

[0026] The focus lens 230 is a lens for changing the focus state of the subject image formed on the image sensor 110 by the optical system. The focus lens 230 is composed of one or more lenses. The focus lens 230 is driven by a focus driver 233.

[0027] The focus driver 233 includes an actuator or a motor, and moves the focus lens 230 along the optical axis of the optical system under the control of the lens controller 240. The focus driver 233 can be realized by a DC motor, a stepping motor, a servo motor, an ultrasonic motor, or the like.

[0028] The OIS lens 220 is a lens for correcting blur of the subject image formed by the optical system of the interchangeable lens 200 in the OIS function. The OIS lens 220 reduces blur of the subject image on the image sensor 110 by moving in a direction that offsets blur of the digital camera 1. The OIS lens 220 is made up of one or more lenses. The OIS lens 220 is driven by an OIS driver 221.

[0029] The OIS driver 221, under control of the OIS processor 223, shifts the OIS lens 220 within a plane perpendicular to the optical axis of the optical system. There are mechanical limitations on the range over which the OIS driver 221 can drive the OIS lens 220. The range over which the OIS driver 221 can drive the OIS lens 220 is referred to as the "lens driving range." The OIS driver 221 can be realized, for example, by a magnet and a flat coil. The position sensor 222 is a sensor that detects the position of the OIS lens 220 within a plane perpendicular to the optical axis of the optical system. The position sensor 222 can be realized, for example, by a magnet and a Hall element. The OIS processor 223 controls the OIS driver 221 based on the output of the position sensor 222 and the output of a gyro sensor 224 (shake detection unit).

[0030] The diaphragm 260 adjusts the amount of light incident on the image sensor 110. The diaphragm 260 is driven by a diaphragm driving unit 262, and the size of the opening is controlled. The diaphragm driving unit 262 includes a motor or an actuator.

[0031] The gyro sensor 184 or 224 detects shake (vibration) in the yaw, pitch, and roll directions based on the change in angle per unit time of the digital camera 1, i.e., the angular velocity. The gyro sensor 184 or 224 outputs an angular velocity signal indicating the amount of shake (angular velocity) detected to the BIS processing unit 183 or OIS processing unit 223. The angular velocity signal output by the gyro sensor 184 or 224 may include a wide range of frequency components caused by camera shake, mechanical noise, etc. Instead of a gyro sensor, another sensor capable of detecting shake of the digital camera 1 may be used. Furthermore, the gyro sensor 224 of the interchangeable lens 200 does not have to detect shake in the roll direction.

[0032] The camera control unit 140 and the lens control unit 240 may be configured with a hardwired electronic circuit, or may be configured with a microcomputer using a program, etc. For example, the camera control unit 140 and the lens control unit 240 can be realized by various processors such as a CPU, an MPU, a GPU, a DSU, an FPGA, or an ASIC.

[0033] 1-3. Image stabilization function configuration The configuration for realizing various image stabilization functions of digital camera 1 in this embodiment will be described with reference to FIGS.

[0034] 1-3-1.BIS processing section The configuration of the BIS processing unit 183 in the camera body 100 will be described using Fig. 3. Fig. 3 is a block diagram showing the configuration of the BIS processing unit 183 in the digital camera 1 of this embodiment. The BIS processing unit 183 includes an HPF (high pass filter) 406, a phase compensation unit 407, an integrator 408, and a PID control unit 410. For example, the BIS processing unit 183 receives a signal from the gyro sensor 184 at a predetermined time interval (for example, 4 kHz).

[0035] The HPF 406 blocks predetermined low frequency components contained in the signal received from the gyro sensor 184, for example, in order to block drift components.

[0036] The phase compensation unit 407 corrects the phase delay of the signal received from the HPF 406 due to the sensor driving unit 181 and the like.

[0037] The integrator 408 integrates the signal indicating the angular velocity of the shake (vibration) input from the phase compensation unit 407 to generate a signal indicating the angle of the shake (vibration) (hereinafter referred to as a "shake detection signal"). The shake detection signal from the integrator 408 is input to a PID control unit 410. Here, the BIS processing unit 183 may use or add a filter configuration other than the above configuration, such as a notch filter for noise processing.

[0038] Based on the output from the position sensor 182 and the output from the integrator 408, the PID control unit 410 generates a drive signal for shifting the image sensor 110 and outputs the drive signal to the sensor drive unit 181. The sensor drive unit 181 drives the image sensor 110 based on the drive signal. Specifically, the sensor drive unit 181 translates the image sensor 110 in the horizontal or vertical direction of the imaging surface within the element movable range, and rotates the image sensor 110 around the optical axis direction as the rotation axis.

[0039] BIS processing unit 183 is configured to be able to communicate data with camera control unit 140. For example, BIS processing unit 183 starts / ends the image stabilization operation in response to a control signal from camera control unit 140. In addition, BIS processing unit 183 transmits various pieces of information related to the image stabilization operation to camera control unit 140.

[0040] For example, the BIS processing unit 183 may calculate blur correction amounts in the horizontal and vertical directions of the imaging surface, respectively, as the movement amount of the image sensor 110 by the sensor driving unit 181, from the angles of blur in the yaw and pitch directions indicated by the generated blur detection signal. The BIS processing unit 183 may acquire a focal length according to the zoom state from the interchangeable lens 200 via the camera control unit 140, and may calculate the blur correction amount by converting the acquired focal length, etc., into a correction angle that offsets the angle of blur into the movement amount of the image sensor 110.

[0041] The OIS processing unit 223 may be configured to drive the OIS driving unit 221 instead of the sensor driving unit 181 in a configuration similar to that of the BIS processing unit 183 described above. The OIS processing unit 223 also operates using the detection results of the gyro sensor 224 in the interchangeable lens 200 instead of the gyro sensor 184 in the camera body 100, for example. The gyro sensor 224 of the interchangeable lens 200 does not need to detect shake in the roll direction.

[0042] 1-3-2.EIS correction mode Digital camera 1 of this embodiment has a plurality of correction modes as operation modes for correcting blur using the image stabilization function. Fig. 4 is a diagram for explaining the correction modes using the EIS function of digital camera 1 of this embodiment.

[0043] In the digital camera 1 of this embodiment, the user can select the correction mode used for image stabilization using the EIS function. Fig. 4 shows an example of a menu screen for setting the correction mode when shooting video in the digital camera 1. In the example of Fig. 4, the LCD monitor 120 displays "large crop," "small crop," "cropless," and "OFF" as menu items corresponding to each correction mode. When the "OFF" menu item is selected, the EIS function is disabled.

[0044] In the "large crop" and "small crop" correction modes, the image correction unit 143 corrects camera shake by changing the area from which the image is cut out in the imaging area of ​​the image sensor 110 in accordance with the camera shake. The EIS function with cropping will be described with reference to FIG. 5. FIG. 5 is a diagram for explaining the EIS function with cropping in the digital camera 1.

[0045] 5, the image correction unit 143 performs processing to cut out an image of a narrower region by a preset cropping amount Eo from the entire image in the image data 10 generated by the image sensor 110. For example, the cropping amount Eo is calculated based on the number of pixels for each of the horizontal direction X and vertical direction Y of the image data 10 according to a predetermined cropping rate, and an image is cropped in which the number of pixels is reduced by the calculated cropping amount Eo in each direction X and Y. For example, various image processes performed by the camera control unit 140 for recording the shooting results are performed on the image data after such cropping. For example, electronic zoom processing may be performed so that the cropped image has the same size as the image before cropping.

[0046] For example, in response to a user operation to select a menu item of "large crop" or "small crop," a predetermined cropping amount Eo stored in advance in the flash memory 142 or the like for each cropped correction mode is set in the image correction unit 143. In the "large crop" correction mode, a larger cropping amount Eo is set than in the "small crop" correction mode. For example, in the "large crop" correction mode, a cropping amount Eo is set such that the number of pixels included in the cropped image is reduced by 20 to 30% from the image before cropping. In the "small crop" correction mode, a cropping amount Eo is set such that the number of pixels before and after cropping is reduced by approximately 8%.

[0047] The image correction unit 143 calculates a blur correction amount as an adjustment amount for the crop position based on the blur detection signal input from the integrator 408 of the BIS processing unit 183. The image correction unit 143 adjusts the position at which the image is cropped by the calculated blur correction amount, thereby realizing the EIS function in the "large crop" or "small crop" correction mode. For example, a reference area 21 is set in advance, using the center position of the entire image in the image data 10 as a reference, in which the image to be cropped is located when the blur correction amount by the EIS function with cropping is zero. For example, the reference area 21 is arranged along the horizontal direction X and vertical direction Y of the image data 10. For example, the area other than the reference area 21 in the image data 10 is an example of a correction area in this embodiment.

[0048] The image correction unit 143 translates the image area 22 cut out from the reference area 21 in the horizontal direction X according to the amount of blur correction for the image sensor 110 in the horizontal direction acquired from the BIS processing unit 183, for example. Similarly, the image correction unit 143 translates the image area 22 in the vertical direction Y according to the amount of blur correction for the image sensor 110 in the vertical direction. Furthermore, the image correction unit 143 rotates the orientation of the image area 22 from the orientation of the reference area 21 on the XY plane according to the amount of blur correction for the roll direction.

[0049] The EIS function with cropping can adjust the position of the image area 22 as described above within the range of the cropping amount Eo. Specifically, the translation of the image area 22 is performed within the range of the cropping amount Eo minus a roll amount as a margin Er. Furthermore, the rotation of the image area 22 is performed within a rotation angle range that falls within the roll margin Er for the cropping amount Eo. The roll margin Er is determined by the camera control unit 140 depending on, for example, the lens state of the interchangeable lens 200. For example, when the interchangeable lens 200 has a wide angle, the amount of movement of the image area 22 that corrects the amount of blur in the yaw direction and pitch direction becomes small, and therefore, the shorter the focal length, the larger the roll margin Er is determined to be.

[0050] In contrast, in the digital camera 1 of this embodiment, when the "cropless" correction mode is selected on the menu screen or the like of FIG. 4, a cropless EIS function is executed, which does not set the crop amount Eo as described above within the range corresponding to the imaging area in the image data 10. In this case, the corrected image obtained has an angle of view similar to that when the EIS function is "OFF," i.e., disabled (the number of pixels in that range). In this way, the cropless EIS function can suppress a reduction in the number of pixels in the image data 10 due to cropping, and perform image stabilization while maintaining the angle of view at the time of image capture. The image correction unit 143 realizes the cropless EIS function in accordance with, for example, the lens characteristics of the interchangeable lens 200, as described below.

[0051] 2.Operation The operation of the digital camera 1 configured as above will now be described.

[0052] 2-1. Distortion Correction In the digital camera 1 of this embodiment, for example, the image correction unit 143 corrects distortion caused by the optical system of the interchangeable lens 200 in the captured image in addition to the operation of image stabilization using the EIS function.

[0053] 6A and 6B are diagrams illustrating correction according to distortion in the optical system of digital camera 1. For example, as shown in Fig. 6A, distortion due to the lens characteristics of interchangeable lens 200 or the like can cause image distortion in image data 10 generated by image sensor 110. In image data 10, the entire range of the image corresponding to the imaging area of ​​image sensor 110 constitutes image formation area 20 according to the number of output pixels in recording the captured image when, for example, various image corrections are not applied.

[0054] 6A shows an example in which the optical system of the interchangeable lens 200 has negative distortion. In the captured image represented by the image data 10 in FIG. 6A, the negative distortion causes so-called barrel aberration, in which the periphery of the image shrinks relative to the center position Pc of the entire image.

[0055] FIG. 6A also illustrates distortion characteristic data 30 indicating the characteristics of distortion in the interchangeable lens 200. The distortion characteristic data 30 associates the image height, based on the center position Pc, with the distortion rate. For example, the distortion rate indicates the rate of change in the image height at which the subject image is formed on the imaging surface of the image sensor 110, from when there is no distortion. The sign of the distortion rate is positive if the direction in which the imaging position of the subject image on the imaging surface changes from when there is no distortion is the direction in which the image height increases, and negative if the direction of change is the direction in which the image height decreases. For example, a wide-angle lens with a relatively wide angle of view is prone to barrel distortion, which is a type of distortion exhibiting a negative distortion rate.

[0056] The characteristics of distortion also change depending on the focal length according to the zoom state of the interchangeable lens 200 and the in-focus position according to the focus state. For example, barrel aberration is likely to occur on the wide-angle side where the focal length is relatively short.

[0057] As shown in FIG. 6B, for example, the image correction unit 143 corrects distortion by deforming the image in the image data 10 of FIG. 6A so that the peripheral portion is stretched outward from the center position Pc from the image formation area 20. The image correction unit 143 performs this distortion correction based on, for example, distortion correction data 31 shown in FIG. 6B. The distortion correction data 31 associates the image height based on the center position Pc with a correction rate. For example, the correction rate indicates the rate of deformation of the image from before the distortion correction.

[0058] According to the distortion correction described above, an enlarged image area 13 is obtained by enlarging the image formation area 20 corresponding to the rectangular imaging area in the image data 10. For example, if the amount of blur correction by the EIS function is zero or the EIS function is disabled, the image correction unit 143 crops and outputs an image of the same size as the range corresponding to the imaging area from the enlarged image area 13 in the image data 10, centered around the center position Pc. The image correction unit 143 in the digital camera 1 of this embodiment realizes a cropless EIS function by using the enlarged image area 13 before and after correction in the image data 10 using this distortion correction.

[0059] The above describes a case where barrel aberration is corrected as distortion correction. However, with pincushion aberration, which exhibits a positive distortion rate, the captured image appears as if the periphery of the image is elongated outward relative to the center position Pc. Digital camera 1 performs distortion correction for such pincushion aberrations by shrinking the periphery of the image. If the enlarged image area 13 cannot be obtained by distortion correction, digital camera 1 of this embodiment does not execute, for example, a cropless EIS function. On the other hand, for pincushion aberration, which is a combination of barrel aberration and pincushion aberration, the cropless EIS function may be applied if the enlarged image area 13 can be obtained by distortion correction, as with barrel aberration correction.

[0060] 2-2. Correcting tilt shake In addition to the distortion aberration described above, in situations where there is relatively large camera shake, such as when a user (photographer) is shooting a video or the like while walking, distortion may occur in the periphery of a captured image due to camera shake caused by tilt that changes the attitude of digital camera 1. Correction of such tilt shake will be described using FIGS. 7 and 8.

[0061] 7A to 7D are diagrams for explaining camera shake caused by tilting the digital camera 1. Each of Fig. 7A to 7D shows the attitude of the digital camera 1 and a captured image Im of the same subject captured by the digital camera 1 in that attitude.

[0062] Fig. 7(A) shows an example in which no tilt blur occurs. Fig. 7(B) shows an example in which the attitude of digital camera 1 has changed in the pitch direction from the state in Fig. 7(A). For example, as shown in Fig. 7(B), in the captured image Im in the case where tilt blur occurs in the pitch direction, trapezoidal distortion occurs in which the subject image is stretched in one of the horizontal direction X and the vertical direction Y and shrunk in the other direction.

[0063] FIG. 7(C) shows an example in which the attitude of digital camera 1 has changed from the state shown in FIG. 7(A) in the yaw direction instead of the pitch direction shown in FIG. 7(B). For example, as shown in FIG. 7(C), in a captured image Im in the event of tilt blur in the yaw direction, trapezoidal distortion occurs, in which the subject image is stretched in one direction in the vertical direction Y and the horizontal direction X while shrinking in the other direction. FIG. 7(D) shows an example in which the attitude of digital camera 1 has changed from the state shown in FIG. 7(A) in both the pitch and yaw directions. In this case, trapezoidal distortion occurs in captured image Im in both the horizontal direction X and the vertical direction Y. It is known that such trapezoidal distortion due to tilt blur is particularly noticeable in captured image Im in wide-angle photography, which has a relatively wide imaging range.

[0064] For example, in the center of the captured image Im, it is expected that blurring in the yaw, pitch, and roll directions can be reduced by translating and rotating the image sensor 110 using the BIS function and / or translating and rotating the image region 22 extracted using the EIS function, with the central position Pc as the reference. On the other hand, even if correction by translation and rotation as described above is performed on the captured image Im, there is a concern that trapezoidal distortion due to tilt blurring will remain, and will be particularly noticeable in the peripheral areas away from the central position Pc.

[0065] For this reason, digital camera 1 corrects trapezoidal distortion caused by tilt shake by performing image transformation processing on image data 10 to restore the trapezoidal distortion. Such image trapezoid correction requires the use of an image area larger than the corrected area, for example, because the transformation processing involves projectively transforming coordinates on the image in accordance with the trapezoidal distortion. Therefore, digital camera 1 of this embodiment corrects trapezoidal distortion caused by tilt shake by using a cropless EIS function to correct the trapezoidal distortion by using an image area 13 enlarged by distortion correction as shown in FIG. 6(B).

[0066] 8 is a diagram illustrating the cropless EIS function of the digital camera 1 of this embodiment. In the digital camera 1 of this embodiment, the image correction unit 143 sets a reference area 11 to be deformed by keystone correction from an image area 13 enlarged by distortion correction in the image data 10. The image correction unit 143 then outputs an image in the output area 12 as a corrected area deformed from the reference area 11. The output area 12 is an area in the image data 10 that is recorded on the memory card 171 or displayed on the LCD monitor 120, for example, and has a number of pixels according to the resolution of the image recorded as the captured image.

[0067] As described above, the digital camera 1 of this embodiment corrects tilt blur in image data 10 that has distortion aberrations such as barrel aberration, using the image area 13 enlarged by distortion correction. In this way, by using the enlarged image area 13, which is wider than the image formation area 20 (FIG. 6) that corresponds to the imaging area of ​​the image sensor 110, to correct keystone distortion caused by tilt blur, it is possible to output an image in the output area 12 that has, for example, the same number of pixels as the image formation area 20. This makes it possible to achieve tilt correction that corrects such image distortion while suppressing a reduction in the angle of view due to image cropping.

[0068] For example, as shown in Figure 5, in the case of an EIS function with cropping that crops an image area 22 from a reference area 21 according to a cropping amount Eo within the image formation area 20, the number of pixels in the output image area 22 is reduced from the number of pixels in the image formation area 20. For this reason, for example, when correction is performed using the EIS function with cropping, an image of the image area 22 cropped to a narrower angle of view than before the correction is output. In contrast, the cropless EIS function can achieve tilt correction and maintaining the angle of view at the same time, as described above.

[0069] While the above description has been given of an example in which the cropless EIS function outputs an image in the output area 12 having the same number of pixels as the image formation area 20, it is sufficient that the number of pixels is approximately the same between the output area 12 and the image formation area 20. For example, even when the number of pixels is reduced in accordance with various image processes performed by the camera control unit 140, the cropless EIS function may be executed so that the rate of reduction in the number of pixels is one-tenth or less of the rate of reduction in the "small crop" correction mode. With this cropless correction mode, image stabilization can be performed without substantially reducing the number of pixels before and after correction.

[0070] 2-3. Overall operation Digital camera 1 of this embodiment performs image stabilization using the BIS function in combination with distortion and tilt correction by image correction unit 143 as described above. The overall operation related to image stabilization in digital camera 1 of this embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart illustrating the overall operation related to image stabilization in digital camera 1 of this embodiment.

[0071] The processing shown in the flowchart of Fig. 9 is started, for example, when the interchangeable lens 200 is attached to the camera body 100 and the cropless correction mode is selected by a user operation using the menu screen of Fig. 4. Each process in this flowchart is executed, for example, by the camera control unit 140 in parallel with operations such as video shooting. The camera control unit 140 repeatedly executes the processing shown in this flowchart, for example, at a predetermined cycle. The predetermined cycle is, for example, a frame cycle, for example, 1 / 30 to 1 / 60 seconds.

[0072] The camera control unit 140 performs data communication with the lens control unit 240 of the interchangeable lens 200 via the body mount 150 and the lens mount 250, and acquires lens state data (S1). The lens control unit 240 reads out the lens state data in response to a request from the camera control unit 140, for example, and transmits the data to the camera body 100.

[0073] The lens state data includes, for example, the focal length according to the zoom state of the interchangeable lens 200, the in-focus position according to the focus state, and the like, and is stored in the RAM 241 of the interchangeable lens 200, etc.

[0074] The camera control unit 140 calculates distortion correction parameters to be used for distortion correction based on the lens state data etc. acquired in step S1 (S2). For example, the distortion correction parameters include distortion correction coefficients etc. used for coordinate transformation for each pixel that transforms the image by distortion correction.

[0075] 10 is a diagram illustrating the calculation process (S2) of distortion correction parameters in digital camera 1. In distortion correction, as shown in Fig. 10(A), for example, the coordinates of pixel R converted from the coordinates of pixel P in accordance with the deformation of the image are calculated based on the distance d from distortion correction center C, which corresponds to the intersection of the optical axis and the imaging surface of image sensor 110, to each pixel P in image data 10.

[0076] 10A, (Cx, Cy) indicate the coordinates of the distortion correction center C, (Px, Py) indicate the coordinates of pixel P, and (Rx, Ry) indicate the coordinates of pixel R. For example, in image data 10, pixel P is located in an image area that is output by distortion correction, and pixel R is located in an image formation area 20 that is referenced in distortion correction (see FIG. 6).

[0077] In step S2, the camera control unit 140 calculates, for each pixel P, a distortion correction coefficient e as shown in FIG. 10B, for example, based on the distance d from the distortion correction center C and the distortion correction data 31 as shown in FIG. 6B. For example, the center position Pc of the entire image in the image data 10 is used as the distortion correction center C. The distance d is calculated as, for example, a Euclidean distance, and indicates the number of pixels corresponding to the image height in the distortion correction data 31 of FIG. 6B.

[0078] 6A may be acquired for each combination of focal length and in-focus position through data communication with the interchangeable lens 200, and stored in the RAM 141, etc. In step S2, the camera control unit 140 may read out the distortion characteristic data 30 corresponding to the focal length and in-focus distance in the lens state data, and may acquire distortion correction data 31 as the inverse characteristics of the distortion aberration characteristics from the distortion characteristic data 30 by appropriate interpolation processing, etc.

[0079] The camera control unit 140 sets, for example, various parameters in the BIS processing unit 183 (FIG. 3) so as to control the image stabilization operation executed by the BIS processing unit 183 in the digital camera 1 (S3). The camera control unit 140 may calculate the angular range in the pitch and yaw directions that can be corrected by the BIS function based on, for example, information indicating the element drive range in the BIS function and the focal length of the interchangeable lens 200 in the lens state data, and set this in the BIS processing unit 183. The information indicating the element drive range may be stored in advance in the flash memory 142, etc. The camera control unit 140 may obtain the current amount of blur correction, such as the amount at the time of execution of step S3, from the BIS processing unit 183 as the amount of blur correction according to the state in which blur is corrected by the BIS function, for example.

[0080] The BIS processing unit 183 generates shake detection signals in the pitch, yaw, and roll directions in the integrator 408 based on the detection results of the gyro sensor 184 of the camera body 100. The BIS processing unit 183 calculates shake correction amounts in each direction as the horizontal and vertical movement amounts of the image sensor 110 in accordance with the shake amounts indicated by the shake detection signals in the yaw and pitch directions, for example, based on the generated shake detection signals. Furthermore, the BIS processing unit 183 calculates a shake correction amount in the roll direction as the movement amount of the image sensor 110 in the roll direction. The BIS processing unit 183 may limit the calculated shake correction amount, for example, to within the correctable angle range set as described above.

[0081] The BIS processing unit 183 causes the sensor driving unit 181 to translate the image sensor 110 in accordance with the calculated amounts of shake correction in the horizontal and vertical directions. The BIS processing unit 183 causes the sensor driving unit 181 to rotate the image sensor 110 in accordance with the amount of shake correction calculated for the amount of shake in the roll direction. The BIS processing unit 183 performs the above-described image stabilization operation as needed in accordance with the detection results of the gyro sensor 184, for example, using various parameters set by the camera control unit 140.

[0082] The camera control unit 140 acquires (S4) the amount of shake indicated by the shake detection signals in the yaw and pitch directions as the amount of tilt caused by the shake of the camera body 100, for example, from the BIS processing unit 183. For example, this amount of tilt shake is acquired based on the shake detection signal at the time of exposure corresponding to the exposure synchronization signal.

[0083] The camera control unit 140 performs a process of calculating a tilt correction parameter used for tilt correction based on the acquired amount of tilt blur, etc. (S5). The tilt correction parameter includes, for example, a parameter of projective transformation that transforms the image according to the amount of tilt blur. In step S5 of this embodiment, the camera control unit 140 does not calculate the tilt correction parameter if the image area 13 after enlargement by distortion correction cannot be obtained. This determination is made based on the distortion correction parameter, etc. calculated in step S2. The tilt correction parameter calculation process (S5) will be described in detail later.

[0084] The camera control unit 140 executes a calculation process (S6) of transforming the image as the image correction unit 143 using the distortion correction parameters calculated in step S2 and the tilt correction parameters calculated in step S5. In this image correction process (S6), the image correction unit 143 of this embodiment simultaneously applies calculation processes for distortion correction and tilt correction to the image data 10 that has undergone image stabilization operation using the BIS function in accordance with the image stabilization control (S3). Furthermore, the image correction process of this embodiment calculates coordinate transformation for each pixel that transforms the image so as to determine, from each pixel in the output area 12 after tilt correction, the corresponding pixel in the image formation area 20 before distortion correction.

[0085] In the perspective correction, a projective transformation is calculated according to the following equation (1) using the perspective correction parameters calculated in step S5, etc. The perspective correction parameters include a rotation matrix R and a translation amount t of the digital camera 1. In equation (1), for example, coordinates (xa, ya, za) indicate three-dimensional coordinates projected onto coordinates (xb, yb) on a two-dimensional image by perspective transformation. The coordinates (xb, yb) indicate coordinates in the output area 12 (FIG. 8) after perspective correction. f indicates the focal length in the lens state data of the interchangeable lens 200.

number

[0086] Furthermore, for example, from the coordinates (xa, ya, za), the coordinates (x, y) corresponding to the coordinates (Px, Py) of pixel P in distortion correction are calculated according to the following equation (2).

number

[0087] In step S6, the image correction unit 143 performs calculations using equations (1) and (2), for example, for the number of pixels in the output area 12. If, for example, tilt correction parameters have not been calculated, the image correction unit 143 does not perform the calculations using equations (1) and (2), but performs only the calculation of distortion correction.

[0088] For example, in distortion correction, the coordinates (Rx, Ry) of pixel R in the image formation area 20 are calculated from the coordinates (Px, Py) of pixel P in the tilt correction reference area 11 using the distortion correction coefficient e calculated in step S2 according to the following equation (3). The image correction unit 143 performs this calculation for the number of pixels in the output area 12 in FIG. 8, for example. Rx=e(Px-Cx)+Cx,Ry=e(Py-Cy)+Cy (3)

[0089] According to the above processing, distortion correction parameters are calculated in accordance with lens state data acquired from the interchangeable lens 200 (S1, S2). Furthermore, image stabilization operation by the BIS function is performed in accordance with image stabilization control (S3) based on the amount of shake correction in the roll direction as well as the horizontal and vertical directions of the image sensor 110. Then, the amount of tilt blur in the yaw direction and pitch direction is acquired (S4), and tilt correction parameters are calculated in accordance with the amount of tilt blur, etc. (S5). The image corrector 143 corrects the image data 10 from the image sensor 110 using the calculated correction parameters (S6). As a result, when a cropless correction mode is selected in the EIS function, distortion correction and tilt correction can be performed efficiently by transforming the image using the calculated correction parameters.

[0090] For example, in this embodiment, after image stabilization operation using the BIS function corrects the image sensor 110 in the horizontal and vertical directions according to the amount of blur in the yaw and pitch directions and corrects the amount of blur in the roll direction, tilt correction and the like are performed by the image corrector 143 (S6). As a result, in addition to distortion correction, the image corrector 143 performs only tilt correction as image stabilization using the cropless EIS function, making it possible to correct image distortion due to tilt blur while suppressing a reduction in the angle of view.

[0091] In the above step S6, an example has been described in which distortion correction and tilt correction are applied to the image data 10 at the same time. However, for example, tilt correction may be performed on the image data 10 after distortion correction has been applied. In this case, distortion correction may be performed before the process of calculating tilt correction parameters (S5). For example, in step S6, a calculation for distortion correction may be performed similarly to the process of step S21, which will be described later. Furthermore, when a cropless correction mode is selected in the digital camera 1, the distortion correction function may be forcibly enabled.

[0092] 2-4. Calculation of tilt correction parameters The calculation process of the tilt correction parameter in step S5 of FIG. 9 will be described with reference to FIGS.

[0093] 11 is a flowchart illustrating the calculation process (S5) of the tilt correction parameters in the digital camera 1. The process shown in this flowchart starts, for example, in a state where lens state data of the interchangeable lens 200 has been acquired in step S1 of FIG. 9, distortion correction parameters have been calculated in step S2, and the amount of tilt blur has been acquired in step S4.

[0094] First, the camera control unit 140 calculates a surplus area resulting from distortion correction in the image data 10 from the image sensor 110 based on the coordinates indicating the position of the image formation area 20 and the distortion correction parameters (S21). Fig. 12 is a diagram for explaining the calculation process (S21) of the surplus area in the calculation process (S5) of the tilt correction parameters. Fig. 12 illustrates an example of a surplus area 15 resulting from distortion correction in the image data 10. In the image data 10, the surplus area 15 is a region of the image area 13 enlarged by distortion correction that does not include the image formation area 20.

[0095] In this embodiment, the camera control unit 140 calculates the coordinates after distortion correction for each of eight pixels, namely, the vertices 41 and the midpoints 42 of each side of the rectangular image formation area 20 as shown in Fig. 12 (S21). In this way, the coordinates of areas that may become the surplus area 15 are calculated partially. This improves the calculation speed required to calculate the surplus area 15, and reduces the processing load on the camera control unit 140.

[0096] In step S21, for example, in a calculation similar to equation (3) for distortion correction in the image correction process (S6), coordinate conversion is performed so that the coordinates of each pixel to be calculated from the image formation area 20 are input as (Px, Py) and the coordinates in the surplus area 15 are output as (Rx, Ry). In this case, contrary to the example in Fig. 10(B), the distortion correction coefficient e changes so as to increase according to the distance d from the distortion correction center C to each pixel to be calculated, and in the correction of barrel aberration, e>1 in the peripheral parts of the image.

[0097] Next, the camera control unit 140 compares each coordinate calculated in step S21 with the corresponding coordinate in the image formation area 20 to determine whether or not a surplus area 15 exists outside the image formation area 20 in the image data 10 (S22). For example, if all of the calculated coordinates are outside the image formation area 20, the camera control unit 140 determines that a surplus area 15 exists (YES in S22).

[0098] If there is a surplus area 15 (YES in S22), the camera control unit 140 calculates a tilt correction parameter (S23). The camera control unit 140 calculates a rotation matrix R and a translation amount t(t x ,t y ,t z ) is calculated according to the following equations (4) and (5). In equations (4) and (5), pitch and yaw indicate the amount of tilt (angle) in the pitch direction and yaw direction, respectively. c ,y c For example, in the image data 10, the horizontal and vertical directions X and Y indicate the positions of the optical axis center corresponding to the intersection between the optical axis of the optical system and the imaging surface of the image sensor 110, with the center position Pc of the entire image as a reference. c ,y c Get, for example, x c ,y cis calculated as a value obtained by converting the output from the integrator 408 into the amount of movement on the imaging surface, or based on the output from the position sensor 182.

number

number

[0099] After calculating the tilt correction parameters (S23), the camera control unit 140 ends the processing of this flowchart and proceeds to step S6 in FIG.

[0100] Moreover, if there is no surplus area 15 (NO in S22), the camera control unit 140 does not calculate the tilt correction parameters (S23) and ends the processing of this flowchart.

[0101] According to the above processing, a surplus area is calculated based on the image formation area 20 in the image data 10 and the distortion correction parameters (S21), and if there is a surplus area 15 resulting from the distortion correction (YES in S22), a tilt correction parameter is calculated (S23). On the other hand, if there is no surplus area 15 (NO in S22), no tilt correction parameter is calculated. As a result, if the distortion correction results in a surplus area 15 outside the image formation area 20, tilt correction can be performed in addition to distortion correction in the image correction processing (S6 in FIG. 9) using the calculated tilt correction parameter.

[0102] If there is a surplus area 15 (YES in S22), tilt correction can be performed by the cropless EIS function using the image area 13 enlarged by distortion correction from the image formation area 20 by the amount of the surplus area 15. This makes it possible to achieve tilt correction while suppressing a reduction in the angle of view.

[0103] In step S23, when the perspective correction parameters are calculated, an upper limit may be set on the amount of blur correction by perspective correction, for example. For example, the upper limit of the blur correction amount is set according to the size of the surplus area 15, and if the blur correction amount that offsets all of the perspective blur amount acquired in step S4 of FIG. 9 is larger than the upper limit, the amount may be limited to the upper limit. Also, in the surplus area calculation process (S21), the number of pixels whose coordinates after distortion correction are calculated from the image formation area 20 is not limited to the above-mentioned eight points. For example, by performing the calculation for more than eight pixels, the upper limit of the blur correction amount can be set with greater accuracy.

[0104] 2-5. Variations In the above-described first embodiment, an example has been described in which image stabilization is performed by the BIS function and tilt correction is performed by the image correction process (S6). Such image stabilization may be performed by cooperative operation of the BIS function and the OIS function. Such a modification of the first embodiment will be described with reference to FIG. 13.

[0105] 13 is a flowchart illustrating the operation of a digital camera 1 according to a modification of embodiment 1. In addition to the same operations (S1 to S2, S4 to S6) as those of digital camera 1 according to embodiment 1, digital camera 1 according to this modification executes a calculation process (S30) of blur correction parameters used in image stabilization operations using the BIS function and the OIS function. In this modification, digital camera 1 controls image stabilization operations using the BIS function and the OIS function (S3A) instead of the control of image stabilization operations using the BIS function in embodiment 1 (S3 in FIG. 9).

[0106] For example, the camera control unit 140 calculates blur correction parameters based on the lens state data of the interchangeable lens 200 acquired in step S1, information indicating the element driving range in the BIS function, and information indicating the lens driving range in the OIS function (S30). The blur correction parameters include, for example, a correction allocation for allocating the amount of blur correction between the BIS processing unit 183 and the OIS processing unit 223. The correction allocation is calculated for each of the horizontal and vertical directions and the roll direction of the image sensor 110. The blur correction parameters may include, for example, a frequency band corresponding to the allocation between the BIS processing unit 183 and the OIS processing unit 223 with respect to frequency components included in the angular velocity signal from the gyro sensor 184 or 224.

[0107] The camera control unit 140 uses the calculated blur correction parameters to control the image stabilization operation using the BIS function and the OIS function (S3A). For example, the camera control unit 140 sets a gain indicating the allocation of the BIS processing unit 183 in the calculated correction allocation to the BIS processing unit 183. The BIS processing unit 183 calculates the amount of blur correction by multiplying the blur detection signal from the integrator 408 by the gain. The camera control unit 140 may set a frequency band to be cut off in the HPF 406 of the BIS processing unit 183, etc. Furthermore, in this modified example, the camera control unit 140 further calculates x in equation (5) based on information acquired from the OIS processing unit 223 when calculating the tilt correction parameters (S5, S23). c ,y c may be calculated.

[0108] Furthermore, the camera control unit 140 transmits the calculated blur correction parameters to the interchangeable lens 200 via the body mount 150 and the lens mount 250. The camera control unit 140 may set the blur correction parameters in the OIS processing unit 223 via the lens control unit 240.

[0109] According to the above process, blur correction parameters are calculated (S3A, S30) so as to cooperatively control the BIS function and the OIS function, and image stabilization operations are performed by each function according to the blur correction parameters. This also makes it possible, for example, to perform image stabilization using the amount of blur correction in the translation direction and roll direction of the image sensor 110 other than tilt correction by the optical image stabilization function, and to perform tilt correction by the cropless EIS function in the image correction process (S6).

[0110] 3. Summary As described above, the digital camera 1 (imaging device) of this embodiment includes an imaging area where a subject image is formed via an optical system, an image sensor 110 (image sensor) that captures the subject image and generates image data, gyro sensors 184, 224 (detection units) that detect the amount of blur in the digital camera 1, and an image correction unit 143 (image processing unit) that performs image blur correction by adjusting a portion of the image data that outputs an image in accordance with the amount of blur detected by the detection unit. The image correction unit 143 performs distortion correction on the image area indicated by the image data 10 in accordance with distortion aberration of the optical system (S2, S6). The image correction unit 143 performs image blur correction using a cropless EIS function as an example of image blur correction in the distortion-corrected image area without using a correction area provided within the image formation area 20 (the range corresponding to the imaging area) to crop an image such as the image in the output area 12 (S5, S6) (see FIG. 8).

[0111] With the imaging device described above, image stabilization is performed using the cropless EIS function in the image area where distortion correction has been performed, without using the correction area provided within the image formation area 20. This allows correction of image distortion caused by camera shake to be performed while suppressing a reduction in the angle of view. For example, the output area 12 using the cropless EIS function can be output as an area in which the number of pixels is not substantially reduced from the image formation area 20, similar to when no cropping amount Eo is set in the setting of the reference area 21 in the image data 10 as shown in FIG.

[0112] In this embodiment, the digital camera 1 further includes an operation unit 130 that inputs a user operation to select a correction mode to be used for image blur correction from a plurality of correction modes (image blur correction modes). The plurality of correction modes include a "cropless" correction mode (first image blur correction mode) that performs image blur correction without using a correction area provided inside the image formation area 20 (the range corresponding to the imaging area), and "large crop" and "small crop" correction modes that perform image blur correction using the correction area, i.e., a correction mode with cropping (second image blur correction mode) (see FIG. 4). This allows the user to select the cropless correction mode and execute the cropless EIS function according to the user's selection.

[0113] In this embodiment, the cropless image blur correction mode performs image blur correction at a rate that is smaller than the rate at which the number of pixels decreases before and after image blur correction in the correction mode with cropping, and outputs an image (partial image) in the output area 12. As a result, the cropless correction mode can output an image with higher resolution than the correction mode with cropping.

[0114] In this embodiment, the image correction unit 143 adjusts the shape of the reference area 11 (area referenced for outputting the image of the output area 12) according to the detected amount of blur using the enlarged image area 13 (area enlarged beyond the range corresponding to the imaging area) that is enlarged beyond the range of the image formation area 20 in the image area where distortion correction has been performed, thereby performing image stabilization using the cropless EIS function without using a correction area that corresponds to a predetermined cropping amount Eo provided within the image formation area 20 (S5, S6) (see FIGS. 8 and 5). This prevents the output area 12 from being narrowed by the amount of the correction area, for example, and allows correction of image distortion due to camera shake to be performed while suppressing a reduction in the angle of view. Furthermore, in this embodiment, when a shake occurs that changes the attitude of the digital camera 1 (image capture device) relative to the subject corresponding to the subject image, such as tilt shake, the image correction unit 143 (image processing unit) performs image stabilization (image blur correction) using the cropless EIS function by adjusting the shape of the reference area 11 (area to be referenced) so as to offset the distortion of the (partial image of) the output area 12 caused by the shake in attitude (S5, S6).

[0115] In this embodiment, the digital camera 1 further includes a sensor driver 181 (element driver) that performs optical image blur correction by moving the image sensor 110 in a plane perpendicular to the optical axis of the optical system, and an OIS driver 221 (lens driver) that performs optical image blur correction by moving an OIS lens 220 (correction lens) included in the optical system in a plane perpendicular to the optical axis. In the optical image blur correction, the image sensor 110 or both the image sensor 110 and the OIS lens are moved to offset the detected amount of blur. The image corrector 143 performs image blur correction by adjusting the shape of the reference area 11 that is referenced to output the image in the output area 12 without moving the portion that outputs the image in the distortion-corrected image area to compensate for the amount of blur that has been offset by the optical image blur correction (S5, S6).

[0116] As described above, the image correction unit 143 does not use the EIS function to correct the amount of blur that has been offset by the optical image stabilization (BIS, OIS) function, which performs optical image blur correction. For example, the EIS function can perform only tilt correction for the amount of tilt blur, separate from the amount of blur correction performed by the optical image stabilization function. This allows correction of image distortion caused by camera shake, such as tilt blur, while suppressing a reduction in the angle of view.

[0117] In this embodiment, the distortion of the optical system is negative at least in the peripheral part of the imaging area, and the image correction unit 143 performs distortion correction by enlarging at least the area corresponding to the peripheral part in the image area indicated by the image data 10 to outside the range of the image formation area 20 corresponding to the imaging area (S2, S6). As a result, an enlarged image area 13 is obtained by the distortion correction, and the cropless EIS function can be executed using the enlarged image area 13.

[0118] In this embodiment, the digital camera 1 further includes a body mount 150 and a lens mount 250 (communication units) that communicate with the optical system, and a camera control unit 140 (control unit) that controls the communication unit and the image correction unit 143. The camera control unit 140 acquires distortion characteristic data 30 (information about distortion aberration) of the optical system from the optical system via the body mount 150 and the lens mount 250, and when a surplus area 15 outside the image formation area 20 corresponding to the image capture area is detected in the image area where distortion has been corrected based on the acquired information and information indicating the image capture area (YES in S22), the camera control unit 140 causes the image correction unit 143 to perform image blur correction without using the correction area provided inside the image formation area 20 (S23, S5, S6) (see FIG. 12 ). As a result, when a surplus area 15 is detected, since the image area 13 enlarged by the distortion correction has been obtained, correction of image distortion due to camera shake can be performed while suppressing a reduction in the angle of view.

[0119] (Embodiment 2) 14 and 15, a second embodiment of the present disclosure will be described below. In the first embodiment, an example has been described in which digital camera 1 performs image stabilization operations other than tilt correction using an optical image stabilization function such as a BIS function, and performs tilt correction using a cropless EIS function. In the second embodiment, a digital camera 1 will be described in which, when operating in such a cropless correction mode, the EIS function also performs image stabilization operations other than tilt correction.

[0120] Below, the description of the configuration and operation similar to those of the digital camera 1 according to the first embodiment will be omitted as appropriate, and the digital camera 1 according to this embodiment will be described.

[0121] 1. Overview In an image Im captured by the digital camera 1, trapezoidal distortion caused by various types of tilt shake, which is a change in posture that tilts the line of sight when viewing a subject from the digital camera 1, is likely to occur in the peripheral parts of the image, as described above, in wide-angle photography (see FIGS. 7B to 7D). Also, in wide-angle photography where the focal length of the interchangeable lens 200 is relatively short, the correction angle corresponding to the amount of movement of the image sensor 110 by the BIS function becomes large, which can also increase distortion in the peripheral parts of the captured image Im. On the other hand, the inventors of the present application have noticed that in telephoto photography where the focal length of the interchangeable lens 200 is relatively long, the influence of distortion in the captured image Im tends to be less likely to occur.

[0122] The inventors of the present application conducted extensive research from this perspective and have come up with the digital camera 1 of this embodiment. For example, when the focal length is relatively long as described above, the effect of distortion due to tilt shake is relatively small in the captured image Im, and it is expected that such distortion can be corrected even if part of the surplus area 15 (see FIG. 12) generated by distortion correction is used for tilt correction.

[0123] Therefore, digital camera 1 of this embodiment utilizes, for example, the area of ​​surplus area 15 that is not used for tilt correction for other image stabilization using the EIS function, such as translation and / or rotation of image area 22 as shown in Fig. 5. This makes it possible to efficiently use image area 13 after enlargement, such as surplus area 15 used for distortion correction, and to perform other image stabilization using the EIS function in addition to tilt correction, while suppressing a reduction in the angle of view due to image cropping.

[0124] The digital camera 1 of this embodiment performs image stabilization by changing the ratio between tilt correction in the EIS function and other image stabilization such as the translation / rotation correction described above, depending on the focal length of the interchangeable lens 200. In this embodiment, the digital camera 1 dynamically determines the image correction ratio by the EIS function depending on the focal length in the zoom state of the interchangeable lens 200. FIG. 14 is a diagram for explaining the operation of the digital camera 1 according to this embodiment.

[0125] FIG. 14 illustrates an example of ratio data D1 that the digital camera 1 refers to when determining the image correction ratio. In FIG. 14, the horizontal axis indicates the focal length of the interchangeable lens 200, and the vertical axis indicates the image correction ratio for the EIS function. For example, as shown in FIG. 14, the ratio data D1 indicates the image correction ratio for each focal length. In this embodiment, the ratio data D1 is generated so that as the focal length becomes longer within a predetermined range (for example, from f1 to f2 in FIG. 14), the image correction ratio R1 for tilt correction decreases and the image correction ratio R2 for translation / rotation correction increases. The operation of the digital camera 1 with respect to these image correction ratios R1 and R2 will be described below.

[0126] 2.Operation 15 is a flowchart illustrating the operation of the digital camera 1 according to embodiment 2. The digital camera 1 according to this embodiment performs processing (S10, S11-S12, S3B, S5A) related to translation / rotation correction in addition to tilt correction using the EIS function, in addition to or instead of the processing (S1-S6) similar to the operation (FIG. 9) of embodiment 1. The processing of this flowchart is started, for example, when the digital camera 1 is started.

[0127] The camera control unit 140 of the digital camera 1 generates ratio data D1 as shown in FIG. 14 based on, for example, distortion characteristic data 30 of the interchangeable lens 200 (see FIG. 6A) (S10). In step S10, the camera control unit 140 acquires distortion characteristic data 30 corresponding to the focal length from the interchangeable lens 200 via the body mount 150 and the lens mount 250. The camera control unit 140 generates ratio data D1 by setting image correction ratios R1 and R2 based on, for example, the distortion characteristics indicated by the distortion characteristic data 30 for each focal length. The image correction ratios R1 and R2 are set according to the focal length under normalization conditions such as a sum of "1."

[0128] For example, the image correction ratios R1 and R2 at each focal length are set from the perspective of the influence of image distortion due to tilt shake, as well as the amount of shake in the horizontal, vertical, and roll directions of the image sensor 110 that can be offset within the element driving range by the BIS function. An example will be described below in which the maximum and minimum values ​​of the image correction ratios R1 and R2 are set at focal lengths f1 and f2 in Fig. 14, assuming a situation in which camera shake is relatively large, such as when a user is shooting a video while walking. Based on the above assumptions, the image correction ratios R1 and R2 in this example are set so that the correction angle is 1.4 degrees or more in the translation directions, i.e., the horizontal and vertical directions, of the image sensor 110, and so that the correction angle is 1.0 degree or more in the roll direction.

[0129] For example, at a focal length f1 (e.g., 20 millimeters), when the correction angle calculated from the amount of movement of image sensor 110 within the element driving range is 3 degrees or greater, the lower limit of the correction angle in the above-described walking shooting situation can be ensured even with the BIS function alone. For this reason, in this example, for the focal length f1, the image correction ratio R1 for tilt correction is set to 100% as the maximum value, and the image correction ratio R2 for translation / rotation correction is set to 0% as the minimum value.

[0130] On the other hand, for example, at a focal length f2 (e.g., 50 mm), when the correction angle within the element driving range by the BIS function is about 1.4 degrees, it may be difficult to ensure the lower limit of the correction angle using only the BIS function. Also, as mentioned above, as the focal length increases, the effects of distortion due to tilt blur become less apparent in the peripheral parts of the captured image Im. In this example, for the focal length f2, the image correction ratio R1 for tilt correction is set to 0%, and the image correction ratio R2 for translation / rotation correction is set to 100%.

[0131] In this example, the image correction ratios R1 and R2 are set so as to change linearly between the respective set values ​​for the focal lengths f1 and f2 within the range from f1 to f2. In this example, the camera control unit 140 sets the image correction ratios R1 and R2 for each focal length as the ratio of the area available for tilt correction and translation / rotation correction in the surplus area 15 by the cropless EIS function. The camera control unit 140 generates ratio data D1 for the EIS function by setting the image correction ratios R1 and R2 as described above, for example, and stores the generated ratio data D1 in the RAM 141 or the like (S10). The image correction ratios R1 and R2 are not limited to the linear change example described above, and may be set so as to change nonlinearly (for example, in a curved line) according to the focal length.

[0132] After generating ratio data D1 of the EIS function (S10), the camera control unit 140 acquires lens state data from the interchangeable lens 200 and calculates distortion correction parameters (S2) in parallel with operations such as video shooting, similar to the operations in the first embodiment (FIG. 9). The lens state data includes a focal length according to the zoom state of the zoom lens 210 of the interchangeable lens 200.

[0133] In this embodiment, the camera control unit 140 determines image correction ratios R1 and R2 corresponding to the focal length in the ratio data D1 for tilt correction and translation / rotation correction using the EIS function based on the focal length of the acquired lens state data (S11).

[0134] Furthermore, the camera control unit 140 performs a process of determining the allocation of correction between the BIS processing unit 183 and the image correction unit 143, for example, for each of the horizontal, vertical, and roll directions of the image sensor 110 (S12). For example, in step S12, the camera control unit 140 acquires information such as the element driving range of the BIS function from the flash memory 142 or the like. The camera control unit 140 determines the allocation of correction based on the acquired information, lens state data, and an image correction ratio R2 for translation / rotation correction (S12). For example, depending on the image correction ratio R2, a cropping amount Eo by the EIS function may be set outside the range of the image formation area 20 as shown in FIG. 5, and the set cropping amount Eo may be used to determine the allocation of correction.

[0135] The correction allocation includes a BIS ratio indicating the allocation to the BIS processing unit 183 and an EIS ratio indicating the allocation to the image correction unit 143, as ratios for pre-allocating the shake correction amounts in the horizontal, vertical, and roll directions of the image sensor 110. The correction allocation is determined, for example, according to the upper limit values ​​of the shake correction amounts that can be allocated to the BIS function and the EIS function under normalization conditions such that the sum of the BIS ratio and the EIS ratio is "1." For example, the BIS ratio and the EIS ratio of the correction allocation may be determined as ratios similar to the respective upper limit values ​​(S12).

[0136] The camera control unit 140 controls the image stabilization operation using the BIS function according to the BIS ratio in the determined correction allocation (S3B). The camera control unit 140 sets, for example, a gain indicating the BIS ratio in the BIS processing unit 183. The BIS processing unit 183 calculates the amount of shake correction by multiplying the shake detection signal from the integrator 408 by the gain set for each cycle of this flowchart. The gains for the shake detection signals in the yaw direction, pitch direction, and roll direction may be the same or may be set separately.

[0137] Thereafter, the camera control unit 140 acquires the amount of tilt shake from the BIS processing unit 183, for example, as in the example of FIG. 9 (S4).

[0138] The camera control unit 140 calculates tilt correction parameters and parameters used for translation / rotation correction by the EIS function as image correction parameters based on the image correction ratios R1, R2, etc. determined in step S12, for example (S5A).

[0139] 9, the camera control unit 140 calculates tilt correction parameters limited to values ​​that allow image deformation within the range of image correction ratio R1 in the surplus area 15 from projective transformation parameters calculated based on the amount of tilt blur, etc. (S5A). The camera control unit 140 also calculates translation / rotation correction parameters limited to values ​​that allow movement of the image area 22 within the range of image correction ratio R2 in the surplus area 15 from gains indicating the EIS ratio of the correction allocation determined in step S12, for example (S5A). For example, the gain calculated according to the EIS ratio as the translation / rotation correction parameter is multiplied by the blur detection signal input from the integrator 408 of the BIS processing unit 183 in the image correction unit 143.

[0140] The camera control unit 140 executes calculation processing as the image correction unit 143 using the distortion correction parameters calculated in step S2 and the image correction parameters calculated in step S5A instead of the tilt correction parameters of the first embodiment (S6). In step S6 of the present embodiment, in addition to processing to transform the image as distortion correction and tilt correction, processing to move the image area 22 by at least one of translational movement and rotational movement as translation / rotation correction is performed. After executing such image correction processing (S6), the camera control unit 140 ends the processing of this flowchart.

[0141] According to the above processing, ratio data D1 including image correction ratios R1, R2 for each focal length in the EIS function is generated from distortion characteristic data 30 corresponding to the focal length of, for example, interchangeable lens 200 (S10). An image correction ratio R1 for tilt correction in the EIS function and an image correction ratio R2 for translation / rotation correction are determined based on this ratio data D1 and the focal length in the lens state data of interchangeable lens 200 (S11). For example, for the horizontal, vertical, and roll directions of image sensor 110, the allocation of correction between BIS processing unit 183 and image correction unit 143 is determined according to image correction ratio R2, etc. (S12).

[0142] Furthermore, image stabilization control is performed using the BIS function according to the BIS ratio in the determined correction allocation (S3B), and the amount of tilt blur is obtained (S4). Then, tilt correction parameters and translation / rotation correction parameters are calculated based on the amount of tilt blur and the image correction ratios R1 and R2 (S5A), and image correction processing is performed using these image correction parameters and distortion correction parameters (S6). As a result, for example, tilt correction and translation / rotation correction using the EIS function can be performed in the surplus area 15 generated by distortion correction according to the image correction ratios R1 and R2, respectively.

[0143] 14, the amount of tilt blur can be offset even when tilt correction using the surplus area 15 generated by distortion correction is performed within the range of image correction ratio r1 in the EIS function using ratio data D1 corresponding to the distortion characteristics, etc., of the interchangeable lens 200. On the other hand, at the focal length f12, even if surplus area 15 not used for tilt correction remains, the surplus area 15 generated by distortion correction, etc. can be efficiently used by performing translation / rotation correction within the range of image correction ratio r2 in the EIS function. In this way, tilt correction, etc. can be performed efficiently while suppressing a reduction in the angle of view using, for example, a cropless EIS function.

[0144] In the above, an example has been described in which the ratio data D1 of the EIS function is generated to include image correction ratios R1 and R2 (S10). The ratio data D1 may include only one of the image correction ratios R1 and R2, or the other may be calculated appropriately based on the relationship of changes in the two ratios R1 and R2 according to the focal length, for example. Furthermore, the ratio data D1 may be generated in advance and stored in flash memory 142 or the like before execution of this flowchart, for example.

[0145] The above describes an example in which image stabilization control (S3B) and image correction processing (S6) are performed using the BIS function at the BIS ratio and EIS ratio of the correction allocation determined in step S12. The corrections performed by the BIS processing unit 183 and the image correction unit 143 are not limited to the above example. For example, if an amount of correction remains even after translation / rotation correction using the EIS function in the image correction processing (S6), image stabilization may be performed using the BIS function for the amount of correction remaining in accordance with the element drive range, etc.

[0146] In the above, an example has been described in which tilt correction and translation / rotation correction are performed using the image correction ratios R1 and R2 in the EIS function. In the EIS function, as an image stabilization operation other than tilt correction according to such an image correction ratio, an image stabilization operation using image processing other than translation / rotation correction may be performed.

[0147] In the above, an example has been described in which the processing of this flowchart is started when the digital camera 1 is started up. This processing may also be started, for example, when the interchangeable lens 200 is attached to the camera body 100.

[0148] 3. Summary As described above, the digital camera 1 (an example of an imaging device) of this embodiment includes an imaging area where a subject image is formed via an optical system, an image sensor 110 (an example of an image sensor) that captures the subject image and generates image data, gyro sensors 184 and 224 (each an example of a detector) that detect the amount of blur in the digital camera 1, an image correction unit 143 (an example of an image processing unit) that performs image blur correction by adjusting a portion of the image data that outputs an image in accordance with the amount of blur detected by the detector, and a camera control unit 140 (an example of a control unit) that controls the image blur correction performed by the image correction unit 143. The image correction unit 143 performs distortion correction on the image area indicated by the image data in accordance with the distortion aberration of the optical system. The camera control unit 140 changes the ratio between tilt correction and translation / rotation correction (an example of first and second image blur corrections, respectively) performed by the image correction unit 143 in the EIS function in accordance with the focal length of the optical system (S11). The tilt correction corrects image distortion in the distortion-corrected image area (see FIG. 8). The translation / rotation correction moves the part of the image area that outputs the image (see FIG. 5).

[0149] According to the digital camera 1 described above, the ratio between tilt correction and translation / rotation correction performed by the EIS function in a distortion-corrected image area, such as the surplus area 15, is changed depending on, for example, the focal length of the optical system in the interchangeable lens 200. This allows image distortion caused by camera shake to be efficiently corrected while suppressing a reduction in the angle of view. For example, in cropless EIS mode, the ratio between tilt correction and translation / rotation correction performed by the EIS function is changed depending on the focal length at which the effect of image distortion due to tilt shake changes, allowing efficient use of the surplus area 15 created by distortion correction.

[0150] In this embodiment, the optical system includes a zoom lens 210 of the interchangeable lens 200 (see FIG. 2). When the focal length is changed by the zoom lens 210, the camera control unit 140 changes the image correction ratios R1 and R2 (each an example of the ratio between tilt correction and translation / rotation correction) (S11, see FIG. 14). This allows the image correction ratios R1 and R2 to be changed according to, for example, the zoom state of the zoom lens 210.

[0151] In this embodiment, the camera control unit 140 changes the image correction ratios R1 and R2 so that the image correction ratio R2 (an example of the proportion of the second image blur correction in the ratio between the first image blur correction and the second image blur correction) of the image correction ratios R1 and R2 increases as the focal length increases within a predetermined range (S11, see FIG. 14). As a result, for example, as the focal length increases, the upper limit of the correction angle by the BIS function decreases, while the influence of image distortion due to tilt blur tends to become less apparent, and therefore tilt correction and translation / rotation correction by the EIS function can be efficiently performed in the surplus area 15.

[0152] In this embodiment, the camera control unit 140 causes the image correction unit 143 to perform tilt correction (S4 to S6) so as to correct image distortion caused by a change in posture that tilts the line of sight when viewing a subject from the digital camera 1 based on the detected amount of shake. This makes it possible to correct image distortion caused by tilt shake during image capture through tilt correction.

[0153] In this embodiment, the image correction unit 143 performs tilt correction using the cropless EIS function in the distortion-corrected image area without using a correction area provided for cutting out an image of a portion such as the image area 22 inside the image formation area 20 (the range corresponding to the imaging area) (S5A, S6) (see FIGS. 5 and 8). This allows correction of image distortion due to tilt blur to be performed while suppressing a reduction in the angle of view.

[0154] In this embodiment, the digital camera 1 further includes a sensor driver 181 (element driver) that performs optical image blur correction by moving the image sensor 110 in a plane perpendicular to the optical axis of the optical system, and an OIS driver 221 (lens driver) that performs optical image blur correction by moving an OIS lens 220 (correction lens) included in the optical system in a plane perpendicular to the optical axis. In this embodiment, the image sensor 110 is moved in the optical image blur correction so as to offset the detected amount of blur (S3B). The image corrector 143 performs translation / rotation correction using the EIS function by moving the image area 22 (an example of a portion that outputs an image) in the distortion-corrected image area according to the detected amount of blur (S5A, S6). As a result, similar to the translation / rotation correction using the EIS function, correction of the image sensor 110 in the translation direction and roll direction can also be performed using the BIS function of the sensor driver 181, for example. Furthermore, the EIS function may perform correction in only one of the translational direction and the rotational direction, and the other correction may be performed by the BIS function, for example.

[0155] (Other embodiments) As described above, embodiment 1 has been described as an example of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to this, and can be applied to embodiments in which appropriate modifications, substitutions, additions, omissions, etc. are made. Furthermore, it is also possible to combine the components described in embodiment 1 above to create new embodiments. Therefore, other embodiments will be described below as examples.

[0156] In the above-described first embodiment, an example has been described in which the digital camera 1 displays the cropless correction mode as selectable on the menu screen shown in Fig. 4, for example, when selecting various correction modes using the EIS function, regardless of the interchangeable lens 200. In this embodiment, the cropless correction mode in the digital camera 1 may be switched between enabled and disabled depending on the attached interchangeable lens 200. For example, when the camera control unit 140 acquires information indicating the type and / or lens characteristics of the interchangeable lens 200 from the interchangeable lens 200 and determines that the enlarged image area 13 can be obtained by distortion correction, the cropless correction mode may be displayed as selectable.

[0157] In the above embodiment, an example has been described in which, when selecting various correction modes using the EIS function, whether or not the cropless correction mode can be selected is switched based on information acquired from the interchangeable lens 200. In this embodiment, for example, if information indicating lens characteristics such as distortion cannot be acquired from the interchangeable lens 200, even if the cropless correction mode is selected, the EIS function may not be executed in that correction mode, and the cropless correction mode may be made unselectable.

[0158] Furthermore, without being limited to the above example, for example, the enable / disable status of the cropless EIS function or the selectability of the cropless correction mode may be managed for each interchangeable lens 200 using information indicating whether the function is usable. Such information may be stored in the flash memory 142 of the camera body 100, or may be acquired from the interchangeable lens 200. For example, the enable / disable status may be set according to the peripheral illumination limit of the interchangeable lens 200. Furthermore, in the digital camera 1, when the cropless EIS function is enabled, a function for correcting peripheral illumination may also be enabled. Furthermore, for example, when the amount of reduction in peripheral illumination due to peripheral shading in the optical system exceeds a predetermined allowable value, the element drive range of the BIS function may be limited.

[0159] In the above-described second embodiment, an example was described in which the digital camera 1 changes the image correction ratios R1, R2 of tilt correction and translation / rotation correction in the EIS function according to the focal length of the interchangeable lens 200. In this embodiment, for example, correction by the EIS function may be performed on a portion of the amount of blur that has not been canceled out by the BIS function. The camera control unit 140 may acquire the remaining correction amount by the BIS function among the amounts of blur in the yaw direction and pitch direction from the BIS processing unit 183 and correct the remaining correction amount using, for example, the EIS function with crop. In this embodiment, for example, the cropless EIS function may be executed when the reduction rate of the number of pixels of an image output for recording and / or display before and after tilt correction is equal to or less than one-tenth of the reduction rate in the "small crop" correction mode.

[0160] In the above-described second embodiment, an example has been described in which translation / rotation correction using the EIS function is performed in a cropless correction mode. However, translation / rotation correction using the EIS function may also be performed in a cropped correction mode, and may be performed, for example, not only on the surplus area 15 resulting from distortion correction but also on the enlarged image area 13.

[0161] In this embodiment, for example, when the cropless and cropped EIS functions are used together, the image area 22 output by the cropped EIS function does not need to be moved to account for the amount of blur that is offset by an optical image stabilization function such as a BIS function. This also makes it possible to correct image distortion caused by camera shake while suppressing a reduction in the angle of view that would be caused by the cropped EIS function. Furthermore, even in the cropped EIS function, the image area 13 enlarged by distortion correction can be used to cut out the image area 22.

[0162] In the above embodiment, an example was described in which tilt correction was performed using a cropless EIS function, and blur amounts in the yaw, pitch, and roll directions were corrected using a cropped EIS function. In this embodiment, image distortion due to, for example, a rolling shutter phenomenon may also be corrected using the EIS function. For example, the correction may be performed using multiple projective transformation matrices according to the characteristics of such distortion as correction parameters.

[0163] In the above-described second embodiment, an example has been described in which image stabilization control using the BIS function (S3B) and image correction processing using the EIS function (S6) are performed based on the correction allocation determined between the BIS processing unit 183 and the image correction unit 143 (S11). In this embodiment, for example, image stabilization may be further performed by the OIS processing unit 223 for the amount of blur in the translational direction of the image sensor 110. For example, the BIS ratio in step S11 of FIG. 15 may be further distributed based on the correction allocation between the BIS processing unit 183 and the OIS processing unit 223, similar to the blur correction parameter (S30 of FIG. 13) in the modified example of the first embodiment. In this embodiment, in optical image blur correction, at least one of the image sensor 110 and the OIS lens 220 is moved so as to cancel out the detected amount of blur.

[0164] In the above-described second embodiment, an example was described in which the interchangeable lens 200 included a zoom lens 210. In this embodiment, even for an interchangeable lens such as a prime lens that does not include a zoom lens, tilt correction and translation / rotation correction may be performed in the EIS function using image correction ratios R1 and R2 according to the focal length of the interchangeable lens, as in the second embodiment. In this embodiment, for example, only the image correction ratios R1 and R2 for the focal length of the interchangeable lens may be calculated, and the ratio data D1 for the EIS function may not be generated.

[0165] In the above-described embodiments, examples have been described in which the digital camera 1 performs image stabilization separate from tilt compensation using the cropless EIS function, using only the BIS function or both the BIS function and the OIS function. In this embodiment, for example, image stabilization separate from tilt compensation may be performed using only the OIS function. In this manner, the digital camera 1 of this embodiment performs image stabilization using an amount of shake compensation separate from the amount of shake compensation used in tilt compensation, using at least one of the OIS function and the BIS function. The digital camera 1 of this embodiment is not limited to the example shown in FIG. 1 , and may have only either the BIS function or the OIS function.

[0166] That is, in this embodiment, digital camera 1 may include at least one of a sensor driver 181 (element driver) that performs optical image blur correction by moving image sensor 110 in a plane perpendicular to the optical axis of the optical system, and an OIS driver 221 (lens driver) that performs optical image blur correction by moving OIS lens 220 (correction lens) included in the optical system in a plane perpendicular to the optical axis. In this embodiment, at least one of image sensor 110 and the OIS lens may be moved in the optical image blur correction so as to cancel out the detected amount of blur.

[0167] In the above embodiments, an example has been described in which the center position Pc of the entire image in the image data 10 is used as the center of distortion correction in calculating the distortion correction parameters (S2). In this embodiment, a position obtained by shifting the center position Pc in the image data 10 in accordance with, for example, the amount of blur correction performed by the optical image stabilization function may be used as the center of distortion correction. As a result, even if, for example, the optical image stabilization function causes the center position Pc to deviate from the intersection point between the optical axis and the imaging surface of the image sensor 110, it is possible to obtain the enlarged image area 13 by performing distortion correction in accordance with the lens characteristics of the interchangeable lens 200.

[0168] In the above embodiments, examples have been described in which the digital camera 1 executes the cropless EIS function when shooting moving images in accordance with the selected correction mode of the EIS function. The digital camera of this embodiment may also execute the cropless EIS function when shooting still images.

[0169] In the above embodiments, a digital camera with an interchangeable lens has been described as an example of an imaging device, but the imaging device of the present embodiment may also be a digital camera that is not particularly an interchangeable lens type. Furthermore, the concept of the present disclosure is applicable not only to digital cameras but also to movie cameras, and to electronic devices with various imaging functions such as camera-equipped mobile phones, smartphones, and PCs.

[0170] As described above, the embodiments have been described as examples of the technology in the present disclosure, and for that purpose, the accompanying drawings and detailed description have been provided.

[0171] Therefore, the components shown in the accompanying drawings and detailed description may include not only essential components for solving the problem, but also components that are not essential for solving the problem in order to illustrate the above technology. Therefore, the fact that these non-essential components are shown in the accompanying drawings or detailed description should not be interpreted as immediately indicating that these non-essential components are essential.

[0172] Furthermore, since the above-described embodiments are intended to illustrate the technology of the present disclosure, various modifications, substitutions, additions, omissions, etc. may be made within the scope of the claims or their equivalents.

[0173] (Summary of aspects) Various aspects of the present disclosure are listed below.

[0174] A first aspect of the present disclosure is an imaging device including an imaging element having an imaging area where a subject image is formed via an optical system and capturing the subject image to generate image data, a detection unit that detects the amount of blurring of the imaging device, and an image processing unit that performs image blur correction by adjusting a portion of the image data that outputs an image in accordance with the amount of blurring detected by the detection unit. The image processing unit performs distortion correction on the image area indicated by the image data in accordance with distortion aberration of the optical system. The image processing unit performs image blur correction on the distortion-corrected image area without using a correction area that is provided to cut out a partial image within a range corresponding to the imaging area.

[0175] In a second aspect, the imaging device of the first aspect further includes an operation unit that inputs a user operation to select an image blur compensation mode to be used for image blur compensation from a plurality of image blur compensation modes, including a first image blur compensation mode that performs image blur compensation without using a correction area provided within a range corresponding to the imaging area, and a second image blur compensation mode that performs image blur compensation using the correction area.

[0176] In a third aspect, in the imaging device of the second aspect, the first image blur correction mode performs image blur correction at a rate that is smaller than the rate at which the number of pixels decreases before and after image blur correction in the second image blur correction mode, and outputs an image of the portion.

[0177] In a fourth aspect, in an imaging device according to any one of the first to third aspects, the image processing unit performs image blur correction without using a correction area provided within the range corresponding to the imaging area by adjusting the shape of the area to be referenced for outputting a partial image in accordance with the detected amount of blur using an area in the image area where distortion correction has been performed that is expanded beyond the range corresponding to the imaging area.

[0178] In a fifth aspect, the imaging device of any of the first to fourth aspects further includes at least one of an element driver that performs optical image blur correction by moving the image sensor within a plane perpendicular to the optical axis of the optical system, and a lens driver that performs optical image blur correction by moving a correction lens included in the optical system within a plane perpendicular to the optical axis. In the optical image blur correction, at least one of the image sensor and the correction lens is moved so as to cancel out the detected amount of blur. In the image area where distortion correction has been performed, the image processor does not move the portion that outputs the image to compensate for the amount of blur that has been canceled out by the optical image blur correction, but instead adjusts the shape of the area that is referenced to output the partial image.

[0179] In a sixth aspect, in an imaging device according to any one of the first to fifth aspects, the distortion aberration of the optical system is negative at least in the peripheral part of the imaging area, and the image processing unit performs distortion correction by expanding the area corresponding to at least the peripheral part in the image area indicated by the image data to outside the range corresponding to the imaging area.

[0180] In a seventh aspect, the imaging device of any of the first to sixth aspects further includes a communication unit that communicates with the optical system, and a control unit that controls the communication unit and the image processing unit. The control unit acquires information about distortion of the optical system from the optical system via the communication unit, and, when an area outside the range corresponding to the imaging area is detected in the image area where distortion correction has been performed, causes the image processing unit to perform image blur correction without using the correction area provided within the range corresponding to the imaging area, based on the acquired information and information indicating the imaging area.

[0181] An eleventh aspect of the present disclosure is an imaging device comprising: an imaging element having an imaging area where a subject image is formed via an optical system and capturing the subject image to generate image data; a detection unit that detects the amount of blur of the imaging device; an image processing unit that performs image blur correction by adjusting a portion of the image data that outputs an image in accordance with the amount of blur detected by the detection unit; and a control unit that controls the image blur correction performed in the image processing unit. The image processing unit performs distortion correction in accordance with distortion aberration of the optical system on the image area indicated by the image data. The control unit changes the ratio between first and second image blur corrections performed by the image processing unit in the distortion-corrected image area in accordance with the focal length of the optical system; the first image blur correction corrects image distortion in the distortion-corrected image area, and the second image blur correction moves the portion that outputs the image in the image area.

[0182] The movement in the second image blur correction includes, for example, both translational movement and rotational movement, but may be either translational movement or rotational movement.

[0183] In a twelfth aspect, in the imaging device of the eleventh aspect, the optical system includes a zoom lens, and the control unit changes the ratio between the first image blur correction and the second image blur correction when the focal length is changed by the zoom lens.

[0184] In a thirteenth aspect, in the imaging device of the eleventh or twelfth aspect, the control unit changes the ratio between the first image blur correction and the second image blur correction so that the proportion of the second image blur correction increases as the focal length increases within a predetermined range.

[0185] In a fourteenth aspect, in an imaging device according to any one of the eleventh to thirteenth aspects, the control unit causes the image processing unit to perform a first image blur correction so as to correct image distortion caused by a change in posture that tilts the line of sight when viewing the subject from the imaging device at the detected amount of blur.

[0186] In a 15th aspect, in an imaging device according to any one of the 11th to 14th aspects, the image processing unit performs the first image blur correction in the distortion-corrected image area without using a correction area that is provided to cut out a partial image within a range corresponding to the imaging area.

[0187] In a 16th aspect, in the imaging device of the 15th aspect, the image processing unit performs first image blur correction without using a correction area provided within the range corresponding to the imaging area by using an area in the distortion-corrected image area that is expanded outside the range corresponding to the imaging area and adjusting the shape of the area to be referenced to output the partial image in accordance with the detected amount of blur.

[0188] In a seventeenth aspect, the imaging device of any of the eleventh to sixteenth aspects further includes at least one of an element driver that performs optical image blur correction by moving the imaging element in a plane perpendicular to the optical axis of the optical system, and a lens driver that performs optical image blur correction by moving a correction lens included in the optical system in a plane perpendicular to the optical axis. In the optical image blur correction, at least one of the imaging element and the correction lens is moved so as to cancel out the detected amount of blur. The image processor performs second image blur correction by moving a portion that outputs an image in the distortion-corrected image area according to the detected amount of blur.

[0189] In an 18th aspect, in an imaging device according to any one of the 11th to 17th aspects, the distortion aberration of the optical system is negative at least in the peripheral part of the imaging area, and the image processing unit performs distortion correction by expanding the area corresponding to at least the peripheral part in the image area indicated by the image data to outside the range corresponding to the imaging area. [Industrial Applicability]

[0190] The concept of the present disclosure can be applied to electronic devices (imaging devices such as digital cameras and camcorders, mobile phones, smartphones, etc.) that have an imaging function and an image stabilization function. [Explanation of symbols]

[0191] 1. Digital camera 100 camera body 110 Image Sensor 140 Camera control unit 143 Image Correction Unit 181 Sensor drive unit 183 BIS Processing Department 184 Gyro Sensor 200 interchangeable lenses 220 OIS lens 221 OIS drive unit 223 OIS processing section 224 Gyro Sensor

Claims

1. An imaging device, an image sensor having an imaging area where a subject image is formed via an optical system, and capturing the subject image to generate image data; a detection unit that detects the amount of blurring of the imaging device; an image processing unit that performs image blur correction by adjusting a portion of the image data that is to be output as an image in accordance with the amount of blur detected by the detection unit; a control unit that controls the image blur correction performed in the image processing unit, the image processing unit performs distortion correction according to distortion aberration of the optical system on an image area indicated by the image data; the control unit changes a ratio between the first and second image blur corrections performed by the image processing unit in the distortion-corrected image area according to a focal length of the optical system, the first image blur correction corrects distortion of the image in the distortion-corrected image area; The second image blur correction is performed by moving a portion of the image area where the image is output. Imaging device.

2. the optical system includes a zoom lens; The control unit changes a ratio between the first image blur correction and the second image blur correction when the focal length is changed by the zoom lens. The imaging device according to claim 1 .

3. The control unit changes the ratio between the first image blur correction and the second image blur correction so that a proportion of the second image blur correction increases as the focal length increases within a predetermined range. The imaging device according to claim 1 .

4. The control unit causes the image processing unit to perform the first image blur correction so as to correct distortion of the image caused by a change in posture in which a line of sight when viewing the subject from the imaging device is tilted in accordance with the detected amount of blur. The imaging device according to claim 1 .

5. The image processing unit performs the first image blur correction in the distortion-corrected image area without using a correction area provided for cutting out an image of the portion within a range corresponding to the imaging area. The imaging device according to claim 1 .

6. The image processing unit performs the first image blur correction without using the correction area provided inside the range corresponding to the imaging area by adjusting the shape of the area to be referenced for outputting the image of the portion in accordance with the detected amount of blur, using an area in the distortion-corrected image area that is expanded to outside the range corresponding to the imaging area. The imaging device according to claim 5 .

7. the optical system further includes at least one of an element driving unit that performs optical image blur correction by moving the image sensor within a plane perpendicular to the optical axis of the optical system, and a lens driving unit that performs optical image blur correction by moving a correction lens included in the optical system within a plane perpendicular to the optical axis, In the optical image blur correction, at least one of the image sensor and the correction lens is moved so as to cancel out the detected amount of blur; The image processing unit performs the second image blur correction so as to move a portion of the image area where the image is output in accordance with the detected amount of blur. The imaging device according to claim 1 .

8. the distortion of the optical system is negative at least in the peripheral portion of the imaging area; The image processing unit performs the distortion correction by expanding at least a region corresponding to the peripheral portion in the image region indicated by the image data to a range outside the range corresponding to the imaging region. The imaging device according to claim 1 .

Images