Imaging device

Abstract

The present invention provides an imaging device capable of appropriately correcting image blur when using an optical system that compresses the subject image in one direction more than in other directions. [Solution] The imaging device comprises an image sensor, an optical correction unit, an electronic correction unit, and a control unit that controls the optical correction unit and the electronic correction unit. The image sensor captures a subject image formed through the optical system and generates image data. The optical correction unit corrects image blur by moving the image sensor in a plane perpendicular to the optical axis of the optical system. The electronic correction unit corrects image blur by image processing of the image data. The optical system compresses the subject image more in a first direction perpendicular to the optical axis than in a second direction. The optical correction unit corrects rotational blur that occurs in the rotational direction around the optical axis. The electronic correction unit corrects shear strain caused by the compression of the subject image by the optical system after the rotational blur has been corrected by the optical correction unit.

Description

[Technical Field]

[0001] This disclosure relates to an imaging device having an image stabilization function. [Background technology]

[0002] Patent Document 1 discloses an image processing device that corrects image blur around the optical axis for images taken using a special lens that forms an optical image compressed in a certain direction. This image processing device has an image deformation means that applies geometric deformation processing, including correction processing and shear processing, to image data, which is caused by rotational deformation of the subject image due to device blur around the optical axis. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-150737 [Overview of the project] [Problems that the invention aims to solve]

[0004] This disclosure provides an imaging device that can appropriately correct image blur when using an optical system that compresses the subject image in one direction more than in other directions. [Means for solving the problem]

[0005] An imaging device according to one aspect of this disclosure comprises an image sensor, an optical correction unit, an electronic correction unit, and a control unit that controls the optical correction unit and the electronic correction unit. The image sensor captures an image of a subject formed through an optical system and generates image data. The optical correction unit corrects image blur by moving the image sensor in a plane perpendicular to the optical axis of the optical system. The electronic correction unit corrects image blur by image processing of the image data. The optical system compresses the subject image more in a first direction perpendicular to the optical axis than in a second direction. The optical correction unit corrects rotational blur that occurs in the rotational direction around the optical axis. The electronic correction unit corrects shear strain caused by the compression of the subject image by the optical system after the rotational blur has been corrected by the optical correction unit. [Effects of the Invention]

[0006] According to the imaging device described herein, when using an optical system that compresses the subject image in one direction more than in other directions, image blur can be appropriately corrected. [Brief explanation of the drawing]

[0007] [Figure 1] Perspective view of a digital camera according to Embodiment 1 of this disclosure [Figure 2] Block diagram showing the configuration of the digital camera according to Embodiment 1 [Figure 3] Block diagram showing the configuration of the BIS processing unit and EIS processing unit in the digital camera according to Embodiment 1. [Figure 4] Block diagram showing the configuration of the OIS processing unit in the digital camera according to Embodiment 1 [Figure 5] A schematic diagram illustrating the challenges of correcting rotational shake in a digital camera equipped with an anamorphic lens. [Figure 6] A schematic diagram illustrating the operation for correcting rotational shake in a digital camera according to Embodiment 1. [Figure 7] Flowchart illustrating the optical correction operation by a digital camera in Embodiment 1 [Figure 8]Flowchart exemplifying the electronic correction operation by the digital camera in Embodiment 1 [Figure 9] Diagram for explaining the optical correction processing of rotational shake in Embodiment 1 [Figure 10A] Graph showing an example of the temporal change of rotational shake in the roll direction when a photographer takes a still picture using a conventional digital camera [Figure 10B] Graph showing an example of the temporal change of rotational shake in the roll direction during walking shooting with a conventional digital camera [Figure 10C] Graph showing an example of the temporal change of rotational shake in the roll direction during running shooting with a conventional digital camera [Figure 11] Schematic diagram for explaining the manifestation of blur in a conventional digital camera [Figure 12] Flowchart exemplifying the optical correction processing by the digital camera in Embodiment 2 [Figure 13] Graph showing an example of the relationship between the exposure time and the EIS ratio in the digital camera of Embodiment 2 [Figure 14] Flowchart exemplifying the operation of the digital camera according to the first modification [Figure 15] Diagram for explaining the optical correction processing of rotational shake in the third modification [Figure 16] Flowchart for explaining the distribution processing of the shake correction amount in the seventh modification

Mode for Carrying Out the Invention

[0008] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings as appropriate. However, a more detailed description than necessary may be omitted. The accompanying drawings and the following description are provided for those skilled in the art to fully understand the embodiments of the present disclosure, and are not intended to limit the subject matter described in the claims.

[0009] (Embodiment 1) 1. Configuration Figure 1 is a perspective view of a digital camera 1 according to Embodiment 1 of this disclosure. Figure 2 is a block diagram showing the configuration of the digital camera 1 according to Embodiment 1. The digital camera 1 is an example of an imaging device consisting of a camera body 100 and a detachable interchangeable lens 200.

[0010] Furthermore, in the following explanation, the function that corrects image blur by moving the corrective lens within the interchangeable lens 200 is called the "OIS (Optical Image Stabilizer) function." The function that corrects image blur by moving the image sensor within the camera body 100 is called the "BIS (Body Image Stabilizer) function." Image blur correction by the OIS function and the BIS function are collectively called "optical correction." In addition, correcting image blur by applying image processing to the image data from the image sensor is called "electronic correction," and the function that performs electronic correction is called the "EIS (Electronic Image Stabilizer) function."

[0011] Furthermore, in the following explanation, the direction around the X-axis corresponding to the horizontal direction of the digital camera 1 (i.e., the tilt direction) will be referred to as the pitch direction, and the direction around the Y-axis corresponding to the vertical direction (i.e., the pan direction) will be referred to as the yaw direction (see Figure 1). Also, the direction in which the imaging surface of the image sensor in the digital camera 1 rotates in a plane perpendicular to the optical axis (the direction around the Z-axis) will be referred to as the roll direction (see Figure 1).

[0012] 1-1. Camera body The camera body 100 includes an image sensor 110, an LCD monitor 120, an operating unit 130, a camera control unit 140, RAM 141, flash memory 142, a body mount 150, a card slot 170, and a shutter 180. The camera body 100 also includes, for example, an EIS processing unit 143 that implements EIS functionality.

[0013] The camera control unit 140 controls the operation of the entire digital camera 1 by controlling components such as the image sensor 110 in response to instructions from the release button. The camera control unit 140 transmits a vertical sync signal to the timing generator (TG) 112. In parallel with this, the camera control unit 140 generates an exposure sync signal. The camera control unit 140 periodically transmits the generated exposure sync signal to the lens control unit 240 via the body mount 150 and lens mount 250. The camera control unit 140 uses the RAM 141 as work memory during control operations and image processing operations.

[0014] The image sensor 110 is an example of an image sensor that captures an image of a subject incident through the interchangeable lens 200 and generates image data. The image sensor 110 is, for example, a CCD, a CMOS image sensor, or an NMOS image sensor. The generated image data is digitized by an AD converter (ADC) 111. The digitized image data is subjected to predetermined image processing by the camera control unit 140. Predetermined image processing includes, for example, gamma correction processing, white balance correction processing, scratch correction processing, YC conversion processing, electronic zoom processing, and / or JPEG compression processing.

[0015] The image sensor 110 operates at timings controlled by the timing generator 112. The image sensor generates still images, moving images, or through images for recording. Through images are mainly moving images and are displayed on the liquid crystal monitor 120 for the user to determine the composition for capturing still images.

[0016] The liquid crystal monitor 120 displays images such as through images and various information such as menu screens. The liquid crystal monitor 120 is an example of a display unit in this embodiment. Other types of display devices, such as an organic EL display device, may be used instead of the liquid crystal monitor.

[0017] The control unit 130 includes various operating components such as a release button for instructing the start of shooting, a mode dial for setting the shooting mode, and a power switch. The control unit 130 also includes a touch panel superimposed on the LCD monitor 120.

[0018] RAM 141 is a recording medium that functions as the work memory of the camera control unit 140. RAM 141 temporarily stores (i.e., holds or buffers) image data generated by the image sensor 110 and various setting information in the digital camera 1, for example. Flash memory 142 is a non-volatile recording medium. For example, flash memory 142 stores predetermined setting values ​​in the digital camera 1. RAM 141 and flash memory 142 are examples of storage units in the digital camera 1 in this embodiment.

[0019] The card slot 170 can accommodate a memory card 171 and controls the memory card 171 based on control from the camera control unit 140. The digital camera 1 can store image data in the memory card 171 and read image data from the memory card 171.

[0020] The shutter 180 adjusts the exposure time of the light incident on the image sensor 110. The shutter 180 is driven by a drive system such as a DC motor or a stepping motor, according to a control signal issued from the camera control unit 140. For example, the camera control unit 140 can adjust the shutter speed (exposure time) by controlling the drive speed at which the shutter 180 is driven.

[0021] The body mount 150 is mechanically and electrically connectable to the lens mount 250 of the interchangeable lens 200. The body mount 150 can send and receive data to and from the interchangeable lens 200 via the lens mount 250. The body mount 150 transmits the exposure synchronization signal received from the camera control unit 140 to the lens control unit 240 via the lens mount 250. It also transmits other control signals received from the camera control unit 140 to the lens control unit 240 via the lens mount 250. Furthermore, the body mount 150 transmits signals received from the lens control unit 240 via the lens mount 250 to the camera control unit 140.

[0022] Furthermore, the camera body 100 is configured to implement the BIS function and further includes a gyro sensor 184 (shake detection unit) for detecting camera body 100 shake, and a BIS processing unit 183 for controlling shake correction processing based on the detection result of the gyro sensor 184. In addition, the camera body 100 is equipped with a sensor drive unit 181 for moving the image sensor 110 and a position sensor 182 for detecting the position of the image sensor 110.

[0023] The sensor drive unit 181 can be implemented, for example, by a magnet and a flat coil. The sensor drive unit 181 may also include other motors or actuators. The position sensor 182 is a sensor that detects the position of the image sensor 110 in a plane perpendicular to the optical axis of the optical system. The position of the image sensor 110 can be defined as a position on a two-dimensional plane perpendicular to the optical axis. For example, the position of the image sensor 110 can be defined as the amount of displacement along two orthogonal axes (X axis, Y axis) from a predetermined reference position of the image sensor 110. The position sensor 182 can be implemented, for example, by three sets of magnets and Hall elements. For example, two of the three sets are arranged in the X-axis direction and one set in the Y-axis direction (or two sets in the Y-axis direction and one set in the X-axis direction), and each detects the position in the X-axis direction or the Y-axis direction. Based on the relationship of each detection result, the position sensor 182 detects the position of the image sensor 110 in the X-axis direction, the Y-axis direction and the rotational direction along the rotation axis along the optical axis relative to the reference position.

[0024] In the position sensor 182, each Hall element is attached to the image sensor 110, and each magnet is fixedly positioned relative to the camera body 100. For example, consider a pair of magnets and Hall elements in the X-axis direction. The Hall element detects the magnetic flux density according to its relative position to the magnet. By pre-establishing a correspondence between the magnitude of the magnetic flux density and the relative position, the relative position corresponding to the magnitude of the magnetic flux density detected by the Hall element can be obtained. The obtained relative position represents the position of the image sensor 110 in the X-axis direction. For example, the position of the image sensor 110 can be similarly obtained in the Y-axis direction and the rotation direction. As a result, for example, the position sensor 182 outputs a signal (also called a "position signal") indicating the detected position of the image sensor 110 to the BIS processing unit 183.

[0025] The BIS processing unit 183 controls the sensor drive unit 181 based on signals from the gyro sensor 184 and the position sensor 182 to shift the image sensor 110 into a plane perpendicular to the optical axis in order to compensate for camera body 100 shaking. There are mechanical limitations to the range in which the image sensor 110 can be driven by the sensor drive unit 181. In the BIS function, the range in which the image sensor 110 can be driven by the sensor drive unit 181 is called the "element drive range".

[0026] 1-2. Interchangeable lenses The interchangeable lens 200 comprises an optical system, a lens control unit 240, and a lens mount 250. The optical system includes an anamorphic lens 270, a zoom lens 210, an OIS (Optical Image Stabilizer) lens 220, a focus lens 230, and an aperture 260. Such an optical system, including the anamorphic lens 270, is sometimes called an "anamorphic optical system."

[0027] The anamorphic lens 270 compresses the subject image more in a first direction perpendicular to the optical axis than in a second direction perpendicular to the optical axis. For example, the first and second directions are perpendicular to each other. In this way, the anamorphic lens 270 is a lens that forms an image by transforming the aspect ratio of the subject image.

[0028] In this embodiment, the anamorphic lens 270 does not compress the subject image in the Y direction as shown in Figure 1, but compresses the subject image in the X direction. Thus, the anamorphic lens 270 may be an optical system that compresses the subject image in a specific direction. The anamorphic lens 270 is composed of one or more lenses.

[0029] The zoom lens 210 is a lens for changing the magnification of the subject image formed by the optical system. The zoom lens 210 consists of one or more lenses. The zoom lens 210 is driven by a zoom drive unit 211. The zoom drive unit 211 includes a zoom ring that can be operated by the user. Alternatively, the zoom drive unit 211 may include a zoom lever and an actuator or motor. The zoom drive unit 211 moves the zoom lens 210 along the optical axis of the optical system in response to user operation.

[0030] The focus lens 230 is a lens used in the optical system to change the focus state of the subject image formed on the image sensor 110. The focus lens 230 is composed of one or more lenses. The focus lens 230 is driven by the focus drive unit 233.

[0031] The focus drive unit 233 includes an actuator or motor and moves the focus lens 230 along the optical axis of the optical system based on the control of the lens control unit 240. The focus drive unit 233 can be implemented using a DC motor, stepping motor, servo motor, or ultrasonic motor, etc.

[0032] The OIS lens 220 is a lens used in the OIS function to correct image blur formed in the optical system of the interchangeable lens 200. The OIS lens 220 reduces image blur on the image sensor 110 by moving in a direction that cancels out the blur of the digital camera 1. The OIS lens 220 consists of one or more lenses. The OIS lens 220 is driven by the OIS drive unit 221.

[0033] The OIS drive unit 221 shifts the OIS lens 220 in a plane perpendicular to the optical axis of the optical system, under control from the OIS processing unit 223. There are mechanical limitations to the range in which the OIS lens 220 can be driven by the OIS drive unit 221. The range in which the OIS lens 220 can be driven by the OIS drive unit 221 is called the "lens drive range". The OIS drive unit 221 can be implemented, for example, by a magnet and a flat plate coil. The position sensor 222 is a sensor that detects the position of the OIS lens 220 in a plane perpendicular to the optical axis of the optical system, for example, by the same principle as the position sensor 182 of the camera body 100. The position sensor 222 can be implemented, for example, by a magnet and a Hall element. The OIS processing unit 223 controls the OIS drive unit 221 based on the output of the position sensor 222 and the output of the gyro sensor 224 (shake detection unit).

[0034] The aperture 260 adjusts the amount of light incident on the image sensor 110. The aperture 260 is driven by an aperture drive unit 262, which controls the size of its opening. The aperture drive unit 262 includes a motor or actuator.

[0035] The gyro sensor 184 or 224 detects shake in the yaw, pitch, and roll directions based on the angular change per unit time, i.e., angular velocity, of the digital camera 1. The gyro sensor 184 or 224 outputs an angular velocity signal indicating the amount of shake (angular velocity) detected to the BIS processing unit 183 or OIS processing unit 223. The angular velocity signal output by the gyro sensor 184 or 224 may include a wide range of frequency components caused by camera shake, mechanical noise, etc. Other sensors capable of detecting shake in the digital camera 1 can be used instead of the gyro sensor. Furthermore, the gyro sensor 224 of the interchangeable lens 200 does not need to detect shake in the roll direction.

[0036] The camera control unit 140, lens control unit 240, OIS processing unit 223, and BIS processing unit 183 may be composed of hardwired electronic circuits or of a microcomputer using a program. For example, the camera control unit 140 and lens control unit 240 can be implemented with various processors such as a CPU, MPU, GPU, DSU, FPGA, or ASIC. The camera control unit 140, lens control unit 240, OIS processing unit 223, and BIS processing unit 183 may each be implemented with separate processors or with a single processor. For example, the camera control unit 140, lens control unit 240, OIS processing unit 223, and BIS processing unit 183 are each examples of control units in the digital camera 1 of this embodiment.

[0037] 1-3. Configuration of the image blur correction function The configuration for realizing various image blur correction functions of the digital camera 1 in this embodiment will be explained with reference to Figures 3 to 5.

[0038] Figure 3 is a block diagram showing the configuration of the BIS processing unit 183 and the EIS processing unit 143 in the digital camera 1 of this embodiment. Figure 4 is a block diagram showing the configuration of the OIS processing unit 223 in the digital camera 1.

[0039] As shown in Figure 3, the digital camera 1 of this embodiment includes, for example, an EIS processing unit 143 that implements an EIS function as a functional configuration of the camera control unit 140. The camera control unit 140 sets a correction distribution that includes an OIS ratio indicating the distribution to the OIS processing unit 223, a BIS ratio indicating the distribution to the BIS processing unit 183, and an EIS ratio indicating the distribution to the EIS processing unit 143 as a ratio for distributing the amount of blur correction to correct image blur. For example, the OIS ratio, BIS ratio, and EIS ratio each include components corresponding to the yaw direction, pitch direction, and roll direction, respectively.

[0040] In this embodiment, the digital camera 1 sets the amount of image blur correction (also called "correction amount") by various image blur correction functions using the camera control unit 140 and the respective processing units 183, 223, and 143.

[0041] 1-3-1. OIS Processing Unit For example, the camera control unit 140 transmits information indicating the OIS ratio via each mount 150, 250 to the lens control unit 240 of the interchangeable lens 200, and sets the OIS processing unit 223.

[0042] The configuration of the OIS processing unit 223 in the interchangeable lens 200 will be explained using Figure 4. The OIS processing unit 223 includes a filter 306, a phase compensation unit 307, an integrator 308, a multiplier 309, and a PID control unit 310.

[0043] Filter 306 applies various filtering processes to the signal received from the gyro sensor 224. Filter 306 consists of an HPF (high-pass filter), an LPF (low-pass filter), and / or a BPF (band-pass filter), and for example, to block drift components, it uses an HPF to block predetermined low-frequency components contained in the signal.

[0044] The phase compensation unit 307 corrects the phase delay caused by the OIS drive unit 221 and other factors in the signal received from the filter 306.

[0045] The integrator 308 integrates the signal indicating the angular velocity of the vibration input from the phase compensation unit 307 to generate a vibration detection signal indicating the angle of the vibration. Since the vibration detection signal indicates the amount of vibration, it can also be said to indicate the amount of vibration correction to be applied. The vibration detection signal from the integrator 308 is input to the PID control unit 310 via the multiplier 309. The OIS processing unit 223 may also include filter configurations other than those described above, such as a notch filter for noise processing.

[0046] The multiplier 309 multiplies the shake detection signal input from the integrator 308 by a gain representing the OIS ratio, for example, set by the lens control unit 240, to calculate the OIS correction amount as the shake correction amount by the OIS function. The OIS ratios in the yaw and pitch directions are set to be between "0" and "1" as gains for the shake detection signal in each direction. The OIS ratios may be set separately for each direction.

[0047] The PID control unit 310 performs PID control based on the difference between the OIS correction amount from the multiplier 309 and the position information of the OIS lens 220 indicated by the signal from the position sensor 222, and generates a drive signal for the OIS drive unit 221. The OIS drive unit 221 drives the OIS lens 220 based on the drive signal.

[0048] In this embodiment, for example, the lens control unit 240, in response to a request from the camera control unit 140, obtains the deviation between the OIS correction amount in PID control and the position signal from the position sensor 222 as the OIS error amount from the PID control unit 310. The OIS error amount represents the difference between the OIS correction amount and the amount of movement of the OIS lens 220 indicated by the position information of the OIS lens 220 driven according to the OIS correction amount. The lens control unit 240 transmits the OIS error amount to the camera control unit 140 via each mount 150, 250. Details regarding the error amount will be described later.

[0049] 1-3-2. BIS Processing Unit The configuration of the BIS processing unit 183 in the camera body 100 will be explained using Figure 3. The BIS processing unit 183 includes a filter 406, a phase compensation unit 407, an integrator 408, a multiplier 409, and a PID control unit 410.

[0050] Filter 406 applies various filtering processes to the signal received from the gyro sensor 184. Filter 406 is composed of, for example, an HPF, LPF, and / or BPF, similar to filter 306 of the OIS processing unit 223.

[0051] The phase compensation unit 407 corrects the phase delay caused by the sensor drive unit 181 and other components in the signal received from the filter 406.

[0052] The integrator 408 integrates the signal indicating the angular velocity of the vibration input from the phase compensation unit 407 to generate a vibration detection signal indicating the angle of the vibration. Since the vibration detection signal indicates the amount of vibration, it can also be said to indicate the amount of vibration correction to be applied. The vibration detection signal from the integrator 408 is input to the PID control unit 410 via the multiplier 409. The BIS processing unit 183 may also include filter configurations other than the above configuration, such as a notch filter for noise processing.

[0053] The multiplier 409 multiplies the shake detection signal input from the integrator 408 by a gain representing the BIS ratio, which is set by, for example, the camera control unit 140, to calculate the BIS correction amount as the shake correction amount by the BIS function. The BIS ratio is set separately for, for example, the gain for the shake detection signals in the yaw and pitch directions and the gain for the shake detection signal in the roll direction. In addition to or instead of this, a separate gain may also be set in the multiplier 409 between the shake detection signal in the yaw direction and the shake detection signal in the pitch direction.

[0054] The PID control unit 410 generates a drive signal to shift the image sensor 110 based on the output from the position sensor 182 and the output from the multiplier 409, and outputs it to the sensor drive unit 181. The sensor drive unit 181 drives the image sensor 110 based on the drive signal. For example, the camera control unit 140 obtains the deviation input in PID control from the PID control unit 410 as the BIS error amount. The BIS error amount represents the difference between the BIS correction amount and the amount of movement indicated by the position information of the image sensor 110, which is driven according to the BIS correction amount by the image blur correction operation of the BIS function. The PID control deviation is calculated, for example, by the PID control unit 410 based on the BIS correction amount and the position signal from the position sensor 182.

[0055] 1-3-3. EIS Processing Unit The EIS processing unit 143 will be described using Figure 3. In this embodiment, the EIS processing unit 143 multiplies the shake detection signal input from the integrator 408 of the BIS processing unit 183 by a gain indicating the EIS ratio set by the camera control unit 140. The EIS ratio is set separately, for example, between the gain for the shake detection signals in the yaw and pitch directions and the gain for the shake detection signal in the roll direction. In addition to or instead of this, a separate gain may also be set between the shake detection signal in the yaw direction and the shake detection signal in the pitch direction.

[0056] The EIS processing unit 143 calculates the amount of image stabilization based on the image stabilization detection signal. Then, the EIS processing unit 143 adds, for example, the BIS error amount obtained from the BIS processing unit 183 and the OIS error amount obtained from the OIS processing unit 223 by the lens control unit 240 to the calculated amount of image stabilization, thereby calculating the EIS correction amount (electronic correction amount) as the amount of image stabilization by the EIS function. Hereinafter, the OIS error amount and the BIS error amount will be collectively referred to as the "error amount".

[0057] When performing corrections for the anamorphic lens 270, the EIS correction amount includes a shear strain correction amount to correctly correct the shear strain that occurs when correcting image blur (rotational blur) in the roll direction. In other words, the EIS processing unit 143 calculates the shear strain correction amount to correctly deform the subject image when correcting the rotational deformation component when the anamorphic lens 270 is attached to the digital camera 1. The calculation of the shear strain correction amount will be explained in detail later.

[0058] The EIS processing unit 143 may, for example, perform a process to extract an image of a narrower region from the entire image data generated by the image sensor 110 by a predetermined extraction amount. For example, various image processing for recording the shooting results is applied to the extracted image data. For example, electronic zoom processing may be applied so that the extracted image is the same size as the image before extraction.

[0059] If the roll component of the EIS ratio is greater than 0, the EIS processing unit 143 corrects rotational shake based on the calculated EIS correction amount. If an anamorphic lens 270 is attached to the digital camera 1, the EIS processing unit 143 also corrects shear strain in the correction of rotational shake. If the roll component of the EIS ratio is greater than 0, the electronic correction performed by the EIS processing unit 143 includes the above-described correction of rotational shake and correction of shear strain.

[0060] 2.Operation The operation of the digital camera 1 in this embodiment will be described below.

[0061] 2-1. Challenges of image stabilization when using anamorphic lenses To explain the operation of the digital camera 1 of this embodiment, first, the problems identified by the inventors regarding image stabilization when using an anamorphic lens will be explained using Figure 5.

[0062] Figure 5 is a schematic diagram illustrating a comparative example of the operation for correcting rotational shake in a digital camera equipped with an anamorphic lens. In such a digital camera, when rotational shake occurs, the anamorphic lens compresses (squeezes) the subject image horizontally across the image sensor and records it, and after shooting, the subject image is disc-squeezed and played back or recorded.

[0063] Figures 5(a) and 5(b) are diagrams illustrating rotational blur and schematically show the scenery as seen from a digital camera during shooting. Figure 5(a) shows the scenery during shooting without rotational blur. Figure 5(b) shows the scenery during shooting with rotational blur present. Figure 5(c) schematically shows image I1, which is represented by the image data generated by the image sensor when rotational blur occurs, as shown in Figure 5(b). When rotational blur occurs, the subject image is compressed horizontally by the anamorphic lens and formed on the imaging surface of the image sensor. The dot-hatched rectangle in Figure 5(c) schematically represents such a subject image. The same applies to Figures 5(d) to 5(g).

[0064] Figure 5(d) shows image I2 obtained by applying rotational deformation to image (frame) I1 in Figure 5(c) using image processing (electronic rotational correction). In Figure 5(d), the areas with line hatching indicate regions that were outside the field of view of image I1 before rotational deformation, and are regions that do not contain information such as pixel values. The same applies to Figures 5(e) to 5(g). The subject image in image I2 has a parallelogram shape, as if it had been subjected to horizontal shear image processing. That is, the subject image in image I2 has distortion (shear distortion) as if it had been subjected to shear image processing. Therefore, when disc-easing is performed on image I2 to stretch it horizontally, the image I3 after disc-easing also has shear distortion, as shown in Figure 5(e).

[0065] To prevent the image after disc-easing from having shear distortion, it is effective to apply a shear distortion correction process to image I2 before disc-easing. Figure 5(f) shows image I4 obtained by correcting the shear distortion of image I2 in Figure 5(d). By disc-easing image I4, a corrected image I5 similar to the state without rotational blur can be obtained, as shown in Figure 5(g).

[0066] As described above, in the comparison example in Figure 5, rotation correction is performed on a frame-by-frame basis using electronic correction.

[0067] However, the inventors found a new problem when performing rotational correction on a frame-by-frame basis in a digital camera that performs disc sizing according to an anamorphic lens. Specifically, because the image processing is performed on a frame-by-frame basis by the digital camera, it is not possible to correct image blur that occurs during the exposure of a single frame.

[0068] To solve this problem, the inventor conducted extensive research and devised the digital camera 1 of this embodiment. The digital camera 1 of this embodiment can correct image blur that occurs during the exposure of one frame by performing at least a portion of the rotational correction using optical correction that can be executed at a shorter interval than the frame period.

[0069] 2-2. Overview of Operation of This Embodiment An overview of the operation of the digital camera 1 according to this embodiment will be explained with reference to Figure 6. Figure 6 is a schematic diagram illustrating the operation for correcting rotational shake in the digital camera 1. Figure 6 shows an example where the optical correction ratio of the shake correction amount in the roll direction is 1, that is, an example in which rotational shake is corrected by optical correction and not by electronic correction.

[0070] Figures 6(a) and 6(b), similar to Figures 5(a) and 5(b), schematically show the scenery as seen from the digital camera 1 during shooting. Figure 6(c) schematically shows the optical image after passing through the anamorphic lens 270.

[0071] Figure 6(d) schematically shows image I6, which is generated by the image data of the image sensor 110 after rotational blur correction by optical correction. Rotational blur correction is performed by rotating the image sensor 110 in a plane perpendicular to the optical axis of the optical system, and the subject image is formed on the imaging surface of the rotated image sensor 110. Since image I6 in Figure 6(d) is not an image obtained by rotating and deforming the captured image, it does not include the area that was outside the field of view in the image before rotation and deformation (the area with line hatching in Figure 5(d)), as shown in Figure 5(d). Thus, the captured image of this embodiment has more areas (pixels) containing information such as pixel values ​​compared to the captured image of the comparative example. Therefore, with the digital camera 1 according to this embodiment, it is easier to secure margins in the image for electronic correction, including processing such as rotation and cropping of image data.

[0072] Figure 6(e) shows image I7 obtained by correcting the shear strain of image I6 in Figure 6(d). By disceasing image I7, a corrected image I8 similar to the state without rotational blur can be obtained, as shown in Figure 6(f).

[0073] Figure 7 is a flowchart illustrating the optical correction operation by the digital camera 1 in this embodiment. Figure 8 is a flowchart illustrating the electronic correction operation by the digital camera 1 in this embodiment.

[0074] Each process shown in the flowcharts of Figures 7 and 8 is executed in parallel with operations such as video recording by the camera control unit 140, OIS processing unit 223, and BIS processing unit 183, for example, when the interchangeable lens 200 is attached to the camera body 100.

[0075] The optical correction process in Figure 7 is repeatedly performed, for example, at a predetermined first period. The first period is, for example, 1 / 10000 to 1 / 500 seconds (0.1 milliseconds to 2 milliseconds). The electronic correction process in Figure 8 is repeatedly performed, for example, at a predetermined second period. The second period is, for example, the frame period, and is, for example, 1 / 60 to 1 / 30 seconds. In this embodiment, the first period is shorter than the second period.

[0076] 2-3. Optical Correction Processing The optical correction process shown in Figure 7 will be explained below. First, the BIS processing unit 183 acquires an angular velocity signal from the gyro sensor 184, and the OIS processing unit 223 acquires an angular velocity signal from the gyro sensor 224 (S11).

[0077] The BIS processing unit 183 and the OIS processing unit 223 generate a shake detection signal based on the angular velocity signal and calculate the shake correction amount (S12).

[0078] The camera control unit 140 obtains the image stabilization amount from at least one of the OIS processing unit 223 and the BIS processing unit 183, and distributes the image stabilization amount to the OIS correction amount, the BIS correction amount, and the EIS correction amount (S13). For example, the camera control unit 140 calculates the OIS correction amount, the BIS correction amount, and the EIS correction amount based on the OIS ratio, the BIS ratio, and the EIS ratio. For example, the camera control unit 140 may distribute the image stabilization amount based on a coefficient determined using information such as focal length, or it may distribute it based on frequency information.

[0079] In this embodiment, the OIS correction amount and the BIS correction amount are collectively referred to as the "optical correction amount." In this embodiment, if rotational runout is present, the optical correction amount for the roll direction is greater than 0. That is, the roll direction component of the BIS correction amount is greater than 0. In contrast, the roll direction component of the EIS correction amount may be 0. Below, an example in which rotational runout is present will be described in this embodiment.

[0080] At least one of the OIS processing unit 223 and the BIS processing unit 183 performs optical correction of yaw and pitch shake (S14). The BIS processing unit 183 can also perform optical correction of rotational shake (S15).

[0081] In Figure 7, steps S14 and S15 are shown separately for the sake of explanation, but these steps may be performed integrally. For example, the BIS processing unit 183 may send a drive signal to the sensor drive unit 181 to drive the image sensor 110 to perform a shift corresponding to the yaw, pitch, and roll directions. The sensor drive unit 181 drives the image sensor 110 based on the drive signal. For example, the OIS processing unit 223 may send a drive signal to the OIS drive unit 221 to drive the OIS lens 220 to perform a shift corresponding to the yaw and pitch directions.

[0082] Here, using Figure 9, we will explain the optical correction process for rotational shake in the case where the optical correction ratio for the amount of shake correction in the roll direction is 1, that is, in the case where rotational shake is corrected by optical correction and not by electronic correction.

[0083] As shown in Figure 9, when rotational wobble occurs with a rotation angle θ in the roll direction from the X axis, point A at coordinate (x,0) with the optical center O as the origin moves to point B(xcosθ,xsinθ). If the anamorphic lens 270 of this embodiment is an anamorphic magnification β lens that compresses the subject image by 1 / β in the X direction without compressing it in the Y direction, then point B moves to point C with respect to the image sensor 110. The coordinates of point C are ((x / β)cosθ,xsinθ). Therefore, the angle θ is expressed by the following equation (1). opt By performing rotational correction using this method, rotational shake when the anamorphic lens 270 is attached can be corrected. θ opt =tan -1 (βtanθ) ···(1)

[0084] Returning to Figure 7, the camera control unit 140 calculates the EIS correction amount based on the correction distribution set in step S13 and stores the EIS correction amount in a buffer linked to time information (S16). The time information includes, for example, the time when the angular velocity signal was acquired in step S11. The frame buffer is a storage area provided in, for example, RAM 141, flash memory 142, etc. The buffer only needs to be able to store the EIS correction amount and time information, and may be provided in other parts of the digital camera 1, external storage devices, etc.

[0085] 2-4. Electronic Correction Process The electronic correction process shown in Figure 8 will be described below. The camera control unit 140 sets the variable i (i is a non-negative integer) to an initial value of 0 (S21). In this embodiment, the variable i corresponds to the row (line) number of the image sensor 110. For example, row 0 (i=0) represents the uppermost or lowermost sensor line of the image sensor 110.

[0086] The camera control unit 140 obtains the exposure time of the i-th row of pixels (S22), and based on the obtained exposure time, obtains the EIS correction amount from the buffer (S23). For example, if the first cycle of the optical correction process is equal to the exposure timing cycle of one line (line cycle, i.e., the repeating cycle of steps S22 to S26), the camera control unit 140 obtains the EIS correction amount from the buffer that corresponds to the exposure time obtained in step S22.

[0087] The above processing may be adjusted depending on the relationship between the first period and the line period. For example, if the first period is longer than the line period, the camera control unit 140 searches the buffer for the time closest to the exposure time obtained in step S22 and obtains the EIS correction amount associated with the searched time from the buffer. Alternatively, the camera control unit 140 may obtain EIS correction amounts associated with multiple times around the exposure time from the buffer and determine the EIS correction amount to be used in subsequent processing by interpolation (e.g., linear interpolation) using the obtained EIS correction amounts.

[0088] On the other hand, if the first period is shorter than the line period, there may be multiple EIS correction amounts for a single line period. In this case, the camera control unit 140 determines, for example, a representative value of the EIS correction amount for each line period (for example, the average value of multiple EIS correction amounts).

[0089] The camera control unit 140 derives a projection transformation matrix based on the EIS correction amount obtained in step S23 (S24). Details of the projection transformation matrix will be described later.

[0090] Next, the camera control unit 140 increments the variable i (S25) and repeatedly executes the processes in steps S22 to S26 until the variable i becomes greater than or equal to size(y), which indicates the size of the image data (No in S26). size(y) is, for example, the number of vertical pixels in the image data.

[0091] When the variable i becomes greater than or equal to size(y) (No in S26), the camera control unit 140 performs electronic correction using the projection transformation matrix (S27).

[0092] Furthermore, as mentioned above, electronic correction processing cannot correct image blur that occurs during the exposure of a single frame. To reduce image blur that occurs during the exposure of a single frame, it is necessary to increase the shutter speed or use optical correction. The digital camera 1 according to this embodiment combines rotation correction using optical correction and electronic correction, as described above. Specifically, by performing at least a part of the rotation correction using optical correction that is executed in a first cycle shorter than the second cycle, it is possible to correct image blur that occurs during the exposure of a single frame.

[0093] Furthermore, based on the correction amount linked to time information, which is saved in step S16 of the optical correction shown in Figure 7 performed in the first period, the projection transformation matrix corresponding to the i-th row is obtained in the electronic correction shown in Figure 8, which is performed in the second period (S22-S24). If the first period is shorter than the second period, the amount of image deformation due to the electronic correction (for example, the correction parameter corresponding to the projection transformation matrix) may not be uniform and may vary within the frame. For example, the image deformation amount of the i-th row may be different from the image deformation amount of the (i+1)-th row. Thus, the electronic correction in this embodiment can correct image blur more precisely by utilizing the correction amount information from the optical correction performed in the first period, which is shorter than the second period.

[0094] Furthermore, in this embodiment, by performing at least a portion of the rotation correction using the optical correction method shown in Figure 7, it becomes easier to secure margins in the image for electronic correction, including processing such as rotation and cropping of image data, as described above.

[0095] The above describes an example in which the EIS correction amount is obtained from a buffer on a line-by-line basis and the projection transformation matrix is ​​derived. For example, when a rolling shutter is used, the rolling shutter distortion can be reduced by using the projection transformation matrix corresponding to each of the lines. However, this embodiment is not limited to calculating the projection transformation matrix corresponding to each of the lines. For example, the camera control unit 140 may derive projection transformation matrices for a number of lines less than the number of vertical pixels, for example, at predetermined intervals. In this case, the correction means for the decimated lines may be determined by an interpolation method (e.g., linear interpolation) using information from the lines for which the projection transformation matrix is ​​derived. Even if the number of lines for which the projection transformation matrix is ​​derived is decimated in this way, the rolling shutter distortion can be reduced while reducing the processing load by using the projection transformation matrix corresponding to each of the multiple lines.

[0096] Furthermore, this embodiment is not limited to the example in which electronic correction is performed using projection transformation matrices derived for each line. For example, the camera control unit 140 may derive one representative projection transformation matrix for each frame and perform electronic correction using this representative projection transformation matrix. For example, the camera control unit 140 derives a representative projection transformation matrix based on the projection transformation matrix corresponding to each line. Alternatively, the camera control unit 140 may correct the entire frame (all lines) using the projection transformation matrix for a specific line in the frame (for example, a line located in the center in the vertical direction). In this case, it is not necessary to derive a large number of projection transformation matrices corresponding to each line, and the amount of computation performed by the camera control unit 140 can be reduced.

[0097] 2-4-1. Projection Transformation Operation The projection transformation matrix used in the electronic correction process is described below. The camera control unit 140 applies geometric image transformation processing to the image data using projection transformation techniques.

[0098] Generally, the rotation matrix R representing rotation in the roll direction is given by the following equation (2). TIFF2026091802000002.tif23169

[0099] The rotation in the roll direction (rotation angle θ) performed by the optical correction process in Figure 7. opt Projection transformation matrix H representing ) opt This can be expressed by the following equation (3). TIFF2026091802000003.tif27169

[0100] In the electronic correction process, correction processing is performed to correct the remaining image deformations (rotational blur and shear distortion). The projection transformation matrix H represents the image deformations performed in the electronic correction process. eis This can be expressed by the following equation (4). TIFF2026091802000004.tif33169

[0101] In Equation (4), θ is the rotation angle of the rotational shake in the roll direction. When all corrections for rotational shake are performed by optical correction, the relationship between θ opt and θ is represented by the above Equation (1). β is the anamorphic magnification.

[0102] As shown in Equation (4), the projection transformation matrix H representing the electronic correction process executed by the EIS processing unit 143 of the camera control unit 140 eis includes the projection transformation matrix H representing the optical correction process. As described above with respect to FIGS. 7 and 8, since the first period of the optical correction process and the second period of the electronic correction process are different, the digital camera 1 performs overall image shake correction by coordinating the optical correction process and the electronic correction process as shown in Equation (4). opt

[0103] 3. Summary ​As described above, the digital camera 1 in this embodiment comprises an image sensor 110 (image sensor), an optical correction unit including a BIS processing unit 183 (first image blur correction unit), an EIS processing unit 143 (electronic image correction unit), and a camera control unit 140. The optical correction unit may also include an OIS processing unit 223 (second image blur correction unit). In this embodiment, the OIS processing unit 223, the BIS processing unit 183, and the camera control unit 140 are examples of control units that control image blur correction by the optical correction unit and the EIS processing unit 143. The image sensor 110 captures an image of a subject formed through the optical system and generates image data. The OIS processing unit 223 performs image blur correction by moving the OIS lens 220 (correction lens) included in the optical system in a plane perpendicular to the optical axis of the optical system. The BIS processing unit 183 performs image blur correction by moving the image sensor 110 in a plane perpendicular to the optical axis. The EIS processing unit 143 corrects image blur by image processing of the image data. The optical system compresses the subject image more in the X direction (first direction) perpendicular to the optical axis than in the Y direction (second direction). The optical correction unit corrects rotational blur that occurs in the rotational direction around the optical axis (S15). The EIS processing unit 143 corrects the shear distortion that occurs due to the compression of the subject image by the optical system after the rotational blur has been corrected by the optical correction unit (S27).

[0104] In this embodiment, the optical system includes an anamorphic lens 270.

[0105] According to the digital camera 1 described above, when using the anamorphic lens 270, rotational shake can be corrected appropriately, for example, with high precision, by correcting the optical correction unit.

[0106] In this embodiment, the optical correction unit corrects at least a portion of the rotational blur, and the EIS processing unit 143 may correct the shear distortion corresponding to the rotational blur without rotating the subject image for the portion of the rotational blur corrected by the optical correction unit. This configuration also allows for appropriate correction of image blur.

[0107] In this embodiment, the EIS processing unit 143 may correct shear distortion without correcting rotational shake. With this configuration, when using the anamorphic lens 270, the shear distortion of the subject image after correction of rotational shake by the optical correction unit can be appropriately corrected.

[0108] In this embodiment, the EIS processing unit 143 corrects shear distortion by performing image processing based on the result of the rotational shake correction by the optical correction unit (S27). Electronic correction processing cannot correct image shake that occurs during the exposure of one frame, and if electronic correction processing is performed alone, image shake that occurs during exposure will remain as blur. According to the above configuration of this embodiment, the optical correction unit and the EIS processing unit 143 work together, so that the amount of rotational shake correction performed by the EIS processing unit 143 can be reduced by the amount that the optical correction unit corrects for rotational shake. Therefore, with this configuration, it is possible to correct shear distortion while reducing the effect of image blur occurring during the exposure of one frame remaining in the image.

[0109] In this embodiment, the optical correction unit corrects rotational blur in a first period, and the EIS processing unit 143 corrects shear distortion by performing image processing in a second period. The first period is shorter than the second period. This makes it possible to correct image blur that occurs during the exposure of one frame. In this case, the camera control unit 140 operates the image sensor 110 at a predetermined frame period to generate image data for each frame, and the EIS processing unit 143 performs image processing by changing the amount of shear distortion correction within the range of the frame period according to the rotational blur correction result by the optical correction unit (S22~S27). By utilizing the correction amount information in the optical correction performed in a first period which is shorter than the second period, image blur can be corrected more precisely.

[0110] The digital camera 1 of this embodiment further includes a body mount 150, which is an example of a connection part for detachably connecting an interchangeable lens 200, and the optical system is included in the interchangeable lens 200.

[0111] (Embodiment 2) 1. Overview When using a digital camera to take photos while walking or while running, significant blurring usually occurs. Figure 10A is a graph showing an example of the temporal change in roll rotation blur when the photographer is stationary. Figures 10B and 10C are graphs showing examples of the temporal change in roll rotation blur when taking photos while walking and running, respectively. For example, the magnitude of blur that occurs when taking photos while stationary is about ±0.5°, but in recent years, blur correction performance of about ±1° to 3° is sometimes required when taking photos while walking, and about ±7° when taking photos while running.

[0112] In conventional digital cameras, which cannot adequately correct for large shakes that occur when shooting while walking or running, the impact of shake on the image is greater than the blur (image distortion), and even if blur is present in the image, it is rarely noticeable to the user. In contrast, if some or all of a large shake is corrected by electronic correction, the impact of shake on the image is reduced, but the blur becomes noticeable to the user, and the effect of blur becomes apparent.

[0113] Figure 11 is a schematic diagram illustrating the manifestation of blur as described above. The graph at the bottom of Figure 11 shows an example of the time course of blur with a solid line. The dashed line in the graph of Figure 11 schematically shows the amount of image blur in images A1 to A6, which are shown by the image data after blur correction by electronic correction. Images A1 to A6 are images shown by the image data for each frame (the nth to the (n+5th)th frame), and Δt shown on the time axis of the graph in Figure 11 indicates the frame period. exp This represents the exposure time for each frame.

[0114] Images A1-A6 illustrate the images obtained by capturing a subject image that is circular in the image plane. Images A1-A6 in Figure 11 show subject images that include blur. In electronic correction, image shake that occurs during the exposure of one frame appears as blur. When shake is corrected by electronic correction, as shown by the dashed line in the graph of Figure 11, the effect of the movement of the subject image between frames due to shake becomes smaller, but the blur does not decrease, and the effect of blur on the magnitude of shake becomes larger compared to before correction.

[0115] For example, blur can be reduced by controlling the shutter speed and shortening the exposure time, but shortening the exposure time can cause the subject's position to not connect smoothly between frames, making the subject's movement appear discontinuous in moving images.

[0116] To solve this problem, the inventor diligently conducted research and devised the digital camera 1 of this embodiment. The digital camera 1 of this embodiment can reduce the effect of blur in images by performing at least a portion of rotational correction using optical correction. Performing at least a portion of rotational correction using optical correction is particularly useful when the digital camera 1 can correct rotational shake of 1° or more (rotational shake of 1° or more, or rotational shake of -1° or less). As mentioned above, the effect of blur becomes apparent when such rotational shake is corrected.

[0117] For optical correction, if the optical component for rotation correction (image sensor) can rotate φ1° around the optical axis relative to the housing of the digital camera 1, then it can be said that rotational shake of φ1° can be corrected. For electronic correction, if the image shown by the image data from the image sensor can rotate φ2° around the optical axis, then it can be said that rotational shake of φ2° can be corrected. For the digital camera 1 to be able to correct rotational shake of 1° or more, it means that the absolute value of (φ1 + φ2) is 1 or greater.

[0118] In particular, the effect of blur becomes apparent when electronic correction is performed for rotational shake of 1° or more. In this embodiment, the digital camera 1 can reduce the effect of blur in the image by performing at least a portion of the rotational correction using optical correction when the correctable angle φ2 for each frame for electronic correction is 1° or more.

[0119] 2.Operation Figure 12 is a flowchart illustrating the optical correction process performed by the digital camera 1 in this embodiment. Compared with the optical correction process of Embodiment 1 shown in Figure 7, the optical correction process in Figure 12 includes a process S201 for acquiring the exposure time after steps S11 and S12, and includes step S202 instead of the process S13 for distributing the blur correction amount.

[0120] In step S201, the camera control unit 140 acquires the exposure time. The exposure time may be manually set by the user via a menu screen or dial operation, or it may be automatically set by the camera control unit 140 according to the brightness of the subject. For example, the camera control unit 140 sets the exposure time according to the brightness of the subject. Alternatively, the camera control unit 140 may acquire the exposure time manually set by the user. In these cases, the exposure time may fluctuate during video recording.

[0121] Step S201 does not have to be performed after step S12 as shown in Figure 12; for example, it may be performed between steps S11 and S12, or before step S11. For example, the camera control unit 140 may acquire the exposure time at the same time as or immediately after the frame rate is set. The exposure time may be set according to the frame rate. For example, the exposure time may be set to half of the frame period.

[0122] The camera control unit 140 allocates the amount of blur correction between the optical correction amount and the EIS correction amount based on the exposure time acquired in step S201 (S202). For example, the camera control unit 140 changes the ratio of the optical correction amount to the EIS correction amount according to the exposure time.

[0123] Figure 13 is a graph showing an example of the relationship between exposure time and the EIS ratio of rotational shake correction amount in the digital camera 1 of this embodiment. In the graph of Figure 13, the horizontal axis represents the exposure time (shutter speed), and the vertical axis represents the EIS ratio of rotational shake correction amount. The EIS ratio of rotational shake correction amount represents the ratio of the EIS correction amount for rotational shake to the sum of the optical correction amount for rotational shake and the EIS correction amount for rotational shake ((EIS correction amount for rotational shake) / (optical correction amount for rotational shake)+(EIS correction amount for rotational shake)).

[0124] In the example shown in Figure 13, the camera control unit 140 sets the EIS ratio of the rotational shake correction amount to R when the exposure time is t1 (for example, (1 / 250) seconds). EIS If set to 1 and the exposure time is t2 (e.g., (1 / 30) seconds), the EIS ratio of the rotational shake correction amount is R EIS Set to 2. EIS 1 is R EIS Greater than 2, 1 ≥ R EIS 1>R EIS 2 ≥ 0. For example, R EIS 1 = 0.5, R EIS 2 = 0.1. The camera control unit 140 is R EIS 1 may be set to the ratio of the second angle to the sum of the first angle, which can be corrected by optical correction processing for rotational shake, and the second angle, which can be corrected by electronic correction processing for rotational shake.

[0125] As shown in Figure 13, the camera control unit 140 may decrease the EIS ratio of rotational shake correction amount as the exposure time increases between exposure times t1 and t2.

[0126] In Figure 13, R EIS2 may be 0. In this case, if the exposure time is longer than t2, the digital camera 1 corrects rotational shake using optical correction only. That is, if the exposure time is longer than a predetermined time, the camera control unit 140 corrects rotational shake using optical correction only, and if the exposure time is less than or equal to the predetermined time, it may determine the ratio of the optical correction amount of rotational shake correction to the EIS correction amount of rotational shake correction according to the exposure time. If the exposure time is longer than a predetermined time, the EIS processing unit 143 will not correct rotational shake and will only correct shear strain.

[0127] While the effects of blur become apparent when the exposure time is relatively long, the digital camera 1 in this embodiment can reduce the effects of blur in the image by performing at least a portion of the rotation correction using optical correction in such cases.

[0128] For example, a table showing the relationship between exposure time and rotational shake correction amount EIS ratio, as shown in Figure 13, is pre-stored in a recording medium such as flash memory 142, and the camera control unit 140 extracts the EIS ratio corresponding to the set exposure time from the table. The camera control unit 140 then determines, for example, the optical correction amount and the EIS correction amount so that the extracted EIS ratio is achieved.

[0129] 3. Summary As described above, in the digital camera 1 of this embodiment, the camera control unit 140 may change the ratio between the optical correction amount by the optical correction unit and the electronic correction amount by the EIS processing unit 143 (electronic correction unit) according to the exposure time. This makes it possible to reduce the effect of blur in the image.

[0130] In the digital camera 1 of this embodiment, the camera control unit 140 may reduce the amount of electronic correction relative to the amount of optical correction as the exposure time increases. This makes it possible to further reduce the effect of blur in the image.

[0131] The digital camera 1 in this embodiment may be capable of correcting rotational shake of 1° or more. When rotational shake of 1° or more is corrected, the effect of blur becomes apparent. In such cases, the digital camera 1 in this embodiment can reduce the effect of blur on the image by performing at least a part of the rotational correction using optical correction.

[0132] (Other embodiments) As described above, Embodiments 1 and 2 have been presented as examples of the technology described herein. However, the technology described herein is not limited thereto and can be applied to embodiments that have been modified, substituted, added, or omitted as appropriate. Furthermore, it is possible to combine the components described in the above embodiments to create new embodiments. Therefore, the following describes some modifications as other embodiments.

[0133] (First variation) In the above embodiment, an example was described in which an interchangeable lens 200 including an anamorphic lens 270 is attached to the camera body 100. However, the digital camera 1 according to this disclosure may use different operations depending on whether the interchangeable lens 200 includes an anamorphic lens 270 or does not include an anamorphic lens 270.

[0134] Figure 14 is a flowchart illustrating the operation of the digital camera 1 according to the first modified example. The flowchart in Figure 14 is executed, for example, each time an interchangeable lens is attached to the body mount 150 of the camera body 100.

[0135] In Figure 14, the camera control unit 140 determines whether or not the digital camera 1 is fitted with an anamorphic lens 270 (S31).

[0136] The signal for determination is generated, for example, by the lens control unit 240. For example, if an interchangeable lens 200 containing an anamorphic lens 270 is mounted on the body mount 150, the lens control unit 240 may automatically transmit an anamorphic lens detection signal to the camera control unit 140 via the lens mount 250 and the body mount 150. Alternatively, the camera control unit 140 may ask the lens control unit 240 whether the interchangeable lens 200 contains an anamorphic lens 270, and in response, the lens control unit 240 may transmit an anamorphic lens detection signal to the camera control unit 140. Furthermore, the camera control unit 140 may automatically acquire information indicating the anamorphic magnification β from the interchangeable lens 200.

[0137] If the digital camera 1 is not fitted with an anamorphic lens 270 (No in S31), the camera control unit 140 operates in a normal (i.e., non-anamorphic lens) correction mode (S32). In the normal correction mode, the camera control unit 140 corrects image blur using at least one of the OIS function, BIS function, and EIS function. In the normal correction mode, a known image stabilization function may be implemented.

[0138] If the digital camera 1 is fitted with an anamorphic lens 270 (Yes in S31), the camera control unit 140 operates in a correction mode for the anamorphic lens 270 (hereinafter referred to as "anamorphic lens mode") (S33). In anamorphic lens mode, the camera control unit 140 performs the optical correction process shown in Figure 7 and the electronic correction process shown in Figure 8.

[0139] It is not necessary to automatically determine whether or not the digital camera 1 is equipped with an anamorphic lens 270. For example, the anamorphic lens mode can be set manually by the user on a menu screen. In this case, the information indicating the anamorphic magnification β can be arbitrarily entered by the user. For example, the lens determination process step S31 can be omitted, and the user can set β to an appropriate value (e.g., 2) when using the anamorphic lens 270, and set β to 1 when not using the anamorphic lens 270.

[0140] Thus, in the digital camera 1 according to the first modified example, if the interchangeable lens connected to the body mount 150 does not include an anamorphic lens 270, the camera control unit 140 has the optical correction unit correct rotational shake without having the EIS processing unit 143 correct shear strain. On the other hand, if the interchangeable lens connected to the body mount 150 includes an anamorphic lens 270, the camera control unit 140 has the optical correction unit correct rotational shake and also has the EIS processing unit 143 correct shear strain.

[0141] (Second variation) In the above embodiment, an example was described in which the optical system of the interchangeable lens 200 includes an anamorphic lens 270, but the anamorphic lens does not have to be included in the interchangeable lens 200. For example, the interchangeable lens 200 may not include an anamorphic lens, and a separate anamorphic lens may be attached to the interchangeable lens 200 or the camera body 100. This embodiment is also an example of an embodiment in which the optical system includes an anamorphic lens.

[0142] (Third variation) In the above embodiment, when rotational wobble of a rotation angle θ occurs in the roll direction with respect to the X-axis (horizontal direction), the θ corresponding to θ optAn example of optical correction for rotational shake was explained (see Figure 9). Unlike this example, optical correction may also be used to correct rotational shake in the roll direction with respect to the Y-axis (vertical direction). For example, the user may manually set whether to correct rotational shake based on the horizontal or vertical direction on the menu screen.

[0143] As shown in Figure 15, when rotational wobble occurs with a rotation angle θ in the roll direction from the Y axis, point D at coordinate (0,y) with the optical center O as the origin moves to point E(ysinθ,ycosθ). Due to compression by the anamorphic lens 270 having anamorphic magnification β, point E moves to point F((y / β)sinθ,ycosθ) with respect to the image sensor 110. Therefore, the angle θ is expressed by the following equation (5). opt_v By performing rotational correction using this method, rotational shake when the anamorphic lens 270 is attached can be corrected. θ opt_v =tan -1 ((1 / β)tanθ) ···(5)

[0144] (Fourth variation) In the embodiments described above, an example was described in which the first period is shorter than the second period, but the concept of this disclosure also applies when the first period is equal to the second period. For example, in a digital camera 1 that performs disc sizing according to an anamorphic lens, by performing at least a part of the rotation correction using optical correction, the load on the camera control unit 140, which performs other corrections by electronic correction, can be reduced.

[0145] (Fifth variation) In the embodiments described above, a lens-interchangeable digital camera was explained as an example of an imaging device. However, the imaging device according to this disclosure may be a digital camera with a built-in lens, rather than a lens-interchangeable digital camera. Furthermore, the imaging device according to this disclosure is not limited to a digital camera, but may be any electronic device such as a movie camera, a mobile phone with a camera, a smartphone, or a tablet terminal.

[0146] (Sixth variation) The above embodiment 2 described the manifestation of blur. Even if the camera body 100 is shaken at the same rate, the magnitude of image blur will differ if the focal length of the lens is different. Therefore, whether or not the correction performance of the digital camera 1 is sufficient depends on the focal length. For example, the longer the focal length, the greater the amount of image blur in the yaw and pitch directions on the image plane, making the effect of blur more likely to become apparent. To suppress blur, it is possible to increase the amount of optical correction in the yaw and pitch directions as the focal length increases. To achieve this, it is useful to decrease the amount of optical correction in the roll direction as the focal length increases. Conversely, the shorter the focal length, the more optical correction in the roll direction may be increased.

[0147] Therefore, if the focal length of the digital camera 1 is shorter than a predetermined threshold, rotational blur correction may be performed solely by optical correction. That is, if the focal length is shorter than a predetermined threshold, the camera control unit 140 may perform rotational blur correction solely by optical correction, and if the focal length is greater than or equal to the predetermined threshold, it may determine the ratio of the optical correction amount to the EIS correction amount according to the focal length.

[0148] (Seventh variation) Furthermore, even if the camera body 100 has the same shake, the magnitude of image blur will differ if the anamorphic magnification β of the anamorphic lens 270 is different. When image blur is large, the effect of blur is more likely to become apparent. Therefore, the digital camera 1 may allocate the amount of image blur correction between the optical correction amount and the EIS correction amount based on the anamorphic magnification β.

[0149] Figure 16 is a flowchart illustrating the distribution process of the blur correction amount in this modified example. Compared with the optical correction process of Embodiment 2 shown in Figure 12, the optical correction process in Figure 16 includes steps S301 and S302 instead of steps S201 and S202.

[0150] In step S301, the camera control unit 140 acquires information indicating the anamorphic magnification β of the anamorphic lens 270. For example, when an interchangeable lens 200 including the anamorphic lens 270 is attached to the body mount 150, the lens control unit 240 may transmit information indicating the anamorphic magnification β to the camera control unit 140. The camera control unit 140 may also automatically acquire information indicating the anamorphic magnification β from the interchangeable lens 200. The information indicating the anamorphic magnification β may be made available for manual input by the user via a menu screen or dial operation.

[0151] The camera control unit 140 allocates the image stabilization amount between the optical correction amount and the EIS correction amount based on the anamorphic magnification β acquired in step S302 (S302). For example, the camera control unit 140 changes the ratio between the optical correction amount and the EIS correction amount according to the anamorphic magnification β. As the anamorphic magnification β increases, the horizontal focal length decreases, so for example, the camera control unit 140 decreases the EIS ratio as the anamorphic magnification β increases.

[0152] This reduces the effect of blur in the image.

[0153] (Example of form) The following are examples of the aspects of this disclosure.

[0154] <Aspect 1> An image sensor that captures an image of a subject formed through an optical system and generates image data, An optical correction unit that corrects image blur by moving the image sensor in a plane perpendicular to the optical axis of the optical system, An electronic correction unit that performs image blur correction by image processing on the aforementioned image data, A control unit that controls the optical correction unit and the electronic correction unit, An imaging device comprising, The optical system compresses the subject image more in a first direction perpendicular to the optical axis than in a second direction. The optical correction unit corrects rotational wobble that occurs in the rotational direction around the optical axis, The electronic correction unit corrects the shear distortion caused by the compression of the subject image by the optical system while the rotational shake has been corrected by the optical correction unit. Imaging device.

[0155] <Aspect 2> The imaging apparatus according to embodiment 1, wherein the control unit changes the ratio between the optical correction amount by the optical correction unit and the electronic correction amount by the electronic correction unit according to the exposure time in imaging of the image sensor.

[0156] <Aspect 3> The imaging apparatus according to embodiment 2, wherein the control unit reduces the ratio of the electronic correction amount to the sum of the optical correction amount and the electronic correction amount as the exposure time increases.

[0157] <Aspect 4> The imaging device is an imaging device according to any one of embodiments 1 to 3, which is capable of correcting rotational shake of a magnitude of 1° or more.

[0158] <Aspect 5> The optical correction unit corrects at least a portion of the rotational wobble, The electronic correction unit corrects the shear distortion corresponding to the rotational shake without rotating the subject image, for the portion of the rotational shake corrected by the optical correction unit. An imaging device according to any one of embodiments 1 to 4.

[0159] <Aspect 6> The imaging apparatus according to any one of embodiments 1 to 4, wherein the electronic correction unit corrects the shear strain without correcting the rotational shake.

[0160] <Aspect 7> The imaging apparatus according to any one of embodiments 1 to 6, wherein the optical system includes an anamorphic lens.

[0161] <Aspect 8> The optical correction unit rotates the image sensor in the rotational direction at an angle θ represented by equation (1). opt By rotating only that much, the aforementioned rotational wobble is corrected. θ opt =tan -1 (βtanθ) ···(1) In equation (1), θ is the rotation angle of the rotational shake, and β is the anamorphic magnification of the anamorphic lens. The imaging apparatus described in Embodiment 7.

[0162] <Pattern 9> The imaging apparatus according to any one of embodiments 1 to 8, wherein the electronic correction unit performs the image processing based on the result of the rotational shake being corrected by the optical correction unit to correct the shear strain.

[0163] <Aspect 10> The optical correction unit corrects the rotational wobble in a first cycle, The electronic correction unit performs the image processing in a second cycle to correct the shear strain. The first period is shorter than the second period. An imaging device according to any one of embodiments 1 to 9.

[0164] <Aspect 11> The second period is the frame period during which the image sensor generates image data for each frame. The electronic correction unit performs the image processing by changing the amount of shear strain correction within the frame period range according to the result of the rotational shake correction by the optical correction unit. The imaging apparatus described in embodiment 10.

[0165] <Aspect 12> The imaging device further includes a connection section for detachably connecting interchangeable lenses. The optical system includes the interchangeable lens, An imaging device according to any one of embodiments 1 to 11.

[0166] <Aspect 13> The control unit, If the interchangeable lens connected to the connection part does not include the optical system, the optical correction unit is made to correct the rotational shake without the electronic correction unit correcting the shear distortion. If the interchangeable lens connected to the connection part includes the optical system, the optical correction unit is made to correct the rotational shake, and the electronic correction unit is made to correct the shear strain. The imaging apparatus described in embodiment 12. [Industrial applicability]

[0167] The concept of this disclosure can be applied to electronic devices with imaging capabilities that include image stabilization (imaging devices such as digital cameras and camcorders, mobile phones, smartphones, etc.). [Explanation of Symbols]

[0168] 1 Digital camera 100 Camera Body 110 Image Sensor 140 Camera Control Unit 141 RAM 142 Flash Memory 143 EIS Processing Unit 146 Correction distribution setting unit 181 Sensor drive unit 183 BIS Processing Unit 184 Gyroscope Sensor 200 interchangeable lenses 220 OIS lens 221 OIS drive unit 223 OIS Processing Unit 224 Gyroscope Sensor 270 Anamorphic Lenses

Claims

1. An image sensor that captures an image of a subject formed through an optical system and generates image data, An optical correction unit that corrects image blur by moving the image sensor in a plane perpendicular to the optical axis of the optical system, An electronic correction unit that performs image blur correction by image processing on the aforementioned image data, A control unit that controls the optical correction unit and the electronic correction unit, An imaging device comprising, The optical system compresses the subject image more in a first direction perpendicular to the optical axis than in a second direction. The optical correction unit corrects rotational wobble that occurs in the rotational direction around the optical axis, The electronic correction unit corrects the shear distortion caused by the compression of the subject image by the optical system while the rotational shake has been corrected by the optical correction unit. Imaging device.

2. The imaging apparatus according to claim 1, wherein the control unit changes the ratio between the optical correction amount by the optical correction unit and the electronic correction amount by the electronic correction unit according to the exposure time in imaging of the image sensor.

3. The imaging apparatus according to claim 2, wherein the control unit reduces the ratio of the electronic correction amount to the sum of the optical correction amount and the electronic correction amount as the exposure time increases.

4. The imaging device according to claim 1, wherein the imaging device is capable of correcting rotational shake of 1° or more in magnitude.

5. The optical correction unit corrects at least a portion of the rotational wobble, The electronic correction unit corrects the shear distortion corresponding to the rotational shake without rotating the subject image, for the portion of the rotational shake corrected by the optical correction unit. The imaging apparatus according to claim 1.

6. The imaging apparatus according to claim 1, wherein the electronic correction unit corrects the shear strain without correcting the rotational shake.

7. The imaging apparatus according to claim 1, wherein the optical system includes an anamorphic lens.

8. The optical correction unit rotates the image sensor in the rotational direction at an angle θ represented by equation (1). ｏｐｔ By rotating only that much, the aforementioned rotational wobble is corrected. i ｏｐｔ =tan －１ (betanth) ・・・(1) In equation (1), θ is the rotation angle of the rotational wobble, and β is the anamorphic magnification of the anamorphic lens. The imaging apparatus according to claim 7.

9. The imaging apparatus according to claim 1, wherein the electronic correction unit performs the image processing based on the result of the rotational shake being corrected by the optical correction unit to correct the shear strain.

10. The optical correction unit corrects the rotational wobble in a first cycle, The electronic correction unit performs the image processing in a second cycle to correct the shear strain. The first period is shorter than the second period. The imaging apparatus according to claim 1.

11. The second period is the frame period during which the image sensor generates image data for each frame. The electronic correction unit performs the image processing by changing the amount of shear strain correction within the frame period range according to the result of the rotational shake correction by the optical correction unit. The imaging apparatus according to claim 10.

12. The imaging device further includes a connection section for detachably connecting interchangeable lenses. The optical system includes the interchangeable lens, The imaging apparatus according to claim 1.

13. The control unit, If the interchangeable lens connected to the connection part does not include the optical system, the optical correction unit is made to correct the rotational shake without the electronic correction unit correcting the shear distortion. If the interchangeable lens connected to the connection part includes the optical system, the optical correction unit is made to correct the rotational shake, and the electronic correction unit is made to correct the shear strain. The imaging apparatus according to claim 12.

Images