Imaging apparatus

Abstract

To provide an imaging apparatus capable of facilitating correction even when a camera-shake remarkably occurs.SOLUTION: An imaging apparatus includes an imaging element for generating image data by imaging a subject image through an optical system, an optical image blur correction part including a first image blur correction part for performing image blur correction by moving a correction lens included in the optical system in a plane perpendicular to the optical axis of the optical system and / or a second image blur correction part for performing the image blur correction by moving the imaging element in the plane, a third image blur correction part for performing the image blur correction by adjusting an area to be cut out for producing an image in the image data, and a control part for controlling the image blur correction by the respective image blur correction parts. The control part sets a blur correction amount by the optical image blur correction part with respect to the shake amount of the imaging apparatus and sets the blur correction amount by the third image blur correction part on the basis of an error amount indicating a difference between the blur correction amount and the movement amount of the correction lens or the imaging element during the image blur correction by the optical image blur correction part. The third image blur correction part performs the image blur correction with the set blur correction amount.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to an imaging device having an image stabilization function. [Background technology]

[0002] Patent Document 1 discloses an imaging device that uses angular velocity sensors provided on an interchangeable lens and a camera body to calculate a correction amount for driving each image blur correction unit. In this imaging device, the interchangeable lens transmits a first correction amount calculated based on the shake of the imaging device to the camera body. The camera body acquires a correction error of the second image blur correction unit of the camera body based on a communication delay of the first correction amount. Then, the camera body performs electronic image blur correction (by image processing) by the electronic image blur correction unit based on the correction error. This achieves the objective of reducing the correction error caused by a communication delay of the first correction amount in the camera body that corrects image blur using the first correction amount received from the interchangeable lens. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-125890 A Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides an imaging device that can easily correct camera shake even when it occurs significantly. [Means for solving the problem]

[0005] An imaging device according to an embodiment of the present disclosure includes an imaging element, an optical image blur correction unit, a third image blur correction unit, and a control unit. The imaging element captures an object image formed through an optical system to generate image data. The optical image blur correction unit includes at least one of a first image blur correction unit that performs image blur correction by moving a correction lens included in the optical system in a plane perpendicular to the optical axis of the optical system, and a second image blur correction unit that performs image blur correction by moving the imaging element in a plane perpendicular to the optical axis. The third image blur correction unit performs image blur correction by adjusting an area from which an image is cut out in the image data. The control unit controls the image blur correction by the optical image blur correction unit and the third image blur correction unit. The control unit sets the amount of blur correction by the optical image blur correction unit relative to the amount of blur of the imaging device. The control unit sets the amount of blur correction to be performed by the third image blur correction unit based on an error amount indicating a difference between the set amount of blur correction to be performed by the optical image blur correction unit and an amount of movement of the correction lens or the image sensor during image blur correction by the optical image blur correction unit. The third image blur correction unit performs image blur correction with the set amount of blur correction.

[0006] An imaging device according to another aspect of the present disclosure includes an imaging element, an optical image blur correction unit, a control unit, and an output unit. The imaging element captures an image of a subject formed through an optical system to generate image data. The optical image blur correction unit includes at least one of a first image blur correction unit that performs image blur correction by moving a correction lens included in the optical system within a plane perpendicular to the optical axis of the optical system, and a second image blur correction unit that performs image blur correction by moving the imaging element within a plane perpendicular to the optical axis. The control unit controls the image blur correction by the optical image blur correction unit. The output unit outputs data. The control unit sets the amount of blur correction by the optical image blur correction unit relative to the amount of blur of the imaging device. The control unit associates the amount of blur correction by the optical image blur correction unit that has been set, or an error amount indicating a difference between the amount of blur correction and the amount of movement of the correction lens or the imaging element during image blur correction by the optical image blur correction unit, with image data, and outputs the image blur correction by the output unit. Effect of the Invention

[0007] According to the imaging device of the present disclosure, camera shake can be easily corrected even if it occurs significantly. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a perspective view of a digital camera according to a first embodiment of the present disclosure. [Diagram 2] FIG. 1 is a block diagram showing a configuration of a digital camera according to a first embodiment. [Diagram 3] FIG. 1 is a block diagram showing the configuration of a BIS processing unit and an EIS processing unit in a digital camera according to a first embodiment. [Figure 4] FIG. 1 is a block diagram showing the configuration of an OIS processing unit in a digital camera according to a first embodiment. [Diagram 5] FIG. 1 is a diagram for explaining an EIS function in a digital camera according to a first embodiment; [Figure 6] 1 is a flowchart illustrating an example of the overall operation of image stabilization of a digital camera according to a first embodiment; [Figure 7] 11 is a flowchart illustrating the operation of a digital camera according to a second embodiment. [Figure 8] 11 is a flowchart illustrating a process for setting an EIS correction amount in a digital camera according to a second embodiment of the present invention; [Figure 9] FIG. 11 is a diagram for explaining an imaging system according to a third embodiment. [Figure 10] FIG. 11 is a block diagram showing the configuration of a digital camera according to a third embodiment. [Figure 11] FIG. 11 is a block diagram showing the configuration of a PC in the imaging system according to the third embodiment. [Figure 12] 11 is a flowchart illustrating the operation of a digital camera according to a third embodiment. [Figure 13] FIG. 13 is a diagram for explaining the operation of a digital camera according to a modified example of the first embodiment. [Figure 14] FIG. 13 is a block diagram showing the configuration of a simulation unit in an imaging system according to a modified example of the third embodiment. [Figure 15] 13 is a flowchart illustrating a process for constructing a correction model in an imaging system according to a modified example of the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings as appropriate. However, in the detailed description, unnecessary parts of the description of the prior art and substantially the same configuration may be omitted. This is for the purpose of simplifying the description. In addition, the following description and the accompanying drawings are disclosed so that a person skilled in the art can fully understand the present disclosure, and are not intended to limit the subject matter of the claims.

[0010] (Embodiment 1) In the first embodiment, a digital camera having three types of image stabilization functions will be described as an example of an imaging apparatus.

[0011] 1. Configuration Fig. 1 is a perspective view of a digital camera 1 according to embodiment 1. Fig. 2 is a block diagram showing the configuration of the digital camera 1 according to embodiment 1. The digital camera 1 is an example of an imaging device composed of a camera body 100 and an interchangeable lens 200 that is detachable from the camera body 100.

[0012] In the following description, the function of correcting blur by moving the correction lens in the interchangeable lens 200 is referred to as the "OIS (Optical Image Stabilizer) function." Also, the function of correcting blur by moving the image sensor in the camera body 100 is referred to as the "BIS (Body Image Stabilizer) function." The function of correcting blur by adjusting the area cut out of image data by the image sensor is referred to as the "EIS (Electronic Image Stabilizer) function."

[0013] In the following description, the rotational directions corresponding to the horizontal and vertical directions of the imaging element in digital camera 1 are referred to as the yaw direction and pitch direction, respectively, and the rotational direction about the rotation axis along the optical axis of digital camera 1 is referred to as the roll direction (see Figure 1).

[0014] 1-1.Camera body Camera body 100 includes an image sensor 110, a liquid crystal monitor 120, an operation unit 130, a camera control unit 140, a RAM 141, a flash memory 142, a body mount 150, a card slot 170, and a shutter 180. Camera body 100 also includes, as a functional component of camera control unit 140, an EIS processing unit 143 that implements an EIS function.

[0015] The camera control unit 140 controls the overall operation of the digital camera 1 by controlling components such as the image sensor 110 in response to instructions from the release button. The camera control unit 140 transmits a vertical synchronization signal to a timing generator (TG) 112. In parallel with this, the camera control unit 140 generates an exposure synchronization signal. The camera control unit 140 periodically transmits the generated exposure synchronization signal to the lens control unit 240 via the body mount 150 and the lens mount 250. The camera control unit 140 uses the RAM 141 as a working memory during control operations and image processing operations.

[0016] The image sensor 110 is an example of an imaging element that captures an image of a subject incident via the interchangeable lens 200 and generates image data. The image sensor 110 is, for example, a CCD, a CMOS image sensor, or an NMOS image sensor. The generated image data is digitized by an AD converter (ADC) 111. The digitized image data is subjected to predetermined image processing by the camera control unit 140. The predetermined image processing is, for example, gamma correction processing, white balance correction processing, scratch correction processing, YC conversion processing, electronic zoom processing, and / or JPEG compression processing.

[0017] The image sensor 110 operates at a timing controlled by a timing generator 112. The image sensor generates still images, moving images, or through images for recording. The through images are mainly moving images, and are displayed on the liquid crystal monitor 120 so that the user can decide the composition for capturing a still image.

[0018] The liquid crystal monitor 120 displays various information such as images such as through images and menu screens. The liquid crystal monitor 120 is an example of a display unit in this embodiment. Instead of the liquid crystal monitor, other types of display devices, for example, an organic EL display device, may be used.

[0019] The operation unit 130 includes various operation members such as a release button for instructing the start of shooting, a mode dial for setting a shooting mode, a power switch, etc. The operation unit 130 also includes a touch panel arranged so as to be superimposed on the liquid crystal monitor 120.

[0020] The RAM 141 is a recording medium that functions as a work memory for the camera control unit 140. The RAM 141 is realized by a DRAM (Dynamic Random Access Memory) or the like. The RAM 141 temporarily stores (i.e., holds or buffers) image data generated by the image sensor 110 and various setting information for the digital camera 1, for example. The flash memory 142 is a non-volatile recording medium. For example, the flash memory 142 stores predetermined setting values ​​for the digital camera 1, etc. Each of the RAM 141 and the flash memory 142 is an example of a storage unit of the digital camera 1 in this embodiment.

[0021] The card slot 170 is capable of receiving a memory card 171, and controls the memory card 171 under control of the camera control unit 140. The digital camera 1 is capable of storing image data in the memory card 171 and reading image data from the memory card 171.

[0022] Shutter 180 adjusts the exposure time of light incident on image sensor 110. Shutter 180 is driven by a drive system such as a DC motor or a stepping motor in accordance with a control signal issued from camera control unit 140. For example, camera control unit 140 can control the drive speed (shutter speed or continuous shooting speed) at which shutter 180 is driven.

[0023] The body mount 150 can be mechanically and electrically connected to the lens mount 250 of the interchangeable lens 200. The body mount 150 can transmit and receive data to and from the interchangeable lens 200 via the lens mount 250. The body mount 150 transmits an exposure synchronization signal received from the camera control unit 140 to the lens control unit 240 via the lens mount 250. In addition, the body mount 150 transmits other control signals received from the camera control unit 140 to the lens control unit 240 via the lens mount 250. In addition, the body mount 150 transmits signals received from the lens control unit 240 to the camera control unit 140 via the lens mount 250.

[0024] Moreover, as components for realizing the BIS function, camera body 100 further includes a gyro sensor 184 (shake detection unit) that detects shake of camera body 100, and a BIS processing unit 183 that controls shake correction processing based on the detection result of gyro sensor 184. Furthermore, camera body 100 includes a sensor driving unit 181 that moves image sensor 110, and a position sensor 182 that detects the position of image sensor 110.

[0025] The sensor driving unit 181 can be realized by, for example, a magnet and a flat coil. The sensor driving unit 181 may include other motors, actuators, etc. The position sensor 182 is a sensor that detects the position of the image sensor 110 in a plane perpendicular to the optical axis of the optical system. The position of the image sensor 110 can be defined as a position on a two-dimensional plane perpendicular to the optical axis. For example, the position of the image sensor 110 can be defined as a displacement amount along each of two orthogonal axes (X axis, Y axis) from a predetermined reference position of the image sensor 110. The position sensor 182 can be realized by, for example, three sets of magnets and Hall elements. For example, two of the three sets are arranged in the X-axis direction and one in the Y-axis direction (or two sets in the Y-axis direction and one in the X-axis direction), and each detects the position in the X-axis direction or the Y-axis direction. The position sensor 182 detects each position of the image sensor 110 in the X-axis direction, the Y-axis direction, and the rotation direction by the rotation axis along the optical axis with respect to the reference position from the relationship of each detection result.

[0026] In the position sensor 182, each Hall element is attached to the image sensor 110, and each magnet is fixedly disposed with respect to the camera body 100. For example, a pair of a magnet and a Hall element that are paired in the X-axis direction is considered. The Hall element detects a magnetic flux density according to a relative position with respect to the magnet. By preparing a correspondence relationship between the magnitude of the magnetic flux density and the relative position in advance, it is possible to obtain a relative position according to the magnitude of the magnetic flux density detected by the Hall element. The obtained relative position represents the position of the image sensor 110 in the X-axis direction. For example, the position of the image sensor 110 can be obtained in the Y-axis direction and the rotation direction in the same manner. Thereby, for example, the position sensor 182 outputs a signal (also called a "position signal") indicating the detected position of the image sensor 110 to the BIS processing unit 183.

[0027] The BIS processing unit 183 controls the sensor driving unit 181 based on the signal from the gyro sensor 184 and the signal from the position sensor 182 to shift the image sensor 110 in a plane perpendicular to the optical axis so as to offset the shake of the camera body 100. There is a mechanical limit to the range in which the image sensor 110 can be driven by the sensor driving unit 181. The range in which the image sensor 110 can be driven by the sensor driving unit 181 in the BIS function is called the "element driving range."

[0028] 1-2.Interchangeable lenses The interchangeable lens 200 includes an optical system, a lens control unit 240, and a lens mount 250. The optical system includes a zoom lens 210, an OIS (Optical Image Stabilizer) lens 220, a focus lens 230, and an aperture 260.

[0029] The zoom lens 210 is a lens for changing the magnification of a subject image formed by the optical system. The zoom lens 210 is composed of one or more lenses. The zoom lens 210 is driven by a zoom driving unit 211. The zoom driving unit 211 includes a zoom ring that can be operated by the user. Alternatively, the zoom driving unit 211 may include a zoom lever and an actuator or a motor. The zoom driving unit 211 moves the zoom lens 210 along the optical axis direction of the optical system in response to an operation by the user.

[0030] The focus lens 230 is a lens for changing the focus state of a subject image formed on the image sensor 110 by the optical system. The focus lens 230 is composed of one or more lenses. The focus lens 230 is driven by a focus driver 233.

[0031] The focus driver 233 includes an actuator or a motor, and moves the focus lens 230 along the optical axis of the optical system under the control of the lens controller 240. The focus driver 233 can be realized by a DC motor, a stepping motor, a servo motor, an ultrasonic motor, or the like.

[0032] The OIS lens 220 is a lens for correcting blur of a subject image formed by the optical system of the interchangeable lens 200 in the OIS function. The OIS lens 220 reduces blur of the subject image on the image sensor 110 by moving in a direction that offsets blur of the digital camera 1. The OIS lens 220 is composed of one or more lenses. The OIS lens 220 is driven by an OIS driver 221.

[0033] The OIS driving unit 221 shifts the OIS lens 220 in a plane perpendicular to the optical axis of the optical system under the control of the OIS processing unit 223. There is a mechanical limit to the range in which the OIS lens 220 can be driven by the OIS driving unit 221. The range in which the OIS lens 220 can be driven by the OIS driving unit 221 is called the "lens driving range". The OIS driving unit 221 can be realized, for example, by a magnet and a flat coil. The position sensor 222 is a sensor that detects the position of the OIS lens 220 in a plane perpendicular to the optical axis of the optical system, for example, by the same principle as the position sensor 182 of the camera body 100. The position sensor 222 can be realized, for example, by a magnet and a Hall element. The OIS processing unit 223 controls the OIS driving unit 221 based on the output of the position sensor 222 and the output of a gyro sensor 224 (shake detection unit).

[0034] The diaphragm 260 adjusts the amount of light incident on the image sensor 110. The diaphragm 260 is driven by an diaphragm driving unit 262, and the size of the opening is controlled. The diaphragm driving unit 262 includes a motor or an actuator.

[0035] The gyro sensor 184 or 224 detects shake (vibration) in the yaw direction, pitch direction, and roll direction based on the angle change per unit time of the digital camera 1, i.e., the angular velocity. The gyro sensor 184 or 224 outputs an angular velocity signal indicating the amount of shake (angular velocity) detected to the BIS processing unit 183 or OIS processing unit 223. The angular velocity signal output by the gyro sensor 184 or 224 may include a wide range of frequency components caused by camera shake, mechanical noise, and the like. Instead of the gyro sensor, another sensor capable of detecting shake of the digital camera 1 may be used. Also, the gyro sensor 224 of the interchangeable lens 200 does not need to detect shake in the roll direction.

[0036] The camera control unit 140, the lens control unit 240, the OIS processing unit 223, and the BIS processing unit 183 may be configured with a hardwired electronic circuit, or may be configured with a microcomputer using a program. For example, the camera control unit 140 and the lens control unit 240 can be realized by various processors such as a CPU, an MPU, a GPU, a DSU, an FPGA, or an ASIC. The camera control unit 140, the lens control unit 240, the OIS processing unit 223, and the BIS processing unit 183 are each an example of a control unit in the digital camera 1 of this embodiment.

[0037] 1-3. Image stabilization function configuration The configuration for realizing various image stabilization functions of digital camera 1 in this embodiment will be described with reference to FIGS.

[0038] Fig. 3 is a block diagram showing the configurations of the BIS processing unit 183 and the EIS processing unit 143 in the digital camera 1 of this embodiment. Fig. 4 is a block diagram showing the configuration of the OIS processing unit 223 in the digital camera 1.

[0039] 3, the digital camera 1 of this embodiment includes, for example, an EIS processing unit 143 that implements an EIS function, and a correction allocation setting unit 146 that sets among various image stabilization functions, as functional components of the camera control unit 140. The correction allocation setting unit 146 of this embodiment sets a correction allocation including an OIS ratio indicating the allocation of the OIS processing unit 223, a BIS ratio indicating the allocation of the BIS processing unit 183, and an EIS ratio indicating the allocation of the EIS processing unit 143, as a ratio for allocating the amount of blur correction in advance. In this embodiment, the digital camera 1 sets the amount of blur correction (also referred to as a "correction amount") by the various image stabilization functions, for example, by the correction allocation setting unit 146 and the processing units 183, 223, 143.

[0040] 1-3-1.OIS processing unit For example, the correction allocation setting section 146 in the camera control section 140 transmits information indicating the OIS ratio to the lens control section 240 of the interchangeable lens 200 via each of the mounts 150 and 250 , and sets the OIS processing section 223 .

[0041] 4, the configuration of the OIS processing unit 223 in the interchangeable lens 200 will be described. The OIS processing unit 223 includes a filter 306, a phase compensation unit 307, an integrator 308, a multiplier 309, and a PID control unit 310.

[0042] The filter 306 applies various types of filtering to the signal received from the gyro sensor 224. The filter 306 is composed of a high pass filter (HPF), a low pass filter (LPF), and / or a band pass filter (BPF), and cuts off a predetermined low frequency component included in the signal by the HPF in order to cut off, for example, a drift component.

[0043] The phase compensation unit 307 compensates for the phase delay of the signal received from the filter 306, the phase delay being caused by the OIS driver 221 and the like.

[0044] The integrator 308 integrates the signal indicating the angular velocity of the shake (vibration) input from the phase compensation unit 307 to generate a shake detection signal indicating the angle of the shake (vibration). The shake detection signal from the integrator 308 is input to a PID control unit 310 via a multiplier 309. The OIS processing unit 223 may use a filter configuration other than the above configuration or may add a filter configuration such as a notch filter for noise processing.

[0045] Multiplier 309 multiplies the blur detection signal input from integrator 308 by a gain indicating an OIS ratio set by lens control unit 240, for example, to calculate an OIS correction amount as a blur correction amount by the OIS function. The OIS ratios in the yaw and pitch directions are set to a value between "0" and "1" as a gain for the blur detection signal in each direction. The OIS ratio may be set for each direction.

[0046] PID control unit 310 performs PID control based on the difference between the OIS correction amount from multiplier 309 and position information of OIS lens 220 indicated by a signal from position sensor 222, and generates a drive signal for OIS drive unit 221. OIS drive unit 221 drives OIS lens 220 based on the drive signal.

[0047] In this embodiment, for example, in response to a request from the camera control unit 140, the lens control unit 240 acquires from the PID control unit 310 the deviation between the OIS correction amount in PID control and the position signal from the position sensor 222 as the OIS error amount. The OIS error amount indicates the difference between the OIS correction amount and the movement amount of the OIS lens 220 indicated by the position information of the OIS lens 220 driven according to the OIS correction amount. The lens control unit 240 transmits the OIS error amount to the camera control unit 140 via each of the mounts 150 and 250. The error amount will be described in detail later.

[0048] 1-3-2.BIS Processing Section 3, the configuration of the BIS processing unit 183 in the camera body 100 will be described. The BIS processing unit 183 includes a filter 406, a phase compensation unit 407, an integrator 408, a multiplier 409, and a PID control unit 410.

[0049] The filter 406 applies various types of filtering to the signal received from the gyro sensor 184. The filter 406 is configured with an HPF, an LPF, and / or a BPF, similar to the filter 306 of the OIS processing unit 223, for example.

[0050] The phase compensation unit 407 compensates for the phase delay of the signal received from the filter 406, the phase delay being caused by the sensor driving unit 181 and the like.

[0051] The integrator 408 integrates the signal indicating the angular velocity of the shake (vibration) input from the phase compensation unit 407 to generate a signal indicating the angle of the shake (vibration) (hereinafter referred to as a "shake detection signal"). The shake detection signal from the integrator 408 is input to a PID control unit 410 via a multiplier 409. Here, the BIS processing unit 183 may use a filter configuration other than the above configuration or may add a filter configuration such as a notch filter for noise processing.

[0052] The multiplier 409 multiplies the shake detection signal input from the integrator 408 by a gain indicating a BIS ratio set in the correction distribution setting unit 146, for example, to calculate a BIS correction amount as a shake correction amount by the BIS function. The BIS ratio is set, for example, separately for the gains for the shake detection signals in the yaw and pitch directions and the gain for the shake detection signal in the roll direction. In addition to or instead of this, separate gains may be set in the multiplier 409 between the shake detection signal in the yaw direction and the shake detection signal in the pitch direction.

[0053] Based on the output from the position sensor 182 and the output from the multiplier 409, the PID control unit 410 generates a drive signal for shifting the image sensor 110 and outputs the drive signal to the sensor drive unit 181. The sensor drive unit 181 drives the image sensor 110 based on the drive signal. For example, the camera control unit 140 acquires the deviation input in the PID control from the PID control unit 410 as a BIS error amount. The BIS error amount indicates the difference between the BIS correction amount and the amount of movement indicated by the position information of the image sensor 110 driven according to the BIS correction amount by the image stabilization operation of the BIS function. The deviation of the PID control is calculated, for example, by the PID control unit 410 based on the BIS correction amount and a position signal from the position sensor 182.

[0054] 1-3-3.EIS processing section The EIS processing unit 143 will be described with reference to Figures 3 and 5. Figure 5 is a diagram for explaining the EIS function in the digital camera 1.

[0055] The EIS processing unit 143 of this embodiment includes, for example, a multiplier 144, an adder 147, and an image cropping unit 145 as shown in Fig. 3. In the EIS processing unit 143, a cropping amount Eo is set as shown in Fig. 5 by a user's operation on the operation unit 130 using, for example, a menu screen of the digital camera 1.

[0056] The multiplier 144 multiplies, for example, a gain indicating the EIS ratio by the shake detection signal input from the integrator 408 of the BIS processing unit 183. The EIS ratio is set, for example, separately between the gains for the shake detection signals in the yaw and pitch directions and the gain for the shake detection signal in the roll direction. In addition to or instead of this, separate gains may be set in the multiplier 144 between the shake detection signal in the yaw direction and the shake detection signal in the pitch direction.

[0057] The EIS processing unit 143 calculates a blur correction amount based on the blur detection signal input via the multiplier 144. The adder 147 calculates an EIS correction amount as a blur correction amount by the EIS function by adding, for example, a BIS error amount acquired from the BIS processing unit 183 and an OIS error amount acquired from the OIS processing unit 223 by the lens control unit 240 to the calculated blur correction amount. Hereinafter, the OIS error amount and the BIS error amount are also collectively referred to as "error amount".

[0058] 5, the image cropping unit 145 performs a process of cropping an image of a narrow region by a preset cropping amount Eo from the entire image in the image data 20 generated by the image sensor 110. For example, various image processes for recording the shooting results are performed on the cropped image data. For example, electronic zoom processing may be performed so that the cropped image has the same size as the image before cropping.

[0059] The EIS processing unit 143 realizes the EIS function by adjusting the position at which the image is cut out by the image cutout unit 145 by the calculated EIS correction amount. For example, a reference region 21 in which the image cut out by the image cutout unit 145 is located when the EIS correction amount is zero is set based on the center position of the entire image in the image data 20. The reference region 21 is arranged along the horizontal direction X and vertical direction Y of the image data 20, for example.

[0060] The EIS processing unit 143 translates the image area 22 cut out by the image cutout unit 145 from the reference area 21 in the horizontal direction X in accordance with the amount of blur correction in the yaw direction, for example. Similarly, the EIS processing unit 143 translates the image area 22 in the vertical direction Y in accordance with the amount of blur correction in the pitch direction. Also, the EIS processing unit 143 rotates the orientation of the image area 22 from the orientation of the reference area 21 on the XY plane in accordance with the amount of blur correction in the roll direction.

[0061] The position adjustment of the image region 22 as described above, i.e., the EIS function, can be performed within the range of the cut-out amount Eo. Specifically, the translation of the image region 22 is performed within the range of the cut-out amount Eo minus the amount for roll as a margin Er. Furthermore, the rotation of the image region 22 is performed within the range of rotation angles that fall within the margin Er for roll in the cut-out amount Eo. The margin Er for roll is determined by the correction allocation setting unit 146 according to, for example, the lens state of the interchangeable lens 200, etc.

[0062] 2. Operation The operation of the camera shake correction in the digital camera 1 configured as above will now be described.

[0063] In this embodiment, assuming a situation where camera shake is relatively large, such as when a user (photographer) takes a video or the like while walking or running, the digital camera 1 performs image stabilization operation using the OIS processing unit 223 and / or the BIS processing unit 183 in combination with the EIS processing unit 143. In the following, an example will be described in which the OIS processing unit 223 and the BIS processing unit 183 perform image stabilization operation in parallel in the digital camera 1, and the EIS processing unit 143 performs image stabilization operation depending on the results of these operations. The allocation of the amount of shake compensation shared by each of the processing units 223, 183, 143 (i.e., compensation allocation) is set in advance, for example.

[0064] For example, the OIS processing unit 223 (FIG. 4) generates blur detection signals in the pitch and yaw directions in an integrator 308 based on the detection result of the gyro sensor 224 of the interchangeable lens 200. The OIS processing unit 223 calculates an OIS correction amount based on the blur detection signal and the OIS ratio in a multiplier 309. The OIS processing unit 223 controls the OIS driving unit 221 by a driving signal from a PID control unit 310 to drive the OIS lens 220 so as to correct the OIS correction amount for the amount of blur indicated by the calculated blur detection signal.

[0065] The BIS processing unit 183 (FIG. 3) generates shake detection signals in the pitch, yaw, and roll directions in an integrator 408 based on the detection result of the gyro sensor 184 of the camera body 100. For example, the BIS processing unit 183 calculates the BIS correction amount in each direction based on the shake detection signal and the BIS ratio in a multiplier 409. The BIS processing unit 183 causes the sensor driving unit 181 to translate the image sensor 110 in response to a drive signal from the PID control unit 310 so as to correct the calculated BIS correction amount for the shake amount in the pitch and yaw directions indicated by the shake detection signal. Furthermore, the BIS processing unit 183 causes the sensor driving unit 181 to rotate the image sensor 110 in accordance with the calculated BIS correction amount for the shake amount in the roll direction.

[0066] In such optical image stabilization operations, since the OIS lens 220 or image sensor 110, which has mass, is moved, an error may occur between the amount of blur correction indicated by the drive signal from the PID control units 310, 410 and the actual amount of movement of the OIS lens 220 or image sensor 110. For example, when shooting a video or the like, it is considered that the change in the amount of movement may be delayed with respect to the amount of correction sequentially instructed by the PID control units 310, 410, or the amount of movement may be excessive or insufficient. In addition, when the digital camera 1 is subjected to an external shock or the like, an error may occur in which the position of the OIS lens 220 or image sensor 110 momentarily deviates from the position corresponding to the amount of correction.

[0067] Therefore, the digital camera 1 of this embodiment performs image stabilization by adding the above-mentioned error to the blur correction amount by the EIS processing unit 143. The EIS processing unit 143 (FIG. 3) calculates the EIS correction amount in each direction in the multiplier 144 and the adder 147 based on the blur detection signal calculated from the detection result of the gyro sensor 184, for example, and the OIS error amount and the BIS error amount corresponding to the above-mentioned error. The image area 22 (FIG. 5) cut out by the image cutout unit 145 of the EIS processing unit 143 is translated so that the calculated EIS correction amount is corrected for the blur amount in the pitch and yaw directions. For the blur amount in the roll direction, the EIS processing unit 143 causes the image cutout unit 145 to rotate the image area 22 according to the calculated EIS correction amount.

[0068] According to the above-described operation of the digital camera 1, even if an error occurs between the amount of blur correction calculated based on the detected amount of blur and the actual amount of movement of the OIS lens 220 or the image sensor 110, the EIS processing unit 143 can perform image stabilization operation with an amount of blur correction including the error. This makes it possible to achieve image stabilization that reduces blurring in an image captured by the digital camera 1 as a whole, even in a situation where significant camera shake is likely to occur, such as taking a picture while running.

[0069] 2-1. Overall operation Digital camera 1 of this embodiment performs image stabilization operation so as to correct errors in the OIS function and BIS function as described above using the EIS function. The overall operation of image stabilization in digital camera 1 of this embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart illustrating the overall operation of image stabilization in digital camera 1 of embodiment 1.

[0070] Each process shown in the flowchart of FIG. 6 is executed in parallel with operations such as video shooting by the camera control unit 140, the OIS processing unit 223, and the BIS processing unit 183, for example, when the interchangeable lens 200 is attached to the camera body 100.

[0071] The camera control unit 140 acquires information indicating the distribution of corrections by the respective processing units 223, 183, and 143 from the flash memory 142 (S1). For example, a predetermined setting value according to the lens state such as the focal length of the interchangeable lens 200 is stored in advance in the flash memory 142 as information indicating the distribution of corrections.

[0072] In the process of step S1, the camera control unit 140 may perform data communication with the lens control unit 240 of the interchangeable lens 200 via the body mount 150 and the lens mount 250 to acquire lens state data. The camera control unit 140, as the correction allocation setting unit 146, may acquire information on correction allocation according to the lens state indicated by the data. For example, the lens state data includes a focal length according to the zoom state of the interchangeable lens 200 and a focusing position according to the focus state, and is stored in the RAM 241 of the interchangeable lens 200. The lens control unit 240 reads out the above information in response to a request from the camera control unit 140, for example, and transmits it to the camera body 100.

[0073] (1) Image stabilization processing for OIS / BIS function The camera control unit 140, as the correction allocation setting unit 146, sets the OIS ratio and BIS ratio of the acquired correction allocation to the multipliers 409, 309 of the OIS processing unit 223 and the BIS processing unit 183, respectively. The OIS processing unit 223 calculates an OIS correction amount by multiplication of the OIS ratio in the multiplier 309 based on a blur detection signal resulting from the detection result of the gyro sensor 224, and sets the OIS correction amount in the PID control unit 310 (S2). The BIS processing unit 183 calculates a BIS correction amount by multiplication of the BIS ratio in the multiplier 409 based on a blur detection signal resulting from the detection result of the gyro sensor 184, and sets the BIS correction amount in the PID control unit 410 (S2).

[0074] According to the OIS correction amount and BIS correction amount set as described above, the OIS processing unit 223 and the BIS processing unit 183 each perform an image stabilization operation (S3). The OIS processing unit 223 of this embodiment acquires, for example, position information of the OIS lens 220 driven by the correction operation from the position sensor 222, and calculates the difference between the position indicated by the position information and the drive destination position indicated by the OIS correction amount as a deviation to be used for PID control. For example, the OIS processing unit 223 as the PID control unit 310 performs PID control in the process of step S3 on the image of the next frame based on the calculated deviation.

[0075] Also, similar to the above-mentioned OIS processing unit 223, for example, the BIS processing unit 183 acquires position information of the image sensor 110 from the position sensor 182, and calculates the difference between the position indicated by the position information and the destination position indicated by the BIS correction amount as the deviation of PID control. In this embodiment, the deviation calculated by PID control in this manner is used as the error amount, so that the error amount can be calculated efficiently.

[0076] (2) Processing related to image stabilization of EIS function The camera control unit 140 acquires, for example, deviations in the PID control as described above as an OIS error amount and a BIS error amount (S4). For example, the OIS error amount is acquired from the lens control unit 240 as a difference between the OIS correction amount and the movement amount of the OIS lens 220 for each of the pitch and yaw directions. In response to a request from the camera control unit 140, the lens control unit 240 acquires the difference from the PID control unit 310 of the OIS processing unit 223 and transmits it to the camera body 100.

[0077] In step S4, the BIS error amount is acquired from the PID control unit 410 of the BIS processing unit 183 as a difference between the BIS correction amount and the movement amount of the image sensor 110 for each of the pitch, yaw, and roll directions. Each error amount may be a signed value with a different positive or negative sign depending on the magnitude of the OIS or BIS correction amount and the movement amount of the OIS lens 220 or the image sensor 110.

[0078] The camera control unit 140, for example as the EIS processing unit 143, sets an EIS correction amount based on the blur detection signal from the gyro sensor 184, the EIS ratio of the correction allocation acquired in step S1, and the OIS error amount and the BIS error amount (S5). The camera control unit 140 calculates an EIS correction amount reflecting each error amount by adding the OIS error amount and the BIS error amount in the adder 147 to the blur correction amount calculated by multiplying the EIS ratio by the blur detection signal in the multiplier 144, and sets the EIS correction amount in the image cropping unit 145.

[0079] In step S5, the camera control unit 140 converts the OIS error amount and the BIS error amount obtained in units such as angles into the magnitude of the error in the image, i.e., the error amount in pixel units, and sets the EIS correction amount. For example, information indicating a calculation formula for the conversion may be stored in advance in the flash memory 142 or the like.

[0080] According to the EIS correction amount set as described above, including the OIS error amount and the BIS error amount, the EIS processing unit 143 executes the image stabilization operation (S6). As a result, even if an error occurs in the image stabilization operation by the OIS function and the BIS function, it becomes possible to accurately correct image stabilization for the entire digital camera 1 using the EIS function.

[0081] (3) Processing according to lens condition The camera control unit 140 periodically acquires lens state data from the interchangeable lens 200 during, for example, the above-mentioned image stabilization operation (S6), and determines whether the lens state has changed (S7). For example, the lens state may change when the user of the digital camera 1 performs a zoom operation or focus operation on the interchangeable lens 200. The determination in step S7 is performed, for example, with respect to a change in the focal length of the interchangeable lens 200 as the object. The determination in step S7 may also include a change in the focus position.

[0082] When it is determined that the lens state has not changed (NO in S7), the camera control unit 140 returns to step S3 without particularly changing the correction distribution, thereby continuing the image stabilization operation with the OIS correction amount and the BIS correction amount set in step S2.

[0083] On the other hand, when it is determined that the lens state has changed (YES in S7), the camera control unit 140 performs the processes from step S1 onwards again according to the lens state. This makes it possible to obtain a correction distribution according to the new lens state (S1) and optimize the distribution of the blur correction amount in the camera shake correction operation.

[0084] For example, the camera control unit 140 repeatedly executes the above-mentioned processing at a predetermined cycle. The predetermined cycle is, for example, a frame cycle, for example, 1 / 30 to 1 / 60 seconds.

[0085] (4) Effects, etc. According to the above process, image stabilization can be performed so as to suppress blurring in an image captured by the EIS function according to the OIS error amount and the BIS error amount (S3 to S6). For example, in image stabilization using the OIS function or the BIS function, when the movement amount of the OIS lens 220 or the image sensor 110 is large, the image blurring may also be large due to the error. However, even in such a case, the above process can suppress blurring in the image.

[0086] Also, for example, when the movement amount of the OIS lens 220 or the image sensor 110 follows each of the sequentially calculated correction amounts of the OIS and BIS with a delay, it is possible to increase the control gain in the PID control units 310 and 410 to improve the tracking ability of the control. However, such control can cause problems such as a decrease in control stability in the PID control and an increase in noise caused by driving the OIS driving unit 221 or the sensor driving unit 181. According to the digital camera 1 of this embodiment, the influence of errors in the OIS function and the BIS function can be reduced by using the EIS function, and accurate image stabilization can be achieved while avoiding various problems that occur when the gain in the PID control is increased.

[0087] Although an example of obtaining a correction allocation according to the lens state has been described above (S1), the correction allocation may be determined by referring to the element driving range in the BIS function, the lens driving range in the OIS function, the cropping amount Eo in the EIS function, etc. The element driving range may be stored in advance, for example, in the flash memory 142 of the camera body 100, and the lens driving range may be stored in advance, for example, in the flash memory 242 of the interchangeable lens 200. The cropping amount Eo may be stored in advance in the flash memory 142, or may be input by the user from the operation unit 130 using a setting menu of the digital camera 1, etc.

[0088] In the above, an example has been described in which the OIS error amount and the BIS error amount are acquired from the PID control units 310 and 410, respectively (S4). Each error amount is not limited to the deviation in such PID control, but may be calculated based on the blur correction amount input to the PID control units 310 and 410 and the position information of the OIS lens 220 or the image sensor 110. For example, the camera control unit 140 may acquire the OIS correction amount and the position signal from the position sensor 222 in response to a request from the lens control unit 240, and calculate the OIS error amount as the difference between the OIS correction amount and the movement amount of the OIS lens 220 corresponding to the correction amount. Similarly, the BIS error amount may be calculated as the difference between the BIS correction amount and the movement amount of the image sensor 110 according to the position signal from the position sensor 182.

[0089] In the above, an example has been described in which digital camera 1 performs image stabilization operation using both the OIS function and the BIS function in addition to the EIS function. Digital camera 1 may execute only one of the OIS function and the BIS function as an optical image stabilization operation. For example, correction allocation setting unit 146 may set the OIS ratio or the BIS ratio to "0", or digital camera 1 may have only one of the OIS function and the BIS function. In this case, digital camera 1 may have only one of a configuration related to the OIS function in interchangeable lens 200 and a configuration related to the BIS function in camera body 100.

[0090] 6 is executed by the camera control unit 140, the OIS processing unit 223, or the BIS processing unit 183, but, for example, some or all of the above processes may be executed by the lens control unit 240. For example, the lens control unit 240 may calculate the OIS error amount and transmit it to the camera control unit 140 by data communication via the mounts 150 and 250.

[0091] In the above example, the camera control unit 140 calculated the OIS error amount for each of the pitch and yaw directions, and calculated the BIS error amount for each of the pitch, yaw, and roll directions. The camera control unit 140 may calculate each error amount for only some of these directions. Also, in the above example, the EIS processing unit 143 added each error amount to the blur correction amount calculated from the blur detection signal by the adder 147 in setting the EIS correction amount. The EIS processing unit 143 may set the EIS correction amount by subtraction or addition / subtraction, not limited to addition by the adder 147, depending on the signs of various variables such as the blur correction amount and each error amount.

[0092] 3. Summary As described above, the digital camera 1 in this embodiment includes the image sensor 110 (image sensor), an optical image blur correction unit including at least one of the OIS processing unit 223 (first image blur correction unit) and the BIS processing unit 183 (second image blur correction unit), the EIS processing unit 143 (third image blur correction unit), and the camera control unit 140. In this embodiment, the OIS processing unit 223, the BIS processing unit 183, and the camera control unit 140 are an example of a control unit that controls the image blur correction by the optical image blur correction unit and the third image blur correction unit. The image sensor 110 captures an object image formed via an optical system to generate image data. The OIS processing unit 223 performs image blur correction by moving the OIS lens 220 (correction lens) included in the optical system in a plane perpendicular to the optical axis of the optical system including the OIS lens 220. The BIS processing unit 183 performs image blur correction by moving the image sensor 110 in a plane perpendicular to the optical axis. The EIS processing unit 143 performs image blur correction by adjusting an area from which an image is cut out in the image data. The OIS processing unit 223, the BIS processing unit 183, and the camera control unit 140 set an OIS correction amount and a BIS correction amount as an example of a blur correction amount by the optical image blur correction unit for the blur amount of the digital camera 1 (S2). The camera control unit 140 sets an EIS correction amount as an example of a blur correction amount by the EIS processing unit 143 based on an OIS error amount and a BIS error amount as an example of an error amount indicating a difference between the set OIS correction amount and BIS correction amount and the movement amount of the OIS lens 220 or the image sensor 110 during image blur correction by the optical image blur correction unit (S5). The EIS processing unit 143 performs image blur correction with the set EIS correction amount (S6).

[0093] According to the above-described digital camera 1, the EIS processing unit 143 executes image stabilization operation with an EIS correction amount that is set based on the OIS error amount and the BIS error amount, thereby making it easier to perform image stabilization even if significant camera shake occurs.

[0094] In this embodiment, the digital camera 1 includes gyro sensors 224, 184 as an example of a blur detection unit that detects the amount of blur in the digital camera 1. The OIS processing unit 223 or the BIS processing unit 183 performs image blur correction according to the OIS correction amount or the BIS correction amount set in each processing unit 223, 183 for the amount of blur detected by the gyro sensors 224, 184 (S3). As a result, each processing unit 223, 183 executes a blur correction operation according to the set correction amount, thereby making it possible to perform optical blur correction according to the detected amount of blur.

[0095] In this embodiment, the digital camera 1 includes position sensors 222, 182 as an example of a position detection unit that detects the position of the OIS lens 220 or the position of the image sensor 110 in a plane perpendicular to the optical axis. The OIS processing unit 223 or the BIS processing unit 183 acquires at least one of the movement amount of the OIS lens 220 and the movement amount of the image sensor 110 due to the image stabilization operation of the OIS or BIS function based on the detection result by the position sensor 222, 182, respectively (S3, S4). This allows each processing unit 223, 183 to calculate the amount of error between the correction amount set for the amount of blur and the movement amount of the OIS lens 220 or the image sensor 110 moved in the image stabilization operation.

[0096] In this embodiment, the OIS processor 223, the BIS processor 183, or the camera controller 140 calculates an OIS error amount or a BIS error amount for at least one of the pitch, yaw, and roll directions in the digital camera 1 illustrated in Fig. 1, which correspond to the horizontal and vertical directions in a plane perpendicular to the optical axis and the rotation direction about the rotation axis along the optical axis (S4). This allows correction by the EIS function according to the error amount for at least the direction in which each error amount is calculated (S5, S6).

[0097] In this embodiment, the digital camera 1 includes an interchangeable lens 200 including an optical system and an OIS processing unit 223, a camera body 100 including an image sensor 110 and an EIS processing unit 143, and a lens mount 250 and a body mount 150 as an example of a communication unit that communicates data between the interchangeable lens 200 and the camera body 100. The camera control unit 140 receives the OIS error amount at the time of image blur correction by the OIS processing unit 223 from the interchangeable lens 200 via each of the mounts 250, 150 (S4). Alternatively, the lens control unit 240 may calculate the OIS correction amount and set it in the OIS processing unit 223. The camera control unit 140 may obtain the OIS correction amount set in the OIS processing unit 223 from the interchangeable lens 200.

[0098] (Embodiment 2) Hereinafter, a second embodiment of the present disclosure will be described with reference to Fig. 7 and Fig. 8. In the first embodiment, a digital camera 1 that performs image stabilization using an EIS function according to errors related to the amount of blur correction using the OIS function and the BIS function has been described. In the second embodiment, a digital camera 1 that further includes an EIS function using a motion vector in an image such as a video being captured will be described.

[0099] Hereinafter, the description of the configuration and operation similar to those of the digital camera 1 according to the first embodiment will be omitted as appropriate, and the digital camera 1 according to this embodiment will be described.

[0100] 1. Operation The digital camera 1 of this embodiment calculates a motion vector indicating the movement between frames in the entire area of ​​the image for each frame represented by the image data, based on image data sequentially generated by the image sensor 110, for example, during shooting of a video. It is considered that such an image motion vector reflects the shaking of the digital camera 1 according to the movement of a subject, such as an area in the image or an image object. In addition to the function as the EIS processing unit 143 similar to that of the first embodiment, the camera control unit 140 of this embodiment also realizes the image stabilization operation by the EIS function by setting the EIS correction amount by calculating the motion vector.

[0101] Fig. 7 is a flowchart illustrating the operation of the digital camera 1 according to the second embodiment. In the overall operation similar to that of the first embodiment, the digital camera 1 according to the present embodiment sets the EIS correction amount by selectively using the error amount and the motion vector of a moving image (S5A) instead of setting the EIS correction amount reflecting the OIS error amount and the BIS error amount (S5). The process of setting the IS correction amount (S5A) will be described below with reference to Fig. 8.

[0102] 1-1. EIS correction amount setting process Fig. 8 is a flowchart illustrating the process (S5A) of setting the EIS correction amount in the digital camera 1 of embodiment 2. The process of this flowchart is started in a state in which, for example, the amount of blur indicated by the blur detection signal from the gyro sensor 184 used to calculate the blur correction amount in step S2 of Fig. 7, and the OIS error amount and BIS error amount acquired in step S4 are stored in the RAM 141 or the like.

[0103] The camera control unit 140 determines whether or not the digital camera 1 is stationary (S11) based on, for example, the amount of blur stored in the RAM 141. The camera control unit 140 determines that the digital camera 1 is stationary (YES in S11) when, for example, the amount of blur is equal to or less than a predetermined amount that indicates that the digital camera 1 is stationary.

[0104] When it is determined that the digital camera 1 is in a stationary state (YES in S11), the camera control unit 140 sets an EIS error amount reflecting the OIS error amount and the BIS error amount (S16), similar to step S5 in Fig. 6 in the first embodiment. In this way, while the digital camera 1 of this embodiment is in a stationary state, it does not particularly correct blurring caused by the movement of a subject in an image, for example, but performs image stabilization operation using the EIS function so as to correct each error amount caused by an external shock to the digital camera 1, etc.

[0105] On the other hand, when it is determined that the digital camera 1 is not stationary (NO in S11), the camera control unit 140 determines whether the detected amount of blur is greater than a predetermined amount (S12), for example, by referring to the RAM 141. The predetermined amount in step S12 is preset as an amount of blur that is expected to have a large error due to a tracking delay in the operation of the OIS lens 220 or the image sensor 110 in the image stabilization operation (S3 in FIG. 7) with respect to the OIS correction amount or BIS correction amount that is sequentially set, for example.

[0106] When the amount of blur is greater than the predetermined amount (YES in S12), the camera control unit 140 sets an EIS error amount reflecting each error amount (S16). In this way, when the amount of blur is relatively large, a significant error due to the tracking delay of the OIS lens 220 or the image sensor 110 is expected, and it is more likely that it will be difficult to accurately calculate a motion vector from an image, so correction using the error amount is prioritized over the motion vector.

[0107] If the amount of shaking is not greater than the predetermined amount (NO in S12), the camera control unit 140 judges whether or not an external impact was applied to the digital camera 1 during the image stabilization operation by the OIS function and the BIS function (S3 in FIG. 7) (S13). For example, the camera control unit 140 makes the judgment of step 13 based on the amount of shaking detected by the gyro sensor 184 or the like during the image stabilization operation, depending on whether or not the amount of shaking is equal to or greater than a predetermined amount that can be regarded as an external impact.

[0108] When it is determined that digital camera 1 has been subjected to an impact during the above-described optical image stabilization operation (S3) (YES in S13), camera control unit 140 sets an EIS error amount that reflects each optical error amount. Thus, in this embodiment, when an impact on digital camera 1 is detected, correction is performed that reflects the error amount in the optical image stabilization operation, since it is considered that the error in the optical image stabilization operation is likely to be large.

[0109] On the other hand, when it is determined that there is no impact on the digital camera 1 (NO in S13), the camera control unit 140 determines whether or not the reliability of the motion vector obtained during execution of this flowchart is lower than a predetermined standard (S14). For example, when the ISO sensitivity is higher than a predetermined value at which the ratio of signal to noise (S / N ratio) indicating image data from the image sensor 110 is deemed to be deteriorated, it is determined that the reliability of the motion vector is lower than the predetermined standard (YES in S13). Also, for example, the motion vector for each predetermined area (for example, for each pixel) may be calculated from each frame image at the time of execution and immediately before, and the determination in step S13 may be made according to the degree of dissipation of each motion vector.

[0110] When it is determined that the reliability of the motion vector is low (YES in S14), the camera control unit 140 proceeds to step S16 and sets an EIS error amount reflecting each error amount. In this way, when the reliability of the motion vector is relatively low, it is considered difficult to perform accurate blur correction using the motion vector, so the digital camera 1 of this embodiment performs image stabilization operation using the EIS function reflecting the error amount (S16).

[0111] When the reliability of the motion vector is not lower than a predetermined standard, that is, is relatively high (NO in S14), the camera control unit 140 calculates an EIS correction amount based on the motion vector of the image, and sets the correction amount reflecting the motion vector in the image cropping unit 145 (S15). For example, the camera control unit 140 acquires image data generated by the image sensor 110, and calculates the difference between frames in the image represented by the acquired image data, thereby calculating the motion vector.

[0112] After setting the correction amount using the motion vector or the EIS correction amount using the OIS error amount and the BIS error amount (S15, S16), the camera control unit 140 ends the process of this flowchart. The camera control unit 140 executes the image stabilization operation by the EIS function at the set EIS correction amount (S6 in FIG. 7).

[0113] According to the above process, when the reliability of the motion vector is considered to be relatively high, the EIS correction amount reflecting the motion vector is set (S15). On the other hand, when the reliability of the motion vector is relatively low, or when the error in the optical image stabilization operation is considered to be relatively large, the EIS correction amount reflecting the error amount in the blur correction amount calculated based on the blur detection signal and the correction distribution is set (S16). This makes it possible to selectively execute correction using the motion vector and correction using the error amount in the EIS function, and to avoid problems such as mutual interference when both corrections are executed simultaneously, thereby enabling image stabilization operation to be performed with high accuracy.

[0114] In the above, an example has been described in which the predetermined amount in step S12 is an amount that takes into consideration the tracking delay of the OIS lens 220, etc. The predetermined amount is not limited to the above example, and may be set from the perspective of, for example, determining whether or not the digital camera 1 is in a state that can be regarded as walking or running shots.

[0115] In the above, an example has been described in which it is determined in step S13 whether or not there has been an impact on the digital camera 1 based on the amount of shake indicated by the shake detection signal from the gyro sensor 184, but for example the camera body 100 or the like may be provided with an acceleration sensor for detecting an impact. In this case, the camera control unit 140 may make the determination in step S13 depending on whether or not the acceleration detected by the acceleration sensor is equal to or greater than a predetermined value that can be regarded as an impact.

[0116] 2. Summary In this embodiment, the camera control unit 140 calculates a motion vector of the subject image in the image represented by the image data based on the image data sequentially captured by the image sensor 110, and sets the EIS correction amount so as to switch between the EIS correction amount according to the OIS error amount and the BIS error amount, and the EIS correction amount according to the calculated motion vector (S5A, S11 to S16). As a result, even if a significant camera shake occurs, for example, the camera shake correction operation of the EIS function can be performed with high accuracy by selectively using the EIS correction amount that can be considered to easily correct the shake in the captured image from the correction amount according to the error amount and the correction amount according to the motion vector.

[0117] (Embodiment 3) Hereinafter, a third embodiment of the present disclosure will be described with reference to Figs. 9 to 12. In the first and second embodiments, a digital camera 1 that performs image stabilization as an EIS function by a camera control unit 140 when shooting a moving image or the like has been described. In the third embodiment, a digital camera 1A and an imaging system 10 that can execute the EIS function in an external information processing device or the like will be described.

[0118] Hereinafter, the description of the configuration and operation similar to those of the digital camera 1 according to the first and second embodiments will be omitted as appropriate, and a digital camera 1A and an image capturing system 10 according to this embodiment will be described.

[0119] 1. Configuration 9 is a diagram for explaining an imaging system 10 according to embodiment 3. The imaging system 10 includes a digital camera 1A and a personal computer (PC) 5 as an example of an information processing device. The imaging system 10 is configured to enable data communication for transferring image data, each error amount, and the like from the digital camera 1A to the PC 5.

[0120] Fig. 10 is a block diagram showing the configuration of a digital camera 1A according to the third embodiment. In addition to the same configuration as in the first embodiment, the digital camera 1A of this embodiment further includes a communication unit 152 that performs data communication with the PC 5 in accordance with a predetermined communication standard in the camera body 100. The communication unit 152 is an example of the output unit in the digital camera 1A of this embodiment. As shown in Fig. 10, unlike the first embodiment, the digital camera 1A does not need to include an EIS processing unit 143 as a functional component of the camera control unit 140.

[0121] Fig. 11 is a block diagram showing the configuration of the PC 5 in the imaging system 10 of the embodiment 3. The PC 5 in Fig. 3 includes a display unit 52, an operation unit 53, a calculation unit 50, a storage unit 51, a device interface 54, and a network interface 55. Hereinafter, "interface" will be abbreviated as "I / F".

[0122] Display unit 52 is configured with, for example, a liquid crystal display or an organic EL display. Display unit 52 displays images indicated by image data received from digital camera 1. Display unit 52 may display various icons for operating operation unit 53, information input from operation unit 53, and the like.

[0123] The operation unit 53 is a general term for operation members operated by the user. The operation unit 53 includes, for example, a keyboard, a mouse, a touch pad, buttons and / or switches. The operation unit 53 may configure a touch panel together with the display unit 52. When the operation unit 53 accepts an operation by the user, it transmits an operation signal corresponding to the user operation to the calculation unit 50.

[0124] The calculation unit 50 is electrically connected to each component constituting the PC 5 and controls the operation of each component. The calculation unit 50 includes a CPU and the like, and the CPU executes a program (software) to realize a predetermined function. The calculation unit 50 can be configured with various circuits such as an MPU, a GPU, a DSP, an FPGA, or an ASIC in addition to a CPU, and may be realized only by a hardware circuit designed exclusively to realize the predetermined function. The calculation unit 50 may include an internal memory as a temporary storage area for holding various data and programs.

[0125] In the PC 5 of this embodiment, the calculation unit 50 includes, as a functional configuration, an EIS processing unit 56 that functions similarly to the EIS processing unit 143 in the digital camera 1 of the first embodiment, for example.

[0126] The storage unit 51 is a recording medium that stores data, programs, and the like required to realize a predetermined function in the PC 5. The storage unit 51 is configured, for example, of a hard disk (HDD), a semiconductor storage device (SSD), or a semiconductor memory (RAM). The storage unit 51 may function as a working area for the calculation unit 50, or may be configured as a storage area in the internal memory of the calculation unit 50.

[0127] The device I / F 54 is a circuit for connecting an external device such as the digital camera 1A to the PC 5. The device I / F 54 performs communication in accordance with the USB standard, the Bluetooth (registered trademark) standard, or the like. The device I / F 54 of this embodiment receives various information from an external device such as the digital camera 1 and transmits various information to the external device in accordance with the control of the calculation unit 50. The device I / F 54 may include a connection circuit for mounting a memory card 171, similar to the card slot 170 of the digital camera 1A, for example, and may read various information from the memory card 171 and write various information to the memory card 171.

[0128] The network I / F 55 is a circuit that performs communication in compliance with the communication standard IEEE802.11 or Wi-Fi (registered trademark) standard. The PC 5 can communicate with other devices directly or via an access point via the network I / F 55. The network I / F 55 may be connectable to a communication network. Each of the device I / F 54 and the network I / F 55 is an example of an acquisition unit in the imaging system 10 of this embodiment.

[0129] The above-mentioned configuration of the PC 5 is an example, and the configuration is not limited to the above example. The acquisition unit of the PC 5 may be realized by cooperation with various software in the calculation unit 50, etc., and may acquire various pieces of information by, for example, reading various pieces of information stored in various recording media such as the memory card 171 into the internal memory of the calculation unit 50. Alternatively, the acquisition unit may acquire various pieces of information by generating new data based on various pieces of data by the calculation unit 50.

[0130] 2.Operation The operations of the imaging system 10 and the digital camera 1A configured as above will be described with reference to Fig. 12. Fig. 12 is a flow chart illustrating the operation of the digital camera 1A according to this embodiment.

[0131] In the imaging system 10 of this embodiment, the digital camera 1A acquires the OIS error amount and the BIS error amount (S4) in the same manner as in the first embodiment, and then outputs each error amount and image data during shooting to the PC 5 (S20) instead of setting the EIS correction amount (S5 in FIG. 6). Specifically, the camera control unit 140 associates correction information including each error amount with image data from the image sensor 110 buffered in the RAM 141 or the like as a target for image stabilization by the EIS function, and transmits the correction information to the PC 5 via the communication unit 152 (S20).

[0132] For example, in addition to each error amount, the correction information includes the EIS ratio set by the correction allocation setting unit 146 (FIG. 3) and the amount of blur calculated from the detection result of the gyro sensor 184. In this embodiment, the camera control unit 140 does not particularly execute the processes of steps S5 and S6 in FIG. 6, and proceeds to the process of step S7.

[0133] 9, the calculation unit 50 receives image data and correction information output from the digital camera 1A via, for example, the device I / F 54. The calculation unit 50, as an EIS processing unit 56, sets an EIS correction amount reflecting each error amount of the received correction information by processing similar to steps S5 and S6 in Fig. 6, and performs an image stabilization operation similar to the EIS function of the first embodiment by using the EIS correction amount on an image indicated by the image data.

[0134] According to the imaging system 10 described above, the PC 5 performs image stabilization similar to that of the EIS function, using the EIS correction amount reflecting the amount of error in the image stabilization operation by the OIS and BIS functions of the digital camera 1A. This also makes it possible to correct blurring in a captured image, taking into account errors that occur in the optical image stabilization in the digital camera 1A. The image stabilization operation in the PC 5 is not limited to when a video or the like is being captured, and may be performed after capture in response to a user operation on the operation unit 53, for example, by storing image data and correction information in the storage unit 51.

[0135] In the above description, imaging system 10 outputs correction information and image data in association with each other from digital camera 1A to PC 5. Digital camera 1A may embed correction information in image data as meta information and output the image data. Also, instead of outputting to PC 5 via communication unit 152, digital camera 1A may output correction information and image data via card slot 170 so that the correction information and image data are stored in an external recording medium such as memory card 171. In this case, PC 5 or the like can read out the stored information from memory card 171 and perform an image stabilization operation similar to the EIS function.

[0136] In the above description, the imaging system 10 outputs correction information including an error amount from the digital camera 1A to the PC 5. The digital camera 1A may include the EIS processing unit 143 like the digital camera 1 of the first embodiment, or may include a functional configuration that realizes some of the functions of the EIS processing unit 143. For example, the digital camera 1A may calculate an EIS correction amount and output the EIS correction amount to the PC 5 instead of the error amount.

[0137] Furthermore, in the imaging system 10, the digital camera 1A may output an OIS correction amount and a BIS correction amount to the PC 5 instead of the EIS error amount, and the PC 5 may calculate the EIS error amount based on each correction amount. For example, the PC 5 of the imaging system 10 may calculate the EIS correction amount by acquiring each correction amount from the digital camera 1 as well as position signals from the position sensors 222 and 182. The PC 5 may calculate the EIS error amount from each correction amount by a simulation process as described later in the third embodiment.

[0138] 3. Summary As described above, the digital camera 1A in this embodiment is an example of an imaging device including the image sensor 110 (image sensor), an optical image blur correction unit including at least one of the OIS processing unit 223 (first image blur correction unit) and the BIS processing unit 183 (second image blur correction unit), the camera control unit 140, and the communication unit 152 (output unit) that outputs data. In this embodiment, the OIS processing unit 223, the BIS processing unit 183, and the camera control unit 140 are an example of a control unit that controls the image blur correction by the optical image blur correction unit. The image sensor 110 captures an object image formed through an optical system to generate image data. The OIS processing unit 223 performs image blur correction by moving the OIS lens 220 (correction lens) included in the optical system in a plane perpendicular to the optical axis of the optical system including the OIS lens 220. The BIS processing unit 183 performs image blur correction by moving the image sensor 110 in a plane perpendicular to the optical axis. The OIS processing unit 223, the BIS processing unit 183, and the camera control unit 140 set an OIS correction amount and a BIS correction amount as an example of a blur correction amount by the optical image blur correction unit with respect to the blur amount of the digital camera 1 (S2 in FIG. 6). The camera control unit 140 associates the set OIS correction amount and BIS correction amount, or the OIS error amount and the BIS error amount as an example of an error amount indicating a difference between each correction amount and the movement amount of the OIS lens 220 or the image sensor 110 during image blur correction by the optical image blur correction unit, with image data generated by the image sensor 110, and outputs them via the communication unit 152.

[0139] According to the digital camera 1A described above, the OIS / BIS error amount or OIS / BIS correction amount in the optical image stabilization operation is output together with the image data. This makes it easier for an external PC 5 or the like to perform image stabilization that reflects each error amount in the image data, based on the error amount or each correction amount output from the digital camera 1A.

[0140] In this embodiment, the imaging system 10 includes an equipment I / F 54 (acquisition unit), an EIS processing unit 56, and a calculation unit 50 in the PC 5 (see FIG. 11). The equipment I / F 54 acquires the OIS correction amount and BIS correction amount (blur correction amount) by the optical image blur correction unit, or the OIS error amount and the BIS error amount, and image data output by the digital camera 1A. The EIS processing unit 56 performs image blur correction by adjusting an area to cut out the image in the image data. The calculation unit 50 controls the image blur correction by the EIS processing unit 56. The calculation unit 50 sets an EIS correction amount as an example of the blur correction amount by the EIS processing unit 56 based on the OIS correction amount and the BIS correction amount, or the OIS error amount and the BIS error amount by the optical image blur correction unit. The EIS processing unit 56 performs image blur correction with the set EIS correction amount.

[0141] According to the above-described imaging system 10, the PC 5 can perform image stabilization operation that reflects errors in optical image stabilization, similar to, for example, the EIS processing unit 143 in the digital camera 1 of the first embodiment, making it easier to correct even significant camera shake.

[0142] (Modification of the third embodiment) In the above-described third embodiment, digital camera 1A calculates an OIS error amount based on the OIS correction amount and a position signal from position sensor 222 as in the first embodiment (FIG. 3), and calculates a BIS error amount based on the BIS correction amount and a position signal from position sensor 182 (FIG. 4). Each error amount may be calculated, for example, from each corresponding correction amount by simulation processing without using a position signal. Such a modified example of the third embodiment will be described with reference to FIGS. 14 and 15.

[0143] The imaging system of this modification includes a simulation unit 57 that executes the above-mentioned simulation process as a functional configuration in the calculation unit 50 of the PC 5. Fig. 14 is a block diagram showing the configuration of the simulation unit in the imaging system of the modification of the third embodiment. The simulation unit 57 includes a PID control model 610 that performs numerical calculations similar to those of the PID control unit 410 (Fig. 3) or the PID control unit 310 (Fig. 4), and a correction model 613 that simulates the operations of the BIS processing unit 183 or the OIS processing unit 223, etc. The correction model 613 may also simulate the operations of the position sensor 182 or the position sensor 222, and the sensor driving unit 181 or the OIS driving unit 221.

[0144] For example, the calculation unit 50 as the simulation unit 57 acquires the OIS correction amount or the BIS correction amount in response to a request to the camera control unit 140 via the device I / F 54 or the like. In the following, a simulation of the BIS function will be described as an example, but the OIS function can also be simulated in the same manner as in this example. Also, a simulation may be performed for only one of the OIS function and the BIS function.

[0145] The simulation unit 57 in this example inputs the BIS correction amount to the PID control model 610, causes it to output a signal corresponding to the drive signal to the sensor drive unit 181, and inputs the signal to the correction model 613, causes it to output position information corresponding to the position signal from the position sensor 182. In this modification, the calculation unit 50 acquires a deviation calculated in the PID control model 610 based on the BIS correction amount and position information from the correction model 613 as the BIS error amount. The calculation unit 50 is not limited to the deviation, and may directly calculate the BIS error amount based on the BIS correction amount and position information from the correction model 613.

[0146] According to the above-described simulation processing, for example, in the PC 5, the correction amount and image data by the BIS function or the like when shooting a video are obtained from the digital camera 1, and the error amount is calculated, so that image stabilization using the EIS correction amount including the error amount can be applied to the image data.

[0147] 15 is a flowchart illustrating a process of constructing a correction model 613 in an imaging system according to a modification of the embodiment 3. The process of this flowchart is executed by the calculation unit 50 of the PC 5 before the execution of the process by the simulation unit 57, for example.

[0148] The calculation unit 50 sets a transfer function expressing characteristics (e.g., frequency characteristics) related to the operation of the sensor driving unit 181 in the correction model 613 (S31). For example, the transfer function corresponds to a second-order integral operation in which a force for driving the image sensor 110 by a driving signal from the sensor driving unit 181 is input, and a position to which the image sensor 110 has been moved is output, and information indicating the function is stored in advance in the storage unit 51. The information may be input by a user operation on the operation unit 53. The information may include initial values ​​of various coefficients included in the transfer function. In step S31, the calculation unit 50 sets the transfer function by referring to such information.

[0149] The calculation unit 50 acquires from the storage unit 51 or the like measurement data indicating a previously measured measurement result regarding input / output characteristics such as the relationship between the drive signal input to the sensor drive unit 181 and the position signal of the image sensor 110 output from the position sensor 182 (S32). In the time domain, for example, such input / output characteristics are measured by inputting a white noise signal to the sensor drive unit 181 and acquiring a corresponding position signal from the position sensor 182. Also, in the frequency domain, for example, the input / output characteristics are measured by repeating a process of calculating an amplification factor (gain) of an output signal with respect to an input signal, based on a position signal output from the position sensor 182 when a single frequency signal is input to the sensor drive unit 181, for a plurality of frequencies.

[0150] The calculation unit 50 determines the coefficients of the transfer function based on the measurement data by the least squares method or the like so as to minimize the error between the output signal by the transfer function when an input signal similar to the measurement data is input to the transfer function and the position signal corresponding to the input signal of the measurement data (S33). For example, a linear or nonlinear least squares method can be used. The calculation unit 50 stores the determined coefficients in the memory unit 51 or the like.

[0151] According to the above processing, for example, the correction model 613 can be constructed as a transfer function that simulates the position (or the amount of movement from the position before movement) of the image sensor 110 moved by the sensor driving unit 181 in the BIS function. Similarly to the above processing, the correction model 613 can also be constructed as a function that simulates the position (or the amount of movement) of the OIS lens 220 driven by the OIS driving unit 221 in the OIS function. The transfer function is not limited to the continuous time domain, and may be expressed in the discrete time domain. The correction model 613 is not limited to the transfer function, and may be expressed by a state equation.

[0152] Furthermore, in the imaging system of this modified example, the digital camera may include various sensors (e.g., acceleration sensors) for detecting external impacts. In this case, the correction model 613 may be constructed so as to further input an output signal from the sensor and output a corresponding position signal. Furthermore, the above processing is not limited to being performed by the calculation unit 50 of the PC 5, and some or all of the processing may be performed by the camera control unit 140 or the like in the digital camera 1A. That is, the above-mentioned simulation processing may be performed in the digital camera.

[0153] As described above, in the imaging system of this modified example, the calculation unit 50 controls a simulation process for predicting at least one of the movement amount of the OIS lens 220 and the movement amount of the image sensor 110 during image blur correction by the optical image blur correction unit, based on the OIS correction amount and / or the BIS correction amount (one example of the blur correction amount by the optical image blur correction unit). As a result, even if the imaging system receives each correction amount at the PC 5 instead of, for example, the OIS error amount and / or the BIS error amount, it can calculate each corresponding error amount by using only each correction amount during prediction.

[0154] (Other embodiments) As described above, the first to third embodiments have been described as examples of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are appropriately performed. In addition, it is also possible to combine the components described in the first to third embodiments to form new embodiments. Therefore, other embodiments will be exemplified below.

[0155] In the above-described first embodiment, in the digital camera 1, the camera control unit 140 sets the EIS correction amount based on the blur detection signal, the EIS ratio, the OIS error amount, and the BIS error amount (S5 in FIG. 6). The camera control unit 140 may further set the EIS correction amount based on the shutter speed of the shutter 180. Such a modified example of the first embodiment will be described with reference to FIG.

[0156] FIG. 13 is a diagram for explaining the operation of a digital camera according to a modification of the first embodiment. The digital camera according to this modification has a configuration similar to that of the digital camera 1 according to the first embodiment. When setting the EIS correction amount, for example in a process similar to step S5 in FIG. 6, the camera control unit 140 adjusts the correction amount for each error amount acquired in step S4 according to the shutter speed. FIG. 13 illustrates an example of the relationship between the correction gain, which indicates the ratio at which the correction amount is adjusted, and the shutter speed. The correction gain is preset as a real number between "0" and "1".

[0157] In the example of Fig. 13, when the shutter speed becomes slower than a threshold th2 that can be regarded as being relatively fast (i.e., exposure time is short), the correction gain is set to decrease linearly, and when the shutter speed is at the threshold th1, the correction gain is set to "0". For example, the camera control unit 140 multiplies the correction amount for each error amount by the correction gain, thereby adjusting the correction amount to decrease when the shutter speed is slower than the threshold th2. This allows the EIS correction amount to be set so that the correction amount for the error amount is increased, in accordance with the ease with which the image stabilization of the EIS function can be performed with high accuracy in images captured at a relatively high shutter speed, and the correction amount is decreased in images captured at a relatively low shutter speed.

[0158] The correction gain is not limited to the example in Fig. 13, and may be set to decrease nonlinearly in response to a decrease in shutter speed, or the minimum value may be set to a value greater than "0" (e.g., "0.8"). In addition, while an example in which the maximum value of the correction gain is "1" has been described in Fig. 13, the maximum value of the correction gain may be, for example, less than "1" or a real number greater than "1".

[0159] In the above third embodiment, the PC 5 has been described as an example of an information processing device in the imaging system 10, but the information processing device may be various server computers or the like. In this embodiment, for example, the digital camera 1A may transmit image data and correction information to a server computer on a communication network such as the Internet via the communication network. In this case, the server computer may execute the image stabilization operation of the EIS function based on the received image data and correction information, or the PC 5 may acquire the image data and correction information from the server computer instead of the digital camera 1A of the third embodiment. The information processing device may be realized by distributed computing or the like.

[0160] In each of the above embodiments, a digital camera equipped with a gyro sensor 184 that detects shake of the camera body 100 and a gyro sensor 224 that detects shake of the interchangeable lens 200 has been described as an example of a shake detection section. In this embodiment, the digital camera may be equipped with an acceleration sensor in at least one of the camera body 100 and the interchangeable lens 200 as a shake detection section, instead of the gyro sensors 184, 224. For example, the BIS processing section 183 or the OIS processing section 223 may calculate the second order integral of the acceleration detected by the acceleration sensor as the displacement of shake (vibration), and calculate the BIS correction amount or the OIS correction amount based on the displacement.

[0161] In the above embodiment, a digital camera using an acceleration sensor as an example of a shake detection unit has been described. In the digital camera of this embodiment, the acceleration sensor may be attached to detect the acceleration of the OIS lens 220 or the image sensor 110 that moves due to the image stabilization operation. For example, the lens control unit 240 or the camera control unit 140 of this embodiment may calculate position information of the OIS lens 220 or the image sensor 110 by performing a second-order integral of the detected acceleration based on the detection result of the acceleration sensor instead of the position signal from the position sensor 222, 182.

[0162] In each of the above embodiments, the digital cameras 1, 1A have been described in which the position sensor 182 of the camera body 100 detects the position of the image sensor 110 in the rotational direction of the rotation axis along the optical axis in addition to the two orthogonal axial directions (X-axis, Y-axis) of the image sensor 110. In the digital camera of this embodiment, the position sensor of the camera body 100 may detect only the position in the X-axis and Y-axis directions due to translation of the image sensor 110 from a reference position.

[0163] In each of the above embodiments, a digital camera that performs image stabilization operation using the EIS function when shooting a video or the like has been described. The digital camera of this embodiment may perform image stabilization operation using the EIS function after shooting by, for example, acquiring an OIS error amount and / or a BIS error amount (S4) and storing them in association with image data obtained from the image sensor 110. For example, when playing back a video indicated by the image data, the error amount associated with the image data may be read out to perform correction of the EIS function. In addition, instead of the error amount, for example, the OIS and BIS correction amounts and the position signals from the corresponding position sensors 222 and 182 may be stored together with the image data, and the error amount may be calculated when correcting the EIS function.

[0164] In each of the above embodiments, an example has been described in which the OIS / BIS error amount is calculated by (1) the deviation used in the PID control units 310, 410, (2) the difference between the OIS / BIS correction amount and the position information of the OIS lens 220 or the image sensor 110, or (3) by simulation. Each error amount may be calculated by a combination of two or more of the above (1) to (3), and may further be calculated by using an acceleration sensor or the like to take into account the effect of an external shock on each error amount. Furthermore, in the present embodiment, each error amount may be calculated by a calculation process for numerical prediction by artificial intelligence such as machine learning based on, for example, the deviation as described above, a signal indicating the detection result of acceleration, and / or image data.

[0165] In each of the above embodiments, a digital camera has been described in which an OIS / BIS error amount is acquired (S4) while repeating the operation of Fig. 6 or Fig. 7 in a predetermined cycle such as a frame cycle, and the error amount is used to perform image stabilization operation by the EIS function. In this embodiment, the camera control unit 140 may acquire each error amount in a cycle corresponding to the exposure timing of the shutter 180, for example, and execute image stabilization operation of the EIS function using the error amount according to the exposure timing. This makes it possible to suppress image distortion caused by, for example, the rolling shutter phenomenon.

[0166] In the above embodiments, a digital camera with interchangeable lenses has been described as an example of an imaging device, but the imaging device of the present embodiment may be a digital camera that is not particularly an interchangeable lens type. The concept of the present disclosure is also applicable not only to digital cameras, but also to movie cameras, and to electronic devices with various imaging functions such as camera-equipped mobile phones, smartphones, and PCs.

[0167] As described above, the embodiments have been described as examples of the technology in the present disclosure, and the accompanying drawings and detailed description have been provided for this purpose.

[0168] Therefore, among the components described in the attached drawings and the detailed description, not only are there components essential for solving the problem, but there may also be components that are not essential for solving the problem in order to illustrate the above technology. Therefore, the fact that such non-essential components are described in the attached drawings or the detailed description should not be interpreted as immediately indicating that such non-essential components are essential.

[0169] Furthermore, since the above-described embodiments are intended to illustrate the technology in the present disclosure, various modifications, substitutions, additions, omissions, and the like can be made within the scope of the claims or their equivalents.

[0170] (Summary of aspects) Various aspects of the present disclosure are listed below.

[0171] A first aspect of the present disclosure is an imaging device including an imaging element that captures an object image formed through an optical system to generate image data, an optical image blur correction unit including at least one of a first image blur correction unit that performs image blur correction by moving a correction lens included in the optical system in a plane perpendicular to the optical axis of the optical system and a second image blur correction unit that performs image blur correction by moving the imaging element in a plane perpendicular to the optical axis, a third image blur correction unit that performs image blur correction by adjusting an area to cut out an image in the image data, and a control unit that controls the image blur correction by the optical image blur correction unit and the third image blur correction unit. The control unit sets an amount of blur correction by the optical image blur correction unit with respect to an amount of blur of the imaging device, and sets the amount of blur correction by the third image blur correction unit based on an error amount indicating a difference between the set amount of blur correction by the optical image blur correction unit and an amount of movement of the correction lens or the imaging element during image blur correction by the optical image blur correction unit, and the third image blur correction unit performs image blur correction with the set amount of blur correction.

[0172] In a second aspect, the imaging device of the first aspect includes a blur detection unit that detects the amount of blur in the imaging device, and the first or second image blur correction unit performs image blur correction in accordance with the amount of blur correction set in each image blur correction unit for the amount of blur detected by the blur detection unit.

[0173] In a third aspect, the imaging device of the first or second aspect includes a position detection unit that detects the position of the correction lens or the position of the imaging element in a plane perpendicular to the optical axis, and the control unit obtains at least one of the amount of movement of the correction lens and the amount of movement of the imaging element based on the detection result by the position detection unit.

[0174] In a fourth aspect, in the imaging device of any of the first to third aspects, the control unit calculates a motion vector of a subject image in an image represented by image data based on image data sequentially captured by the imaging element, and sets the amount of blur correction by the third image blur correction unit so as to switch between the amount of blur correction corresponding to the amount of error and the amount of blur correction corresponding to the calculated motion vector.

[0175] In a fifth aspect, in the imaging device of any of the first to fourth aspects, the control unit calculates an amount of error in at least one of the horizontal and vertical directions in a plane perpendicular to the optical axis, and the rotational direction about a rotation axis along the optical axis.

[0176] In a sixth aspect, the imaging device of any of the first to fifth aspects includes an interchangeable lens including an optical system and a first image blur correction unit, a camera body including an image sensor and a third image blur correction unit, and a communication unit that communicates data between the interchangeable lens and the camera body, and the control unit receives at least one of the blur correction amount set in the first image blur correction unit and the amount of error during image blur correction by the first image blur correction unit from the interchangeable lens via the communication unit.

[0177] A seventh aspect of the present disclosure is an imaging device including an image sensor that captures an object image formed via an optical system to generate image data, an optical image blur correction unit including at least one of a first image blur correction unit that performs image blur correction by moving a correction lens included in the optical system within a plane perpendicular to the optical axis of the optical system and a second image blur correction unit that performs image blur correction by moving the image sensor within a plane perpendicular to the optical axis, a control unit that controls the optical image blur correction unit, and an output unit that outputs data. The control unit sets an amount of blur correction by the optical image blur correction unit with respect to an amount of blur of the imaging device, and outputs, via the output unit, an error amount indicating a difference between the set amount of blur correction by the optical image blur correction unit or the amount of blur correction and the amount of movement of the correction lens or the image sensor during image blur correction by the optical image blur correction unit, in association with image data.

[0178] An eighth aspect of the present disclosure is an imaging system including an acquisition unit that acquires the amount of blur correction or the amount of error by the optical image blur correction unit and image data output by the imaging device of the seventh aspect, a third image blur correction unit that performs image blur correction by adjusting an area to cut out an image from the image data, and a calculation unit that controls the image blur correction by the third image blur correction unit. The calculation unit sets the amount of blur correction by the third image blur correction unit based on the amount of blur correction or the amount of error by the optical image blur correction unit, and the third image blur correction unit performs image blur correction with the set amount of blur correction.

[0179] In a ninth aspect, in the imaging system of the eighth aspect, the calculation unit controls a simulation process that predicts at least one of the movement amount of the correction lens during image blur correction by the optical image blur correction unit and the movement amount of the image sensor, based on the amount of blur correction by the optical image blur correction unit. [Industrial Applicability]

[0180] The concept of the present disclosure can be applied to electronic devices having an imaging function with an image stabilization function (imaging devices such as digital cameras and camcorders, mobile phones, smartphones, etc.). [Explanation of symbols]

[0181] 1,1A Digital Camera 100 Camera body 110 Image Sensor 140 Camera control unit 141 RAM 142 Flash Memory 143 EIS Processing Section 146 Correction distribution setting section 181 Sensor drive unit 183 BIS Processing Department 184 Gyro sensor 200 Interchangeable Lenses 220 OIS lens 221 OIS drive unit 223 OIS Processing Unit 224 Gyro sensor 5 PC 51 Storage section 52 Display section 53 Operation section 54 Equipment I / F 55 Network I / F 56 EIS Processing Section 57 Simulation Department 10. Imaging System

Claims

1. an image sensor that captures an object image formed via an optical system and generates image data; an optical image blur correction unit including at least one of a first image blur correction unit that performs image blur correction by moving a correction lens included in the optical system within a plane perpendicular to an optical axis of the optical system, and a second image blur correction unit that performs image blur correction by moving the image sensor within a plane perpendicular to the optical axis; a third image blur correction unit that performs image blur correction by adjusting an area to cut out an image from the image data; a control unit that controls image blur correction by the optical image blur correction unit and the third image blur correction unit, The control unit is setting an amount of blur correction to be performed by the optical image blur correction unit with respect to an amount of blur of the imaging device; setting an amount of blur correction by the third image blur correction unit based on an amount of error indicating a difference between the set amount of blur correction by the optical image blur correction unit and an amount of movement of the correction lens or the image sensor during image blur correction by the optical image blur correction unit; The third image blur correction unit performs image blur correction with a set blur correction amount. Imaging device.

2. a blur detection unit that detects an amount of blur of the imaging device, The first or second image blur correction unit performs image blur correction according to a blur correction amount set in each image blur correction unit for the blur amount detected by the blur detection unit. The imaging device according to claim 1 .

3. a position detection unit that detects a position of the correction lens or a position of the image sensor in a plane perpendicular to the optical axis, The control unit obtains at least one of the movement amount of the correction lens and the movement amount of the imaging element based on a detection result by the position detection unit. The imaging device according to claim 1 .

4. The control unit is Calculating a motion vector of a subject image in an image represented by the image data, based on the image data sequentially captured by the image sensor; The amount of blur correction by the third image blur correction unit is set so as to switch between an amount of blur correction according to the amount of error and an amount of blur correction according to the calculated motion vector. The imaging device according to claim 1 .

5. The control unit calculates the amount of error in at least one of a horizontal direction and a vertical direction in a plane perpendicular to the optical axis, and a rotation direction about a rotation axis aligned with the optical axis. The imaging device according to claim 1 .

6. an interchangeable lens including the optical system and the first image blur correction unit; a camera body including the imaging element and the third image blur correction unit; a communication unit that communicates data between the interchangeable lens and the camera body, The control unit is At least one of the blur correction amount set in the first image blur correction unit and the error amount at the time of image blur correction by the first image blur correction unit is received from the interchangeable lens by the communication unit. The imaging device according to claim 1 .

7. an image sensor that captures an object image formed via an optical system and generates image data; an optical image blur correction unit including at least one of a first image blur correction unit that performs image blur correction by moving a correction lens included in the optical system within a plane perpendicular to an optical axis of the optical system, and a second image blur correction unit that performs image blur correction by moving the image sensor within a plane perpendicular to the optical axis; A control unit that controls the optical image blur correction unit; An imaging device comprising an output unit that outputs data, The control unit is setting an amount of blur correction to be performed by the optical image blur correction unit with respect to an amount of blur of the imaging device; The output unit outputs, by association with the image data, an error amount indicating a difference between the set amount of blur correction by the optical image blur correction unit or the amount of blur correction and a movement amount of the correction lens or the image sensor during image blur correction by the optical image blur correction unit. Imaging device.

8. an acquisition unit that acquires the image data and the blur correction amount or the error amount output by the optical image blur correction unit, the image data being output by the imaging device according to claim 7; a third image blur correction unit that performs image blur correction by adjusting an area to cut out an image from the image data; a calculation unit that controls the image blur correction by the third image blur correction unit, the calculation unit sets an amount of blur correction to be performed by the third image blur correction unit, based on the amount of blur correction performed by the optical image blur correction unit or the amount of error; and The third image blur correction unit performs image blur correction with a set blur correction amount. Imaging system.

9. The calculation unit is A simulation process is controlled to predict at least one of a movement amount of the compensation lens and a movement amount of the imaging element during image blur correction by the optical image blur correction unit, based on a blur correction amount by the optical image blur correction unit. The imaging system according to claim 8 .

Images