Imaging device and control method thereof

Abstract

To obtain an excellent shake correction effect when performing shake correction on each of an interchangeable lens side and an imaging apparatus side.SOLUTION: An imaging apparatus 120 can be detachably attached with an interchangeable lens 100 that performs the first shake correction to the vibration. The imaging apparatus 120 comprises: correction means which performs the second shake correction to the vibration; vector detection means which detects motion vectors of a plurality of regions in an image obtained with imaging; representative decision means which decides a representative vector from the motion vectors of the plurality of regions; and control means which controls the second shake correction according to the representative vector. The representative decision means acquires a second correction amount according to an angle of the shake of the imaging apparatus and a sharing ratio of the second shake correction occupied in the shake correction by the first and second shake corrections, and decides the representative vector on the basis of the second correction amount.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an imaging apparatus having a function for reducing (correcting) image blurring. [Background technology]

[0002] Some imaging systems (hereinafter referred to as camera systems) such as digital still cameras and video cameras are equipped with a shake correction function that corrects image blur caused by camera shake such as hand shake. Some interchangeable lens camera systems perform optical shake correction by moving a correction lens within the interchangeable lens, and also perform electronic shake correction by moving the cut-out range from image data obtained by an image sensor within the camera.

[0003] Patent Document 1 discloses a camera system that controls the ratio of optical shake correction on the interchangeable lens side and electronic shake correction on the camera side based on the amount of camera shake detected by a shake sensor provided in the interchangeable lens. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-219635 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the camera system of Patent Document 1 does not have a configuration for further correcting image blur (residual image blur) that cannot be completely corrected due to detection errors of the shake sensor caused by temperature or delays in tracking the camera shake in driving the correction lens.

[0006] The present invention is intended to provide a favorable shake correction effect when shake correction is performed on both the interchangeable lens side and the camera side. [Means for solving the problem]

[0007] An imaging device according to one aspect of the present invention is capable of attaching and detaching an interchangeable lens that performs a first shake correction against vibration. The imaging device has a correction means that performs a second shake correction against the vibration, a vector detection means that detects motion vectors of multiple regions in an image obtained by imaging, a representative determination means that determines a representative vector from the motion vectors of the multiple regions, and a control means that controls the second shake correction according to the representative vector. The representative determination means determines the representative vector based on a second correction amount acquired based on the angle of shake of the imaging device and the proportion of the second shake correction accounted for in the shake correction by both the first and second shake corrections. 。 Ma In addition, an imaging device according to another aspect of the present invention is an imaging device to which an exchangeable lens can be attached or detached. ,before Interchangeable lenses Rafuri the image processing device according to the present invention is characterized in that it has an acquisition means for acquiring a correction amount for image blur correction, a vector detection means for detecting motion vectors of a plurality of regions in an image obtained by imaging, and a control means for controlling the image blur correction based on a motion vector whose difference from the correction amount is equal to or less than a predetermined value, if the motion vectors of the plurality of regions include a motion vector whose difference from the correction amount is equal to or less than a predetermined value, and for controlling the image blur correction based on the correction amount, if the motion vectors of the plurality of regions include no motion vector whose difference from the correction amount is equal to or less than the predetermined value.

[0008] A control method according to another aspect of the present invention is applied to an imaging device that allows the attachment and detachment of an interchangeable lens that performs a first shake correction against vibration and performs a second shake correction against the vibration. The control method includes a step of detecting motion vectors of a plurality of regions in an image obtained by imaging, a step of determining a representative vector from the motion vectors of the plurality of regions, and a step of controlling the second shake correction according to the representative vector. The representative vector is determined based on a second correction amount acquired based on the angle of shake of the imaging device and the share of the second shake correction in the shake correction by both the first and second shake corrections. 。 Ma A control method according to another aspect of the present invention is applied to an image pickup apparatus to which an interchangeable lens is detachable. , intersection The method includes the steps of acquiring a correction amount for shake correction from an interchangeable lens, detecting motion vectors of multiple regions in an image obtained by imaging, and, if there is a motion vector among the motion vectors of the multiple regions whose difference from the correction amount is equal to or less than a predetermined value, controlling shake correction based on the motion vector whose difference from the correction amount is equal to or less than the predetermined value, and, if there is no motion vector among the motion vectors of the multiple regions whose difference from the correction amount is equal to or less than the predetermined value, controlling shake correction based on the correction amount. A computer program for causing a computer of an imaging device to execute processing according to the above control method also constitutes another aspect of the present invention. Effect of the Invention

[0009] According to the present invention, when the first shake correction is performed on the interchangeable lens side and the second shake correction is performed on the imaging device side, a good shake correction effect can be obtained. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of a camera system according to an embodiment of the present invention. [Diagram 2] 5A and 5B are diagrams showing the relationship between a subject and a camera system when no shake correction is performed in the embodiment. [Diagram 3] 11A and 11B are diagrams showing the relationship between a captured image and a motion vector when no shake correction is performed in the embodiment. [Figure 4] 5A and 5B are diagrams showing the relationship between a subject and a camera system during shake correction in the embodiment. [Diagram 5] 6A and 6B are diagrams showing the relationship between a camera system and a motion vector during shake correction in the embodiment. [Figure 6] 6A and 6B are diagrams showing the amount of movement of a subject image when shake correction is performed and when shake correction is not performed in an embodiment. [Figure 7] FIG. 11 is a histogram diagram showing a method for determining a representative vector in the embodiment. [Figure 8] 11 is a flowchart showing a representative vector determination process in the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0012] 1 shows the configuration of an interchangeable lens camera system according to an embodiment of the present invention. The camera system is composed of an image capture device (hereinafter referred to as camera) 120 such as a digital still camera or video camera, and an interchangeable lens (hereinafter simply referred to as lens) 100 that is detachable from the camera 120. In this embodiment, when vibration such as hand shake (hereinafter referred to as camera shake) occurs in the camera system, optical shake correction is performed as a first shake correction in which a correction lens (optical element) 110 (described later) is moved in the lens 100 to optically reduce (correct) image shake, and electronic shake correction is performed as a second shake correction in which an area to be cut out from image data (frame image) obtained by an image capture element 121 (described later) in the camera 120 is moved to electronically correct image shake.

[0013] In the lens 100, the shake detection unit 101 is a shake sensor that detects vibrations such as hand shake (hereinafter referred to as camera shake), and in this embodiment, it is an angular velocity sensor that outputs a voltage value according to the angular velocity of the camera shake. The A / D converter 102 converts the voltage value output from the shake detection unit 101 into digital data (angular velocity data) and outputs it to the signal processing unit 103. The signal processing unit 103 performs signal processing to generate a shake correction amount from the angular velocity data. The signal processing includes a HPF that removes DC components superimposed on the angular velocity data, an integral filter for converting the angular velocity into an angle, and a cutoff frequency change process of the HPF for removing shake components due to panning and tilting of the camera from the angular velocity data. The data converted into angle units by these signal processes is output to the division unit 104.

[0014] The division unit 104 determines the allocation ratio between optical shake correction on the lens 100 side and electronic shake correction on the camera 120 side (hereinafter referred to as correction allocation ratio), and sets the shake correction amount by optical shake correction on the lens 100 side (first correction amount: hereinafter referred to as optical correction amount) and the shake correction amount by electronic shake correction on the camera 120 side (second correction amount: hereinafter referred to as electronic correction amount) according to the determined correction allocation ratio. The correction allocation ratio is determined based on the ratio between the maximum image shake amount correctable by movement of the correction lens 110 on the lens 100 side (hereinafter referred to as maximum optical correction amount) and the maximum image shake amount correctable by electronic shake correction on the camera 120 side (hereinafter referred to as maximum electronic correction amount).

[0015] Specifically, electronic shake correction limiter 128 in camera 120 transmits a previously stored maximum amount of electronic correction to division unit 104 via camera communication processing unit 127 and lens communication processing unit 113. Division unit 104 determines a correction allocation ratio from the previously stored maximum amount of optical correction and the received maximum amount of electronic correction, and calculates (acquires) an amount of optical correction and an amount of electronic correction according to the correction allocation ratio. The amount of optical correction is calculated based on the allocation ratio of optical shake correction to the shake correction by both optical shake correction and electronic shake correction, and the amount of electronic correction is calculated based on the allocation ratio of electronic shake correction to the shake correction by both of the above.

[0016] For example, when the ratio of the maximum optical correction amount (1 deg) to the maximum electronic correction amount (3 deg) is 1:3, the correction sharing ratio is set to 1:3, and when the camera shake amount is 2 deg, the optical correction amount is set to 0.5 deg and the electronic correction amount is set to 1.5 deg. The optical correction amount set by division unit 104 is output to control filter 105, and the optical correction amount is output to pixel unit conversion unit 112.

[0017] In optical shake correction, a correction lens 110 included in an imaging optical system (not shown) moves in a direction perpendicular to the optical axis of the imaging optical system (including the case of rotating around a point on the optical axis) to correct image shake. To drive the correction lens 110, the position of the correction lens 110 is detected, and feedback control is performed so as to bring the amount of deviation between the detected position and a target position corresponding to the amount of optical correction closer to zero. The amount of deviation, which is the difference between the amount of optical correction output from the division unit 104 and the detected position of the correction lens 110, is input to the control filter 105.

[0018] The control filter 105 is composed of an amplifier that amplifies the input signal with a predetermined gain and a phase compensation filter. The control filter 105 performs amplification and phase compensation on the deviation amount input as a signal to generate motor control data and outputs this to the pulse width modulation unit 106. The pulse width modulation unit 106 converts the input motor control data into a PWM waveform, which is a waveform that changes the duty ratio of a pulse wave, and outputs it to the motor driver 107.

[0019] The motor driver 107 applies a drive voltage to the motor 108 based on the input PWM waveform. The motor 108 is a voice coil type motor, and when a voltage is applied from the motor driver 107 to the coil, a drive force is generated that moves the correction lens 110 in a direction perpendicular to the optical axis.

[0020] As a result of the movement of the correction lens 110, the translational shake of the subject image on the image sensor (imaging surface) 121 caused by camera shake is corrected. The position of the correction lens 110 is detected by a position detection unit 109. The position detection unit 109 is composed of, for example, a magnet and a magnetic sensor (Hall sensor) provided at a position opposite to it, and outputs a voltage that changes according to the movement of the correction lens 110. The A / D converter 111 converts the output voltage from the position detection unit 109 into digital data. The amount of deviation between the digitized detected position of the correction lens 110 and the above-mentioned target position is input to the control filter 105 as deviation amount data. In this way, a feedback control system that controls the drive of the correction lens 110 is configured.

[0021] On the other hand, the amount of electronic correction in electronic shake correction is an amount in pixel units that indicates the position of the cut-out range from the image data (hereinafter referred to as the image cut-out position). The pixel unit conversion unit 112 converts the amount of electronic correction (unit is angle) from the division unit 104 into a value in pixel units. Specifically, assuming that the focal length of the imaging optical system is f, the amount of electronic correction is x (deg), and the distance between pixels (pixel pitch) in the imaging element 121 is g (μm), the value p obtained by converting the amount of electronic correction into pixel units is expressed as p = f × tan(x) / g. However, when x is infinitesimal compared to the focal length f, f × tan(x) can be regarded as f × x, and p can be expressed as p = fx / g.

[0022] The pixel unit conversion unit 112 acquires information on the focal length f from the imaging optical system, and acquires information on the pixel pitch of the imaging element 121 from the camera 120.

[0023] The lens communication processing unit 113 receives an information transmission request from the camera 120 (camera communication processing unit 127) and transmits corresponding information to the camera 120. The amount of electronic correction (in pixel units) from the pixel unit conversion unit 121 is supplied to the camera 120 via the lens communication processing unit 113.

[0024] In the camera 120, light passing through the imaging optical system of the lens 100 is received by an imaging element 121 using a CCD sensor, a CMOS sensor, or the like, and photoelectrically converted. An imaging signal output from the imaging element 121 is subjected to analog signal processing such as AGC (Automatic Gain Control) processing, and then converted into a digital signal by A / D conversion. Digital signal processing such as gamma correction and white balance processing is performed on the digital imaging signal, thereby generating image data. The image data is converted into a video signal (moving image) conforming to the NTSC format, and stored in an image memory 122.

[0025] The motion vector detection unit (vector detection means) 123 detects a motion vector based on the luminance signal contained in the current frame image (or field image) and the previous frame image in the moving image obtained by imaging. A known block matching method or the like is used to detect the motion vector. The block matching method is a method in which a frame image is divided into a plurality of blocks, and similar parts between the previous frame image and the current frame image are detected on a block-by-block basis. In the previous frame image, a block having the highest correlation value with a specific block in the current frame image is determined to be a similar block, and the amount of displacement between the position of the specific block and the position of the similar block is detected as motion information between one frame, i.e., a motion vector. The motion vector detection unit 123 detects such a motion vector for each of a plurality of blocks. Note that the motion vector may be detected by a method other than the block matching method.

[0026] A representative vector detection unit (representative determination means) 125 determines a representative vector from the motion vectors of multiple blocks detected by the motion vector detection unit 123, using the amount of electronic correction obtained from the lens 100. The representative vector in this embodiment is a motion vector used to determine an image cut-out position in electronic shake correction, among the motion vectors detected in multiple blocks.

[0027] The motion vector of the background region in the moving image is caused by camera shake, and image shake occurring between frames in the moving image can be corrected by performing electronic shake correction based on the motion vector of the background region. For this reason, in this embodiment, the motion vector of the background region is calculated as the representative vector. The processing in the representative vector detection unit 125 will be described in detail later.

[0028] The image deformation amount calculation unit (control means) 129 determines the image cut-out position based on the value of the representative vector determined by the representative vector detection unit 125. That is, it controls electronic shake correction. For example, when an image blur of three pixels in the upward direction occurs due to camera shake, the representative vector detection unit 125 determines a motion vector equivalent to three pixels in the upward direction as the representative vector. The image deformation amount calculation unit 129 moves the image cut-out position by three pixels in the downward direction. This makes it possible to absorb the positional shift of the subject image of the above-mentioned three pixels in the upward direction and correct the image blur. Note that when an image blur occurs in the left or right direction due to camera shake, the image cut-out position can be moved in the opposite direction to correct the image blur.

[0029] Note that the maximum electronic correction amount is obtained from the electronic shake correction limiter 128, and if the movement amount to the determined image cut-out position exceeds the maximum electronic correction amount, the movement amount of the image cut-out position is limited to the maximum electronic correction amount.

[0030] The image transformation unit (correction means) 124 cuts out an output image from the frame image according to the image cut-out position determined by the image deformation amount calculation unit 129. The output image cut out by the image transformation unit 124 is stored in a recording medium 126 such as a semiconductor memory or a hard disk, or is displayed on a display device such as a liquid crystal element.

[0031] Rotational image blurring that occurs when camera shake occurs in the rotational direction around the optical axis may be corrected by electronic shake correction.

[0032] Next, a method for detecting a motion vector and a method for determining a representative vector will be described with reference to Figs. 2(A), (B) and 3(A)-(C). Figs. 2(A), (B) and 3(A)-(C) show a case where imaging is performed without image stabilization. Here, a case where horizontal camera shake occurs in actual imaging will be described, but the same applies to a case where vertical camera shake occurs.

[0033] FIG. 2(A) shows the amount of camera shake due to camera shake as a sine wave with a fixed frequency, and the shake angle is shown as θ. FIG. 2(B) shows the relative positions of a person, which is a moving subject, a building in the background, and the camera system. The positive and negative directions of the shake angle θ in FIG. 2(A) respectively indicate camera shake to the right and left from the center in FIG. 2(B). FIG. 2(B) also shows a state in which a person is moving to the right from time T0 to T1, which is also shown in FIG. 2(A). The amount of camera shake from the center of the camera system at time T0 is set to 0.

[0034] FIG. 3(A) shows frame images obtained by capturing images at times T0 and T1 in FIG. 2(A) and (B). FIG. 3(B) shows motion vectors calculated from the two frame images shown in FIG. 3(A). The block matching method described above is used for motion vector detection, and symbols B01 to B09 each indicate nine blocks. The number of blocks may be other than nine. V01 to V09 indicate motion vectors output from motion vector detection unit 123 for each block. The motion vector detected for a moving person is indicated by Va, and the motion vector detected for a building is indicated by Vb. The direction of the motion vector is from a specific block at time T0 toward a similar block at time T1.

[0035] The motion vectors V01, V02, V05, and V08 cannot be detected, so no arrows are shown indicating the motion vectors. For example, when the contrast of the subject is low or the subject has a repeating pattern, it is difficult to associate a specific block with a similar block at time T0 and T1, so that the motion vector cannot be detected. If the motion vector cannot be detected, a detection error may be notified.

[0036] Fig. 3(C) shows a histogram generated from the detected motion vectors. The horizontal axis shows the value of the motion vector, and the vertical axis shows the number of detections of the same motion vector. As shown in Fig. 3(B), motion vector Va is detected in two blocks, and motion vector Vb is detected in three blocks, so the number of detections of motion vector Va is 2, and the number of detections of motion vector Vb is 3.

[0037] There is a method for determining the motion vector with the largest number of detections in this histogram as the representative vector. This is because, in images obtained by general imaging, the ratio of the area occupied by the background is often larger than the ratio of the area occupied by moving subjects such as people, so it can be assumed that the motion vector with the largest number of detections is detected relative to the background. However, in the method for determining the motion vector with the largest number of detections as the representative vector, there is a risk that the representative vector will be erroneously determined to be the motion vector of the moving subject area when the moving subject area is larger than the background area.

[0038] For this reason, in this embodiment, the representative vector is determined by the method described below. Here, as an example, a method for determining a representative vector in a case where the moving subject area and the background area in an image are of equal size, and further where the lens 100 and the camera 120 share the role of shake correction as described above, will be described with reference to Figs. 4(A), (B) and 5(A)-(C). Figs. 4(A) and (B) correspond to Figs. 2(A) and (B), and Figs. 5(A)-(C) correspond to Figs. 3(A)-(C). Figs. 4(A), (B) and 5(A)-(C) show a case where imaging is performed with optical shake correction performed by the lens 100.

[0039] 4(A) shows by a solid line the amount of camera shake due to camera shake also shown in Fig. 2(A), and by a dotted line the amount of optical correction (angle) by correction lens 110. The dotted line shows the amount of optical correction when the maximum amount of optical correction and the maximum amount of electronic correction are the same, and the correction allocation ratio between optical shake correction and electronic shake correction for the amount of camera shake is determined by division unit 104 to be θ / 2 each.

[0040] Fig. 4(B) shows the positional relationship between the moving subject (person), the background (building), and the camera system, similar to Fig. 2(B). In Fig. 4(B), the moving subject moves closer to the camera system than in the positional relationship shown in Fig. 2(B) so that the background area and the moving subject area are of equal size, as described above.

[0041] Fig. 5(A) shows frame images obtained by capturing images at times T0 and T1 in Fig. 4(A) and (B), respectively. In these frame images, the background area and the moving subject area are of equal size. By performing optical shake correction, the amount of shake of the subject image formed on the image sensor 121 is suppressed to θ / 2, and as a result, the amount of movement of the structure in the frame images is reduced.

[0042] Fig. 5(B) shows motion vectors of nine blocks B01 to B09 calculated from the two frame images shown in Fig. 5(A), similar to Fig. 3(B). Motion vectors V01, V04, and V07 are motion vectors Vc detected with respect to a moving subject, and motion vectors V03, V06, and V09 are motion vectors Vd detected with respect to the background.

[0043] FIG. 5(C) shows a histogram generated from the detected motion vectors, similar to FIG. 3(C). Compared with the cases shown in FIGS. 3(A) to (C), the proportion of moving subjects in the frame images is large, so the number of detected motion vectors Vc is three, which is the same as the number of detected motion vectors Vd. In this case, a method of determining the motion vector with the largest number of detections as the representative vector cannot determine one representative vector. Furthermore, if the number of detected motion vectors Vc exceeds the number of detected motion vectors Vd, the representative vector will be erroneously determined. If the image cutout position is changed based on the erroneously determined representative vector, a moving image will be obtained in which the imaging angle of view changes with the movement of the subject.

[0044] In the past, various methods have been proposed to avoid erroneous determination of a representative vector in an image capture scene in which the number of motion vectors detected for a moving subject is the maximum. For example, a method of estimating a motion vector of a background region from the tendency of past motion vectors, a method of estimating a person or animal as a subject from a feature amount in an image to specify a motion vector of the background region, etc. have been proposed. In this embodiment, these conventional methods are not used, but the method of this embodiment described below may be used in combination with the conventional methods to avoid erroneous determination of a representative vector.

[0045] In this embodiment, a method of calculating a representative vector using the amount of electronic correction obtained from the lens 110 will be described. First, the relationship between the amount of optical correction by the correction lens 110 and the amount of movement of the subject between frame images will be described with reference to Figs. 6(A) to 6(C). Figs. 6(A) to 6(C) show the camera system (the correction lens 110 and the image sensor 121) seen from above in actual imaging, and the inclination in the vertical direction in each figure shows the inclination in the horizontal direction in actual imaging. For example, in Fig. 6(B), the camera system is inclined downward in the figure by an angle θ, which is the same as the inclination in the negative direction in Fig. 4(B). Also, Figs. 6(A), (B), and (C) each show the relationship between the camera system and the subject on the left side, and the frame image obtained by imaging on the right side. The subject does not move.

[0046] Fig. 6(A) shows a state where the amount of camera shake of the camera system is 0, which corresponds to the state at time T0 described above. When camera shake of angle θ is added to the camera system from this state, the subject image moves by a movement amount d within the frame image, that is, on the image sensor 121, as shown in Fig. 6(B). The movement amount d of the subject image on the image sensor 121 is expressed as d=f×tan θ, where f is the focal length of the imaging optical system.

[0047] 6(C) shows a state in which optical shake correction has been performed by the correction lens 110 when camera shake of an angle θ occurs in the camera system. As described above, the correction share of optical shake correction is θ / 2, so a movement amount d' = f × tan(θ / 2) of the subject image on the image sensor 121 occurs. The movement of the subject image equivalent to this movement amount d' is detected as a motion vector between frame images. In other words, the amount of electronic correction transmitted from the lens 100 to the camera 120 can be associated with the motion vector detected between frame images.

[0048] Based on the above explanation, a method for determining, as a representative vector, the motion vector Vd detected for the background, rather than the motion vector Vc detected for the moving subject, in the representative vector detection unit 125, will be explained using Fig. 7. Fig. 7 is a histogram showing the number of detected motion vectors Vc and Vd (each 3), the same as Fig. 5(C).

[0049] 6(C), when a camera shake of angle θ is applied to the camera system and image shake of an angle θ / 2 is corrected by correction lens 110, the correction share of electronic shake correction output from pixel unit conversion section 112 of lens 100 is expressed as d' / g using the movement amount d' of the subject image and the pixel pitch g of image sensor 121. In the following explanation, d' / g will be referred to as the electronic correction amount E. The unit of electronic correction amount E is the number of pixels.

[0050] If the amount of electronic correction E is plotted on the histogram in Fig. 7, the amount of electronic correction E will be a value close to the motion vector Vd. This is for the following reason. The amount of electronic correction E corresponds to the image blur component that is not corrected by optical shake correction, and appears as the amount of movement of the subject image on the image sensor 121 and is detected as a motion vector. Since the movement of the subject image on the image sensor 121 is caused only by camera shake, the detected motion vector can be considered to be equivalent to the motion vector of the background, and E ≈ Vd holds.

[0051] In this way, by determining the motion vector with the greatest number of detections as the representative vector from among the motion vectors with values ​​substantially equal to the amount of electronic correction E, it is possible to identify the motion vectors of the background without being affected by the moving subject.

[0052] As shown in FIG. 7, it is preferable to determine the representative vector from motion vectors whose values ​​are within a predetermined range E±th including the electronic correction amount E. In other words, it is preferable to determine the representative vector from motion vectors whose difference from the electronic correction amount E is equal to or less than a predetermined value th. Taking into consideration the shake correction remaining due to the tracking performance of the optical shake correction by the correction lens 110 against camera shake and the detection error by the shake detection unit 101, there is a possibility that a motion vector equal to or greater than the electronic correction amount E determined by the division unit 104 may be detected, so ±th is provided. th is a predetermined number of pixels. This prevents the motion vector detected by the representative vector detection unit 125 from being excluded from the candidates for the representative vector even if the detected motion vector is larger than the electronic correction amount E by a few pixels equal to or less than the predetermined number of pixels.

[0053] The flowchart in Fig. 8 shows the process in which the representative vector detection unit 125 determines a representative vector. The representative vector detection unit 125, which is configured by a computer together with the motion vector detection unit 123, the image deformation amount calculation unit 129, the image deformation unit 124, etc., executes this process according to a computer program. The execution cycle of this process is synchronized with the frame rate cycle of the moving image. As a result, the execution timing of the representative vector determination process is after the detection of the motion vector in all blocks in the frame image is completed.

[0054] First, in step S 100 , the representative vector detection unit 125 acquires the amount of electronic correction E from the lens 100 via the camera communication processing unit 127 .

[0055] Next, in step S101, the representative vector detection unit 125 determines whether the electronic correction amount E is 0 or not. The electronic correction amount E is 0 when the amount of camera shake detected by the shake detection unit 101 in the lens 100 is 0 or when the user intentionally turns off the shake correction function of the camera system. If the electronic correction amount E is 0, the representative vector detection unit 125 proceeds to step S107 to set the value of the representative vector to 0, and then proceeds to step S108. If the electronic correction amount E is not 0, the representative vector detection unit 125 proceeds to step S102.

[0056] In step S102, the representative vector detection unit 125 obtains motion vectors of all blocks between frame images.

[0057] Next, in step S103, the representative vector detection unit 125 generates a histogram of the number of detected motion vectors as shown in FIG. 7 from the acquired motion vectors of all blocks. Next, in step S104, the representative vector detection unit 125 determines whether or not there is a motion vector in the generated histogram whose value is within the range of the electronic correction amount E±th. If such a motion vector is present, the representative vector detection unit 125 proceeds to step S105, and if such a motion vector is not present, the representative vector detection unit 125 proceeds to step S106.

[0058] In step S105, the representative vector detection unit 125 determines the value of the motion vector that is detected the greatest number of times among the motion vectors whose values ​​are within the range of the electronic correction amount E±th as the value of the representative vector, and then proceeds to step S .

[0059] In step S106, the representative vector detection unit 125 sets the value of the representative vector to the amount of electronic correction E. If a motion vector with a value within the range of E±th is not detected, it may be the case that the motion vector cannot be detected due to low contrast, or a moving subject is captured over the entire frame image. In such cases, even if a motion vector is detected, it is unreliable, and therefore electronic shake correction can be performed more accurately by setting the image cutout position based on the amount of electronic correction E obtained from the lens 100. For this reason, in step S106, the amount of electronic correction E is used as the value of the representative vector. Then, the process proceeds to step S108.

[0060] In step S108, the representative vector detection unit 125 outputs the value of the representative vector determined in step S105, S106, or S107 to the image deformation amount calculation unit 129. This causes the electronic shake correction by the image deformation unit 124 to be controlled according to the representative vector. After this, the representative vector detection unit 125 ends this process.

[0061] As described above, in this embodiment, in a camera system in which the lens 100 and the camera 120 cooperate to perform optical shake correction and electronic shake correction, image shake including the shake correction residue from optical shake correction can be corrected by electronic shake correction, compared to a case in which electronic shake correction is performed using the amount of electronic shake correction calculated by the lens 100 as is. This makes it possible to improve the shake correction effect of the entire camera system.

[0062] In the above embodiment, the case where the shake detection unit 101, the division unit 104, and the pixel unit conversion unit 112 are provided in the lens 100 has been described, but these may also be provided in the camera. When the division unit and the pixel unit conversion unit are provided in the camera, the camera acquires information on the maximum optical correction amount from the lens, determines a correction share ratio from the maximum optical correction amount and the maximum electronic correction amount in the division unit (acquisition means), calculates (acquires) the optical correction amount and the electronic correction amount, and transmits the optical correction amount to the lens 100 to cause optical shake correction to be performed. In this configuration, as in the above embodiment, the camera determines a representative vector using the electronic correction amount calculated from the correction share ratio.

[0063] In the above embodiment, the electronic blur correction is performed by moving the image extraction position in the camera 120, but the image sensor may be moved in a direction perpendicular to the optical axis to perform the blur correction optically. In this case, the amount of movement of the image sensor may be set according to the representative vector. The representative vector corresponding to each frame may be stored, and the representative vector of the corresponding frame may be predicted from the transition of the stored representative vector, or a sensor capable of acquiring the motion vector during the exposure for acquiring the image may be used.

[0064] In addition to electronic image stabilization that moves the image cutout position for each frame, electronic image stabilization may be performed in which a plurality of images whose image cutout positions have been changed according to the amount of shake are combined to generate an image for one frame. In this case, the amount of movement of the image cutout position may be set according to the representative vector. Image stabilization may also be performed in which a plurality of images whose image stabilization has been performed by moving the imaging element are combined to generate an image for one frame. (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-mentioned embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0065] The embodiments described above are merely representative examples, and various modifications and alterations are possible for each embodiment when implementing the present invention. [Explanation of symbols]

[0066] 100 Interchangeable Lenses 101 Shake detection unit 104 Division 110 Correction Lenses 112 Pixel unit conversion section 120 Imaging device (camera) 121 Image sensor 123 Motion Vector Detector 124 Image transformation section 125 Representative Vector Detection Unit 129 Image deformation amount calculation unit

Claims

1. An imaging device to which an interchangeable lens can be attached and detached, the interchangeable lens performing a first shake correction for vibration, a correction means for performing a second vibration correction on the vibration; A vector detection means for detecting motion vectors of a plurality of regions in an image obtained by imaging; A representative vector determining means for determining a representative vector from the motion vectors of the plurality of regions; a control unit that controls the second shake correction in accordance with the representative vector; The representative determination means is An imaging device characterized in that the representative vector is determined based on a second correction amount obtained based on a shake angle of the imaging device and a share of the second shake correction in the shake correction by both the first and second shake corrections.

2. The imaging device according to claim 1, characterized in that the representative determination means determines the motion vector detected the most frequently as the representative vector among motion vectors whose values ​​are within a predetermined range including the second correction amount or motion vectors whose difference from the second correction amount is less than a predetermined value.

3. 3. The imaging device according to claim 2, wherein the control means controls the second shake correction using the second correction amount when there is no motion vector whose value is within the predetermined range or whose difference is less than or equal to the predetermined value.

4. In the first shake correction, an optical element is moved; 4. The imaging apparatus according to claim 1, wherein in the second image blur correction, a position where the image is cut out is moved.

5. In the first shake correction, an optical element is moved; 4. The imaging apparatus according to claim 1, wherein the second image blur correction comprises moving an image sensor that captures an image of the subject.

6. the interchangeable lens acquires a first correction amount and a second correction amount, which are correction amounts of the first shake correction, based on a sharing ratio between the first shake correction and the second shake correction and the angle; 6. The imaging apparatus according to claim 1, wherein the representative determination unit acquires the second correction amount from the interchangeable lens.

7. A detection means for detecting the angle; an acquisition unit that acquires a first correction amount and a second correction amount, which are correction amounts of the first shake correction, based on the angle detected by the detection unit and the allocation ratio, The imaging apparatus according to claim 1 , wherein the representative determination unit acquires the second correction amount from the acquisition unit.

8. An imaging device with a detachable interchangeable lens, An acquisition unit that acquires a correction amount of shake correction from the interchangeable lens; A vector detection means for detecting motion vectors of a plurality of regions in an image obtained by imaging; and a control means for controlling the shake correction based on the motion vector whose difference from the correction amount is equal to or less than a predetermined value when the motion vectors of the plurality of regions include a motion vector whose difference from the correction amount is equal to or less than a predetermined value, and for controlling the shake correction based on the correction amount when the motion vectors of the plurality of regions include no motion vector whose difference from the correction amount is equal to or less than the predetermined value.

9. A control method for an imaging device in which an interchangeable lens that performs a first shake correction for vibration can be attached and detached and which performs a second shake correction for the vibration, comprising: Detecting motion vectors of a plurality of regions in an image obtained by capturing an image; determining a representative vector from the motion vectors of the plurality of regions; and controlling the second shake correction in accordance with the representative vector. In the step of determining the representative vector, A method for controlling an imaging device, comprising: determining the representative vector based on a second correction amount obtained based on a shake angle of the imaging device and a proportion of the second shake correction accounted for in the shake correction by both the first and second shake corrections.

10. A method for controlling an imaging device to which an interchangeable lens can be attached and detached, comprising the steps of: acquiring a correction amount of shake correction from the interchangeable lens; Detecting motion vectors of a plurality of regions in an image obtained by capturing an image; a step of controlling the shake correction based on the motion vector that is equal to or less than the predetermined value when the motion vectors of the plurality of regions include a motion vector whose difference from the correction amount is equal to or less than the predetermined value, and a step of controlling the shake correction based on the correction amount when the motion vectors of the plurality of regions include no motion vector whose difference from the correction amount is equal to or less than the predetermined value.

11. 10. A computer program product for causing a computer of an imaging device that is capable of attaching and detaching an interchangeable lens that performs a first shake compensation against vibration and that performs a second shake compensation against the vibration to execute processing in accordance with the control method according to claim 9.

12. A computer program for causing a computer of an image pickup apparatus to which an interchangeable lens is detachably attached to execute a process according to the control method recited in claim 10.

Images