Imaging apparatus, lens device, and control method

Abstract

To appropriately perform shake correction even when a motion vector is not used for shake detection.SOLUTION: An imaging apparatus allows a lens device to be removably attached thereto, and the imaging apparatus has: first detection means that detects a shake added to the imaging apparatus; second detection means that detects a shake from an image acquired by an image pick-up device; first correction means that performs shake correction by using output from at least one of the first detection means and the second detection means; restriction means that restricts the use of output from the second detection means; and setting means that sets a system when performing the shake correction. When the lens device comprises second correction means that performs the shake correction and the use of output from the second detection means for the shake correction is restricted, the setting means sets a system with a higher correction effect, from a first system of increasing a correction ratio of the first correction means compared to a correction ratio of the second correction means to perform the shake correction, and a second system of increasing the correction ratio of the second correction means compared to the correction ratio of the first correction means to perform the shake correction.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an imaging device and a lens device that have a correction unit that performs shake correction. [Background technology]

[0002] Conventionally, there has been known an interchangeable lens camera that corrects shake detected in an interchangeable lens by moving a correction lens provided as part of the lens group, and then detects and corrects the remaining amount of shake that could not be corrected by the interchangeable lens in the camera body using a motion vector. However, when a moving subject such as a person, animal, or car (hereinafter referred to as a moving object) is photographed, the motion vector is affected by the movement of the moving object, making it impossible to accurately detect shake. Patent Document 1 discloses a configuration that does not use motion vectors in a shooting mode in which moving objects are likely to be photographed, i.e., a shooting mode in which erroneous detection of motion vectors is likely to occur. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2018-159957 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the above-mentioned camera, if the use of motion vectors is stopped as in the configuration of Patent Document 1, the camera body cannot detect the remaining amount of shake and cannot perform shake correction, which results in a decrease in the shake correction performance of the camera as a whole.

[0005] An object of the present invention is to make it possible to appropriately perform shake correction even when a motion vector is not used for shake detection. [Means for solving the problem]

[0006] An imaging device according to one aspect of the present invention is an imaging device with a detachable lens device, comprising: a first detection means for detecting shake applied to the imaging device; a second detection means for detecting shake from an image acquired by an image sensor; a first correction means for performing shake correction using the output of at least one of the first detection means and the second detection means; a restriction means for restricting use of the output of the second detection means; and a setting means for setting a method for performing shake correction, wherein when the lens device is equipped with a second correction means for performing shake correction and use of the output of the second detection means for shake correction is restricted, the setting means sets the method with the higher correction effect from among a first method for performing shake correction by making the correction ratio of the first correction means larger than that of the second correction means, and a second method for performing shake correction by making the correction ratio of the second correction means larger than that of the first correction means. [Effects of the Invention]

[0007] According to the present invention, it is possible to appropriately perform shake correction even when motion vectors are not used for shake detection. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a camera system according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating the relationship between a shake correction method and factors that determine the shake correction method. [Figure 3] 10 is a flowchart showing a process performed by a shake correction method determination unit. [Figure 4] FIG. 10 is an explanatory diagram of a camera shake correction mode. [Figure 5] FIG. 10 is a diagram illustrating an example of the configuration of a camera according to a second embodiment. [Figure 6] FIG. 10 is a diagram showing the results of histogram processing of a face region and a motion vector. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same components, and redundant explanations will be omitted. First Embodiment 1 is a diagram showing an example of the configuration of a camera system according to this embodiment. The camera system includes a camera (imaging device) 100 and an interchangeable lens (lens device) 101. The camera 100 is an interchangeable lens camera to which the interchangeable lens 101 is detachable. The interchangeable lens 101 is one of the lenses that can be attached to the camera 100.

[0010] The interchangeable lens 101 includes a lens unit 102 that includes a focus lens 103 for adjusting focus, a zoom lens 104 with adjustable focal length, a shift lens (second correction means) 105 for image blur correction, and an aperture 106. The shift lens 105 is movable in directions that include a component perpendicular to the optical axis direction, and by moving in response to shake applied to the interchangeable lens 101, it is possible to optically correct image shake relative to the optical axis. The interchangeable lens 101 is provided with ring operating members (not shown) for operating the focus lens 103, zoom lens 104, and aperture 106. By operating the ring operating members, the photographer can change the focal length, focus on the subject, and change the exposure by adjusting the aperture. The focus lens 103 and zoom lens 104 are each equipped with a position detection encoder (not shown). This allows the current position of each lens to be determined, and the subject distance and focal length to be calculated. The aperture 106 changes its aperture diameter by moving its aperture blades with a stepping motor (not shown). In this embodiment, the current aperture diameter information can be acquired. For example, the aperture diameter information can be acquired based on rotation angle information from a sensor that detects the rotation angle of a stepping motor. An imaging light beam from a subject passes through the lens unit 102 and enters the image sensor 116 of the camera 100.

[0011] The angular velocity sensor (detection means) 107 detects shake applied to the interchangeable lens 101 as an angular velocity and outputs a signal corresponding to the angular velocity. Note that shake may be detected using a sensor that detects acceleration or angular acceleration instead of the angular velocity sensor 107. Furthermore, if the angular velocity sensor 107 is an analog sensor that changes its output voltage according to the detected angular velocity, the signal from the angular velocity sensor 107 is converted into digital data via an A / D converter and sent to the lens correction amount calculation unit 108. If the angular velocity sensor 107 is a digital sensor, angular velocity information acquired as digital data by a serial communication unit (not shown) is sent to the lens correction amount calculation unit 108.

[0012] The lens correction amount calculation unit 108 calculates a movement target position of the shift lens 105 for performing shake correction. The lens correction amount calculation unit 108 performs, for example, high-pass filter calculation processing to remove deviations from the reference value of the output of the angular velocity sensor 107, and integration processing to calculate an angle from the angular velocity. The lens correction amount calculation unit 108 also performs unit conversion processing to convert information in angle units to the unit of position information of the shift lens 105, and correction processing of the movement target position of the shift lens 105 according to the focal length and subject distance.

[0013] The operation switching control unit 109 outputs movement target position information for the shift lens 105 to the motion compensation lens control unit 110 to execute or stop the motion compensation function in accordance with the output of a stop instruction unit 114 (described later). When executing motion compensation, the operation switching control unit 109 transmits the output of the lens correction amount calculation unit 108 to the motion compensation lens control unit 110. When stopping motion compensation, the operation switching control unit 109 fixes the shift lens 105 to the optical axis center position or outputs a movement target position for maintaining the current position. The motion compensation lens control unit 110 controls the shift lens 105 to move to a predetermined position through feedback control. The motion compensation lens control unit 110 calculates the difference between the movement target position obtained from the operation switching control unit 109 and the current position of the shift lens 105 obtained from the position detection sensor 112, and calculates the motor drive amount based on the difference. In addition to the process of calculating the difference between the current position and the movement target position described above, the motion compensation lens control unit 110 also performs an amplification process to amplify the calculated difference by a predetermined gain and a phase compensation filter process to compensate the phase of the amplified difference. The data output from the image stabilization lens control unit 110 is modulated by a pulse width modulator (not shown) into a PWM waveform that changes the duty ratio of the pulse wave, and is supplied to the motor 111. The motor 111 includes an H-bridge circuit and a voice coil motor, and generates a driving force that moves the shift lens 105 by applying a voltage to the terminals of the voice coil motor based on the PWM waveform and changing the amount of current flowing through the coil.

[0014] Position detection sensor 112 includes a magnet and a Hall sensor provided opposite it. As shift lens 105 moves, the magnetic flux acting on the Hall sensor changes, causing a change in the output voltage from the Hall sensor. Position detection sensor 112 may also include a differential amplifier circuit that amplifies the output of the Hall sensor. The output of position detection sensor 112 is input to blur correction lens control unit 110 via an A / D converter (not shown), and as described above, blur correction lens control unit 110 recalculates the difference between the input output of position detection sensor 112 and the target position information, and recalculates the motor drive amount. In this way, feedback control is configured to constantly correct the difference between the current position and the target position.

[0015] The lens IF 113 performs serial communication with the camera 100 via the mount contact at a predetermined timing, transmitting data to the camera 100 and receiving data from the camera 100. The camera 100 can instruct the interchangeable lens 101 to stop shake correction, and the instruction to stop shake correction from the camera 100 is input to the stop instruction unit 114 via the lens IF 113. The interchangeable lens 101 may also be equipped with a locking mechanism for fixing the shift lens 105 in a predetermined position. In this case, upon receiving an instruction to stop shake correction from the camera 100, the interchangeable lens 101 controls the locking mechanism via the stop instruction unit 114 and fixes and holds the shift lens 105 with the locking mechanism. The locking mechanism can also determine a stationary state based on the output of the angular velocity sensor 107, and lock the shift lens 105 based on the determination of the interchangeable lens 101.

[0016] The image sensor 116 is composed of a CCD, CMOS, or other element, and photoelectrically converts an optical image formed by the lens unit 102. An image signal representing a captured image made up of multiple pixels undergoes predetermined signal processing in a signal processing unit 117. The signal processing unit 117 performs noise reduction, gain adjustment, and AD conversion on the image signal read from the image sensor 116, converting it into digital data, and then performs image processing such as pixel interpolation and color conversion to generate image data for each frame. The image data for each frame is stored in an image memory 118. An image transformation unit 121 changes the range read from the image memory 118 and the read position of each pixel. The image transformation unit 121 corrects misalignment between frames by changing the range read from the image memory 118 and the read position of each pixel in a direction that cancels out the misalignment between images caused by shake of the camera 100. In other words, the image transformation unit 121 functions as a correction unit (first correction unit) that realizes electronic image stabilization by controlling the image read position. The video data output from the image transformation unit 121 is supplied to a recording control unit 122. The recording control unit 122 acquires instructions from the photographer to start or stop recording from the operation unit 130 or the like, and when an instruction to record a video signal is given, the recording control unit 122 converts the video data supplied from the image transformation unit 121 into a format suitable for recording to the recording medium 123, and records the converted data on the recording medium 123. The output of the image transformation unit 121 is also supplied to a display control unit 124. The display control unit 124 drives a display device 125, and the display device 125 displays an image using a liquid crystal display element (LCD) or the like.

[0017] The motion vector detection unit (second detection means) 119 detects a motion vector of an image based on the luminance signal contained in the current video signal generated by the signal processing unit 117 and the luminance signal contained in the video signal of the previous frame stored in the image memory 118. A block matching method, for example, may be used as a method for detecting a motion vector. The block matching method divides a captured image into regions called blocks, and detects similar portions between the captured image of the previous frame and the current captured image on a block-by-block basis. The location within an arbitrary range in the captured image of the previous frame that has the largest correlation value with an arbitrary block in the current captured image is determined as the similar block position. The amount of displacement between the arbitrary block position in the current captured image and the similar block position in the captured image of the previous frame is calculated, and motion information between the captured image frames, i.e., the motion vector, is detected.

[0018] Furthermore, the motion vector detection unit 119 determines one representative motion vector from the entire captured image based on the motion vectors for each block. For example, a method for calculating the representative motion vector may be used in which a histogram (frequency distribution) showing the frequency (frequency) of the magnitude (class) of the motion vector for each block is generated, and the motion vector with the highest frequency is determined as the representative motion vector. In the following description, the term "motion vector" refers to the representative motion vector.

[0019] The image deformation amount calculation unit 120 determines the amount of deformation of the image, such as the read position from the image memory 118, so as to correct the discrepancy in the subject position between frames based on the input motion vector, and the image deformation unit 121 applies deformation processing to the image based on the amount of deformation.

[0020] Like the lens IF 113, the camera IF 128 performs serial communication with the interchangeable lens 101 via the mount contact at predetermined times, transmitting data to and receiving data from the interchangeable lens 101. The shake correction method determination unit (setting means) 129 acquires information from the interchangeable lens 101 and determines (sets) the shake correction method. Specifically, it acquires information on whether the currently attached interchangeable lens 101 has a shake correction device, and if the interchangeable lens 101 has a shake correction device such as the shift lens 105, the camera 100 calculates the amount of image deformation based on the output of the motion vector detection unit 119. As a result, after shake correction using the shift lens 105 is performed, the amount of shake remaining after shake correction can be detected based on the motion vector. If the interchangeable lens 101 does not have a shake correction device, the camera 100 performs shake correction using an angular velocity sensor (first detection means) 126. The angular velocity sensor 126 detects shake applied to the camera 100 as an angular velocity and outputs a signal corresponding to the angular velocity. Instead of the angular velocity sensor 126, a sensor that detects angular acceleration or acceleration may be used to detect shake. The output of the angular velocity sensor 126 is converted by a unit conversion unit 127 from angular velocity units to pixel units output by the motion vector detection unit 119. It is assumed that high-pass filter calculation processing for removing reference value deviations in the output of the angular velocity sensor 107 is also performed within this processing. The operation unit 130 is a user interface that allows the photographer to perform various menu operations, mode switching operations, and the like. As described above, the operation unit 130 can also issue instructions for shooting and recording, and select whether or not to use motion vectors in image stabilization. In other words, the operation unit 130 functions as a restriction unit that restricts the use of motion vectors.

[0021] When the photographer selects whether or not to use motion vectors in shake correction, camera 100 switches the shake correction method depending on whether or not the attached interchangeable lens 101 has a shake correction function. The shake correction methods for these combinations are explained below. Figure 2 is a diagram showing the relationship between shake correction methods and the factors that determine the shake correction method.

[0022] If the attached interchangeable lens 101 does not have a shake correction means such as a shift lens 105, shake correction is performed by the camera 100 alone. In this case, electronic shake correction is performed by the image deformation amount calculation unit 120 based on the shake detected by the angular velocity sensor 126. This method is referred to as the first shake correction method.

[0023] If the attached interchangeable lens 101 has a shake correction mechanism, the shake correction method varies depending on whether the photographer selects whether to use a motion vector. If the photographer selects to use a motion vector, the interchangeable lens 101 detects shake using the angular velocity sensor 107 and performs shake correction using the shift lens 105. The camera 100 detects the shake correction remaining from the shift lens 105 using the motion vector detection unit 119 and calculates the amount of image deformation based on this. At this time, the angular velocity sensor 126 is not used. This method is referred to as the second shake correction method. If the interchangeable lens 101 has a shake correction mechanism and the photographer selects not to use a motion vector, the method is determined based on the magnitude (height) of the shake correction effect between the camera 100 and the interchangeable lens 101. In this embodiment, the magnitude of the shake correction effect refers to the angle at which shake correction is possible. However, the magnitude of the shake correction effect may also be determined based on other factors, such as the resolution of the shake correction control and the variable frequency band of the shake correction mechanism.

[0024] If the effect of shake correction on the interchangeable lens 101 side is large, the interchangeable lens 101 performs shake correction using the shift lens 105, as in the second shake correction method, but the camera 100 stops shake correction. This method is referred to as the third shake correction method (second method). Methods for stopping shake correction in the camera 100 include, for example, a method in which the image deformation amount calculation unit 120 outputs an image so that no deformation is performed, and a method in which no deformation is performed on the image output from the image memory 118, regardless of the amount of deformation acquired by the image deformation unit 121. Another method is to stop output from the motion vector detection unit 119 and the unit conversion unit 127 and set the image deformation amount to an invalid value.

[0025] If the shake correction effect on the camera 100 side is large, the camera 100 performs shake correction using the angular velocity sensor 126, as in the first shake correction method, without using the output of the motion vector detection unit 119. The camera 100 also sends a command to the interchangeable lens 101 to stop shake correction, causing the interchangeable lens 101 to stop shake correction. This method is referred to as the fourth shake correction method (first method). The fourth shake correction method is equivalent to the first shake correction method in that shake correction is performed only by the camera 100, without using the shake correction of the interchangeable lens 101, but differs from the first shake correction method in that the camera 100 explicitly commands the interchangeable lens 101 to stop shake correction.

[0026] In this embodiment, if the attached interchangeable lens 101 does not have a shake correction unit, the first shake correction method is used. However, even in this case, the photographer may change the processing depending on whether or not to use a motion vector. For example, the photographer can change whether or not to use a motion vector to correct the reference value deviation of the angular velocity sensor 126. The angular velocity sensor 126 generally generates a reference value deviation (a state in which the output does not become zero when no angular velocity is generated). As described above, a method may be used to attenuate the reference value deviation using a high-pass filter, or a method may be used in which the reference value deviation is estimated based on the motion vector and corrected by subtracting or adding the estimated deviation from the output of the angular velocity sensor 126. The use of a motion vector to estimate the reference value of the angular velocity sensor 126 may be linked to whether or not the photographer uses the motion vector.

[0027] The flow of the process for switching between the first to fourth image stabilization methods will be described below. Fig. 3 is a flowchart showing the processing by the image stabilization method determination unit 129. The flow of Fig. 3 may be started in synchronization with the frame rate or the like, or may be started in conjunction with communication with the interchangeable lens 101 or operation of the operation unit 130 by the photographer.

[0028] In step S101, the image stabilization method determination unit 129 acquires information about the interchangeable lens 101 from the lens information notification unit 115 via the camera IF 128. The acquired information here is information about whether the interchangeable lens 101 has a motion compensation device and information about the angle at which motion compensation is possible for the shift lens 105.

[0029] In step S102, shake correction method determination unit 129 acquires information about camera 100. Shake correction method determination unit 129 acquires shake correction possible angle information on the camera 100 side from a camera memory (not shown) that is built into camera 100 and stores various data such as parameters used for control in camera 100.

[0030] In step S103, shake correction method determination unit 129 acquires information (operation information) from operation unit 130. The information from operation unit 130 is information on whether or not a motion vector is to be used for shake correction. If the photographer has not performed any menu operations after purchasing camera 100, the initial value of camera 100 is acquired. The initial value is set to either whether or not a motion vector is to be used. Furthermore, the photographer's menu operations are stored in the camera memory described above, and the previous operation information is also stored the next time the camera is started up.

[0031] In step S104, the image stabilization method determination unit 129 determines whether or not the interchangeable lens 101 has image stabilization means. If the image stabilization method determination unit 129 determines that the interchangeable lens 101 has image stabilization means, it executes the process of step S105, and if it determines that the interchangeable lens 101 does not have image stabilization means, it executes the process of step S107.

[0032] In step S105, the shake correction method determination unit 129 determines the shake correction method to be the first shake correction method.

[0033] In step S106, the shake correction method determination unit 129 determines whether or not to use a motion vector. If it determines that a motion vector should be used, the shake correction method determination unit 129 executes the process of step S107, and if it determines that a motion vector should not be used, the shake correction method determination unit 129 executes the process of step S108.

[0034] In step S107, the shake correction method determination unit 129 determines the shake correction method to be the second shake correction method.

[0035] In step S108, the shake correction method determination unit 129 determines whether the shake correction effect of the interchangeable lens 101 is greater than that of the camera 100. If the shake correction method determination unit 129 determines that the shake correction effect of the interchangeable lens 101 is greater than that of the camera 100, it executes the processing of step S109, and if it determines that this is not the case, it executes the processing of step S110. Note that if the shake correction effect of the interchangeable lens 101 is equal to that of the camera 100, it is possible to arbitrarily set which step of processing to execute.

[0036] In step S109, the shake correction method determination unit 129 determines the shake correction method to be the third shake correction method.

[0037] In step S110, the shake correction method determination unit 129 determines the shake correction method to be the fourth shake correction method.

[0038] After this flow is completed, as described above, the process of step S101 is executed again when the next frame image is generated or when the photographer performs an operation or the like.

[0039] Note that the method for determining the shake compensation method is not limited to the method described in FIG. 3. As an example, a method for determining the shake compensation method when camera 100 has two camera shake compensation modes with different shake compensation effects will be described. Here, the mode with the smallest shake compensation effect is defined as the first camera shake compensation mode, and the mode with the largest shake compensation effect is defined as the second camera shake compensation mode. Depending on the camera shake compensation mode, camera 100 changes the cropping range within the image and changes the surplus area for shake compensation. The camera shake compensation mode can be selected by the photographer using operation unit 130.

[0040] FIG. 4 is an explanatory diagram of the camera shake correction modes. FIG. 4(a) shows the captured image, recorded image, and surplus area for shake correction in the first camera shake correction mode. FIG. 4(b) shows the captured image, recorded image, and surplus area for shake correction in the second camera shake correction mode. The range of the recorded image is smaller in the second camera shake correction mode than in the first camera shake correction mode, so a larger surplus area for shake correction is obtained. The larger the surplus area, the more the crop position can be changed, and therefore the larger the surplus area obtained, the greater the angle of shake that can be corrected.

[0041] The recorded images of FIGS. 4(a) and 4(b) shown in FIGS. 4(c) and 4(d) are recorded on a recording medium and displayed on display device 125. Images recorded in first camera shake compensation mode are wider-angle images than images recorded in second camera shake compensation mode. In other words, the first camera shake compensation mode provides a smaller shake compensation effect but wider-angle shooting, while the second camera shake compensation mode provides a telephoto angle but greater shake compensation effect. Here, the shake compensation angle available for interchangeable lens 101 is assumed to be larger than that in the first camera shake compensation mode but smaller than that in the second camera shake compensation mode. In this way, if the magnitude relationship between the shake compensation effect and interchangeable lens 101 is determined in advance by the camera shake compensation mode, shake compensation method determination unit 129 can omit obtaining information about the shake compensation angle of interchangeable lens 101. Meanwhile, operation unit 130 also obtains information about whether motion vectors are used, as well as information about the camera shake compensation mode selected by the photographer. 3, but if the first camera shake compensation mode is selected in step S108, the process of step S109 is executed, and if the second camera shake compensation mode is selected, the process of step S110 is executed. In this way, if camera 100 is equipped with multiple compensation modes with different shake compensation effects, and the magnitude relationship between the shake compensation effects of interchangeable lens 101 and camera 100 is determined in advance depending on the selected mode, the shake compensation method may be determined by the camera shake compensation mode.

[0042] Furthermore, the shake correction methods when motion vectors are not used are not limited to the methods described above. While the third shake correction method stops shake correction in the camera 100, and the fourth shake correction method stops shake correction in the interchangeable lens 101, a method in which the camera 100 and the interchangeable lens 101 share the amount of correction may also be used. For example, in the fourth shake correction method, if the detected shake angle exceeds the shake correction possible angle on the camera 100 side, the interchangeable lens 101 may be made to share the shake correction. The shake correction method determination unit 129 determines the respective sharing ratios (correction ratios) based on the ratio of the shake correction possible angle on the camera 100 side to the shake correction possible angle on the interchangeable lens 101 side, and notifies the interchangeable lens 101 of the sharing ratio. The interchangeable lens 101 calculates the correction amount in the lens correction amount calculation unit 108 by multiplying the output of the angular velocity sensor 107 by a predetermined gain according to the sharing ratio, and the camera 100 calculates the correction amount in the unit conversion unit 127 by multiplying the output of the angular velocity sensor 126 by a gain according to the sharing ratio. Furthermore, in the third shake correction method, the correction ratio of the shake correction of the interchangeable lens 101 may be made larger and the correction ratio of the shake correction of the camera 100 may be made smaller than in the fourth shake correction method. Also, in the second shake correction method, a method may be used in which the amount of correction is shared between the camera 100 and the interchangeable lens 101 based on the correction ratio. In that case, in the third shake correction method, the correction ratio of the shake correction of the interchangeable lens 101 may be made larger and the correction ratio of the shake correction of the camera 100 may be made smaller than in the second shake correction method. Furthermore, in the fourth shake correction method, the correction ratio of the shake correction of the interchangeable lens 101 may be made smaller and the correction ratio of the shake correction of the camera 100 may be made larger than in the second shake correction method.

[0043] As described above, with the configuration of this embodiment, if the photographer wishes to stop using motion vectors, the shake correction method is determined according to the shake correction effects of the camera 100 and the interchangeable lens 101. This makes it possible to provide the photographer with a higher quality shake correction function.

[0044] In this embodiment, the interchangeable lens 101 performs optical shake correction using the shift lens 105, and the camera 200 performs electronic shake correction, but other shake correction means may also be used. For example, the image sensor 116 may be mounted on a movable stage, and the position of the image sensor 116 may be moved based on detected shake. In this case, the image sensor 116 may function as a first correction means. Furthermore, if the camera 200 is configured to perform both electronic shake correction and shake correction that moves the position of the image sensor 116 (in-camera optical shake correction), both electronic shake correction and in-camera optical shake correction may be performed when electronic shake correction is performed. In this case, the shake correction method determination unit 129 may determine the sharing ratio between the electronic shake correction and the in-camera optical shake correction based on the correctable angle for each. <Second embodiment> FIG. 5 is a diagram showing an example of the configuration of camera 200 of this embodiment. Unlike camera 100 of the first embodiment, camera 200 has a function that prompts the photographer to select to stop using motion vectors when there is a possibility of erroneous detection of a motion vector. Specifically, because erroneous detection of a motion vector is likely to occur when a person is photographed large within the screen, an icon or the like is displayed on the display device in accordance with the detected face size to notify that there is a high possibility of erroneous detection of a motion vector. Note that components of camera 200 that are the same as those of camera 100 will be designated by the same reference numerals as those already used, and detailed description thereof will be omitted.

[0045] The signal processing unit 201 also outputs the video signal to the face detection unit 202. The face detection unit 202 outputs the result of determining whether a face is present in the video, and, if a face is present, information on the size of the face area and the position of the face. Face detection is performed using a process of determining whether the image matches a template in which the eyes, nose, mouth, etc. that make up the face are arranged.

[0046] The motion vector detection unit 203 also outputs information about the motion vector to the motion vector reliability determination unit 204. Similar to the motion vector detection unit 119, the motion vector detection unit 203 determines a representative motion vector for the entire captured image from the motion vector for each block. The representative motion vector is output to the image deformation amount calculation unit 120, but information about the motion vector for each block is output to the motion vector reliability determination unit 204. The motion vector reliability determination unit 204 determines the reliability of the motion vector based on face information and motion vector information.

[0047] A warning display determination unit (notification unit) 205 determines whether or not to display a warning display depending on the reliability of the motion vector. A display control unit 206 generates data for displaying a warning icon or the like on a display device 207 based on the output of the warning display determination unit 205. The display device 207 displays the warning display icon or the like superimposed on the video.

[0048] The following describes the determination made by the motion vector reliability determination unit 204. Figure 6 shows the results of histogram processing of face regions and motion vectors. Figure 6(A) shows an image in which the person in the image is captured prominently. Figure 6(B) shows an image in which the person is captured small. Figure 6(C) shows a histogram of motion vectors in each block detected from the image in Figure 6(A). Figure 6(D) shows a histogram of motion vectors in each block detected from the image in Figure 6(B). The histograms in Figures 6(C) and 6(D) have frequency on the vertical axis and class on the horizontal axis. Here, frequency represents the number of detected motion vectors, and class represents the magnitude of the motion vector. In Figure 6(C), the person occupies a larger proportion of the screen than fixed objects such as buildings, so motion vectors related to the movement of the person (hereinafter referred to as "person vectors") are detected in more blocks than vectors of fixed objects (hereinafter referred to as "background vectors"). In Figure 6(D), background vectors are detected in more blocks than the person vector. If the representative motion vector were determined based on the number of detected vectors, the person's motion vector would be determined as the representative motion vector in FIG. 6(C). In this case, the person's vector is input as the representative motion vector to the image deformation amount calculation unit 120. If electronic image stabilization were performed based on this information, the cropping position would change in accordance with the person's movement, resulting in incorrect image stabilization. In other words, it is not desirable to use a representative motion vector determined from an image in which a person is prominent, as shown in FIG. 6(A), for image stabilization. To avoid this situation, the motion vector reliability determination unit 204 sets a low value for the reliability of the representative motion vector for the image in FIG. 6(A).

[0049] One method for determining the reliability of a motion vector is to estimate the proportion of a person in a captured image and compare that proportion with the proportion of fixed subjects. The motion vector reliability determination unit 204 estimates the size of the torso based on the size of the face region and calculates the estimated size of the entire person. The torso size can be estimated by multiplying the size of the face region by a predetermined coefficient. The face detection unit 202 may classify the person as an adult or child based on the positions of the eyes, nose, and mouth within the face region, and may have multiple coefficients for torso estimation accordingly. Furthermore, a face region detected as a profile may be smaller than a face facing forward. In this case, a method may be used in which the size of the face region is converted to a size equivalent to a forward-facing face before estimating the torso size. If the estimated size of the person calculated in this way occupies more than half of the captured image, the reliability of the motion vector is reduced. If the estimated size of the person, as shown in Figure 6(A), is more than half of the captured image, the motion vector reliability determination unit 204 outputs a motion vector reliability value less than a predetermined value. The warning display determination unit 205 determines to display a warning display when the reliability value is less than a predetermined value.

[0050] The determination method used by the motion vector reliability determination unit 204 is not limited to the method described above. If the motion vector detection unit 203 can identify a building from the characteristics of the captured image, the reliability of the motion vector may be determined by comparing the size of the person with the size of the building. Buildings can be determined by calculating the edges of objects in the captured image by binarizing the pixels in the captured image and determining whether they are man-made buildings based on the amount of linearity contained in the edges. In the captured images of Figures 6(A) and (B), if the area other than the person and building is the entire sky or a concrete ground, the contrast between the sky and ground is low, making it difficult to identify similar areas using block matching. If block matching is not possible, the motion vector detection unit 203 outputs motion vector detection error information from that block. The motion vector reliability determination unit 204 compares the size of the area estimated to be a person with the size of the area estimated to be a building within the captured image, excluding the area where the motion vector detection error was output.

[0051] Other determination methods by the motion vector reliability determination unit 204 are described below. Assume that the interchangeable lens 101 is attached to the camera 200, and that reliability is determined based on depth-of-field information. Assume that the interchangeable lens 101 can calculate information on the focal length, subject distance, and aperture diameter, and that the warning display determination unit 205 can acquire this information via the lens IF 113 and the camera IF 128. The warning display determination unit 205 calculates depth-of-field information from the focal length, subject distance, and aperture diameter information, and outputs a warning if the depth-of-field is shallow. When a person is focused and the depth-of-field is set to a shallow value, areas other than the person become blurred, making it highly likely that a motion vector cannot be detected. In this case, since a motion vector detection error is likely to occur, the reliability of the motion vector may be determined based on an area excluding the motion vector detection error area, as described above, or the reliability may be determined directly from the depth-of-field information.

[0052] In this embodiment, the camera 200 determines the possibility of erroneous detection of a motion vector, and notifies the photographer of the determination result, thereby enabling more appropriate operation of the camera 200.

[0053] In this embodiment, the motion vector reliability determination unit 204 determines the reliability of a motion vector using the area of ​​a person included in the captured image, but the present invention is not limited to this. The camera 100 may be provided with a means capable of detecting the area of ​​another moving object, such as an animal or vehicle, included in the captured image, and the reliability may be determined using the detected area of ​​the moving object included in the captured image.

[0054] In addition, in this embodiment, the warning display determination unit 205 determines whether to display a warning display based on the reliability of the motion vector, but the present invention is not limited to this. The warning display determination unit 205 may be configured to determine that a warning display should be displayed when the size of the moving object area matches an area in the captured image where the motion vector is highly reliable. Here, "match" includes not only a strict match but also a substantial match.

[0055] In addition, in each embodiment, the motion vectors are stopped by the photographer operating the camera, but the present invention is not limited to this. For example, the camera may automatically switch to a control mode that does not use motion vectors based on the results of the motion vector reliability determination. In this case, the camera functions as a limiting means for limiting the use of motion vectors.

[0056] The disclosure of this embodiment includes the following configuration. (Configuration 1) An imaging device with a detachable lens device, a first detection means for detecting a shake applied to the imaging device; a second detection means for detecting a shake from an image acquired by the imaging element; a first correction means for performing blur correction using an output of at least one of the first detection means and the second detection means; a limiting means for limiting use of the output of the second detecting means; a setting means for setting a method for performing shake correction; and when the lens device is equipped with a second correction means for performing image blur correction and use of the output of the second detection means for image blur correction is restricted, the setting means selects a method with a higher correction effect from among a first method for performing image blur correction by making the correction ratio of the first correction means larger than that of the second correction means, and a second method for performing image blur correction by making the correction ratio of the second correction means larger than that of the first correction means. (Configuration 2) In the first method, the first correction means performs blur correction using an output of the first detection means, The imaging device according to configuration 1, wherein in the second method, the second correction means performs shake correction based on an output from a detection means provided in the lens device that detects shake applied to the lens device. (Configuration 3) 3. The imaging device according to configuration 1 or 2, wherein the correction effect is an angle at which blur correction is possible. (Configuration 4) 4. The imaging device according to any one of configurations 1 to 3, wherein in the first method, the first correction means performs shake correction, and the second correction means does not perform shake correction. (Configuration 5) 5. The imaging device according to any one of configurations 1 to 4, wherein the limiting means limits use of the output of the second detecting means in response to an instruction from a photographer. (Configuration 6) The imaging device according to any one of configurations 1 to 5, wherein the first correction means performs electronic image stabilization by controlling the readout position of the image using the output of at least one of the first detection means and the second detection means. (Configuration 7) The imaging device according to any one of configurations 1 to 5, wherein the first correction means performs blur correction by moving the imaging element using the output of at least one of the first detection means and the second detection means. (Configuration 8) The imaging device according to any one of configurations 1 to 7, further comprising a notification unit that issues a notification urging restriction of use of the output of the second detection means depending on the area of ​​a moving object in the image. (Configuration 9) 9. The imaging device according to configuration 8, wherein the notification section issues the notification when a ratio of a size of the area of ​​the moving object to a size of the image is greater than a predetermined value. (Configuration 10) 10. The imaging device according to configuration 8 or 9, wherein the notification unit performs the notification when the size of the area of ​​the moving object matches an area in the image where the motion vector is highly reliable. (Configuration 11) 11. The imaging device according to any one of configurations 8 to 10, wherein the notification unit issues the notification when the depth of field is smaller than a predetermined value. (Configuration 12) A lens device that is detachable from an imaging device, a first detection means for detecting a vibration applied to the lens device; a correction unit that performs blur correction using an output of at least one of the first detection unit and a second detection unit that detects blur from an image acquired by an image sensor, The correction means changes whether or not to perform blur correction depending on the correction effect of the correction means when use of the output of the second detection means for blur correction is restricted. (Configuration 13) A lens device that is detachable from an imaging device, a first detection means for detecting a vibration applied to the lens device; a correction unit that performs blur correction using an output of at least one of the first detection unit and a second detection unit that detects blur from an image acquired by an image sensor, The lens device is characterized in that the correction means changes a correction ratio of the correction means depending on whether or not use of the output of the second detection means for image blur correction is restricted. (Configuration 14) The lens device described in configuration 13, wherein the correction means changes the correction ratio of the correction means depending on whether or not use of the output of the second detection means for image blur correction is restricted and on the correction effect of the correction means.

[0057] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0058] 100 Camera (imaging device) 101 Interchangeable lenses (lens devices) 116 image sensor 119 Motion vector detection unit (second detection means) 121 Image transformation unit (first correction means) 126 Angular velocity sensor (first detection means) 129 Image stabilization method determination unit (setting means) 130 Operation unit (restriction means)

Claims

1. An imaging device with a detachable lens device, a first detection means for detecting a shake applied to the imaging device; a second detection means for detecting a shake from an image acquired by the imaging element; a first correction means for performing blur correction using an output of at least one of the first detection means and the second detection means; a limiting means for limiting use of the output of the second detecting means; a setting means for setting a method for performing shake correction; and when the lens device is equipped with a second correction means that performs image blur correction and use of the output of the second detection means for image blur correction is restricted, the setting means selects a method with a higher correction effect from among a first method that performs image blur correction by making the correction ratio of the first correction means larger than the correction ratio of the second correction means, and a second method that performs image blur correction by making the correction ratio of the second correction means larger than the correction ratio of the first correction means.

2. In the first method, the first correction means performs blur correction using an output of the first detection means, 2. The imaging device according to claim 1, wherein in the second method, the second correction means performs blur correction based on an output from a detection means provided in the lens device that detects a shake applied to the lens device.

3. 3. The imaging device according to claim 1, wherein the correction effect is an angle at which blur correction is possible.

4. 3. The imaging apparatus according to claim 1, wherein in the first method, the first correction means performs blur correction, and the second correction means does not perform blur correction.

5. 3. The imaging apparatus according to claim 1, wherein the limiting means limits use of the output of the second detecting means in response to an instruction from a photographer.

6. 3. The imaging device according to claim 1, wherein the first correcting means performs electronic blur correction by controlling the read position of the image using the output of at least one of the first detecting means and the second detecting means.

7. 3. The imaging device according to claim 1, wherein the first correcting means performs blur correction by moving the imaging element using the output of at least one of the first detecting means and the second detecting means.

8. 3. The imaging device according to claim 1, further comprising a notification unit that issues a notification to prompt the user to restrict use of the output of the second detection unit in accordance with a region of a moving object in the image.

9. 9. The imaging device according to claim 8, wherein the notification unit issues the notification when a ratio of a size of the area of ​​the moving object to a size of the image is greater than a predetermined value.

10. 9. The imaging device according to claim 8, wherein the notification unit issues the notification when a size of the area of ​​the moving object matches an area in the image where a motion vector has high reliability.

11. The imaging device according to claim 8 , wherein the notification unit issues the notification when the depth of field is smaller than a predetermined value.

12. A lens device that is detachable from an imaging device, a first detection means for detecting a vibration applied to the lens device; a correction unit that performs blur correction using an output of at least one of the first detection unit and a second detection unit that detects blur from an image acquired by an image sensor, a correction means for changing whether or not to perform blur correction depending on the correction effect of the correction means when use of the output of the second detection means for blur correction is restricted.

13. A lens device that is detachable from an imaging device, a first detection means for detecting a vibration applied to the lens device; a correction unit that performs blur correction using an output of at least one of the first detection unit and a second detection unit that detects blur from an image acquired by an image sensor, The lens device according to claim 1, wherein the correction means changes a correction ratio of the correction means depending on whether or not use of the output of the second detection means for image blur correction is restricted.

14. 14. The lens device according to claim 13, wherein the correction means changes a correction ratio of the correction means depending on whether or not use of the output of the second detection means for image blur correction is restricted and on the correction effect of the correction means.

15. A control method for an imaging device to which a lens device is detachable, comprising: a correction step of performing shake correction using an output of at least one of a first detection means for detecting shake applied to the imaging device and a second detection means for detecting shake from an image acquired by an imaging element; a limiting step of limiting use of the output of the second detection means; a setting step of setting a method for performing image stabilization; a setting step of setting a method for controlling an imaging device that has a higher correction effect out of a first method for performing shake correction by making a correction ratio of a first correction means used by the imaging device to be larger than a correction ratio of a second correction means used by the lens device to perform shake correction, and a second method for performing shake correction by making a correction ratio of the second correction means to be larger than the correction ratio of the first correction means, when use of the output of the second detection means for shake correction is restricted.

16. A method for controlling a lens device that has a correction unit that performs shake correction and is detachable from an imaging device, comprising: a correction step of performing shake correction by the shake correction means using an output of at least one of a first detection means for detecting shake applied to the lens device and a second detection means for detecting shake from an image acquired by an image sensor; and a changing step of changing whether or not to perform image blur correction in accordance with the correction effect of the correction means, when use of the output of the second detection means for image blur correction is restricted.

17. A method for controlling a lens device that has a correction unit that performs shake correction and is detachable from an imaging device, comprising: a correction step of performing shake correction by the shake correction means using an output of at least one of a first detection means for detecting shake applied to the lens device and a second detection means for detecting shake from an image acquired by an image sensor; and a changing step of changing a correction ratio of the correction means depending on whether or not use of the output of the second detection means for image blur correction is restricted.

Images