Control apparatus, image capturing apparatus, control method, and storage medium

Abstract

To provide a control device capable of acquiring a smooth and high-quality image in a scene where a plurality of subjects exist.SOLUTION: A control apparatus (1) includes an object detector (26) configured to detect main object information, a motion vector detector (27) configured to detect motion vector information in an image, a shake detector (13, 14) configured to detect shake information, and a controller (15) configured to determine whether or not to make a shake correction control area different from a focus control area based on the main object information, the motion vector information, and the shake information.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a control device, an imaging device, a control method, and a program. [Background technology]

[0002] The blur that occurs in the subject of a captured image (subject blur) can be caused by the movement of the imaging device, such as camera shake, or by the movement of the subject. To correct subject blur and track the subject (main subject) intended by the photographer to provide a stable composition, it is necessary to detect the movement of the main subject separately from the movement of other subjects such as the background, and correct it so that the main subject continues to be captured in approximately the same position on the screen.

[0003] Patent Document 1 discloses a method for easily performing panning by driving a correction unit to correct image blur that occurs in a captured image based on a shake detection signal that indicates shake applied to the imaging device and the motion vector of the subject. Patent Document 2 discloses a method for suppressing the flickering sensation that occurs when the AF frame is moved by changing the drawing position of the AF frame and the position of the focus detection area with different characteristics. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-36366 [Patent Document 2] Japanese Patent Application Laid-Open No. 2010-204585 [Patent Document 3] Patent No. 3143173 Summary of the Invention [Problem to be solved by the invention]

[0005] In the method of Patent Document 1, the position of the motion vector detection area is set according to the focus position, so if it becomes difficult to track the correction target, an inappropriate subject (area) may be selected as the target for the image stabilization operation. In the method of Patent Document 2, the area that should actually be controlled may not match the changed area. For this reason, the methods of Patent Documents 1 and 2 cannot capture smooth, high-quality images in scenes with multiple subjects.

[0006] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a control device, an imaging device, a control method, and a program that are capable of capturing smooth, high-quality images in a scene where multiple subjects exist. [Means for solving the problem]

[0007] A control device according to one aspect of the present invention includes a subject detection means for detecting main subject information, a motion vector detection means for detecting motion vector information in an image, a blur detection means for detecting blur information using an inertial sensor, and a control means for determining whether or not to make a blur correction control area different from a focus control area based on the main subject information, the blur information, and the motion vector information.

[0008] Other objects and features of the present invention will be described in the following embodiments. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a control device, an imaging device, a control method, and a program that are capable of acquiring a smooth, high-quality image in a scene where multiple subjects exist. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 2 is a block diagram of an imaging system according to each embodiment. [Figure 2] 10 is a flowchart of a main subject detection process in each embodiment. [Figure 3] 10 is a flowchart of a photographing process in each embodiment. [Figure 4] 4 is a flowchart of a shake correction process according to the first embodiment. [Figure 5] FIG. 3 is an explanatory diagram relating to setting of a vector detection frame in the first embodiment. [Figure 6] 10 is a flowchart of a shake correction process according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0012] (First embodiment) First, an imaging device according to a first embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a block diagram of an imaging system 100. The imaging system 100 is configured to include an imaging device (camera body) 1 and an interchangeable lens (lens device) 31 that is detachable from the imaging device 1.

[0013] Reference numeral 2 denotes a lens mount to which an interchangeable lens 31 is attached, and 3 denotes a sensor (image pickup element) that photoelectrically converts an image of a subject (optical image) that passes through the imaging optical system. 4 denotes an imaging circuit that generates a predetermined image signal by performing various image processing on the electrical signal photoelectrically converted by sensor 3, and 5 denotes an A / D conversion circuit that converts the analog image signal generated by imaging circuit 6 into a digital image signal. 6 denotes a VRAM (memory) such as a buffer memory that temporarily stores the digital image signal output from A / D conversion circuit 7, and 7 denotes a D / A conversion circuit that reads out the image signal stored in VRAM 6, converts it into an analog signal, and converts it into an image signal suitable for playback output. Reference numeral 8 denotes an image display device such as an LCD (liquid crystal display device) that displays the image signal output from the D / A conversion circuit 7, and 10 denotes a storage memory that stores image data and is made up of a semiconductor memory or the like. Reference numeral 9 denotes a compression / expansion circuit that has a compression circuit and an expansion circuit. The compression circuit reads out the image signal temporarily stored in the VRAM 6 and performs compression processing and encoding processing on the image data to make it in a form suitable for storage in the storage memory 10. The expansion circuit performs decoding processing, expansion processing, etc. on the image data stored in the storage memory 10 to make it in a form optimal for playback and display, etc.

[0014] Reference numeral 11 denotes an AE processing circuit that performs automatic exposure (AE) processing on the output signal from the A / D conversion circuit 7. Reference numeral 12 denotes an AF processing circuit that generates an AF evaluation value and detects defocus amount information for performing automatic focus (AF) processing on the output signal from the A / D conversion circuit 5. Reference numeral 14 denotes a shake detection sensor that detects movement of the imaging device 1, such as camera shake, and is composed of inertial sensors such as a gyro sensor and an accelerometer, and detects multi-axial shake by using multiple inertial sensors. Reference numeral 13 denotes a shake detection circuit that processes the signal from the shake detection sensor 14.

[0015] Reference numeral 15 denotes a CPU (control means) with built-in memory for calculations that controls the imaging device 1, 16 a TG (timing generator) that generates predetermined timing signals, 17 a sensor driver, and 18 an operation switch (operation SW) consisting of various switch groups. Reference numeral 19 denotes an EEPROM, an electrically rewritable read-only memory that pre-stores programs for various controls and data used to perform various operations. Reference numeral 20 denotes a battery, 21 a communication driver for communicating with the interchangeable lens 31, and 22 an LED (display element) for displaying warnings and other information. Reference numeral 25 denotes a sensor movement motor for moving the sensor 3 in the horizontal and vertical directions, and 24 a sensor movement control circuit for controlling the movement of the sensor movement motor 25. Reference numeral 27 denotes a motion vector detection circuit that uses output signals from the A / D conversion circuit 7 to detect the motion vector of the subject. Reference numeral 26 denotes a main subject detection circuit that uses output signals from the motion vector detection circuit 27, the A / D conversion circuit 7, and the CPU 15 to perform main subject detection processing. Reference numeral 28 denotes an image transformation and cutting circuit that performs image processing such as image rotation, enlargement, reduction, trimming (cutting), etc. Reference numeral 23 denotes a speaker for issuing various notifications and warnings.

[0016] Reference numeral 32 denotes a motion compensation lens, 33 denotes a focus lens, 34 denotes a diaphragm (light intensity adjustment means) that controls the amount of light passing through the imaging optical system consisting of the motion compensation lens 32 and the focus lens 33, and 35 denotes a communication driver for communicating with the imaging device 1. 36 denotes a control circuit that controls an diaphragm drive motor (not shown) that drives the diaphragm 34, a focus drive motor (not shown) that drives the focus lens 33, and a motion compensation lens drive motor (not shown) that drives the motion compensation lens 32. 37 denotes an EEPROM, an electrically rewritable read-only memory that stores data used to perform various operations. The interchangeable lens 31 has an imaging optical system including the motion compensation lens 32, the focus lens 33, and the diaphragm 34, and is also equipped with a zoom ring (not shown) that changes the focal length of the interchangeable lens 31, a manual focus ring that adjusts the focus, and the like.

[0017] The storage memory, which is a storage medium for image data, etc., is not limited to fixed semiconductor memories such as flash memories, or semiconductor memories such as card-shaped or stick-shaped card-type flash memories that are detachably attached to the device. For example, various types of storage media, such as hard disks and floppy disks, can be used.

[0018] The operation switches 18 include a main power switch that starts up the imaging device 1 and supplies power, a release switch that starts video shooting operations (recording operations), etc., a playback switch that starts playback operations, an exposure compensation amount change dial, an exposure time change dial, an aperture value change dial, etc. The release switch is a two-stage switch with a first stroke (SW1) that generates a command signal to start AE processing and AF processing that are performed prior to the shooting operation, and a second stroke (SW2) that generates a command signal to start the actual exposure operation.

[0019] Next, the operation of the imaging system 100 will be described. First, a subject light beam (subject image) passing through the interchangeable lens 31, the light intensity of which has been adjusted, is focused on the light receiving surface of the sensor 3. The subject image is converted into an electrical signal by photoelectric conversion processing by the sensor 3 and output to the imaging circuit 4. The imaging circuit 4 performs various signal processing on the input signal to generate a predetermined image signal. This image signal is output to the A / D conversion circuit 5 and converted into a digital signal (image data), which is then temporarily stored in the VRAM 6. The image data stored in the VRAM 6 is output to the D / A conversion circuit 7 and converted into an analog signal, which is then converted into an image signal in a format suitable for display, and then displayed as an image on the LCD 8. Meanwhile, the image data stored in the VRAM 6 is also output to the compression / expansion circuit 9. After compression processing by the compression circuit in the compression / expansion circuit 9, the image data is converted into image data in a format suitable for storage and stored in the storage memory 10.

[0020] Furthermore, for example, when a playback switch (not shown) among the operation switches 18 is operated and turned on, playback operation begins. At this time, the image data stored in compressed form in the storage memory 10 is output to the compression / expansion circuit 9, where it is subjected to decoding and expansion processes, etc., and then output to the VRAM 6 and temporarily stored. Furthermore, this image data is output to the D / A conversion circuit 7 and converted into an analog signal, which is then converted into an image signal in a form suitable for display, and then displayed as an image on the LCD 8.

[0021] Meanwhile, the TG 16 outputs a predetermined timing signal to the CPU 15, the imaging circuit 4, and the sensor driver 17, and the CPU 15 performs various controls in synchronization with this timing signal. The imaging circuit 4 also receives the timing signal from the TG 16 and performs various image processing such as color signal separation in synchronization with this. Furthermore, the sensor driver 17 receives the timing signal from the TG 16 and drives the sensor 3 in synchronization with this.

[0022] On the other hand, the image data digitized by the A / D conversion circuit 5 is also output to the AE processing circuit 11, the AF processing circuit 12, the motion vector detection circuit 27, the main subject detection circuit 26, and the image transformation and cutting circuit 28, in addition to the VRAM 6.

[0023] The AE processing circuit 11 receives the input digital image signal, calculates an AE evaluation value according to the brightness of the subject, and outputs the AE evaluation value to the CPU 15. Based on the AE evaluation value, the CPU 15 calculates the exposure time of the sensor 3 and the aperture value of the diaphragm 34, and sends these to the interchangeable lens 31 via the communication driver 21. In the interchangeable lens 31, a control circuit 36 ​​performs diaphragm drive processing and adjusts the aperture value of the diaphragm 34 so that it is appropriate.

[0024] The AF processing circuit (defocus amount detection means) 12 performs image correction on the image signal acquired by the sensor 3 having imaging pixels for focus adjustment, and performs correlation calculations on the corrected image signal to detect defocus amount information. The CPU 15 determines the amount and direction of drive of the focus lens 33 and sends this to the interchangeable lens 31 via the communication driver 21. The interchangeable lens performs drive processing on the focus lens 33, enabling AF control to achieve a focused state.

[0025] The motion vector detection circuit 27 calculates the motion vector of the subject based on the input digital image signal (base image) and the digital image signal (reference image) from the previous frame, according to the regions divided by instructions from the CPU 15. That is, the difference between the base image and the reference image is calculated while shifting the reference image by a predetermined number of pixels in the horizontal and vertical directions, and the pixel shift amount that gives the highest correlation (smallest difference amount) is taken as the motion amount of the subject in that region, and the horizontal and vertical pixel shift direction at that time is taken as the motion direction. In this way, the motion vector of the subject within the region between frames is calculated. Note that details are described in Patent Document 3, etc., and therefore will not be described here.

[0026] Main subject detection circuit 26 acts as a main subject detection means and detects the position of the main subject within the screen (main subject information) as follows. First, main subject detection circuit 26 determines whether the main subject area has been designated by the photographer. For example, it determines the area where the AF point has been designated by the photographer, or the area where the photographer has performed a touch operation for a predetermined period of time, if a touch panel is attached to LCD 8, as the main subject area.

[0027] Furthermore, main subject detection circuit 26 searches for face-characterizing features such as pupils and eyebrows on the image based on the output signal from A / D conversion circuit 5. Main subject detection circuit 26 then detects the position of the person's face on the image from this positional relationship, and further determines the size and tilt of the face from the positional relationship such as the spacing between the face-characterizing features.

[0028] Furthermore, main subject detection circuit 26 detects the position of a person present within the screen by searching the image for parts having shapes corresponding to the parts that characterize a person, such as the head or torso, and evaluating the relative positions of these parts, based on the output signal from A / D conversion circuit 5. For example, if a shape close to a circle is detected, and below it there is a first rectangular shape, and a second rectangle with a shorter side than the first rectangle exists, and the two are adjacent to each other, it may be determined that a person is present.

[0029] Furthermore, main subject detection circuit 26 detects clusters of similar color and brightness based on the output signal from A / D conversion circuit 5, detects their size and position on the screen, and estimates the likelihood of them being the main subject from the results. For example, if a cluster of similar color and brightness with a size equal to or larger than a predetermined size is present near the center of the screen, it is determined to be the likely main subject. Then, from among those whose size and position on the screen satisfy predetermined conditions, the area with the highest likelihood of being the main subject, calculated from the center of gravity coordinates with the center of the screen as the origin and the size of the cluster, is determined to be the main subject area.

[0030] Furthermore, main subject detection circuit 26 acquires the distance or defocus amount for each AF point within the screen from the processing results of AF processing circuit 12. Then, using the processing results of motion vector detection circuit 27 and the results of blur detection circuit 13, it detects a subject that is moving in real space and that the photographer is trying to capture within the screen. Main subject detection circuit 26 also obtains information such as the processing results of AE processing circuit 11, AF point information set by the photographer including that set by touching the LCD screen, shooting mode, shutter speed, and aperture value (F-number) from CPU 15. Main subject detection circuit 26 comprehensively detects multiple main subject areas from the detection results obtained in this way, and sends the ranked results to the CPU.

[0031] In this embodiment, if the reliability of the main subject detected by main subject detection circuit 26 is high, that main subject is selected as the target for shake correction control. Then, after the motion vector of that main subject is detected by motion vector detection circuit 27, the detected motion vector is used to correct shake occurring in the main subject. If multiple main subjects are detected, the main subject with the highest rank is selected as the main subject, and shake correction is performed in the same manner. Conversely, if there are multiple subjects with the same rank and the main subject cannot be determined, or if the main subject is temporarily not detected, the main subject detected by main subject detection circuit 26 is not selected as the target for shake correction control, and instead another subject, such as the background, is selected as the target for shake correction control.

[0032] Next, the main subject detection process in this embodiment will be described with reference to Fig. 2. Fig. 2 is a flowchart of the main subject detection process. Each step in Fig. 2 is mainly executed by main subject detection circuit 26 based on instructions from CPU 15. In this process, the earlier a main subject is detected, the higher the priority of that main subject.

[0033] When processing begins, all main subject detection flags are first turned off. Then, in step S201, main subject detection circuit 26 determines whether a main subject area (AF measuring point) has been designated by the photographer. This is performed by examining the AF point information sent from CPU 15 and determining whether an AF point has been designated by the photographer via a menu or touch operation on the LCD screen. If a main subject area has been designated, the process proceeds to step S202. In step S202, main subject detection circuit 26 determines the main subject area to be the AF point designated by operating operation switch 18 or the main subject area selected by the photographer by touching the LCD screen, and an area nearby where a similar subject exists.

[0034] On the other hand, if a main subject region has not been specified, the process proceeds to step S203. In step S203, main subject detection circuit 26 determines whether a face has been detected. If a face has been detected, the face detection flag is turned on, and then in step S221 it is determined whether the detected face can be considered to be the same subject as the main subject up to the previous frame. If it can be considered to be the same subject, the process proceeds to step S204, and the detected face region is set as the main subject region. On the other hand, if it cannot be considered to be the same subject, the process proceeds to step S205. Note that the determination of whether it can be considered to be the same subject can be made using a function of main subject detection circuit 26 that tracks subjects that are presumed to be the same.

[0035] In step S205, main subject detection circuit 26 determines whether a person has been detected. If a person has been detected, the person detection flag is turned on, and then in step S222 it is determined whether the detected person can be considered the same subject as the person that was the main subject up until the previous frame. If the person can be considered the same subject, the process proceeds to step S206, where the detected person area is set as the main subject area. On the other hand, if the person cannot be considered the same subject, the process proceeds to step S207.

[0036] In step S207, main subject detection circuit 26 determines whether a moving object (a subject moving in real space that the photographer is trying to capture within the frame) exists (whether a moving object has been detected). If a moving object exists, the moving object detection flag is turned on, and then in step S222 it is determined whether the detected moving object can be considered the same subject as the main subject up to the previous frame. If it can be considered the same subject, the process proceeds to step S208, where the moving object is set as the main subject area. On the other hand, if it cannot be considered the same subject, the process proceeds to step S209. The determination of whether a moving object exists is made using output signals from blur detection circuit 13, motion vector detection circuit 27, and AF processing circuit 12.

[0037] If the output signal of the shake detection circuit 13 is small (if all of the detected values ​​for multiple axes are less than a predetermined value), that is, if the photographer is not intentionally moving the imaging device 1, it is determined whether there is an area where the amount of movement of the motion vector detected by the motion vector detection circuit 27 is equal to or greater than a predetermined value. If there is an area where the amount of movement of the motion vector is equal to or greater than the predetermined value, that area is determined to be the main subject area.

[0038] On the other hand, if the output signal of the shake detection circuit 13 is large (if any of the detection values ​​of the multiple axes is equal to or greater than a predetermined value), that is, if the photographer is intentionally moving the imaging device 1, the area exhibiting the same movement as this intentional movement is determined to be the main subject area. That is, if there is an area where the amount of movement of the motion vector detected by the motion vector detection circuit 27 is equal to or less than a predetermined value, that area is determined to be the main subject area.

[0039] If a moving object cannot be detected by processing using the output signal of the motion vector detection circuit 27, a determination is made as to whether there is a moving object moving in the distance direction (optical axis direction). This can be done by checking whether there are any AF points whose distance or defocus amount obtained from the AF processing circuit 12 changes in the same direction over time (for example, AF points whose distance decreases over five consecutive frames). The changes in distance or defocus amount for each frame are checked for all AF points obtained from the AF processing circuit 12, and AF points whose distance changes in the same direction by a predetermined amount over a predetermined number of frames or more are extracted. These AF points are considered to be areas where a moving object exists, and adjacent AF points are merged. If there are multiple AF point areas whose distance changes in the same direction by a predetermined amount over a predetermined number of frames or more, a measurement point area closest to the center of the screen is selected.

[0040] In step S209, main subject detection circuit 26 determines whether there is an area that can be considered a main subject area with a high degree of likelihood of being the main subject among the clusters of similar colors and brightness. If there is an area that can be considered a main subject area, the main subject detection flag is turned on, and then in step S224 it is determined whether the main subject in the detected main subject area can be considered the same subject as the main subject up until the previous frame. If it can be considered the same subject, proceed to step S210, and that area is designated as the main subject area. The degree of likelihood of being the main subject is determined based on the position and size on the screen of clusters of similar colors and brightness. Detected clusters that do not touch two of the four sides of the screen are selected, and of these, those that are larger than a predetermined size are deemed to be clusters with a high degree of likelihood of being the main subject. If there are multiple such clusters, the one with the center of gravity closest to the center of the screen is selected.

[0041] If the subject cannot be determined to be the same as the subject that was the main subject up to the previous frame, the process proceeds to step S225. In step S225, a detection flag is checked to determine whether a main subject candidate has been detected in the processing up to that point. If any detection flag is on, main subject detection circuit 26 determines that a main subject candidate has been detected, and the process proceeds to step S226.

[0042] In step S226, main subject detection circuit 26 selects the detection area with the highest priority among the detection areas whose detection flags are on, and designates that area as the main subject area. If a face is detected, the main subject area is the face detection area; if a face is not detected but a person is detected, the main subject area is the person detection area; if a moving object is detected without a face or person, the main subject area is the moving object area; and if neither a face nor a person is detected, the main subject area is the detection area determined to be the main subject.

[0043] If the detection flag is not on, no main subject candidate has been detected in the processing up to that point, and the process proceeds to step S211. In step S211, main subject detection circuit 26 checks the processing results of AF processing circuit 12 to see if there is a subject with a high proportion of AF points within the frame that can be AFed and at a different distance. If the condition is met, the process proceeds to step S212, and the AF point that shows the closest AF result among the multiple AF points is determined to be the main subject area. In this case, since the entire frame is not the subject, as in landscape photography, but rather commemorative photography with the landscape as the background, the closest subject is determined to be the main subject area. On the other hand, if the condition is not met, in step S213, the area to perform AF is determined based on the AE processing result, shooting mode, shutter speed, aperture value, and flash ON / OFF information obtained from CPU 15.

[0044] First, the area for AF is determined based on the shooting mode as shown in Table 1 below.

[0045] [Table 1]

[0046] If the above conditions are not met, the main subject area is determined as shown in Table 2 below.

[0047] [Table 2]

[0048] If multiple faces, people, or moving objects are detected, the main subject area is determined based on their detected position and size (faces) or their detected position (people / moving objects). Main subjects with different priorities may also be detected, and the main subject area is determined according to the priority. Initially, the main subject area is determined in this way, but from the next frame onwards, priority is given to the same subject.

[0049] The main subject detection circuit 26 has the function of tracking a subject that is estimated to be the same as the subject detected as the main subject in the previous frame. If this function determines that the subject detected as the main subject in the above procedure is the same as the main subject up to the previous frame, that subject is set as the highest-ranked main subject. Even if a main subject is detected earlier in the above procedure, the main subject estimated to be the same is given priority. Therefore, blur correction is performed on this main subject estimated to be the same. Note that this function is only performed when an arbitrary AF point is specified (steps S204, S206, S208, and S210 in FIG. 2). If the closest subject is set as the main subject area (steps S212 and S213 in FIG. 2), the subject is not determined to be the same as the subject.

[0050] Furthermore, when the photographer specifies an AF point using a menu or the like to explicitly specify a main subject, the subject at the specified AF point is considered to be the main subject. Therefore, it is determined that a new main subject has been detected when the photographer explicitly specifies a main subject (step S202) and when a main subject that is no longer estimated to be the same subject and a main subject with a higher ranking is detected at the same time (step S212). Here, when a main subject that is no longer estimated to be the same subject and a main subject with a higher ranking is detected at the same time, such as when the main subject has moved significantly up until then and its position on the screen changes significantly without the photographer intentionally tracking that subject.

[0051] Subsequently, the image transformation and cropping circuit 28 uses the output signals of the main subject detection circuit 26 and the motion vector detection circuit 27 to correct the changes in the image based on information about the up / down / left / right movement of the main subject and the rotation of the imaging device 1 calculated by the CPU 15. To achieve this, image processing such as transformations like image rotation and cropping of a portion of the image is performed. For example, the face of a person detected as the main subject may significantly change position on the screen due to the person's movement or the photographer's camera shake, or the main subject may move diagonally. If such shaking or movement of the main subject in the horizontal, vertical, or diagonal directions occurs between frames of a video, the main subject may not be located in the on-screen position desired by the photographer, or unnatural movement may occur. If such phenomena occur frequently between frames, recorded video images may be very difficult to view.

[0052] Therefore, a motion vector representing the horizontal and vertical movement of the main subject between frames is detected by motion vector detection circuit 27, and information for correcting the image is calculated from the motion vector by CPU 15. Then, the image is transformed and corrected by image transformation and cutout circuit 28 according to the calculated amount of correction, and the generated image is recorded in a predetermined area of ​​VRAM 6.

[0053] Next, the photographing process of the imaging device 1 will be described with reference to Fig. 3. Fig. 3 is a flowchart of the photographing process. Each step in Fig. 3 is executed by each unit of the imaging device 1 mainly based on instructions from the CPU 15. When the main power switch of the imaging device 1 is on and the operation mode of the imaging device 1 is in the photographing (recording) mode, the photographing process sequence is executed.

[0054] First, in step S301, the CPU 15 performs initialization processing such as initializing variables used in processing and moving drive members to their initial positions, and then determines whether or not an interchangeable lens 31 is attached. If an interchangeable lens 31 is attached, the CPU 15 acquires information about the image stabilization lens 32, focus lens 33, aperture 34, focal length, etc. On the other hand, if an interchangeable lens 31 is not attached, the process proceeds to step S302 without acquiring interchangeable lens information.

[0055] In step S302, CPU 15 displays the image formed on sensor 3 through interchangeable lens 31 as an image on the LCD. That is, the subject image formed on sensor 3 is photoelectrically converted by sensor 3 into an electrical signal, which is then output to imaging circuit 4. The input signal is subjected to various signal processing to generate a predetermined image signal, which is then output to A / D conversion circuit 5, converted into a digital signal (image data), and temporarily stored in VRAM 6. The image data stored in VRAM 6 is output to D / A conversion circuit 7, converted into an analog signal, and converted into an image signal in a format suitable for display, and then displayed as an image on LCD 8. Note that if the video recording mode is set and the image stabilization setting is ON, an image that has been subjected to image stabilization processing (image stabilization control) may be displayed on LCD 8.

[0056] Next, in step S303, CPU 15 determines whether the camera is set to moving image recording mode or still image shooting mode. If the camera is set to moving image recording mode, the process proceeds to step S304. On the other hand, if the camera is set to still image shooting mode, the process proceeds to step S321.

[0057] In step S321, CPU 15 checks the state of the release switch. If CPU 15 checks that the photographer has operated the release switch and that SW1 (first stroke of the release switch) has been turned on, the process proceeds to step S322. If the photographer has set the camera to perform shake correction processing, CPU 15 performs shake correction processing. This processing can be performed using the shake correction processing used in conventional still image shooting, as described in Patent Document 2, for example, and therefore detailed description of the processing will be omitted.

[0058] Next, in step S323, CPU 15 executes AF processing and AE processing, drives focus lens 33 to the in-focus position, and determines the aperture value, exposure time, etc. for still image shooting. Next, in step S324, CPU 15 superimposes the AE processing result (high brightness / low brightness warning, etc.) and AF processing result (AF success or failure, etc.) on the image formed on sensor 3 and displays them as an image on the LCD. The AE and AF processing results may also be notified to the photographer by, for example, lighting and flashing LED 22 or emitting a focus or out-of-focus sound from the speaker.

[0059] Next, in step S325, CPU 15 checks SW2 (the first stroke of the release switch). If SW2 is on, the process proceeds to step S326, where CPU 15 executes exposure processing. If the photographer has set image stabilization during exposure processing, CPU 15 executes image stabilization processing in the same manner as in step S322. After the exposure processing is completed, the process proceeds to step S310.

[0060] On the other hand, if the moving image recording mode is set in step S303, the process proceeds to step S304, where the CPU 15 executes AF processing and AE processing. The CPU 15 drives the focus lens 33 to the in-focus position, determines the aperture value and exposure time, and controls the driving of the aperture 34 and the exposure time (accumulation time) of the sensor 3.

[0061] Next, in step S305, CPU 15 determines whether shake correction processing has been set by the photographer. If shake correction processing has been set (if IS is on), CPU 15 performs shake correction processing in step S306. On the other hand, if shake correction processing has not been set (if IS is off), CPU 15 proceeds to step S307. Details of the processing in step S306 will be described later.

[0062] In step S307, CPU 15 superimposes the AE processing result (high brightness / low brightness warning, etc.) and the AF processing result (AF success / failure, etc.) on the image and displays it on LCD 8. This image is the image that has been subjected to shake correction created in the processing of step S306 if shake correction processing has been set, and is the image that was read out in step S302 if shake correction processing has not been set.

[0063] Next, in step S308, CPU 15 determines whether a moving image recording instruction has been issued by the photographer. If a moving image recording instruction has been issued, the process proceeds to step S309, where CPU 15 executes moving image recording processing, and then proceeds to step S310. This moving image recording processing is performed by recording the image that has been subjected to the shake correction processing created in the processing of step S306, if shake correction processing has been set. On the other hand, if shake correction processing has not been set, the image to be displayed in step S307 (the image read out in step S302) is recorded.

[0064] If no moving image recording instruction has been issued, proceed to step S310. In step S310, CPU 15 determines whether the state of the main power switch or playback switch has changed, or whether the lens has been replaced. If there has been a change in the state of any of the switches or if the lens has been replaced, the process ends. Otherwise, return to step S302.

[0065] The shake correction process of step S306 will now be described with reference to Figures 4 and 5. Figure 4 is a flowchart of the shake correction process in this embodiment. Figure 5 is an explanatory diagram related to setting of the vector detection frame. The steps in Figure 4 are mainly executed by CPU 15, or by various parts such as main subject detection circuit 26 and motion vector detection circuit 27 based on instructions from CPU 15.

[0066] First, in step S401, main subject detection circuit 26 determines whether a subject presumed to be the same as the subject previously detected as the main subject has been detected. If a subject presumed to be the same as the subject previously detected as the main subject has not been detected, the process proceeds to step S402. On the other hand, if a subject presumed to be the same as the subject previously detected as the main subject has been detected, i.e., if it is determined that a so-called tracking state is in effect, the process proceeds to step S421. This determination is made using the function of main subject detection circuit 26 to track a subject presumed to be the same as the subject detected as the main subject in the previous frame. That is, if a main subject region has been determined in steps S204, S206, S208, and S210 of FIG. 2, it is determined that a subject presumed to be the same as the subject previously detected as the main subject has been detected. Otherwise, that is, if a main subject region has been determined in steps S202, S226, S212, and S213 of FIG. 2, it is determined that the subject has not been detected.

[0067] In step S402, main subject detection circuit 26 determines whether a new main subject has been detected. If a main subject region has been determined in steps S202 and S226 of Fig. 2, it determines that a new main subject has been detected, and proceeds to step S403. On the other hand, if a main subject region has been determined in steps S212 and S213 of Fig. 2, it determines that a new main subject has not been detected, and proceeds to step S422.

[0068] In step S403, main subject detection circuit 26 determines whether the subject (focus control area) targeted for focus control has been designated by the photographer. If the main subject area has been determined in step S202 of Fig. 2, it is determined that the photographer has designated it, and the process proceeds to step S421. If the main subject area has been determined in any other step of Fig. 2, it is determined that the photographer has not designated it, and the process proceeds to step S404.

[0069] In step S404, main subject detection circuit 26 determines whether the photographer is tracking the subject by panning or the like. If the subject is being tracked, proceed to step S421. On the other hand, if the subject is not being tracked, proceed to step S405. The determination of whether the subject is being tracked is made using the output signal of shake detection circuit 13 and the output signal of motion vector detection circuit 27. That is, if the output signal of shake detection circuit 13 is large and the movement of the main subject on sensor 3 calculated by motion vector detection circuit 27 is small, it is determined that the photographer is tracking the subject by moving imaging device 1 so that the desired main subject does not move on sensor 3.

[0070] In step S405, main subject detection circuit 26 uses the face recognition function to determine whether the newly detected person is the same person. If it is determined that it is the same person, the process proceeds to step S421. On the other hand, if it is determined that it is not the same person, the process proceeds to step S406.

[0071] In step S406, main subject detection circuit 26 determines whether multiple subjects exist and the main subject cannot be determined. If the main subject cannot be determined, the process proceeds to step S422. On the other hand, if the main subject can be determined, the process proceeds to step S407. This determination is made using the size and number of subject areas detected by main subject detection circuit 26. Main subject detection circuit 26 detects subjects such as human faces, people themselves, and moving objects, and may detect multiple subjects. If multiple subjects are detected with the same priority, the main subject area is determined based on their detected position and size (faces) or their detected position (people / moving objects). In this case, if there is no clear difference in the detected position and size, the main subject cannot be clearly determined from the multiple subjects.

[0072] Therefore, when the difference in the sizes of the detected faces is less than a predetermined ratio, that is, when the size of the second-largest face is greater than or equal to a predetermined ratio (Cr1) with respect to the size of the largest face (when the ratio of the sizes of multiple subjects is greater than or equal to the first predetermined ratio), the main subject cannot be clearly determined. In this case, it is determined that multiple subjects exist and the main subject cannot be determined.

[0073] Similarly, when the size of the second-largest face is greater than or equal to a predetermined ratio (Cr2: Cr2 < Cr1) and the difference between the sensor center position of the largest face and the position of the second-largest face is less than a predetermined value (Cd1), the main subject cannot be clearly determined. That is, when the ratio of the sizes of multiple subjects is greater than or equal to the second predetermined ratio and the difference in the positions of the multiple subjects is less than the first predetermined value, it is determined that the main subject cannot be clearly determined.

[0074] Similarly, when the difference between the position of the detected largest subject (person or moving object) with the sensor center as the origin and the position of the second-largest subject is less than a predetermined value (Cd2) (when the difference in the positions of multiple subjects is less than the second predetermined value), the main subject cannot be clearly determined. In this case as well, it is determined that multiple subjects exist and the main subject cannot be determined.

[0075] In step S407, the motion vector detection circuit 27 determines whether a subject with a different motion on sensor 3 has crossed the previous main subject (old subject). If the subjects have crossed, the process proceeds to step S422. On the other hand, if the subjects have not crossed, the process proceeds to step S408. This determination is made using the detection result of the motion vector detection circuit 27. The motion vector detection circuit 27 detects the motion vector, that is, the amount of motion on sensor 3, for the multiple subjects detected by the main subject detection circuit 26. When multiple subjects are detected, motion vectors are detected for the multiple subjects in order.

[0076] Therefore, the motion vectors and positions of each of the multiple subjects from several frames before to the current frame are recorded. If the difference in the positions of two of the multiple subjects is smaller than a predetermined value (Cd3) and the difference in the motion vectors of the subjects between the frames immediately before is equal to or greater than a predetermined value (Cv1), it is determined that the subject, whose movement is different on the sensor 3, has crossed over with the previous main subject. In other words, if the difference in the positions of the multiple subjects is smaller than a third predetermined value and the difference in the motion vectors is equal to or greater than a first predetermined vector difference, it is determined that the subjects have crossed over.

[0077] In step S408, main subject detection circuit 26 determines whether the position of the detected main subject on sensor 3 is at an edge position (whether it is outside a certain range from the center of sensor 3). If the position of the main subject is outside the certain range from the center of sensor 3, the process proceeds to step S422. On the other hand, if the position of the main subject is within the certain range from the center of sensor 3, the process proceeds to step S409. This determination is made if the position of the main subject on sensor 3 is away from the center position of sensor 3 by a predetermined value (Cp1) or more, it is determined to be outside the certain range, and the process proceeds to step S422. In other words, if the position of the most recently detected main subject is outside a first predetermined range from the center position, the process proceeds to step S422.

[0078] In step S409, main subject detection circuit 26 determines whether the previous main subject has temporarily become undetectable. If a main subject was not detected near the previous detection position of the main subject, and main subject detection circuit 26 detects a new main subject, but that new main subject is now farther away and the size of the main subject area is significantly different, it is determined that the main subject has temporarily become undetectable, and processing proceeds to step S423. On the other hand, if a main subject has been detected, processing proceeds to step S410. This determination assumes that the previous main subject has temporarily become undetectable because it has been temporarily hidden by another subject.

[0079] When the face is hidden by other subjects in the main subject detection circuit 26, the face detection may fail. As a countermeasure, when the face is not detected by the main subject detection circuit 26, the CPU 15 determines the main subject assuming that the face is present at the previous face detection position for several frames. If a new face (a face of another person) is detected during this process while the previous main subject is still hidden, the main subject detection circuit 26 notifies the CPU 15 that the face of the other person is the main subject.

[0080] Therefore, when a new main subject is notified, the sizes and detection positions of the face detected until then and the newly detected face are compared. If the ratio of the face sizes (Rf = size of the face detected until then / size of the newly detected face) is outside the predetermined range (Crf1 < Rf < Crf2), it is determined that the face has temporarily disappeared. Or, if the difference Df in the detection positions of the face detected until then and the newly detected face is greater than the predetermined value (Cd4), it is determined that the face has temporarily disappeared. That is, when the ratio of the sizes of multiple subjects is less than or equal to the first comparison value or greater than or equal to the second comparison value, or when the difference in the positions of multiple subjects is greater than the fourth predetermined value, it is determined that the main subject has temporarily disappeared, and the process proceeds to step S423. The same applies to cases such as people. After the temporary period of several frames ends, it is determined that this condition is not met, and the process proceeds to step S410.

[0081] In step S410, the main subject detection circuit 26 determines whether all of the detected multiple subjects are within a predetermined defocus range (whether the difference in the defocus amounts of the multiple subjects is less than or equal to a predetermined value). If all of the multiple subjects are within the predetermined defocus range, the process proceeds to step S423. On the other hand, if not all of the multiple subjects are within the predetermined defocus range (if the multiple subjects are within the first predetermined defocus range or within the first distance range), the process proceeds to step S423. This determination is made using the distance or defocus amount obtained from the AF processing circuit 12.

[0082] For multiple subjects detected by the main subject detection circuit 26, the distance or defocus amount of the area where the subject is located is obtained from the AF processing circuit 12. If all of the main subject areas are found to be within a predetermined range using this value, they are determined to be within the predetermined defocus range, and the process proceeds to step S423. On the other hand, if any area is not within the predetermined range, the process proceeds to step S421. This can be done by finding the defocus amounts (or distances) of the nearest and farthest areas and comparing the difference with the predetermined range (distance range). The predetermined range is determined based on depth.

[0083] In step S421, CPU 15 sets the main subject region detected by main subject detection circuit 26 as the target region for shake correction (shake correction control region) and sets a vector detection frame in motion vector detection circuit 27. For example, as shown in FIG. 5(A), the entire image is divided into 9×7 regions, and regions that partially overlap with regions included in the main subject region (gray areas in FIG. 5(A)) are set as detection frames for detecting vectors to correct subject shake. Vectors are then detected in the vector detection frames of each region. Then, vector clustering is performed to separate vectors in the vector detection frames within the main subject frame that are caused by subject movement from those that are not, and the amount of subject shake that has occurred is detected by detecting motion vectors caused by subject movement.

[0084] In step S422, the background is set as the target region for shake correction (shake correction control region), and a vector detection frame in motion vector detection circuit 27 is set in the region excluding the main subject region detected by main subject detection circuit 26. For example, as shown in FIG. 5(A), the entire image is divided into 9×7 regions, and the region excluding the gray parts is set as the detection frame for detecting vectors to correct shake. Vectors are then detected in the vector detection frame of each region. Then, vector clustering is performed to separate the vectors in the vector detection frame excluding the main subject frame into background blur caused by camera shake and other vectors, and the amount of background blur caused by camera shake is detected by detecting the motion vectors of the background.

[0085] In step S423, a vector detection frame is set in motion vector detection circuit 27, with the area where the previous main subject (old main subject) was present as the target area for blur correction. However, because main subject detection circuit 26 is temporarily not detecting the previous main subject or has detected another subject as the main subject, it is also necessary to detect the motion vectors of the background and other subjects. In this case, for example, multiple subjects, multiple subjects and the background, the previously detected subject (old subject), or the old subject and the background are set as the blur correction control area. Therefore, a vector detection frame in motion vector detection circuit 27 is set to the entire screen. Then, three areas are provided: an area including the previous subject (old subject) obtained from main subject detection circuit 26, an area including the newly detected subject (new subject), and an area excluding these, and a vector detection frame in motion vector detection circuit 27 is set for each of them.

[0086] For example, as shown in Figure 5(B), the entire image is divided into 9x7 regions. Three regions are defined: a region that partially overlaps with the region included in the old main subject region (the gray region in Figure 5(B)), a region that partially overlaps with the region included in the new main subject region (the white region in Figure 5(B)), and a region excluding both. These regions are used as detection frames for detecting vectors for blur correction. Next, vector detection frames are detected for each region. Vector clustering is then performed to separate the vectors in the vector detection frames within the previous and new main subject region frames into those caused by subject movement and those not, thereby detecting motion vectors caused by subject movement. This allows the amount of subject blur to be detected. Furthermore, the vectors in the vector detection frames excluding the two subject frames are separated into those caused by background blur due to camera shake and those not, thereby detecting the amount of background blur.

[0087] Thereafter, the amount of blur to be corrected is calculated from the three amounts of blur thus obtained in the following manner. (1) If the difference between the three obtained amounts of blur is within a predetermined value (Cb1), the average of the three amounts of blur is calculated and set as the amount of blur to be corrected. (2) If the difference between the amount of blur of the main subject and the amount of blur of the background is within a predetermined value (Cb1), the average of the two amounts of blur is calculated and set as the amount of blur to be corrected. (3) If the difference between the amount of blur of the main subject up to that point and the amount of blur of the new main subject is within a predetermined value (Cb1), the average of the two amounts of blur is calculated and used as the amount of blur to be corrected. (4) If the amount of blur of the main subject up to that point is greater than a predetermined value (Cs1), the amount of blur of the main subject up to that point is set as the amount of blur to be corrected. (5) If the amount of blur in the background is greater than a predetermined value (Cs1), the amount of blur in the background is set as the amount of blur to be corrected. (6) In all other cases, the amount of blur of the main subject up to that point is the amount of blur to be corrected.

[0088] Then, in step S424, CPU 15 performs shake correction processing. First, the amount of correction (shake correction amount) for correcting the shake occurring in the captured image is calculated from the amount of shake to be corrected calculated in steps S421 to S423. After that, the image transformation and cutting out circuit 28 cuts out the image based on the calculated correction amount, thereby generating a shake-corrected image.

[0089] In this embodiment, main subject detection circuit 26 serves as a main subject detection means that detects the position of the main subject (main subject information). Motion vector detection circuit 27 serves as a motion vector detection means that detects the movement of the subject (motion vector information). Shake detection sensor 14 and shake detection circuit 13 serve as a shake detection means that detects the movement of imaging device 1 (shake information). CPU 15 functions as a calculation means that calculates the amount of correction used to correct shake that occurs in the subject of the captured image. Operation switch 18, LCD 8 equipped with a touch panel, and CPU 15 serve as a means for designating the target area for focus control (focus control area).

[0090] CPU 15 also functions as a determination means (control means) that determines whether or not to make the shake correction control area different from the focus control area based on main subject information, blur information, and motion vector information. CPU 15 also has a function that determines the target area for shake correction control based on the determination result of the determination means. As a result, based on the subject detection results, the defocus amount of each area, and the subject movement, it determines whether the subject (area on the screen) to be targeted for shake correction control should be a different subject from the subject to focus control, and determines the target for shake correction control based on the result.

[0091] If a previous subject is being tracked, that subject continues to be the subject of control. When a new main subject is detected, the subject of focus control is set as the subject of shake correction control if the subject of focus control is specified by the photographer. Alternatively, the output signal of the shake detection means is large and the movement of the subject calculated from the motion vector detection means and the shake detection means is small (i.e., when the photographer moves the imaging device but it can be determined that the photographer is tracking the subject so that it does not move on the sensor). That is, when the output signal of the shake detection means is larger than the first output signal and the motion vector information is smaller than the first motion vector. Alternatively, when a newly detected main subject, such as a face recognition function, can be considered the same as the previous main subject. This sets the subject of shake correction control to the same subject as the subject of focus control, resulting in a good image that is in focus and has little subject blur.

[0092] When a new main subject is detected, the subject of image stabilization control is not the same as the subject of focus control, for example, in the following cases (1) to (5). (1) When it is not possible to determine the main subject from multiple subjects. (2) When it can be determined that a different motion has intersected from the motion of the subject calculated by the motion vector detection means and the blur detection means. (3) When the main subject is detected near the edge of the screen. (4) When the main subject area detection means temporarily fails to detect the subject. (5) The difference in the defocus amounts of the detected multiple subjects is equal to or less than a predetermined value.

[0093] Since focus control is always necessary, a single control target must be determined, but with image stabilization control, it is possible to perform image stabilization processing (camera stabilization) on the background without determining a specific subject to control. Therefore, if the subject detected as the main subject is not one for which the photographer wants to correct subject blur, the subject to image stabilization control can be set to a different subject from the subject to focus control. This makes it possible to provide smooth, high-quality images even in scenes with multiple mixed subjects.

[0094] (Second embodiment) Next, a second embodiment of the present invention will be described with reference to Fig. 6. This embodiment is the same as the first embodiment except for the processing in step S423 in Fig. 4. Fig. 6 is a flowchart of the image stabilization processing in this embodiment. The steps in Fig. 6 are mainly executed by CPU 15 or by various parts such as main subject detection circuit 26 based on instructions from CPU 15.

[0095] First, in step S601, CPU 15 determines whether the old and new main subjects are within the depth of field at the aperture value used during video recording. If the new and old main subjects are within the depth of field, the process proceeds to step S608, where the area where the old main subject is present is designated as the target area for image stabilization control. On the other hand, if the new and old main subjects are outside the depth of field, the process proceeds to step S602. That is, CPU 15 acquires from AF processing circuit 12 the defocus amounts for the area where the old main subject is present, detected by main subject detection circuit 26, and the area where the new main subject is present, detected by main subject detection circuit 26. CPU 15 then determines whether the difference between these values ​​is within the depth of field. The depth of field is calculated by multiplying the aperture value (aperture value information) obtained by communicating with interchangeable lens 31 via communication driver 21 by a predetermined permissible circle of confusion.

[0096] In step S602, CPU 15 determines whether a focusing operation (driving focus lens 33) on a new main subject has started. If a focusing operation on a new main subject has started, the process proceeds to step S603. On the other hand, if a focusing operation on a new main subject has not started, the process proceeds to step S608.

[0097] In step S603, CPU 15 determines whether the area where the former main subject exists is still within the depth of field from the drive position of focus lens 33 obtained by communicating with interchangeable lens 31 via communication driver 21. If the area where the former main subject exists is within the depth of field, the process proceeds to step S608. On the other hand, if the area where the former main subject exists is outside the depth of field, the process proceeds to step S604.

[0098] In step S604, CPU 15 determines whether the focusing operation on the new main subject has been completed. If the focusing operation on the new main subject has been completed, the process proceeds to step S605. On the other hand, if the focusing operation on the new main subject has not been completed, the process proceeds to step S608. If the new main subject is not yet in focus, even if the area where the new main subject exists is set as the target area for image stabilization control, a good image that is in focus and has little subject blur cannot be obtained. Therefore, by setting the area where the previous main subject exists as the target area for image stabilization control, a continuity of image is provided.

[0099] In step S605, CPU 15 determines whether the aperture value of interchangeable lens 31 can be changed. If the aperture value can be changed, the process proceeds to step S606. In step S606, CPU 15 narrows aperture 34 of interchangeable lens 31. Subsequently, in step S607, CPU 15 determines whether narrowing aperture 34 can bring the old main subject within the depth of field. If it is possible to bring the old main subject within the depth of field, the process proceeds to step S608. On the other hand, if it is not possible to bring the old main subject within the depth of field, the process proceeds to step S609. In step S609, CPU 15 designates the area where the new main subject exists as the target area for image stabilization control. Note that if it is not possible to change the aperture value of interchangeable lens 31 in step S605, the process proceeds to step S609, where the area where the new main subject exists is designated as the target area for image stabilization control.

[0100] In this embodiment, main subject detection circuit 26 serves as a main subject detection means for detecting the location of the main subject (main subject information). CPU 15 has a function for determining the image stabilization control area based on the determination result of the determination means. As a result, if the subject detected as the main subject may not necessarily be the subject for which the photographer wants to correct subject blur, or if the previous main subject is within the depth, the subject targeted for image stabilization control is set to a different subject from the subject targeted for focus control. This makes it possible to provide smooth, high-quality images even in scenes with multiple subjects.

[0101] While each embodiment has been described using an interchangeable lens imaging device as an example, the invention can also be applied to image stabilization in digital video cameras, integrated lens digital cameras, etc. In each embodiment, the CPU is responsible for the functions of the calculation means for calculating the correction amount, the means for designating the focus control target area, the determination means for determining whether the image stabilization control area should be different from the focus control area, and the function for determining the image stabilization control area based on the determination result of the determination means. However, dedicated circuits for performing these functions may also be provided.

[0102] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0103] In each embodiment, the subject (area on the screen) that is the target of image stabilization control is switched between the same subject as the subject of focus control or a different subject based on predetermined conditions. Therefore, each embodiment can provide a control device, an imaging device, a control method, and a program that can capture smooth, high-quality images in scenes with multiple subjects.

[0104] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0105] The shake correction process in each embodiment is a process using electronic image stabilization (image stabilization by trimming), but is not limited to this. For example, the shake correction process can also be performed by performing an optical image stabilization process, such as moving a correction lens (image stabilization lens) of the imaging optical system in a direction perpendicular to the optical axis, or by moving the sensor 3 in a direction perpendicular to the optical axis. [Explanation of symbols]

[0106] 1. Imaging device (control device) 13 Shake detection circuit (shake detection means) 14 Shake detection sensor (shake detection means) 15 CPU (control means) 26 Main subject detection circuit (main subject detection means) 27 Motion vector detection circuit (motion vector detection means)

Claims

[Claim 1] a subject detection means for detecting main subject information; a motion vector detection means for detecting motion vector information in an image; a shake detection means for detecting shake information using an inertial sensor; and control means for determining whether or not to make a blur correction control area different from a focus control area based on the main subject information, the blur information, and the motion vector information.

Images