Image capture apparatus, electronic device, and control method therefor

Abstract

To provide an image capture apparatus capable of properly applying a subject blur correction function, and a control method of the imaging apparatus.SOLUTION: The image capture apparatus performs subject blur correction for suppressing a positional change of a main subject in an image. The apparatus determines if a motion of the main subject is a predetermined specific motion. In a case where the motion of the main subject is determined to be the specific motion, the apparatus suspends execution of the subject blur correction.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an imaging device, an electronic device, and a control method thereof. [Background technology]

[0002] A subject blur correction function is known that suppresses changes in the position of a subject (subject blur) in a captured image by controlling a cut-out position according to the position of a specific subject or by driving a correction lens (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-215350 A Summary of the Invention [Problem to be solved by the invention]

[0004] If the moving direction and speed of the specific subject are constant, the accuracy of detecting or predicting the subject position is high, and the accuracy of subject blur correction is also high. On the other hand, if the moving direction and speed of the specific subject are not constant, the accuracy of detecting or predicting the subject position is low. For example, if the specific subject is performing a motion that includes reciprocating motion, such as repeating a jump motion, the accuracy of detecting or predicting the subject position is low each time the moving direction is reversed. As a result, there is a risk that the positions of both the specific subject and the background in the captured video will become unstable.

[0005] In one aspect, the present invention provides an imaging apparatus, an electronic device, and a control method thereof that are capable of appropriately applying a subject blur correction function. [Means for solving the problem]

[0006] In one aspect, the present invention provides an imaging device capable of subject blur correction that suppresses changes in the position of a main subject in an image, the imaging device comprising: a discrimination means for discriminating whether or not the movement of the main subject is a predetermined specific movement; and a control means for controlling the imaging device to interrupt execution of subject blur correction when the discrimination means discriminates that the movement of the main subject is a specific movement. Effect of the Invention

[0007] According to one aspect of the present invention, it is possible to provide an imaging device, an electronic device, and a control method thereof that are capable of appropriately applying a subject blur correction function. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an example of the functional configuration of a digital camera as an example of an imaging apparatus according to an embodiment; [Diagram 2] Flowchart for controlling the subject blur correction in the embodiment [Diagram 3] 1 is an exemplary circuit diagram of a low pass filter that can be used in an embodiment; [Figure 4A] 1 is a flowchart showing a main subject motion determination operation in the first embodiment. [Figure 4B] 1 is a flowchart showing a main subject motion determination operation in the first embodiment. [Diagram 5] 11 is a flowchart for determining the movement of a main subject in the second embodiment. [Figure 6] 13 is a flowchart showing the operation of determining the movement of a main subject in the third embodiment. [Figure 7] FIG. 10 shows a method for determining the movement of a main subject according to a third embodiment. [Figure 8] 13 is a flowchart for determining the movement of a main subject in the fourth embodiment. [Figure 9] FIG. 10 is a diagram showing a method 1 for determining the movement of a main subject in the third embodiment. [Figure 10] FIG. 13 is a diagram showing an example of output of a judgment index in the first embodiment. [Figure 11] FIG. 13 is a diagram showing an example of a time series change in a main subject position and an LPF output thereof in the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The present invention will be described in detail below based on its exemplary embodiments with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. In addition, although multiple features are described in the embodiments, not all of them are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numbers are used for the same or similar configurations, and duplicated explanations are omitted.

[0010] In the following embodiment, the present invention will be described with respect to a case where the present invention is implemented in a digital camera. However, the present invention can also be implemented in any electronic device having an imaging function. Such electronic devices include video cameras, computer devices (personal computers, tablet computers, media players, PDAs, etc.), mobile phones, smartphones, game consoles, robots, drones, and drive recorders. These are merely examples, and the present invention can also be implemented in other electronic devices.

[0011] ●(First embodiment) (Configuration of Digital Camera 1) 1 is a block diagram showing an example of the functional configuration of a lens-interchangeable digital camera 1, which is an example of an imaging device capable of implementing the present invention. In FIG 1, an interchangeable lens 31 is attached to the digital camera 1.

[0012] The interchangeable lens 31 is detachably attached to a lens mount 2 provided on the digital camera 1. The interchangeable lens 31 and the lens mount 2 are provided with terminals configured to come into contact when the interchangeable lens 31 is attached to the lens mount 2. Power is supplied from the digital camera 1 to the interchangeable lens 31 through the terminals. In addition, communication is possible between the CPU 15 of the digital camera 1 and a control circuit 36 ​​of the interchangeable lens 31 through the terminals.

[0013] The interchangeable lens 31 has an imaging optical system that forms an optical image of a subject on the imaging plane of the image sensor 3. The imaging optical system is composed of multiple lenses including a movable lens. Here, a motion compensation lens 32 and a focus lens 33 are shown as examples of the movable lens.

[0014] The aperture size of the diaphragm 34 is adjustable. The aperture size of the diaphragm 34 is controlled by the control circuit 36 ​​in accordance with commands from the CPU 15. The aperture size (F value) of the diaphragm 34 can also be changed manually by the user.

[0015] The communication driver 35 supports communication with the communication driver 21 on the camera side.

[0016] The control circuit 36 ​​is, for example, a microprocessor, and controls the drive of the movable lens and the aperture opening by loading and executing programs stored in the EEPROM 37 into its internal memory, and transmits information about the interchangeable lens 31 to the digital camera 1.

[0017] The EEPROM 37 stores unique information and setting values ​​of the interchangeable lens 31, programs executed by the control circuit 36, and the like.

[0018] The imaging element 3 may be, for example, a known CCD or CMOS color image sensor having a primary color Bayer array color filter. The imaging element 3 has a pixel array in which a plurality of pixels are arranged two-dimensionally, and a peripheral circuit for reading out signals from each pixel. Each pixel accumulates electric charge according to the amount of incident light by photoelectric conversion. By reading out a signal having a voltage according to the amount of electric charge accumulated during the exposure period from each pixel, a group of pixel signals (analog image signals) representing the subject image formed on the imaging plane by the imaging optical system of the interchangeable lens 31 can be obtained.

[0019] The imaging circuit 4 applies noise reduction processing, gain, etc. to the analog image signal.

[0020] The image processing circuit 5 A / D converts the signal output from the imaging circuit 4 into image data, and applies predetermined image processing to generate display image data or recording image data.

[0021] Image processing that can be applied by the image processing circuit 5 can include, for example, pre-processing, color interpolation processing, correction processing, data processing, special effect processing, and the like. Pre-processing may include signal amplification, reference level adjustment, defective pixel correction, and so on. Color interpolation processing is performed when a color filter is provided on an image sensor, and is a process for interpolating values ​​of color components that are not included in the individual pixel data that constitutes the image data. Color interpolation processing is also called demosaic processing. The correction processing can include white balance adjustment, gradation correction, correction of image degradation caused by optical aberration of the imaging optical system 101 (image restoration), correction of the effects of peripheral light falloff of the imaging optical system 101, color correction, and the like. The data processing may include processing such as synthesis, scaling, etc. The generation of image data for display or image data for recording is also included in the data processing. The special effects processing may include adding a blur effect, changing color tones, relighting, and the like. Note that these are examples of processes that can be applied by the image processing circuit 5, and do not limit the processes applied by the image processing circuit 5. The image data processed by the image processing circuit 5 is supplied to various functional blocks depending on the application.

[0022] The VRAM 6 stores image data for display, and is also used as a buffer memory for image data for recording, etc. The image data for display stored in the VRAM 6 is converted by a D / A conversion circuit 7 into an analog image signal in a format suitable for display, and then displayed on the LCD 8, which is a display device.

[0023] The compression / decompression circuit 9 reduces the amount of data by encoding the image data for recording stored in the VRAM 6. The compression / decompression circuit 9 also generates a data file that stores the encoded image data and records it in the storage memory 10. The compression / decompression circuit 9 further reads out and decodes the image data recorded in the storage memory 10 and stores it in the VRAM 6.

[0024] The storage memory 10 may be a removable storage medium such as a memory card, and is used to store image data captured by the digital camera 1.

[0025] The AE processing circuit 11 calculates a predetermined AE evaluation value based on the image data output by the image processing circuit 5. Then, the AE processing circuit 11 performs automatic exposure control (AE) processing to determine the imaging conditions based on the AE evaluation value. The AE processing circuit 11 notifies the CPU 15 of the determined imaging conditions. The CPU 15 transmits a target F-value of the aperture 34 to the interchangeable lens 31 via the communication driver 21. When the control circuit 36 ​​of the interchangeable lens 31 receives this command via the communication driver 35, it drives the aperture 34 to the F-value according to the command.

[0026] The AF processing circuit 12 calculates a predetermined AF evaluation value based on the image data output by the image processing circuit 5. Then, the AF processing circuit 12 performs auto focus detection (AF) processing to determine a target position of the interchangeable lens 31 based on the AF evaluation value. The AF processing circuit 12 notifies the CPU 15 of the determined target position. The CPU 15 transmits a command to the interchangeable lens 31 via the communication driver 21 to move the focus lens 33 to the target position. When the control circuit 36 ​​of the interchangeable lens 31 receives this command via the communication driver 35, it moves the focus lens 33 to a position according to the command.

[0027] The shake detection sensor 14 outputs a signal corresponding to the movement of the digital camera 1. The shake detection sensor 14 is, for example, a gyro sensor or an acceleration sensor, and outputs a signal corresponding to the movement in three orthogonal axial directions and around three axes.

[0028] The shake detection circuit 13 processes the output signal of the shake detection sensor 14 and supplies the detected movement of the digital camera 1 to the CPU 15 and the main subject detection circuit 26 .

[0029] The CPU 15 is a control unit of the digital camera 1. The CPU 15 loads a program stored in the EEPROM 19 into the RAM 29 and executes it to control the operation of each unit of the digital camera 1 and realize the functions of the digital camera 1. The CPU 15 also communicates with a control circuit 36 ​​of the interchangeable lens 31 and controls the operation of the interchangeable lens 31.

[0030] In this embodiment, the CPU 15 determines whether the main subject is moving back and forth based on the subject position detected by the main subject detection circuit 26, and stops the subject blur correction for the main subject when it is determined that the main subject is moving back and forth. Details will be described later. Back and forth movement is a movement in which the direction of movement is repeatedly reversed, and in the case of a human subject, it is a movement such as swinging the body from side to side or repeatedly jumping upward.

[0031] The EEPROM 19 stores programs executed by the CPU 15, various setting values ​​of the digital camera 1, GUI data, etc. The RAM 29 is a main memory used when the CPU 15 loads and executes programs. The RAM 29 and the VRAM 6 may be separate areas in the same memory space.

[0032] A timing generation circuit (TG) 16 generates timing signals for controlling the operations of the image sensor 3, the image pickup circuit 4, and the CPU 15. The sensor driver 17 controls the operation of the imaging element 3 in response to a timing signal output from the TG 16 .

[0033] The operation switch (SW) 18 is a general term for input devices (buttons, switches, dials, etc.) provided for the user to input various instructions to the digital camera 1. The input devices constituting the operation SW 18 have names according to the functions assigned to them. For example, the operation SW 18 includes a release switch, a video recording switch, a shooting mode selection dial for selecting a shooting mode, a menu button, direction keys, and an enter key. The release switch is a switch for recording still images, and the CPU 15 recognizes the half-pressed state of the release switch as an instruction to prepare for shooting and the full-pressed state as an instruction to start shooting. In addition, the CPU 15 recognizes the pressing of the video recording switch in a shooting standby state as an instruction to start recording a video, and the pressing of the switch during video recording as an instruction to stop recording.

[0034] The battery 20 is, for example, a battery pack, and supplies the power necessary for the digital camera 1 and the interchangeable lens 31 to operate. The communication driver 21 supports communication with a communication driver 35 included in an interchangeable lens 31 attached to the lens mount 2 .

[0035] The LED 22 is a display element used to display, for example, the operating state of the digital camera 1, warnings, and the like. The speaker 23 is used to output an alarm, audio guidance, and the like. The sensor moving motor 25 moves the imaging element 3 in the horizontal direction, the vertical direction and the rotation direction. The operation of the sensor moving motor 25 is controlled by the CPU 15 via a sensor movement control circuit 24.

[0036] Motion vector detection circuit 27 detects the motion vector of a subject in an image based on image data output by image processing circuit 5, for example, data of multiple frames of a moving image. Motion vector detection circuit 27 detects a motion vector, for example, for each region obtained by dividing an image, and outputs the detection result to main subject detection circuit 26 and CPU 15.

[0037] Main subject detection circuit 26 detects a main subject region in an image from image data output by image processing circuit 5. Main subject detection circuit 26 outputs information such as the position, size, and reliability of the detected main subject region.

[0038] For example, main subject detection circuit 26 can separate an image into a background region and a subject region based on the motion vector detected by motion vector detection circuit 27, and detect, for example, the subject region closest to the center as the main subject region.

[0039] Alternatively, if the type of the main subject is specified in advance, main subject detection circuit 26 may detect a feature region corresponding to the main subject and determine one of the detected feature regions as the main subject region. For example, main subject detection circuit 26 may detect a human face region or head region as a feature region using a known method, and determine the feature region that is the largest, closest to the center, or to which a focus detection region is set as the main subject region.

[0040] Furthermore, when information on a main subject region designated by a user is obtained through CPU 15, main subject detection circuit 26 may detect the main subject region in the next frame based on a motion vector detected for the main subject region. Alternatively, main subject detection circuit 26 may detect a region having a specific color as a characteristic region.

[0041] In addition, the main subject detection circuit 26 may obtain information on the amount of defocus in the image from the AF processing circuit 12, and detect the area with the smallest difference in the amount of defocus from the immediately previous frame (the area with the most similar degree of focus) as the characteristic area.

[0042] The method is not limited to the one exemplified here, and main subject detection circuit 26 can detect the main subject region by any known method using one or more of a motion vector, a defocus amount, a set AF frame, a region designated by the user, detection results of a characteristic region, etc. Note that main subject detection circuit 26 may detect multiple main subject regions. In this case, main subject detection circuit 26 may assign a rank to the main subject regions based on their resemblance to the main subject.

[0043] The image transformation and cutting circuit 28 applies rotation, enlargement, reduction, trimming (cutting) and the like to the image data output from the image processing circuit 5.

[0044] Details will be described later, but in this embodiment, subject blur is corrected by the CPU 15 controlling the position where the image transformation and cut-out circuit 28 cuts out the image area based on the position of the main subject area. In other words, the change in the position of the main subject is suppressed by changing the position where a partial image including the main subject is cut out from the image acquired using the image sensor 3. Note that when multiple main subject areas are detected, the main subject area with the highest ranking of main subject resemblance is targeted. The image transformation and cut-out circuit 28 stores the processed image data in an area of ​​the VRAM 6 depending on the purpose.

[0045] It should be noted that the CPU 15 can perform camera shake correction by, for example, moving the motion compensation lens 32 and / or the imaging element 3 based on the output of the motion compensation circuit 13.

[0046] (shake compensation control) Next, the image stabilization control operation performed by digital camera 1 of this embodiment will be described with reference to the flowchart shown in Fig. 2. This operation is realized by CPU 15 executing a program to control the operation of each unit. This operation is executed when digital camera 1 is shooting video and the subject blur correction function is enabled. Note that the video being shot may be for live view display or for recording.

[0047] When shooting video for recording, in addition to video data for recording, video data for display is also generated and displayed as a live view on the LCD 8. Note that the operation related to image stabilization will be described here, and descriptions of other processes executed during video shooting, such as AE processing, AF processing, display processing, and storage processing, will be omitted.

[0048] While a moving image is being shot, image data for each frame is supplied from the image processing circuit 5 to, for example, the AE processing circuit, the AF processing circuit 12, the motion vector detection circuit 27, the main subject detection circuit 26, and the image transformation and cut-out circuit 28. Each of these circuits then executes the above-mentioned processing. Note that each of these circuits does not need to execute processing for all frames to which image data is supplied from the image processing circuit 5.

[0049] In S201, the CPU 15 sets the value of a status flag, which is set to 1 when subject blur compensation is to be suspended, to 0 in order to make subject blur compensation executable.

[0050] In S202, the CPU 15 executes subject blur correction. As described above, the CPU 15 corrects subject blur by controlling the position at which the image transformation and cut-out circuit 28 cuts out the image so as to suppress a change in the position of the main subject area in the image.

[0051] A specific subject blur correction operation will now be described. CPU 15 acquires the output signal of blur detection circuit 13 and the detection result of the main subject region by main subject detection circuit 26. CPU 15 applies low-pass filter processing (LPF processing) to position information (e.g., image coordinates corresponding to the center or center of gravity of the region) included in the detection result of the main subject region. By applying LPF processing, it is possible to suppress the influence on the detection result of errors in the detection process and movements of the main subject that should not be reflected in the detection result (e.g., movements such as momentarily lowering one's face).

[0052] The cutoff frequency of the LPF processing can be determined in consideration of the characteristics of the movement of the main subject, etc. (for example, about 2 to 3 Hz). Furthermore, the LPF processing can be realized by the CPU 15 applying a calculation to the position information to obtain a low-pass filter output (LPF_OUT) using a digital filter configured as shown in FIG.

[0053] The CPU 15 obtains the LPF output (LPF_OUT) by applying the calculation shown in the following formula to the horizontal coordinate and vertical coordinate of the position information of the main subject region. LPFoutX(n) = K1·[ CX(n) - LPFoutX(n-1)] + LPFoutX(n-1) LPFoutY(n) = K1·[ CY(n) - LPFoutY(n-1)] + LPFoutY(n-1) Where: CX(n): The horizontal coordinate of the main subject area at time n CY(n): Vertical coordinate of the main subject area at time n LPFoutX(n): Horizontal LPF output at time n LPFoutY(n): Vertical LPF output at time n The coordinates are values ​​in a predetermined image coordinate system. The value of K1 is determined by the cutoff frequency. For example, if the cutoff frequency is set to 2 Hz and the sampling frequency is set to 30 Hz, the value of K1 will be approximately 0.3463.

[0054] The cutoff frequency can be determined based on the characteristics of the reciprocating movement of the main subject to be detected. The frame rate and exposure time during video shooting do not need to be taken into consideration when determining the cutoff frequency. For example, the frequency of the up and down reciprocating movement that occurs in a person walking or running may be measured in advance through an experiment or the like, and a cutoff frequency that does not cut the measured frequency but cuts unnecessary movements such as a momentary face-down movement may be stored in the EEPROM 19 in advance. Alternatively, a measurement period may be set to measure the characteristics of the reciprocating movement of the main subject to be detected, and a change in the motion vector of the main subject may be measured based on the image data of each frame acquired during the measurement period, and an appropriate cutoff frequency may be calculated.

[0055] Using the LPF output values ​​(LPFoutX(n), LPFoutY(n)) as the position of the main subject region, CPU 15 controls the cut-out position of image transformation cut-out circuit 28. For example, CPU 15 calculates the cut-out position (target position) of the image so that the position of the main subject region maintains the center of the image or the position at the start of recording. Then, CPU 15 outputs the calculated cut-out position (target position) to image transformation cut-out circuit 28 as subject blur correction position information. Note that the amount of change from the previous cut-out position may also be used as subject blur correction position information.

[0056] When subject blur correction is performed so that the main subject region is located at the center of the image, the subject blur correction position information is, for example, the coordinates [LPFoutX(n)-r·X / 2, LPFoutY(n)-r·Y / 2]. In addition, when subject blur correction is performed so as to maintain the position (SX, SY) of the main subject area when recording began, the subject blur correction position information is, for example, the coordinates [(1-r)X / 2+LPFoutX(n)-SX, (1-r)Y / 2+LPFoutY(n)-SY]. Here, r is the cropping ratio with respect to the size of the entire image, and for example, when cropping an area that is 70% in both the horizontal and vertical directions, r = 0.7 (r < 1). Also, X and Y are the number of pixels in the horizontal and vertical directions of the image. The minimum value of the coordinates constituting the subject blur correction position information is 0, and the maximum value is (1-r)·X (horizontal direction) or (1-r)·Y (vertical direction).

[0057] The image transformation and cutting out circuit 28 cuts out a part of the image in accordance with the subject blur correction position information output from the CPU 15 and stores it in the VRAM 6.

[0058] In S203, CPU 15 determines whether the main subject is making a specific movement, specifically whether the main subject is moving back and forth, and sets a value corresponding to the determination result to a state flag for the subject blur correction operation. Details of the operation of S203 will be described later. Here, the specific movement is not limited to back and forth movement, but is any movement for which subject blur correction may cause a deterioration in the quality of a captured image. It is possible to predict in advance what kind of movement corresponds to the specific movement depending on the specific method of subject blur correction, and therefore a detection method can be included in the program executed by CPU 15 and implemented in advance.

[0059] In S204, CPU 15 refers to the value of the status flag of the subject blur correction operation, and executes S205 if the main subject is making a specific movement. Here, if the status flag has a value indicating that a movement including a reciprocating movement in the vertical direction or both directions has been detected, CPU 15 considers that the main subject is making a specific movement. On the other hand, if the main subject is not making a specific movement, CPU 15 returns the process to S202 and continues or resumes the execution of subject blur correction.

[0060] In S205, CPU 15 suspends execution of subject blur correction. CPU 15 also determines the cut-out position in image transformation cut-out circuit 28 while subject blur correction is suspended. The cut-out position while subject blur correction is suspended is a fixed value. This fixed value may be, for example, the final cut-out position, or may be determined by CPU 15 according to another condition. Note that instead of suspending execution of subject blur correction, the strength of subject blur correction (strength for suppressing positional changes in the main subject area in the image) may be weakened compared to S202.

[0061] CPU 15 can perform camera shake correction, for example, by moving blur correction lens 32 and / or image sensor 3 based on the output of blur detection circuit 13, independent of subject blur correction. While subject blur correction is suspended, CPU 15 can change the image cut-out position for the purpose of camera shake correction. CPU 15 can, for example, change the image cut-out position so as to correct movement in a range that cannot be corrected by driving blur correction lens 32 and / or image sensor 3.

[0062] In S206, the CPU 15 outputs the subject blur correction position information consisting of the fixed value obtained in S205 to the image transformation and cutting out circuit 28. The image transformation and cutting out circuit 28 cuts out the image according to the fixed position indicated by the subject blur correction position information and stores it in the VRAM 6. The CPU 15 repeats the process from S203. Note that the process of S205 may be skipped while the specific movement of the main subject is continuously detected.

[0063] In this way, CPU 15 suspends subject blur correction while a specific movement of the main subject is detected.

[0064] The motion discrimination process in S203 will be described in detail with reference to the flowcharts shown in FIGS. 4A and 4B. In S401, CPU 15 calculates an index relating to the movement of the main subject in the vertical direction of the image. Specifically, CPU 15 first calculates an integrated value of the absolute difference between the vertical coordinate CY() of the main subject region and the vertical LPF output LPFoutY().

[0065] Here, two indexes with different integration periods are calculated. Specifically, the first integration period is 10 frames (for example, 1 / 3 seconds when the frame rate of the video being shot is 30 fps), and the second integration period is 30 frames (1 second). The integration value of the first integration period is Def10Y(n), and the integration value of the second integration period is Def30Y(n).

[0066] The length of the integration period is important, and when the integration period is specified by the number of frames, it is a variable value based on a predetermined reference frame rate. Therefore, when the actual frame rate is different from the reference frame rate, the number of frames that specifies the integration period is converted according to the actual frame rate. For example, the reference frame rate is 30 fps, and the integration period is specified as 10 frames. In this case, if the actual frame rate is 60 fps, the integration period is set to 20 frames.

[0067] However, when the actual frame rate is lower than the reference frame rate (24 fps), the number of frames may not be converted, and the integration period may be based on the reference frame rate (10 frames in this case).

[0068] Furthermore, when the exposure time of one frame is longer than the readout interval based on the frame rate set by the photographer, the exposure time generally takes priority, and the actual frame rate becomes lower than the set frame rate. For example, when the set frame rate is 60 fps, the readout interval is 1 / 60 seconds. However, when the exposure time is 1 / 40 seconds (>1 / 60 seconds), the upper limit of the frame rate is limited to 40 fps. When the digital camera 1 automatically sets the exposure time, the exposure time depends on the brightness of the shooting scene. Therefore, the frame rate may depend on the brightness of the shooting scene.

[0069] Furthermore, when the photographed scene is dark, noise in the captured image increases, which may reduce the accuracy of the index related to the movement of the main subject. Therefore, when the brightness of the photographed scene is below a threshold (or when the photographing sensitivity is above a threshold), the integration period may be extended to suppress the reduction in the accuracy of the index.

[0070] For example, consider a case where noise in a captured image affects the accuracy of an index when the shooting sensitivity exceeds ISO800. When the shooting sensitivity exceeds ISO800 by user setting or automatic setting, the CPU 15 can extend the integration interval for calculating the index (increase the number of frames). For example, when a shooting sensitivity exceeding ISO800 is required to compensate for a lack of exposure time in order to maintain a frame rate equal to or higher than a threshold, the CPU 15 can extend the integration interval so as to obtain an index equivalent to a proper amount of exposure while keeping the shooting sensitivity at ISO800. In this way, the integration interval can be changed in consideration of one or more of the frame rate, exposure time, and brightness of the shooting scene so as to obtain an index with high accuracy.

[0071] Then, the CPU 15 normalizes the integrated value according to each integration period. The CPU 15 normalizes the integrated value Def10Y(n) for the first integration period by the ratio of the actual frame rate to the reference frame rate. This is because the threshold value applied to the integrated value Def10Y(n) is a value assuming that the integrated value is integrated at the reference frame rate.

[0072] When a frame rate faster than the reference frame rate (e.g. 30 fps) is set, the accumulation interval (number of frames) increases as described above. To make the value correspond to the number of frames at the reference frame rate, the accumulation value is normalized by the ratio of the actual frame rate to the reference frame rate. For example, if the actual frame rate is 60 fps, the accumulation value is set to 1 / 2 (= 30 fps / 60 fps).

[0073] Even if the frame rate has changed from the reference frame rate due to prioritizing the exposure time, if the new frame rate is faster than the reference frame rate, the integrated value is normalized according to the frame rate ratio. Even if the integration interval has been changed taking into account the brightness of the shooting scene, if the integration interval (number of frames) has increased from the reference frame rate, the value is normalized to the value at the reference frame rate according to the frame rate ratio.

[0074] On the other hand, the integrated value Def30Y(n) of the second integration period is normalized by dividing it by the ratio of the second integration period to the first integration period. Since the second integration period / the first integration period=3 in this example, the CPU 15 normalizes the integrated value Def30Y(n) by dividing it by 3.

[0075] The CPU 15 clips the normalized integrated value to the maximum value within a certain period equal to the integration period, and sets the index as the index Def10YMAX(n) and Def30YMAX(n).

[0076] In S402, CPU 15 calculates an index relating to the movement of the main subject in the horizontal direction of the image. The calculation is performed in the same manner as S401, except that the integrated value of the absolute difference between the horizontal coordinate CX() of the main subject region and the horizontal LPF output LPFoutX() is used. The obtained indexes are defined as Def10XMAX(n) and Def30XMAX(n).

[0077] In S403, CPU 15 calculates indices Def10WMAX(n) and Def30WMAX(n) relating to the movement of the main subject in both the vertical and horizontal directions (ie, diagonal directions) of the image by the following formulas. Def10WMAX(n) = ( Def10YMAX(n) 2 + Def10XMAX(n) 2 ) 1 / 2 Def30WMAX(n)=( Def30YMAX(n) 2 + Def30XMAX(n) 2 ) 1 / 2

[0078] Based on the various indices thus obtained, the CPU 15 determines the movement of the main subject in S404 and subsequent steps. First, in S404, the CPU 15 determines whether or not both of the vertical indicators Def10YMAX(n) and Def30YMAX(n) exceed the first threshold value. If it is determined that both of the vertical indicators exceed the first threshold value, the CPU 15 executes S405, and if not, the CPU 15 executes S406.

[0079] The first threshold value can be determined in advance, for example, experimentally depending on the frame rate. Def10WMAX(n) and Def30WMAX(n) are normalized by the integration interval, so they do not depend on the integration interval. However, the indices (Def10YMAX(n), Def30YMAX(n), Def10XMAX(n), Def30XMAX(n)) are calculated using the absolute difference between the coordinates of the main subject area and its LPF output, so they depend on the frame rate. Specifically, when the frame rate slows down, the LPF effect occurs, so the difference between Def10YMAX(n) and Def30YMAX(n) (the difference between Def10XMAX(n) and Def30XMAX(n)) tends to become smaller. In addition, Def10WMAX(n) and Def30WMAX(n), which are the indices for judgment, also tend to become smaller.

[0080] 10(A) to 10(C) show examples of the indices Def10YMAX(n) and Def30YMAX(n) for determining each frame rate. As shown in FIG. 10(A) to 10(C), as the frame rate becomes slower, the index value and the difference between the index values ​​in the section where a specific movement, such as a back and forth movement, occurs that may cause the subject blur correction to degrade the quality of the captured image becomes smaller. Therefore, an appropriate first threshold value is determined for each of a plurality of frame rates.

[0081] In S405, the CPU 15 determines that the movement of the main subject includes a reciprocating movement in the vertical direction, and sets the variable Ver_repe to 1. After that, the CPU 15 executes S408.

[0082] In S406, the CPU 15 determines whether or not both of the vertical indicators Def10YMAX(n) and Def30YMAX(n) are equal to or less than a second threshold. The second threshold is smaller than the first threshold, and can be determined experimentally in advance depending on the frame rate, for example. If it is determined that both of the vertical indicators are equal to or less than the second threshold, the CPU 15 executes S407, and if not, executes S408 without changing the value of Ver_repe. The first threshold and the second threshold may be equal to each other.

[0083] In S407, CPU 15 determines that the movement of the main subject does not include reciprocating movement in the vertical direction, and clears the variable Ver_repe (sets it to 0). After that, CPU 15 executes S408.

[0084] In S408, the CPU 15 determines whether or not both of the indicators Def10WMAX(n) and Def30WMAX(n) in both directions exceed the first threshold value, similarly to S404. If it is determined that both of the indicators in both directions exceed the first threshold value, the CPU 15 executes S409, and if not, executes S410.

[0085] In S409, CPU 15 determines that the movement of the main subject includes reciprocating movement in both directions, and sets the variable Both_repe to 1. After that, CPU 15 executes S421 (FIG. 4B).

[0086] In S410, the CPU 15 determines whether or not both of the indicators Def10WMAX(n) and Def30WMAX(n) in both directions are equal to or less than the second threshold value, similarly to S406. If it is determined that both of the indicators in both directions are equal to or less than the second threshold value, the CPU 15 executes S411, and if not, executes S421 (FIG. 4B) without changing the value of Both_repe.

[0087] In S411, CPU 15 determines that the movement of the main subject does not include reciprocating movement in both directions, and clears the variable Both_repe (sets it to 0). After that, CPU 15 executes S421 (FIG. 4B).

[0088] Moving on to FIG. 4B, in the processing from S421 onwards, the CPU 15 sets the state flag for the subject blur correction operation to a value according to the values ​​of the variables Ver_repe and Both_repe. In S421, the CPU 15 judges whether or not the variables Ver_repe and Both_repe are both 0. If it is judged that both are 0, the CPU 15 executes S431, and if not, the CPU 15 executes S422.

[0089] In S431, CPU 15 determines that the movement of the main subject does not include reciprocating movement in either the vertical direction or both directions, and clears the status flag (sets it to 0). Then, CPU 15 ends the process of S203.

[0090] In S422, the CPU 15 determines whether or not the condition that the value of the status flag is not 2 (is 0 or 1) and the value of the variable Ver_repe is 1 is met, and if it is determined that the condition is met, executes S432, and if not, executes S423.

[0091] At S432, CPU 15 determines that the movement of the main subject does not include reciprocating movement in both directions, but does include reciprocating movement in the vertical direction. Based on the determination, CPU 15 sets the status flag to a value of 1 (flag=1), which indicates that reciprocating movement in the vertical direction has been detected. Then, CPU 15 ends the process of S203.

[0092] In S423, the CPU 15 determines whether or not the conditions that the value of the status flag is 2 and the value of the variable Both_repeat is 1 are met, and if it is determined that the conditions are met, executes S433, and if not, executes S424.

[0093] In S433, CPU 15 determines that the state in which the movement of the main subject includes reciprocating movement in both directions continues. Based on the determination, CPU 15 sets the status flag to a value of 2 (flag=2), which indicates that reciprocating movement in both directions has been detected. Then, CPU 15 ends the process of S203.

[0094] In S424, the CPU 15 determines whether or not the condition that the value of the status flag is not 1 (is 0 or 2) and the value of the variable Ver_repe is 1 is met, and if it is determined that the condition is met, executes S434, and if not, executes S425.

[0095] In S434, CPU 15 determines that the movement of the main subject has changed from a state that does not include reciprocating movement in the vertical direction to a state that includes it. Based on the determination, CPU 15 sets the state flag "flag" to a value of 1 (flag=1), which indicates that reciprocating movement in the vertical direction has been detected. Then, CPU 15 ends the process of S203.

[0096] In S425, the CPU 15 determines whether or not the condition is met that the value of the status flag is not 2 (is 0 or 1) and the value of the variable Both_repe is 1, and if it is determined that the condition is met, it executes S435, and if not, it ends the processing of S203. If it is not determined that the condition is met, the value of the status flag is not changed.

[0097] In S434, CPU 15 determines that the movement of the main subject has changed from a state that does not include reciprocating movement in both directions to a state that does include it. Based on the determination, CPU 15 sets the state flag "flag" to a value of 2 (flag=2), which indicates that reciprocating movement in both directions has been detected. Then, CPU 15 ends the process of S203.

[0098] 4A and 4B, the value of the status flag is set based on an indicator for vertical movement and an indicator for movement in both directions (diagonal directions). This is to control the execution or interruption of subject blur correction according to the vertical reciprocating movement of the main subject, which occurs more frequently than the vertical reciprocating movement of the main subject in the vertical and horizontal directions. For example, it is possible to determine in advance whether the vertical or horizontal reciprocating movement is to be prioritized according to the type of main subject, and to change whether the vertical or horizontal reciprocating movement is to be prioritized according to the type of main subject that is the target of subject blur correction.

[0099] As described above, in this embodiment, the threshold for determining the specific movement of the main subject is changed taking into consideration one or more of the shooting frame rate, the exposure time of one frame, and the brightness of the shooting scene, thereby making it possible to detect the specific movement of the main subject with high accuracy.

[0100] Furthermore, when a shooting mode in which a moving object is not likely to be photographed is set, CPU 15 can set a threshold value for determining a specific movement of the main object smaller than normal. Similarly, CPU 15 can set a threshold value for determining a specific movement of the main object smaller as the image magnification (or focal length) increases. By setting a smaller threshold value, the object blur correction is more likely to be interrupted, and background fluctuations caused by the movement of digital camera 1, such as camera shake, can be made less noticeable. Shooting modes in which a moving object is not likely to be photographed may include, but are not limited to, a macro shooting mode, a flower shooting mode, and a building shooting mode.

[0101] Moreover, the CPU 15 can reduce the threshold value for determining the specific movement of the main subject as the contrast of the background is higher. When the contrast of the background is high, the background fluctuation becomes more noticeable, so the threshold value is reduced to make it easier to interrupt the subject blur correction. Note that a high-pass filter (or a band-pass filter) is applied to the luminance value signal of the captured image other than the area where the main subject exists, and the maximum value or integrated value of the output can be obtained as an index of the contrast of the background. The method of separating the background image from the image may be any known method. The index regarding the contrast of the background area may be an index used in the autofocus detection of the contrast method. The CPU 15 can also apply, for example, FFT to the background area to obtain a spatial frequency spectrum, and determine that the contrast of the background area is high if the ratio of spatial frequency components equal to or higher than a predetermined threshold is equal to or higher than the threshold.

[0102] It should be noted that if the frame rate does not satisfy a predetermined condition, the decision to suspend subject blur correction need not be made. In this embodiment, an index for determining the presence or absence of a specific movement of the main subject is calculated using the absolute difference between the coordinates of the main subject area obtained by main subject detection circuit 26 and the LPF output. Therefore, as described above, if the frame rate decreases, the accuracy of the decision using the index may decrease.

[0103] Figures 11(A)-(D) show examples of the changes over time in the coordinates of the main subject area (subject position) and the signal value of its LPF output when the same scene is shot at frame rates of 15 fps, 13 fps, 10 fps, and 8 fps. At a frame rate of 15 fps, the periodic movement of the main subject is clear, and the difference between the coordinates and the LPF output is also clearly visible. However, when the frame rate is reduced to 13 fps or less, the amplitude of the change in coordinates of the main subject decreases and the distortion of the waveform increases. Furthermore, the difference between the coordinate change and the LPF output also decreases, and at a frame rate of 8 fps there is almost no difference.

[0104] Thus, as the frame rate decreases, it becomes more difficult to detect certain movements that may cause the subject blur correction to degrade the quality of the captured image. Therefore, if the frame rate is equal to or lower than the threshold, the CPU 15 may suspend the subject blur correction without making a determination. The frame rate may not be equal to or lower than the threshold due to an explicit setting of the frame rate. For example, the frame rate may be equal to or lower than the threshold when the exposure time per frame is longer than a predetermined number of seconds due to a user setting or automatic exposure control.

[0105] Similarly, when the photographing sensitivity is equal to or greater than the threshold value, the CPU 15 may unconditionally suspend the subject blur correction instead of adjusting the length of the integration interval.

[0106] Furthermore, when a shooting mode in which a moving object is not likely to be photographed is set, CPU 15 may unconditionally suspend subject blur correction from the viewpoint of preventing erroneous detection, instead of setting a threshold value for determining a specific movement of the main subject smaller than normal.

[0107] Also, if the focal length of the interchangeable lens 31 is shorter than the specified focal length (wide-angle side) or if the focal length of the zoom lens is changed to a wider angle side than the specified focal length, the decision to interrupt subject blur correction is not made in order to prevent erroneous detection. This applies to cases where a specific lens such as a fisheye lens is attached or where the zoom is operated to a wider angle side than the specified focal length. When using the function of the present invention with a lens with such a short focal length, it is assumed that the subject will be photographed close up, such as when photographing a miniature. Therefore, even if a moving object moves significantly so as to go outside the specified range, the movement is not likely to be noticeable because the background will be photographed small.

[0108] As described above, in this embodiment, when it is detected that the movement of the main subject targeted by the subject blur correction is a specific movement, such as a reciprocating movement, for which the subject blur correction may degrade the quality of the captured image, the execution of the subject blur correction is suspended, thereby making it possible to prevent the quality of the captured image from being degraded by the subject blur correction.

[0109] In addition, under conditions where the accuracy of determining a specific movement may decrease, the execution of subject blur correction is more likely to be interrupted than under other conditions, or the execution of subject blur correction is interrupted unconditionally, thereby suppressing deterioration in the quality of the captured image caused by erroneous determination.

[0110] ●(Second embodiment) Next, a second embodiment of the present invention will be described. This embodiment can be implemented by the digital camera 1, and differs from the first embodiment in the operations in S203 and S204 in Fig. 2. Therefore, the following description will focus on the differences from the first embodiment.

[0111] Fig. 5 is a flowchart showing details of the main subject motion determination process (S203 in Fig. 2) in this embodiment. In Fig. 5, steps that perform the same operations as in the first embodiment are given the same reference numbers as in Fig. 4A, and descriptions thereof will be omitted.

[0112] In steps S401 to S407, the presence or absence of reciprocating movement in the vertical direction is determined based on an index relating to movement in the vertical direction, and the value of the variable Ver_repe is set, which is the same as in the first embodiment. Note that in this embodiment, an index relating to movement in both directions is not used, so S403 is not executed. After executing S405 or S407, the CPU 15 executes S508.

[0113] In S508, the CPU 15 determines whether or not both of the horizontal indicators Def10XMAX(n) and Def30XMAX(n) exceed the first threshold value. If it is determined that both of the horizontal indicators exceed the first threshold value, the CPU 15 executes S509, and if not, the CPU 15 executes S510.

[0114] In S509, CPU 15 determines that the movement of the main subject includes a reciprocating movement in the horizontal direction, and sets variable Hor_repe to 1. After that, CPU 15 ends the process of S203.

[0115] In S510, the CPU 15 determines whether or not both of the horizontal indices Def10XMAX(n) and Def30XMAX(n) are equal to or less than the second threshold value. If it is determined that both of the horizontal indices are equal to or less than the second threshold value, the CPU 15 proceeds to S511. If it is determined that both of the horizontal indices are equal to or less than the second threshold value, the CPU 15 ends the process of S203 without changing the value of Hor_repe.

[0116] In S511, CPU 15 determines that the movement of the main subject does not include reciprocating movement in the horizontal direction, and clears the variable Hor_repe (sets it to 0). After that, CPU 15 ends the process of S203.

[0117] In S204, CPU 15 refers to the values ​​of variables Ver_repe and Hor_repe instead of the subject blur correction operation status flag, and executes S205 if the main subject is making a specific movement. Here, if the values ​​of both or either of variables Ver_repe and Hor_repe are 1, CPU 15 determines that the main subject is making a specific movement. On the other hand, if the main subject is not making a specific movement, CPU 15 returns to the process of S202 and continues or resumes execution of subject blur correction.

[0118] In S205 and S206, the CPU 15 suspends subject blur correction only in the vertical direction if only the variable Ver_repe is 1, and suspends subject blur correction only in the horizontal direction if only the variable Hor_repe is 1. Similarly, if both the variables Ver_repe and Hor_repe are 1, the CPU 15 suspends subject blur correction in both the vertical and horizontal directions, as in the first embodiment.

[0119] When subject blur correction is to be suspended only in the vertical or horizontal direction, CPU 15 generates subject blur correction position information having a fixed value for the direction in which subject blur correction is to be suspended and a value obtained in the same manner as in S202 for the direction in which subject blur correction is not to be suspended. The fixed value can be obtained in the same manner as in S205 of the first embodiment.

[0120] In S206, the CPU 15 outputs the cut-out position determined in S205 to the image transformation cut-out circuit 28. The image transformation cut-out circuit 28 cuts out the image in accordance with the position indicated by the subject blur correction position information and stores it in the VRAM 6.

[0121] In this embodiment, whether or not the main subject is making a specific movement is determined independently in the vertical and horizontal directions. Also, subject blur correction is suspended independently for each direction depending on the determination result. In this embodiment, the same effect as in the first embodiment can be achieved. Also, by continuing subject blur correction for directions in which no specific movement is occurring, the effect of subject blur correction can be maintained.

[0122] ●(Third embodiment) Next, a third embodiment of the present invention will be described. In this embodiment, as in the second embodiment, the movement of the main subject is determined separately in the vertical and horizontal directions, and subject blur correction is interrupted separately for each direction. In this embodiment, the movement of the main subject is determined using a number of methods, and a final determination is made based on the results of each method.

[0123] Specifically, in this embodiment, whether or not the main subject is making a specific movement is determined using the following methods 1 to 3. Note that methods 1 to 3 are merely examples, and other methods may be used. Furthermore, it is sufficient that two or more methods are used.

[0124] Method 1: Determine whether the main subject is reciprocating using the frequency characteristics of the detection result (position) of main subject detection circuit 26. The frequency characteristics can be found by a known method such as fast Fourier transform (FFT). FFT may be performed by CPU 15 executing a program, or a dedicated circuit such as a microprocessor having a product-sum calculator may be provided to perform FFT at high speed. In either case, CPU 15 determines whether the main subject is reciprocating based on the frequency characteristics related to the position of the main subject.

[0125] Method 2: Whether or not the main subject is moving back and forth is determined using a moving average of the detection result of main subject detection circuit 26. Specifically, CPU 15 integrates the absolute value of the difference between the detection result (position) of main subject detection circuit 26 and its moving average for different integration periods (for example, the first integration period and the second integration period in the first embodiment). Then, CPU 15 determines whether or not the main subject is moving back and forth based on each integrated value.

[0126] Method 3: CPU 15 determines whether or not the main subject is moving back and forth based on the interval between extreme values ​​(maximum and minimum values) of the detection result (position) of main subject detection circuit 26.

[0127] Each of these methods 1 to 3 has its own advantages and disadvantages. Method 1 can achieve high detection accuracy, but it takes a certain amount of time to achieve sufficient detection accuracy. For example, if an output signal of 128 samples is required, it takes about 4.2 seconds. In addition, since the calculation load is high, it requires a dedicated processing circuit and cannot be used when resource contention occurs. In addition, the discrimination accuracy is not high for main subjects that do not move back and forth many times, such as low-frequency high-speed moving objects.

[0128] Method 2 has the advantage that it can quickly distinguish between reciprocating movements of various moving objects and has a low computational load. However, the accuracy varies depending on the speed of movement, and the discrimination accuracy is likely to decrease for objects with medium frequency and medium speed. In addition, it is easily affected by the amplitude of the reciprocating movement, and dynamic change of the judgment parameters is required to deal with reciprocating movements with small amplitude.

[0129] Like method 2, method 3 can quickly identify reciprocating movement, but it is vulnerable to noise, and small fluctuations in the main subject detection results may be mistaken for extreme values, and the criteria for identifying reciprocating movement may be ambiguous.

[0130] In this way, since each method has different characteristics, in this embodiment, the final discrimination result is obtained by taking into account the discrimination results of each method. For example, by taking a weighted average of the discrimination results, a final discrimination result that takes into account the discrimination results of each method can be obtained.

[0131] The main subject movement determination process (S203 in FIG. 2) in this embodiment will be described in detail with reference to the flowchart in FIG 6. Here, it is assumed that the determination results have been obtained for image data of the same frame by the above-mentioned methods 1 to 3.

[0132] In S601, the CPU 15 determines a weighting coefficient for each method. Here, the weighting coefficient for method 1 is wft, the weighting coefficient for method 2 is wma, and the weighting coefficient for method 3 is wle (Local extremum). For example, the CPU 15 can determine these weights as follows.

[0133] Since the discrimination accuracy of method 1 increases with increasing elapsed time, the CPU 15 also increases the weighting coefficient wft with increasing elapsed time. Specifically, the CPU 15 starts measuring time when the FFT starts. Then, If the elapsed time is less than the first predetermined time (for example, 2 seconds), then wft=0 When the elapsed time is the second predetermined time (> the first predetermined time) (for example, 4 seconds), wft is the first weighting factor (for example, 0.55), In the section of first predetermined second≦elapsed time<second predetermined second, wft is determined by linear interpolation.

[0134] Moreover, the CPU 15 determines the weighting coefficients for methods 2 and 3 based on the weighting coefficient wft for method 1 as follows. wma=(1-wft)×2÷3 wle = (1-wft) ÷ 3

[0135] Here, in the section where the elapsed time is equal to or greater than the second predetermined time, the discrimination accuracy of method 1 is deemed to be the highest, and the weighting coefficients for methods 2 and 3 are determined. Note that the measurement value of the elapsed time is reset when subject blur correction is resumed, or when FFT resources become unavailable while subject blur correction is interrupted.

[0136] Here, a method for calculating the discrimination index in Methods 1 to 3 will be described with reference to Fig. 7. The weighting for each method described above is applied to the discrimination index for each method.

[0137] The CPU 15 calculates the discrimination index P_FFT by method 1 based on the ratio between the integrated value of the frequency components equal to or higher than the discrimination frequency and the integrated value of the frequency components equal to or lower than the discrimination frequency. The discrimination frequency can be determined based on the frequency at which subject blur correction should be interrupted among the frequency components of the reciprocating movement of the main subject.

[0138] For example, when FFT results for each frame rate as shown in FIG. 9A are obtained, the CPU 15 calculates the determination index P_FFT as follows. If subject blur correction should be suspended for movements with a frequency of 2 Hz or more, the discrimination frequency should be set to, for example, around 1.8 Hz. However, since the obtained frequency changes depending on the frame rate (FFT sampling frequency), the discrimination frequency is set for each frame rate.

[0139] 9B shows the integrated values ​​of the intensities of the frequency bands equal to or higher than the discrimination frequency when the frame rates are 30 fps, 15 fps, and 7.5 fps. Note that the DC component is not taken into consideration in the calculation of the ratio described above.

[0140] If the discrimination frequency for 30 fps is 1.76 Hz, the discrimination frequencies at which roughly the same ratios are obtained at 15 fps and 7.5 fps are 1.54 Hz at 15 fps and 1.19 Hz at 7.5 fps. If the motion frequency of the subject to be detected is 2 Hz or higher, and a condition that the discrimination frequency is 1.5 Hz or higher is added, the discrimination frequency for 7.5 fps becomes 1.55 Hz. Since the discrimination frequencies for 30 fps and 15 fps are originally 1.5 Hz or higher, the above values ​​can be used as is. In this way, the parameters (discrimination frequency) for index calculation are changed according to the frame rate.

[0141] Then, the CPU 15 can calculate the discrimination index by referring to the relationship between the value of the discrimination index stored in advance and the above-mentioned ratio, for example, as shown in Fig. 7(A). As shown in Fig. 7(A), in the section where the ratio of the integrated value of the intensity of the frequency band equal to or higher than the discrimination frequency is equal to or lower than Ra1 (for example, 1 / 3), the discrimination index P_FFT is set to 0, and in the section where the ratio is equal to or higher than Ra2 (for example, 2 / 3), the discrimination index P_FFT is set to 1. For the ratio between Ra1 and Ra2, the discrimination index P_FFT is calculated by linear interpolation. The relationship shown in Fig. 7(A) can be stored in the EEPROM 19 as, for example, a function or a table. In the example of Fig. 9(A), the ratio of 69% is larger than Ra2, so the CPU 15 determines the discrimination index P_FFT to be 1.

[0142] The period for performing the FFT is a fixed period (e.g., 6 seconds) from the start of measuring the elapsed time until 1.5 times the second predetermined time has elapsed. After the fixed period has elapsed, the period for applying the FFT is shifted back.

[0143] The CPU 15 calculates the discrimination index P_Ma by method 2 based on a discrimination threshold based on an integrated value of the absolute value of the difference between the moving average of the movement of the main subject at which the above-mentioned subject blur correction should be interrupted. In other words, the calculation is based on how close the value is to a threshold value corresponding to the repetitive movement of the main subject in question.

[0144] CPU 15 integrates the absolute value of the difference between the detection result (position) of main subject detection circuit 26 and its moving average. CPU 15 calculates discrimination index P_Ma according to the magnification of the integrated value for the first integration period (first integrated value) with respect to the discrimination threshold. For example, as shown in Fig. 7(B), CPU 15 sets discrimination index P_Ma to 0 if the integrated value is 0.5 times the discrimination threshold or less, and sets discrimination index P_Ma to 1 if the integrated value is 1.5 times or more. CPU 15 also performs linear interpolation to determine discrimination index P_Ma for a range exceeding 0.5 times and less than 1.5 times.

[0145] CPU 15 similarly obtains discrimination index P_Ma for the integrated value for the second integration period (second integrated value). In this case, the discrimination threshold for the second integration period is used. The discrimination threshold used here is changed according to the frame rate. As described above, a decrease in frame rate reduces the amplitude of the position change of the main subject area in a section where a specific movement such as a reciprocating movement occurs, and the difference between the subject position and its moving average value also tends to become smaller.

[0146] Then, the CPU 15 averages the discrimination index P_Ma calculated for the first integration period and the second integration period to obtain the discrimination index P_Ma by method 2. This integration interval (number of frames) is changed according to the frame rate, as in the first embodiment.

[0147] The CPU 15 determines the discrimination index P_Le according to Method 3 as follows. CPU 15 detects the maximum and minimum values ​​of the detection result (position) of main subject detection circuit 26. CPU 15 then sequentially obtains the intervals between alternatingly adjacent maximum and minimum values, and calculates the average value of the extreme value intervals obtained over a predetermined period according to the most recent frame rate (for example, 2 seconds if the video frame rate is 30 fps).

[0148] This predetermined period can be experimentally determined, taking into consideration the accuracy of detecting the intervals between the maximum and minimum values ​​within that period and calculating the average value. As the frame rate decreases, the accuracy of the intervals between the maximum and minimum values ​​decreases, so in order to maintain accuracy, it is necessary to increase the number of intervals obtained.

[0149] Therefore, the predetermined period for the reference frame rate is used as a reference value, and the predetermined period for the other frame rates is calculated by multiplying the reference value by the ratio to the reference frame rate. However, upper and lower limits are set for the predetermined period, taking into consideration the main frequency of the movement of the subject to be detected and the time required for detection.

[0150] For example, if the predetermined period for a reference frame rate of 30 fps is 2 seconds, then the predetermined period for 48 fps is 2 x 5 / 8 = 1.25 seconds. If 1.25 seconds is set as the lower limit in consideration of the movement of the subject, then the predetermined period for frame rates exceeding 48 fps will be 1.25 seconds. On the other hand, the predetermined period for 15 fps will be 4 seconds. If 4 seconds is set as the upper limit in consideration of the time required for detection, then the predetermined period for frame rates below 15 fps will be 4 seconds.

[0151] Then, CPU 15 calculates discrimination index P_Le according to the magnification of the average value with respect to a predetermined reference time based on the frequency of the movement of the main subject at which subject blur correction should be suspended. For example, as shown in Fig. 7C, if the average value is 0.5 times the reference time or less, CPU 15 sets discrimination index P_Le to 1, and if it is 1.5 times or more, sets discrimination index P_Le to 0. Furthermore, CPU 15 performs linear interpolation to determine discrimination index P_Le for a range exceeding 0.5 times and less than 1.5 times.

[0152] For example, if the frequency of the main subject's movement at which subject blur correction should be suspended is 2 Hz or higher, the reference time can be set to 0.25 seconds. In this case, if the average value of the extreme intervals is 0.125 seconds or less, the discrimination index P_Le is 1, and if it is 0.375 seconds or more, the discrimination index P_Le is 0.

[0153] If the output of main subject detection circuit 26 at time t is d[t], CPU 15 can determine the maximum and minimum values ​​of the output as follows: Here, Th is a predetermined threshold value for determining the extreme value.

[0154] d[t]>=d[t+1]+Th and d[t]>=d[t+2] +2×Th and d[t+1]>d[t+2] and d[t]>=d[t-1]+Th and d[t]>=d[t-2] +2×Th and d[t-1]>d[t-2] The output d[t] that satisfies the following is defined as the maximum value.

[0155] Also, d[t]<=d[t+1]-Th and d[t]<=d[t+2]-2×Th and d[t+1] <d[t+2] and d[t]<=d[t-1]-Th and d[t]<=d[t-2]-2×Th and d[t-1]><[t-2] The output d[t] that satisfies the following is defined as the minimum value.

[0156] For methods 1 to 3, the CPU 15 obtains the discrimination indexes for the horizontal and vertical directions.

[0157] Returning to FIG. 6, in S602, the CPU 15 calculates a horizontal direction judgment value JudX and a vertical direction judgment value JudY based on the weighting coefficient and the discrimination index for each method using the following formulas. Horizontal direction judgment value JudX= wft×P_FFT(x)+wma×P_Ma(x)+wle×P_Lew1(x) Vertical direction judgment value JudY= wft×P_FFT(y)+wma×P_Ma(y)+wle×P_Lew1(y) The subscript (x) indicates the discrimination index in the horizontal direction, and (y) indicates the discrimination index in the vertical direction. In this way, a judgment value is calculated for each direction by adding the discrimination indices of methods 1 to 3 using the weights for each method.

[0158] Next, in S604, the CPU 15 determines whether or not the judgment value JudY exceeds a first threshold value (a threshold value for judging that the main subject is moving back and forth), and if it is determined that it exceeds the first threshold value, executes S605, and if not, executes S606.

[0159] In S605, the CPU 15 determines that the movement of the main subject includes a reciprocating movement in the vertical direction, and sets the variable Ver_repe to 1. After that, the CPU 15 executes S608.

[0160] In S606, the CPU 15 determines whether the judgment value JudY is equal to or less than a second threshold value (a threshold value for determining that there is no reciprocating movement in the subject movement) which is smaller than the first threshold value, and if it is determined that it is equal to or less than the second threshold value, executes S607, and if it is not determined that it is equal to or less than the second threshold value, executes S608.

[0161] In S607, the CPU 15 determines that the movement of the main subject does not include reciprocating movement in the vertical direction, and clears the variable Ver_repe (sets it to 0). After that, the CPU 15 executes S608.

[0162] In S608, the CPU 15 determines whether the judgment value JudX exceeds a first threshold value (a threshold value for judging that the main subject is moving back and forth), and if it is determined that it exceeds the first threshold value, executes S609, and if not, executes S610.

[0163] In S609, the CPU 15 determines that the movement of the main subject includes a reciprocating movement in the horizontal direction, sets the variable Hor_repe to 1, and ends the process of S203.

[0164] In S610, the CPU 15 determines whether the judgment value JudX is equal to or less than a second threshold value (a threshold value for determining that there is no reciprocating movement in the subject movement) which is smaller than the first threshold value, and if it is determined that it is equal to or less than the second threshold value, the CPU 15 terminates the processing of S611, and if it is not determined that it is equal to or less than the second threshold value, the CPU 15 terminates the processing of S203.

[0165] In S611, the CPU 15 determines that the movement of the main subject does not include reciprocating movement in the horizontal direction, clears the variable Hor_repe (sets it to 0), and ends the process of S203.

[0166] Thereafter, the CPU 15 executes S204 to S206 in the same manner as in the second embodiment.

[0167] According to this embodiment, it is possible to obtain the same effect as the second embodiment. In addition, whether or not the main subject is making a specific movement is determined in consideration of the determination results by a plurality of methods. Therefore, it is possible to improve the reliability of the movement determination compared to the second embodiment.

[0168] ●(Fourth embodiment) Next, a fourth embodiment of the present invention will be described. In this embodiment, as in the second and third embodiments, whether or not the main subject is making a specific movement is determined independently in the vertical and horizontal directions. Also, as in the third embodiment, the movement of the main subject is determined using methods 1 to 3.

[0169] The main subject movement determination process (S203 in FIG. 2) in this embodiment will be described in detail with reference to the flowchart in FIG. 8. Here, it is assumed that the determination results have been obtained for image data of the same frame by the above-mentioned methods 1 to 3.

[0170] The operation shown in Fig. 8 is performed in common for the horizontal and vertical directions, so the following description will not distinguish between the directions. In reality, the operation shown in Fig. 8 is performed separately for the horizontal and vertical directions, and a discrimination result is obtained for each direction.

[0171] In S801, CPU 15 acquires a detection result from main subject detection circuit 26 and associates it with the value of a counter for measuring elapsed time. Since this counter is different from a counter that measures the elapsed time of FFT (FFT counter), it will be referred to as a normal counter hereinafter.

[0172] In S802, the CPU 15 determines whether or not resources for executing the FFT are available. If it is determined that the resources are available, the process proceeds to S803. If not, the process proceeds to S804.

[0173] In S803, the CPU 15 obtains the value of the FFT counter. In S804, the CPU 15 resets the value of the FFT counter.

[0174] In S805, the CPU 15 judges whether the value of the FFT counter is equal to or greater than the second predetermined time, and if it is judged to be equal to or greater than the second predetermined time, executes S820, otherwise executes S806. As a result, if the FFT execution time is equal to or greater than the second predetermined time, method 1, which has sufficiently high detection accuracy, is executed preferentially, and if it is less than the second predetermined time, method 2 or 3 is executed preferentially.

[0175] In S806, the CPU 15 obtains the first integrated value and the second integrated value according to the method 2 as described in the third embodiment.

[0176] In S807, the CPU 15 judges whether the value of the normal counter is equal to or greater than a first predetermined time that is shorter than the second predetermined time, and if it is judged to be equal to or greater than the first predetermined time, executes S811, and if it is not, executes S808. As a result, if the elapsed time from the start of processing (number of processed frames) is equal to or greater than the first predetermined time, method 2 is executed with priority, and if it is less than the first predetermined time, method 3 and method 2 are executed with priority.

[0177] In the processes from S808 onwards, CPU 15 determines the movement of the main subject by method 3. In S808, CPU 15 obtains the average value of the intervals between the maximum and minimum values ​​in the output of main subject detection circuit 26 according to method 3 as described in the third embodiment.

[0178] In S809, the CPU 15 determines whether or not the conditions that the average value calculated in S808 is equal to or less than the first interval threshold and both the first and second integrated values ​​calculated in S806 are equal to or greater than the first integrated value threshold are satisfied. If it is determined that the conditions are satisfied, the CPU 15 executes S830, and if not, the CPU 15 executes S810.

[0179] In S830, the CPU 15 determines to suspend the subject blur correction. The CPU 15 sets, for example, the variables Ver_repe and Hor_repe to 1, and ends the process of S203.

[0180] In S810, the CPU 15 determines whether or not the condition is satisfied that the average value calculated in S808 is equal to or greater than a second interval threshold longer than the first interval threshold, and both the first and second integrated values ​​calculated in S806 are equal to or less than a second integrated value threshold smaller than the first integrated value threshold. If it is determined that the condition is satisfied, the CPU 15 executes S831. If it is not determined that the condition is satisfied, the CPU 15 ends the process of S203 without changing the state of the subject blur correction.

[0181] In S831, the CPU 15 determines to resume the subject blur correction. The CPU 15 sets, for example, the variables Ver_repe and Hor_repe to 0, and ends the process of S203.

[0182] In S811 and thereafter, the movement of the main subject is determined by Method 2. In S811, CPU 15 determines whether or not both the first and second integrated values ​​are equal to or greater than the first integrated value threshold, and if it is determined that they are equal to or greater than the first integrated value threshold, executes S830, and if not, executes S812.

[0183] In S812, the CPU 15 determines whether the first and second integrated values ​​are both equal to or less than a second integrated value threshold that is smaller than the first integrated value threshold, and if it is determined that they are equal to or less than the second integrated value, it executes S831. If it is not determined that the condition is satisfied, the CPU 15 ends the process of S203 without changing the state of the subject blur correction.

[0184] In the processing from S820 onwards, CPU 15 discriminates the movement of the main subject by method 1. In S820, CPU 15 obtains the ratio of the integrated value of frequency components equal to or lower than the discrimination frequency to the integrated value of frequency components equal to or higher than the discrimination frequency according to method 1, as described in the third embodiment.

[0185] In S821, the CPU 15 determines whether or not the ratio calculated in S820 is equal to or greater than the first FFT threshold, and if it is determined that it is equal to or greater than the first FFT threshold, executes S830, and if not, executes S822.

[0186] In S822, the CPU 15 judges whether the ratio calculated in S820 is equal to or less than a second FFT threshold smaller than the first FFT threshold, and if it is judged to be equal to or less than the second FFT threshold, executes S831, and if it is not, executes S806. In this way, if a specific movement cannot be identified by the first method, the specific movement is identified by method 2 or 3.

[0187] Thereafter, the CPU 15 executes S204 to S206 in the same manner as in the second embodiment.

[0188] In this embodiment, parameters for judgment are also changed according to shooting conditions such as the frame rate. The second predetermined time period to be compared with the value of the normal counter is changed according to the frame rate. This is because the accuracy of the average value calculated by detecting the interval between the maximum value and the minimum value changes according to the frame rate. As in the third embodiment, the predetermined time period for the reference frame rate is used as a reference value, and the predetermined time periods for other frame rates are calculated by multiplying the reference value by the ratio to the reference frame rate. Note that the second predetermined time period is related to the accuracy of the FFT, and is therefore independent of the frame rate.

[0189] The first and second interval thresholds used in steps S808 and S809 are the same in terms of seconds, but the number of data points is changed according to the frame rate. For convenience, the equation for determining the maximum and minimum output values ​​is a function of time t, but it can also be expressed as a function of the number of data points. In that case, the number of data points is changed according to the frame rate so as to take into account the same number of data points per time.

[0190] The first and second determination thresholds used in S808, S809, S811, and S812 can be determined in advance, for example, experimentally depending on the frame rate, as in the first embodiment.

[0191] This embodiment can also provide the same effects as the second embodiment. In addition, whether or not the main subject is making a specific movement is determined using a plurality of methods. Therefore, it is possible to improve the reliability of the movement determination compared to the second embodiment. Furthermore, by changing the method to be preferentially executed according to the elapsed time, it is possible to execute the movement determination using an appropriate method.

[0192] (Other embodiments) In the second to fourth embodiments, instead of finding the moving average by method 2, a filter having a delay effect may be used to find the integrated value of the absolute value of the difference between the output of main subject detection circuit 26 and the output of the filter.

[0193] In addition, in the above-described embodiment, when it is determined that the user has intentionally changed the main subject (for example, when a panning operation is detected), the CPU 15 may suspend subject blur correction until a new main subject is detected.

[0194] CPU 15 can determine that the user has intentionally changed the main subject, for example, when the amount of change in the position of the main subject area obtained from the output of main subject detection circuit 26 is equal to or greater than a predetermined threshold value. A similar determination can also be made when the difference between the movement of the main subject obtained from the outputs of motion vector detection circuit 27 and blur detection circuit 13 and the movement of digital camera 1 is equal to or greater than a predetermined threshold value.

[0195] In the above embodiment, an example has been described in which the present invention is implemented in a lens-interchangeable digital camera 1. However, the present invention can be implemented in electronic devices in general that use fixed lenses.

[0196] At least a part of the operations described in the above embodiment as being performed by the CPU 15 may be performed by other hardware such as an ASIC.

[0197] In the above-described embodiment, the result of determining whether the main subject is moving in a reciprocating manner is used to control subject blur correction. However, the result of determination may be used for other purposes. Therefore, the present invention may be embodied as an image processing device or image processing method that independently executes the motion determination process in the above-described embodiment.

[0198] In the above embodiment, the image transformation and cropping circuit 28 corrects the subject blur by controlling the position where the image is cropped, but the subject blur may be corrected by moving at least one of the optical system included in the interchangeable lens and the image sensor 3 included in the digital camera 1. Alternatively, the subject blur may be corrected by using a drive device that is integrated with the digital camera 1 or is detachable from the digital camera 1 and that can rotate the digital camera 1 around the pan axis and the tilt axis using at least one motor. In a configuration in which the subject blur correction is performed using a drive device for an interchangeable lens or a pan-tilt, a microprocessor provided in the drive device for the interchangeable lens or the pan-tilt may control the subject blur correction. In other words, it is preferable that the detection of the subject blur is performed by the imaging device, but the electronic device that controls the correction of the detected subject blur includes devices that do not have an imaging function in addition to the imaging device.

[0199] The control of the above-described embodiment can be expressed as follows. CPU 15 (control means) controls a process for suppressing a change in the position of the main subject between multiple images of a video captured using image sensor 3 (imaging means). A first movement is a movement of the main subject repeatedly moving alternately in a first direction and a second direction opposite to the first direction relative to the imaging means. When the main subject is performing the first movement during video capture, the control means controls the process for suppressing the change so that the degree of suppression of the change in the position of the main subject is smaller than when the main subject is performing a second movement different from the first movement relative to the imaging means during video capture.

[0200] The effect of the above-mentioned embodiment can be confirmed by carrying out the following comparison. First, the digital camera 1 is fixed to a tripod or the like and is set in a state of not moving. Next, while the digital camera 1 is not moving, a video (first video) is taken in which the main subject moves in a predetermined period and a predetermined amount of movement (example of the first movement) in a first direction and a second direction opposite to the first direction with respect to the digital camera 1. Next, while the digital camera 1 is not moving, a video (second video) is taken in which the main subject moves in the first direction with respect to the digital camera 1 by a predetermined amount of movement (example of the second movement). When the above-mentioned embodiment is implemented, the change in the position of the main subject between images is larger in the first video than in the second video. Here, it is preferable that the first direction and the second direction are closer to a direction perpendicular to the optical axis of the digital camera 1 than a direction parallel to the optical axis of the digital camera 1.

[0201] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0202] The disclosure of the present embodiment includes the following imaging device, a control method for the imaging device, an image processing device, and a program. (Item 1) An imaging device capable of subject blur correction that suppresses changes in the position of a main subject in an image, a determination means for determining whether or not the movement of the main subject is a predetermined specific movement; a control means for controlling the imaging device to interrupt execution of the subject blur correction when the determination means determines that the movement of the main subject is the specific movement; An imaging device comprising: (Item 2) 2. The imaging device according to item 1, wherein the control means controls the imaging device to resume execution of the subject blur correction when the determination means no longer determines that the movement of the main subject is the specific movement. (Item 3) a detection means for detecting an area of ​​a main subject to be subjected to the subject blur correction from an image; and acquiring a target position of the subject blur correction based on the position of the area detected by the detecting means, the determining means determines whether or not the movement of the main subject is the specific movement based on a difference between the position of the area detected by the detecting means and the target position. 3. The imaging device according to item 1 or 2, (Item 4) 4. The imaging device according to item 3, wherein the acquisition means acquires the target position by setting a position obtained by applying a low-pass filter process to the position of the area detected by the detection means as the position of the main subject. (Item 5) 5. The imaging device according to item 3 or 4, wherein the discrimination means discriminates whether the movement of the main subject is the specific movement or not based on the magnitude of the integrated value of the absolute values ​​of the differences. (Item 6) 6. The imaging device according to any one of items 3 to 5, wherein the discrimination means discriminates whether the movement of the main subject is the specific movement or not based on the magnitudes of multiple integrated values ​​having different integration periods. (Item 7) 7. The imaging device according to any one of items 1 to 6, wherein the determination means determines whether the movement of the main subject is the specific movement for each direction. (Item 8) 7. The imaging device according to any one of items 1 to 6, wherein the determination means determines whether the movement of the main subject is the specific movement by giving priority to a vertical direction over a horizontal direction. (Item 9) 9. The imaging device according to any one of items 1 to 8, wherein the determining means determines whether the movement of the main subject is the specific movement by using a plurality of methods. (Item 10) The imaging device described in item 9, characterized in that the discrimination means discriminates whether the movement of the main subject is the specific movement based on an index obtained by weighting and adding indexes indicating the discrimination results of each of the multiple methods. (Item 11) 11. The imaging device according to item 9 or 10, wherein the plurality of methods include two or more of a first method using an index based on a frequency component related to a position of an area of ​​a main subject that is a target of the subject blur correction in an image, a second method using an index based on a moving average of the position, and a third method using an index based on a maximum value and a minimum value of the position. (Item 12) the plurality of methods includes the first method, The imaging device described in item 11, characterized in that if the discrimination means can determine whether the movement of the main subject is the specific movement using the first method, it does not perform discrimination using another method, and if the discrimination means cannot determine whether the movement of the main subject is the specific movement using the first method, it performs discrimination using another method. (Item 13) 13. The imaging device according to any one of items 1 to 12, wherein the specific movement is a reciprocating movement. (Item 14) 2. The imaging device according to item 1, wherein one or more of the parameters used to determine whether the movement of the main subject is the predetermined specific movement has a variable value. (Item 15) 15. The imaging device according to item 14, wherein the parameters include a parameter having a fixed value. (Item 16) 15. The imaging device according to item 14, wherein the parameter having a variable value is either an integration interval of values ​​relating to the position of the main subject, or a decision threshold applied to an integration value in the integration interval. (Item 17) 17. The imaging device according to item 16, wherein the parameter having a variable value has a value according to any one of a shooting frame rate, an exposure time per frame, and a brightness of a shooting scene. (Item 18) The imaging device according to item 1, characterized in that, when a shooting mode for shooting an image of a stationary subject is set, the control means controls the imaging device so that execution of the subject blur correction is more likely to be interrupted than when the shooting mode is not set. (Item 19) 2. The imaging device according to item 1, wherein the control means controls the imaging device so that the execution of the subject blur correction is more likely to be interrupted the greater the image magnification or focal length during shooting. (Item 20) 2. The imaging device according to item 1, wherein the control means controls the imaging device so that the execution of the subject blur correction is more likely to be interrupted the higher the contrast of the background of the main subject. (Item 21) 21. The imaging device according to item 20, wherein the control means determines that the contrast of the background is high if a proportion of spatial frequency components equal to or higher than a predetermined threshold in the spatial frequency spectrum of the background region of the captured image is equal to or higher than a threshold. (Item 22) 22. The imaging device according to any one of items 18 to 21, wherein the control means controls the imaging device to interrupt execution of the subject blur correction, instead of controlling the imaging device to make the execution of the subject blur correction easier to interrupt. (Item 23) The imaging device according to item 1, wherein the control means controls the imaging device to suspend execution of the subject blur correction when any one of the following conditions is met: a shooting frame rate is equal to or lower than a predetermined value, an exposure time per frame is equal to or longer than a predetermined number of seconds, a luminance of the shooting scene is equal to or lower than a threshold, or shooting sensitivity exceeds a predetermined sensitivity. (Item 24) An acquisition means for acquiring a position of a region of a main subject detected from an image; 24. The imaging device according to any one of items 1 to 23, further comprising a discrimination means for discriminating whether or not the main subject is moving in a manner that includes reciprocating motion, based on an integrated value of an absolute value of a difference between the position and a value obtained by applying a low-pass filter process to the position. (Item 25) a control means for controlling a process for suppressing a change in the position of a main subject between a plurality of images of a moving image acquired by the imaging means; the control means controls the processing so that, when the main subject is performing a first movement relative to the imaging means during video shooting in which the main subject moves alternately and repeatedly in a first direction and a second direction opposite to the first direction, the degree to which changes in the position of the main subject are suppressed is smaller than when the main subject is performing a second movement relative to the imaging means during video shooting that is different from the first movement. (Item 26) 26. The electronic device according to item 25, wherein the processing suppresses a change in the position of the main subject by changing a position at which a partial image including the main subject is cut out from an image acquired using the imaging means. (Item 27) 27. The electronic device described in item 25 or 26, wherein the control means reduces a degree of suppression of the change in the position of the main subject in the processing when the movement of the main subject changes from the second movement to the first movement. (Item 28) 28. The electronic device according to item 27, wherein the control means decreases the degree and then increases the degree when the movement of the main subject changes from the first movement to the second movement. (Item 29) 26. The electronic device according to item 25, wherein the processing is performed when the main subject is performing the second movement, and the processing is interrupted when the movement of the main subject changes from the second movement to the first movement. (Item 30) 1. A control method executed by an imaging device capable of subject blur correction that suppresses a change in a position of a main subject in an image, comprising: determining whether the movement of the main subject is a predetermined specific movement; controlling the imaging device to interrupt execution of the object blur correction when the movement of the main object is determined to be the specific movement; 13. A method for controlling an imaging apparatus comprising: (Item 31) A control method executed by an electronic device that controls a process for suppressing a change in a position of a main subject between a plurality of images of a moving image acquired by an imaging means, comprising: A method for controlling an electronic device, characterized in that, when the main subject is performing a first movement relative to the imaging means during video shooting in which the main subject moves alternately and repeatedly in a first direction and a second direction opposite to the first direction, the processing is controlled so that the degree to which changes in the position of the main subject are suppressed is smaller than when the main subject is performing a second movement relative to the imaging means during video shooting that is different from the first movement. (Item 32) A program for causing a computer included in an imaging device capable of subject blur correction that suppresses changes in the position of a main subject in an image to function as each of the means included in the imaging device described in items 1 to 24.

[0203] The present invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Therefore, the following claims are appended to disclose the scope of the invention. [Explanation of symbols]

[0204] 1...imaging device, 3...imaging element, 5...image processing circuit, 13...shake detection circuit, 14...shake detection sensor, 15...CPU, 18...operation switch, 26...main subject detection circuit, 27...motion vector detection circuit, 28...image transformation and extraction circuit

Claims

1. An imaging device capable of subject blur correction that suppresses changes in the position of a main subject in an image, a determination means for determining whether or not the movement of the main subject is a predetermined specific movement; a control means for controlling the imaging device to interrupt execution of the subject blur correction when the determination means determines that the movement of the main subject is the specific movement; An imaging device comprising:

2. 2. The imaging apparatus according to claim 1, wherein the control means controls the imaging apparatus to resume execution of the subject blur correction when the determination means no longer determines that the movement of the main subject is the specific movement.

3. a detection means for detecting an area of ​​a main subject to be subjected to the subject blur correction from an image; and acquiring a target position of the subject blur correction based on the position of the area detected by the detecting means, the determining means determines whether or not the movement of the main subject is the specific movement based on a difference between the position of the area detected by the detecting means and the target position.

2. The imaging device according to claim 1 .

4. 4. The imaging apparatus according to claim 3, wherein the acquisition means acquires the target position by setting a position obtained by applying a low-pass filter process to the position of the area detected by the detection means as the position of the main subject.

5. 4. The image pickup apparatus according to claim 3, wherein said determining means determines whether or not the movement of said main subject is the specific movement based on the magnitude of an integrated value of the absolute values ​​of said differences.

6. 4. The imaging apparatus according to claim 3, wherein said determining means determines whether or not the movement of said main subject is the specific movement based on the magnitudes of a plurality of integrated values ​​having different integration periods.

7. 2. The image pickup apparatus according to claim 1, wherein said determining means determines whether or not the movement of said main subject is the specific movement for each direction.

8. 2. The image pickup apparatus according to claim 1, wherein the determining means determines whether the movement of the main subject is the specific movement by giving priority to a vertical direction over a horizontal direction.

9. 2. The image pickup apparatus according to claim 1, wherein the determining means determines whether the movement of the main subject is the specific movement by using a plurality of methods.

10. 10. The imaging device according to claim 9, wherein the discrimination means discriminates whether the movement of the main subject is the specific movement based on an index obtained by weighting and adding indexes indicating the discrimination results of each of the plurality of methods.

11. 10. The imaging device according to claim 9, wherein the plurality of methods include two or more of: a first method using an index based on a frequency component related to a position of an area of ​​a main subject that is a target of the subject blur correction in an image; a second method using an index based on a moving average of the position; and a third method using an index based on a maximum value and a minimum value of the position.

12. the plurality of methods includes the first method; The imaging device according to claim 11, characterized in that if the discrimination means is able to determine whether the movement of the main subject is the specific movement using the first method, it does not perform discrimination using another method, and if the discrimination means is unable to determine whether the movement of the main subject is the specific movement using the first method, it performs discrimination using another method.

13. 2. The imaging apparatus according to claim 1, wherein the specific movement is a reciprocating movement.

14. 2. The image capturing apparatus according to claim 1, wherein one or more of the parameters used to determine whether the movement of the main subject is the predetermined specific movement has a variable value.

15. 15. The imaging apparatus according to claim 14, wherein the parameters include a parameter having a fixed value.

16. 15. The image pickup apparatus according to claim 14, wherein the parameter having a variable value is either an integration interval of a value relating to the position of the main subject, or a determination threshold applied to an integrated value in the integration interval.

17. 17. The image pickup apparatus according to claim 16, wherein the parameter having a variable value has a value corresponding to any one of a shooting frame rate, an exposure time per frame, and brightness of a shooting scene.

18. 2. The imaging device according to claim 1, wherein the control means controls the imaging device so that, when a shooting mode for shooting an image of a stationary subject is set, the execution of the subject blur correction is more likely to be interrupted than when the shooting mode is not set.

19. 2. The image pickup apparatus according to claim 1, wherein the control means controls the image pickup apparatus so that the execution of the subject blur correction is more likely to be interrupted the greater the image magnification or focal length during shooting.

20. 2. The image pickup apparatus according to claim 1, wherein the control means controls the image pickup apparatus so that the execution of the subject blur correction is more likely to be interrupted the higher the contrast of the background of the main subject.

21. 21. The imaging device according to claim 20, wherein the control means determines that the contrast of the background is high if a ratio of spatial frequency components equal to or higher than a predetermined threshold in a spatial frequency spectrum of a background region of the captured image is equal to or higher than a threshold.

22. 20. The imaging device according to claim 18, wherein the control means controls the imaging device to interrupt execution of the subject blur correction, instead of controlling the imaging device to make execution of the subject blur correction easier to interrupt.

23. 2. The imaging device according to claim 1, wherein the control means controls the imaging device to suspend execution of the subject blur correction when any one of the following conditions is met: a shooting frame rate is equal to or lower than a predetermined value, an exposure time per frame is equal to or longer than a predetermined number of seconds, a luminance of the shooting scene is equal to or lower than a threshold value, or shooting sensitivity exceeds a predetermined sensitivity.

24. An acquisition means for acquiring a position of a region of a main subject detected from an image; 2. The imaging device according to claim 1, further comprising a determination means for determining whether or not the main subject is moving in a reciprocating manner based on an integrated value of an absolute value of a difference between the position and a value obtained by applying a low-pass filter process to the position.

25. a control means for controlling a process for suppressing a change in the position of a main subject between a plurality of images of a moving image acquired by the imaging means; The electronic device is characterized in that, when the main subject is performing a first movement in which the main subject moves alternately and repeatedly in a first direction and a second direction opposite to the first direction relative to the imaging means during video shooting, the control means controls the processing so that the degree of suppression of change in the position of the main subject is smaller than when the main subject is performing a second movement different from the first movement relative to the imaging means during video shooting.

26. 26. The electronic device according to claim 25, wherein the process suppresses a change in the position of the main subject by changing a position at which a partial image including the main subject is cut out from the image acquired by the imaging means.

27. 27. The electronic device according to claim 25, wherein the control means reduces a degree of suppression of the change in position of the main subject in the processing when the movement of the main subject changes from the second movement to the first movement.

28. 28. The electronic device according to claim 27, wherein said control means decreases said degree, and then increases said degree when the movement of said main subject changes from said first movement to said second movement.

29. 26. The electronic device according to claim 25, wherein the processing is performed while the main subject is performing the second movement, and the processing is interrupted when the movement of the main subject changes from the second movement to the first movement.

30. 1. A control method executed by an imaging device capable of subject blur correction that suppresses a change in a position of a main subject in an image, comprising: determining whether the movement of the main subject is a predetermined specific movement; controlling the imaging device to interrupt execution of the object blur correction when the movement of the main object is determined to be the specific movement; 13. A method for controlling an imaging apparatus comprising:

31. A control method executed by an electronic device that controls a process for suppressing a change in a position of a main subject between a plurality of images of a moving image acquired by an imaging means, comprising: A method for controlling an electronic device, characterized in that, when the main subject is performing a first movement in which the main subject moves alternately and repeatedly in a first direction and a second direction opposite to the first direction relative to the imaging means during video shooting, the processing is controlled so that the degree to which changes in the position of the main subject are suppressed is smaller than when the main subject is performing a second movement different from the first movement relative to the imaging means during video shooting.

32. A program for causing a computer included in an imaging device capable of subject blur correction for suppressing a change in the position of a main subject in an image to function as each of the means included in the imaging device according to any one of claims 1 to 24.

Images