Swing detection device and method, electronic equipment, program, and storage medium

Abstract

To compatibly avoid a malfunction of image blur correction due to failure in detection of a motion vector and have image blur correction.SOLUTION: A swing detection device has: motion detection means which generates first information related to a motion in an image obtained by photography; acquisition means which acquires second information related to a swing of a device having photographed the image; subject detection means which detects a predetermined subject in the image and generates subject detection information related to the detected subject; and determination means which determines whether the subject is in motion based upon the first information, the second information, and the subject detection information, wherein the determination means determines that the subject is in motion when the difference between the first information and the second information is equal to or larger than a predetermined threshold, and determines a parameter to be used to determine whether the subject is in motion based upon the size of the subject in the subject detection information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a shake detection device and method, an electronic device, a program, and a storage medium, and more particularly to a technique for determining the movement of a subject during shake correction. [Background technology]

[0002] When shooting a video using an imaging device such as a video camera, for example, if the imaging device is handheld and not fixed to a tripod, camera shake occurs in the captured video due to shaking of the imaging device. To address this issue, a so-called electronic image stabilization method is known, which involves shooting with a wide angle of view in advance, detecting camera shake from changes in feature points (motion vectors) in the captured video, and correcting the camera shake by controlling the crop position of frame images so as to cancel out the detected camera shake.

[0003] However, when photographing a moving subject such as a person or an animal, the motion vector detected is not just the camera shake of the imaging device, but also the movement of the subject, which makes it impossible to perform appropriate camera shake correction.

[0004] In response to this, Patent Document 1 discloses a technology that uses subject detection information to determine that the reliability of a motion vector is low if the proportion of the subject occupying the screen is equal to or greater than a predetermined size, and does not perform camera shake correction using the motion vector. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-90216 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the conventional technology disclosed in Patent Document 1 does not mention determining whether the subject being photographed is moving. Therefore, even if the detected subject is not moving and there is no erroneous detection of a motion vector, if the subject occupies a certain proportion of the screen, image stabilization based on the motion vector is not performed. In other words, even in scenes where image stabilization based on the motion vector is effective, not using the motion vector may reduce the effectiveness of image stabilization.

[0007] The present invention has been made in consideration of the above problems, and has an object to achieve both prevention of malfunction of image stabilization due to erroneous detection of motion vectors and good image stabilization performance. [Means for solving the problem]

[0008] In order to achieve the above object, the shake detection device of the present invention comprises a motion detection means that generates first information regarding motion in a captured image, an acquisition means that acquires second information regarding the shake of the device that captured the image, a subject detection means that detects a predetermined subject from the image and generates subject detection information regarding the detected subject, and a determination means that determines whether or not the subject is moving based on the first information, the second information, and the subject detection information, wherein the determination means determines that the subject is moving if the difference between the first information and the second information is equal to or greater than a predetermined threshold, and determines a parameter to be used for determining whether or not the subject is moving based on the size of the subject from the subject detection information. [Effects of the Invention]

[0009] According to the present invention, it is possible to avoid malfunctions in image stabilization due to erroneous detection of a motion vector, while also achieving good image stabilization performance. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing the functional configuration of an imaging apparatus according to a first embodiment of the present invention. [Figure 2] 6 is a flowchart of subject movement determination processing in the first and second embodiments. [Figure 3] FIG. 4 is a graph illustrating threshold values ​​used in the subject movement determining unit in the first and second embodiments. [Figure 4] 5 is a flowchart of a shake correction amount calculation process according to the first embodiment. [Figure 5] FIG. 10 is a block diagram showing the functional configuration of an imaging apparatus according to a second embodiment. [Figure 6] FIG. 10 is a block diagram showing the functional configuration of an imaging system according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0012] First Embodiment A first embodiment of the present invention will be described below. In the following embodiments, the present invention will be described as being implemented in an imaging device such as a digital camera. However, the present invention can be applied to any electronic device capable of performing image processing on captured images. Such electronic devices include video cameras, computer devices (personal computers, tablet computers, media players, PDAs, etc.), mobile phones, smartphones, game consoles, robots, drones, drive recorders, etc. These are merely examples, and the present invention can also be applied to other electronic devices.

[0013] FIG. 1 is a block diagram showing the configuration of an image capturing apparatus 100 according to an embodiment of the present invention. The image sensor 102 receives an image of a subject incident through the imaging lens 101 and converts the received image into an electrical signal. Note that while the imaging lens 101 is shown as a single lens in FIG. 1, it is actually made up of a number of lenses including, for example, a fixed lens, a focus lens, a zoom lens, and an anti-vibration lens, as well as an aperture. The imaging lens 101 may be a lens that is integral with the imaging device 100, or may be an interchangeable lens that is detachable from the imaging device 100. The image sensor 102 is made up of, for example, a CMOS image sensor, a CCD image sensor, or the like.

[0014] The image signal processing unit 103 performs predetermined processing on the electrical signal output by the image sensor 102 to generate an image signal, and outputs the image signal to the motion vector detection unit 104 , the subject detection unit 105 , and the image clipping unit 109 .

[0015] The motion vector detection unit 104 detects a motion vector based on the image signal obtained from the image signal processing unit 103. A known method for detecting a motion vector can be used, such as a matching method that compares representative points or pixels of two temporally consecutive images. Note that the present invention is not limited to the matching method, and any known method for detecting a motion vector can be used.

[0016] The subject detection unit 105 detects a subject in an image based on the image signal obtained from the image signal processing unit 103, and outputs the detected subject as subject detection information. The subject detection information includes, for example, the type of subject (person, animal, vehicle, etc.), the part of the subject (face, eyes, body), the size of the subject image, the center position of the subject image, etc. Subject detection methods include, for example, well-known methods such as deep learning technology, in which multiple images including the subject to be recognized are input to a system and a neural network is constructed to learn the subject. Note that the present invention is not limited to deep learning technology, and any well-known subject detection method may be used.

[0017] The angular velocity sensor 106 is a sensor that physically detects the angular velocity applied to the image capture device 100, and outputs the detected angular velocity as an angular velocity signal.

[0018] Based on the motion vector, the subject detection information, and the angular velocity signal, the subject motion determination unit 107 determines whether the captured subject is moving or not, and outputs the result as a subject motion determination result. Details of the subject motion determination process in the subject motion determination unit 107 will be described later.

[0019] The shake correction amount calculation unit 108 calculates a shake correction amount for correcting image shake in the captured image based on the motion vector, the angular velocity signal, and the subject motion determination result, and outputs the amount to the image cropping unit 109. Details of the shake correction amount calculation process by the shake correction amount calculation unit 108 will be described later.

[0020] The image cropping unit 109 outputs an image (hereinafter referred to as a "cropped image") cropped from the image signal obtained from the image signal processing unit 103 at a cropping position changed based on the shake correction amount obtained from the shake correction amount calculation unit 108 and having a predetermined size.

[0021] In this embodiment, the so-called electronic image stabilization method has been described, which includes the image cropping unit 109 and corrects image shake in a captured image by changing the crop position of the image, but the present invention is not limited to this. For example, a so-called optical image stabilization method may be used, which optically corrects image shake by driving a correction member such as the imaging lens 101 or the image sensor 102 based on the shake correction amount obtained from the shake correction amount calculation unit 108, without using the image cropping unit 109. In this case, the image signal output from the image signal processing unit 103 is output to the display control unit 110 and the recording control unit 112 without changing the angle of view. Alternatively, image stabilization may be performed using both the electronic image stabilization method and the optical image stabilization method.

[0022] The display control unit 110 controls the display device 111 to display the cropped image output by the image cropping unit 109. The display device 111 is configured by, for example, an LCD panel. Based on an instruction from an operation unit 114, a recording control unit 112 records the cut-out image output by the image cut-out unit 109 in a recording medium 113 configured by, for example, a semiconductor memory, a hard disk, or the like. The operation unit 114 is configured with, for example, a touch panel, switches, etc., and acquires various operation information including start and stop of video recording by the user of the imaging device 100 .

[0023] Next, the subject movement determination process in subject movement determination section 107 will be described with reference to the flowchart in Fig. 2. Note that the process shown in Fig. 2 is repeatedly executed at an arbitrary cycle.

[0024] In S101, the subject movement determination unit 107 acquires a motion vector Vt, an angular velocity signal θt, and subject detection information at time t. Here, the motion vector Vt and the angular velocity signal θt respectively represent the pixel change amount and angular velocity in an arbitrary period. In S102, in order to match the units of the angular velocity signal θt and the motion vector Vt, unit conversion of the angular velocity signal θt is performed. For example, if the motion vector Vt is the pixel change per unit time (pixel / s), the angular velocity signal θt (rad / s) can be converted into the pixel change amount (pixel / s) using equation (1) based on the focal length information f (mm) of the imaging lens 101 and the cell pitch p (mm / pixel) of the imaging element 102. If the angular velocity signal after unit conversion is Gt, then Gt=(ftanθt) / p …(1) This becomes:

[0025] In S103, the subject movement determination unit 107 calculates the sum of the sizes of the subjects based on the acquired subject detection information. For example, if the subject is a person's face and n people's faces are detected at time t, the size of the i-th face is calculated as x i , the orientation of the i-th face is d iThen, the total size of the subject S can be calculated using equation (2). Here, the face direction d i is a number that increases from 0 when facing forward to the side, and c is an arbitrary constant. TIFF2025155346000002.tif658

[0026] In this embodiment, the total size S of the subjects is calculated as the total size of one or more detected faces, but this is not limited to the subject. For example, the total size S of the subjects may be calculated as the total size of one or more animals or the total size of one or more vehicles. Furthermore, the total size S may be the size of one main subject selected based on predetermined conditions from the detected subjects.

[0027] In S104, the subject movement determination unit 107 calculates a threshold value Th to be used in the determination process in the next S105 based on the total size S of the subjects calculated in S103. In this embodiment, the threshold value Th is calculated so that it decreases as the total size S of the detected subjects increases, for example, and is calculated using equation (3). Here, a and b are arbitrary constants. Furthermore, Smin and Smax are the lower and upper limits of the total size S of the subjects that can be detected by the subject detection unit 105.

[0028] Th=-aS+b (Smin <S<Smax) …(3) In this embodiment, the threshold value Th is calculated using a linear function with the sum S of the subject sizes as a variable, but the calculation formula is not limited to this, and for example, it may be calculated using a higher-order function or an exponential function with the sum S of the subject sizes as a variable.

[0029] Fig. 3 is a graph showing the relationship between the sum S of the subject sizes and the threshold value Th. Fig. 3(a) shows an example of a graph in which the threshold value Th is calculated using the linear function shown in equation (3), Fig. 3(b) shows a quadratic function, and Fig. 3(c) shows an example of a graph in which the threshold value Th is calculated using the exponential function. In all cases, when the sum S of the subject sizes is a first value, the threshold value Th is smaller than when the sum S is a second value smaller than the first value.

[0030] In S105, subject motion determination unit 107 performs subject motion determination based on the motion vector Vt, angular velocity signal Gt, and threshold value Th. In subject motion determination, the absolute value of the difference between the motion vector Vt and angular velocity signal Gt is calculated, and if the absolute value of the difference is equal to or greater than threshold value Th, it is determined that the subject is moving, and if the absolute value of the difference is less than threshold value Th, it is determined that the subject is not moving, and the subject motion determination result is output.

[0031] That is, the subject movement determination result Mt at time t can be expressed by equation (4): where 1 indicates that the subject is moving, and 0 indicates that the subject is not moving.

[0032] Mt=1 (|Vt-Gt|≧Th) Mt=0 (|Vt-Gt| <Th) …(4)

[0033] When the process of S105 ends, one iteration of the subject movement determination process ends.

[0034] Next, the shake correction amount calculation process in the shake correction amount calculation unit 108 will be described with reference to the flowchart in Fig. 4. Note that the process shown in Fig. 4 is repeatedly executed at an arbitrary cycle.

[0035] In S201, the shake correction amount calculation unit 108 acquires the motion vector Vt, angular velocity signal θt, and subject motion determination result Mt at time t. In S202, in order to match the units of the angular velocity signal θt and the motion vector Vt, unit conversion of the angular velocity signal θt is performed. Note that the unit conversion process performed in S202 is the same as the unit conversion process performed by the subject motion determination unit 107 in S102, and therefore a description thereof will be omitted here. Note that in S201, instead of the angular velocity signal θt, the angular velocity signal Gt after unit conversion by the subject motion determination unit 107 may be acquired. In this case, the process of S202 is unnecessary.

[0036] In S203, the shake correction amount calculation unit 108 refers to the subject movement determination result Mt, and if the subject is moving, the process proceeds to S204, and if the subject is not moving, the process proceeds to S205.

[0037] In S204 and S205, the shake correction amount calculation unit 108 calculates weights for calculating the shake correction amount. In S204, the weight Wv of the motion vector Vt relative to the weight Wg of the angular velocity signal Gt is set to a ratio of 1:0. Meanwhile, in S205, as weights for calculating the shake correction amount, the weight Wv of the motion vector Vt relative to the weight Wg of the angular velocity signal Gt is set to a ratio of m:1-m, where m is any number between 0 and 1, excluding 0 and 1.

[0038] In S206, the shake correction amount calculation unit 108 calculates the shake correction amount based on the weights Wg and Wv determined in S204 or S205. The shake correction amount Ht at time t can be expressed by equation (5), where C is an arbitrary constant.

[0039] Ht = C(Wg × Gt + Wv × Vt) ​​…(5)

[0040] When the process of S204 is executed, Ht=CGt, and the shake correction amount is calculated based only on the angular velocity signal. This means that when the subject is moving, the shake correction amount is not calculated using the motion vector Vt.

[0041] The shake correction amount Ht calculated in S206 is output to the image cropping unit 198.

[0042] In this embodiment, the shake compensation amount is calculated based on a signal obtained by combining an angular velocity signal and a motion vector at a predetermined ratio. However, the method for calculating the shake compensation amount is not limited to this. For example, if the subject is not moving, the shake compensation amount may be calculated by combining the high-frequency component of the angular velocity signal with the low-frequency component of the motion vector. If the subject is moving, the shake compensation amount may be calculated based only on the high-frequency component of the angular velocity signal without using the low-frequency component of the motion vector. Alternatively, if the subject is not moving, the low-frequency component of the angular velocity signal may be corrected based on the motion vector, and the shake compensation amount may be calculated based on the corrected angular velocity signal. If the subject is moving, the shake compensation amount may be calculated based on the angular velocity signal whose low-frequency component has not been corrected by the motion vector. Furthermore, in S204, the weight Wv of the motion vector Vt relative to the weight Wg of the angular velocity signal Gt may be set to a ratio of n:1-n (n is any number between 0 and 1 greater than m, excluding 0 and 1).

[0043] As described above, according to the first embodiment, a threshold value for determining whether a subject is moving is determined based on an angular velocity signal, a motion vector, and the size of the subject, and the determined threshold value is used to determine whether the subject is moving during shooting. If the subject is moving during shooting, image stabilization is performed by using a lower ratio of motion vectors than if the subject is not moving. This makes it possible to avoid malfunctions of image stabilization due to erroneous detection of motion vectors while also achieving good image stabilization performance.

[0044] Although the method of changing the threshold value has been described as a method of making it easier to determine that a subject is moving as the subject's size increases, a method of changing a parameter other than the threshold value depending on the size of the subject may also be used. For example, without changing the threshold value Th, the absolute value of the difference between the motion vector Vt and the angular velocity signal Gt may be multiplied by a coefficient α, and the value of the coefficient α may be compared with the threshold value Th, and the larger the size of the subject, the larger the value of the coefficient α may be. Alternatively, without changing the threshold value Th, the absolute value of the difference between the motion vector Vt multiplied by a coefficient β and the angular velocity signal Gt may be compared with the threshold value Th, and the larger the size of the subject, the larger the value of the coefficient β may be. In this way, the parameters used to determine whether a subject is moving can be determined so that the subject is more likely to be determined to be moving when the sum S of the subject's sizes is a first value than when it is a second value smaller than the first value.

[0045] <Second embodiment> Next, a second embodiment of the present invention will be described. Fig. 5 is a block diagram showing the functional configuration of the image capture device 200 according to the second embodiment. In the configuration shown in Fig. 5, the same components as those of the image capture device 100 described in the first embodiment with reference to Fig. 1 are denoted by the same reference numerals, and the description thereof will be omitted.

[0046] Compared to the imaging device 100, the imaging device 200 according to the second embodiment does not include the shake correction amount calculation unit 108 and the image cropping unit 109. A display control unit 210 controls the display device 111 to display the image signal obtained from the image signal processing unit 103.

[0047] Furthermore, based on instructions from the operation unit 114, the recording control unit 212 temporally associates the image signal obtained from the image signal processing unit 103, the motion vector obtained from the motion vector detection unit 104, the angular velocity signal obtained from the angular velocity sensor 106, and the subject motion determination result obtained from the subject motion determination unit 107, and records them on the recording medium 113. For example, the recording control unit 212 saves the motion vector, angular velocity signal, and subject motion determination result, which are synchronized for each frame image of the image signal, on the recording medium as a moving image file to which they are added as metadata.

[0048] Using the video and metadata recorded on the recording medium in this way, an external system (not shown) having a shake correction amount calculation unit and an image cropping unit can perform image stabilization. For example, using video editing software on a computer, the shake correction amount is calculated based on the motion vector and angular velocity signal recorded as metadata, and an image of a predetermined size is cropped at a crop position changed based on the calculated shake correction amount. This makes it possible to perform image stabilization on video recorded after shooting.

[0049] As the image blur correction processing method, the method in the first embodiment described above with reference to FIGS. 2 to 4 can be used, and therefore a description thereof will be omitted here. As described above, according to the second embodiment, even if the imaging device does not have an image stabilization mechanism, by recording the motion vector, angular velocity signal, and subject motion determination result in synchronization with the video, it becomes possible to perform image stabilization later using an external device. In this case, by not using the motion vector for image stabilization if the subject is moving, it becomes possible to achieve both good image stabilization performance and avoidance of image stabilization malfunctions.

[0050] <Third embodiment> Next, a third embodiment of the present invention will be described. In the second embodiment described above, a case has been described in which a motion vector is calculated within the imaging device 200, and the motion vector, angular velocity signal, and subject motion determination result are recorded in synchronization with the video. In contrast, in the third embodiment, a case will be described in which a motion vector is calculated by an information processing device such as a personal computer, and subject motion determination is performed.

[0051] Fig. 6 shows an imaging system according to the third embodiment, with Fig. 6(a) being a block diagram showing the functional configuration of an imaging device 300 according to the third embodiment, Fig. 6(b) being a block diagram showing the functional configuration of an information processing device 600, and Fig. 6(c) being a block diagram showing the functional configuration of an image processing unit 601. In the configuration shown in Fig. 6, components similar to those of the imaging devices 100 and 200 described in the first and second embodiments with reference to Figs. 1 and 5 are designated by the same reference numerals, and descriptions thereof will be omitted.

[0052] Compared to the imaging device 200, the imaging device 300 shown in Fig. 6(a) has a configuration that does not include the motion vector detection unit 104 and the subject motion determination unit 107. Although the subject detection unit 105 is not shown in Fig. 6(a), it does not matter whether it is included or not. Also, the imaging device 300 has a communication unit 315 for communicating with external devices including the information processing device 600, and performs communication via wired or wireless communication.

[0053] Then, based on an instruction from the operation unit 114, the recording control unit 312 temporally associates the image signal obtained from the image signal processing unit 103 with the angular velocity signal obtained from the angular velocity sensor 106 and records them on the recording medium 113. For example, the recording control unit 312 adds, as metadata, the angular velocity signal synchronized with each frame image of the image signal, and stores the result as a moving image file on the recording medium. Alternatively, the communication unit 315 transmits, for example, a moving image file to which the angular velocity signal synchronized with each frame image of the image signal and added as metadata is added, to the information processing device 600.

[0054] 6(b), the information processing device 600 includes an image processing unit 601, a ROM 607, a RAM 608, a CPU 609, a disk device 610, a bus 611, a communication unit 612, an I / F 613, an external storage device 614, a display control unit 615, and a display device 616.

[0055] The external storage device 614 is for driving an external storage medium such as the recording medium 113, and reads various data stored in the storage medium from the storage medium. The communication unit 612 can communicate with external devices including the imaging device 300 by wired or wireless communication, directly or via a network, to acquire various data.

[0056] The ROM 607 stores various application programs, and the RAM 608 provides a storage area necessary for the programs to run. The CPU 609 performs processing in accordance with the programs stored in the ROM 607.

[0057] A bus 611 connects the above components and enables data to be exchanged between the components. An I / F 613 connects to an operation unit 620 such as a mouse, keyboard, or touch panel.

[0058] As shown in FIG. 6( c ), the image processing unit 601 includes a motion vector detection unit 602 , a subject detection unit 603 , a subject movement determination unit 604 , a shake correction amount calculation unit 605 , and an image clipping unit 606 .

[0059] In the information processing device 600 having the above configuration, a moving image file is acquired from the imaging device 300 via the communication unit 612 or the external storage device 614, and image signals and angular velocity signals synchronized for each frame image are input to the image processing unit 601. The motion vector detection unit 602, the subject detection unit 603, the subject movement determination unit 604, the shake correction amount calculation unit 605, and the image cropping unit 606 use the input image signals and angular velocity signals to perform processing similar to that performed by the motion vector detection unit 104, the subject detection unit 105, the subject movement determination unit 107, the shake correction amount calculation unit 108, and the image cropping unit 109 described in the first embodiment, respectively, and output cropped images.

[0060] The display control unit 615 controls the display device 616 to display the cut-out image output from the image processing unit 601 .

[0061] With the above configuration, the information processing device 600 can also determine a threshold value to be used for subject movement determination based on the total size of the subject, and control whether or not to use a motion vector for calculating a shake correction value depending on the result of subject movement determination using the determined threshold value.

[0062] As described above, according to the third embodiment, it is possible to avoid malfunctions in image stabilization due to erroneous detection of motion vectors, while also achieving good image stabilization performance.

[0063] In the configuration shown in FIG. 6(b), the image processing unit 601 is described as an independent component, but the CPU 609 may load a program stored in the ROM 607 into the RAM 608 and execute it to perform the processing performed by the image processing unit 601.

[0064] <Other embodiments> The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0065] <Summary> The disclosure of this embodiment includes the following configuration. (Item 1) a motion detection means for generating first information relating to motion in a captured image; an acquisition means for acquiring second information relating to the shake of a device that captured the image; a subject detection means for detecting a predetermined subject from the image and generating subject detection information relating to the detected subject; a determination means for determining whether the subject is moving or not based on the first information, the second information, and the subject detection information, The determination means determining that the subject is moving when a difference between the first information and the second information is equal to or greater than a predetermined threshold; Among the subject detection information, a parameter used to determine whether the subject is moving is determined based on the size of the subject. A vibration detection device characterized by: (Item 2) 2. The shake detection device according to item 1, wherein the determination means determines the threshold value based on the size of the subject. (Item 3) 3. The shake detection device according to item 2, wherein the threshold value is set smaller when the size of the subject is a first size than when the size is a second size smaller than the first size. (Item 4) The shake detection device according to any one of items 1 to 3, further comprising a calculation means for calculating a shake correction amount by lowering the ratio of using the first information when it is determined by the determination means that the subject is moving compared to when it is not determined that the subject is moving. (Item 5) 5. The shake detection device according to item 4, wherein the calculation means calculates the shake correction amount based on the second information without using the first information when the determination means determines that the subject is moving. (Item 6) 6. The shake detection device according to item 5, wherein the calculation means calculates a shake correction amount based on the first information and the second information when the determination means determines that the subject is not moving. (Item 7) the first information is a motion vector, the second information is an angular velocity signal, The shake detection device described in any one of items 1 to 3 is characterized in that it further comprises a calculation means that, when the determination means determines that the subject is not moving, calculates a shake correction amount by combining the low-frequency components of the first information and the high-frequency components of the second information, and, when it determines that the subject is moving, calculates the shake correction amount based on the high-frequency components of the second information. (Item 8) the first information is a motion vector, the second information is an angular velocity signal, The shake detection device described in any one of items 1 to 3 further comprises a calculation means for correcting low-frequency components of the second information based on the first information when the determination means determines that the subject is not moving, and calculating a shake correction amount based on the corrected second information, and for calculating a shake correction amount based on the second information when the determination means determines that the subject is moving. (Item 9) 9. The shake detection device according to any one of items 4 to 8, further comprising a cutout means for cutting out an image of a predetermined size from the image while changing the cutout position based on the shake correction amount. (Item 10) 10. The shake detection device according to any one of items 4 to 9, further comprising a driving means for driving the shake correction member based on the shake correction amount. (Item 11) 11. The shake detection device according to any one of items 1 to 10, further comprising a recording means for adding the first information, the second information, and the determination result by the determination means to the image and recording the same. (Item 12) 10. The shake detection device according to any one of items 1 to 9, further comprising a readout means for reading out the image and the second information from a recording medium. (Item 13) 10. The shake detection device according to any one of items 1 to 9, further comprising a communication means for acquiring the image and the second information from an external device via communication. (Item 14) The determining means determines a coefficient to be multiplied by the difference between the first information and the second information based on the size of the subject. 2. The shake detection device according to item 1, (Item 15) The determining means determines a coefficient to be multiplied by the first information based on the size of the subject. 2. The shake detection device according to item 1, (Item 16) A shake detection device according to any one of items 1 to 15, An imaging means; a vibration detection means for detecting vibration of the device; a vibration correction member for correcting vibration; An electronic device comprising: (Item 17) a motion detection step of generating first information relating to motion in the captured image; an acquisition step of acquiring second information related to shake of a device that captured the image; a subject detection step of detecting a predetermined subject from the image and generating subject detection information related to the detected subject; a determining step of determining whether the subject is moving based on the first information, the second information, and the subject detection information, In the determination step, determining that the subject is moving when a difference between the first information and the second information is equal to or greater than a predetermined threshold; A vibration detection method comprising determining a parameter used to determine whether or not the subject is moving based on the size of the subject from among the subject detection information. (Item 18) A program for causing a computer to function as each of the means of the shake detection device according to any one of items 1 to 15. (Item 19) Item 19. A computer-readable storage medium storing the program described in item 18. [Explanation of symbols]

[0066] 100, 200, 300: imaging device, 101: imaging lens, 102: imaging element, 103: image signal processing unit, 104, 602: motion vector detection unit, 105, 603: subject detection unit, 106: angular velocity sensor, 107, 604: subject movement determination unit, 108, 605: shake correction amount calculation unit, 109, 606: image cropping unit, 110, 210, 615: display control unit, 111, 616: display device, 112, 212, 312: recording control unit, 113: recording medium, 114, 620: operation unit, 600: information processing device, 601: image processing unit, 607: ROM, 608: RAM, 609: CPU, 610: disk, 611: bus, 612: communication unit, 613; I / F, 614: external storage device

Claims

1. a motion detection means for generating first information relating to motion in a captured image; an acquisition means for acquiring second information relating to a shake of a device that captured the image; a subject detection means for detecting a predetermined subject from the image and generating subject detection information relating to the detected subject; a determination means for determining whether the subject is moving or not based on the first information, the second information, and the subject detection information, The determination means determining that the subject is moving when a difference between the first information and the second information is equal to or greater than a predetermined threshold; Among the subject detection information, a parameter used to determine whether the subject is moving is determined based on the size of the subject. A vibration detection device characterized by:

2. The determining means determines the threshold value based on the size of the subject.

2. The vibration detection device according to claim 1.

3. 3. The shake detection device according to claim 2, wherein the threshold value is set smaller when the size of the subject is a first size than when the size is a second size smaller than the first size.

4. 2. The shake detection device according to claim 1, further comprising a calculation unit that, when it is determined by the determination unit that the subject is moving, calculates a shake correction amount by lowering the ratio of use of the first information compared to when it is not determined that the subject is moving.

5. 5. The shake detection device according to claim 4, wherein when the determination means determines that the subject is moving, the calculation means calculates the shake correction amount based on the second information without using the first information.

6. 6. The shake detection device according to claim 5, wherein the calculation means calculates a shake correction amount based on the first information and the second information when the determination means determines that the subject is not moving.

7. the first information is a motion vector, the second information is an angular velocity signal, 2. The shake detection device according to claim 1, further comprising: a calculation means for calculating a shake correction amount by combining a low-frequency component of the first information and a high-frequency component of the second information when the determination means determines that the subject is not moving; and for calculating a shake correction amount based on the high-frequency component of the second information when the determination means determines that the subject is moving.

8. the first information is a motion vector, the second information is an angular velocity signal, 2. The shake detection device according to claim 1, further comprising: a calculation unit that, when it is determined by the determination unit that the subject is not moving, corrects low-frequency components of the second information based on the first information, calculates a shake correction amount based on the corrected second information, and, when it is determined that the subject is moving, calculates the shake correction amount based on the second information.

9. 5. The shake detection device according to claim 4, further comprising: cutting means for cutting out an image of a predetermined size from the image while changing a cutting position based on the shake correction amount.

10. 5. The vibration detection device according to claim 4, further comprising a driving means for driving the vibration correction member based on the vibration correction amount.

11. 2. The shake detection device according to claim 1, further comprising a recording means for recording the first information, the second information, and the determination result by the determining means by adding them to the image.

12. 2. The shake detection device according to claim 1, further comprising a readout means for reading out the image and the second information from a recording medium.

13. 2. The shake detection device according to claim 1, further comprising a communication unit for acquiring the image and the second information from an external device through communication.

14. The determining means determines a coefficient to be multiplied by the difference between the first information and the second information based on the size of the subject.

2. The vibration detection device according to claim 1.

15. The determining means determines a coefficient to be multiplied by the first information based on the size of the subject.

2. The vibration detection device according to claim 1.

16. A shake detection device according to any one of claims 1 to 15; An imaging means; a vibration detection means for detecting vibration of the device; a vibration correction member for correcting vibration; An electronic device comprising:

17. a motion detection step of generating first information relating to motion in the captured image; acquiring second information relating to shake of a device that captured the image; a subject detection step of detecting a predetermined subject from the image and generating subject detection information related to the detected subject; a determining step of determining whether the subject is moving based on the first information, the second information, and the subject detection information, In the determination step, determining that the subject is moving when a difference between the first information and the second information is equal to or greater than a predetermined threshold; A vibration detection method comprising determining a parameter used to determine whether or not the subject is moving based on the size of the subject from among the subject detection information.

18. A program for causing a computer to function as each of the means of the shake detection device according to any one of claims 1 to 15.

19. A computer-readable storage medium storing the program according to claim 18.

Images