Motion vector calculation device, imaging device, and motion vector calculation method

Abstract

To provide a motion vector calculation device capable of improving the calculation cycle of a motion vector.SOLUTION: An optical device acquires event data on the basis of an output of an event sensor 180 that detects a luminance change in a subject's image and maps event data acquired over a mapping time to generate frames. The optical device controls the mapping to the event data to overlap a portion of a plurality of frames and calculates motion vectors on the basis of the plurality of frames having differences in the mapping time at the mapping start time.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a motion vector calculation device, an imaging device, and a motion vector calculation method. [Background technology]

[0002] An event-based vision sensor (hereinafter referred to as "event sensor") has been proposed, which has pixels that detect changes in the brightness of incident subject light and output an event signal. Optical devices such as imaging devices can acquire event data, which is data related to the event signal. Compared to existing CMOS (Complementary Metal Oxide Semiconductor) sensors, event sensors are characterized by high-speed operation, a wide dynamic range, and low power consumption. Patent Document 1 discloses a method of generating frames by mapping event data generated over a predetermined period of time and calculating motion vectors by comparing the generated frames. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2020-522067 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the technology disclosed in Patent Document 1, the period for calculating motion vectors is limited by the predetermined time for mapping event data (hereinafter referred to as "mapping time"). The mapping time can be set to any value, but since the value at which a motion vector can be calculated with high accuracy varies depending on the shape of the subject and the magnitude of the motion vector, it is not desirable to change the mapping time to match a desired calculation period. An object of the present invention is to provide a motion vector calculation device that enables an improvement in the calculation period for motion vectors. [Means for solving the problem]

[0005] A motion vector calculation device according to one embodiment of the present invention , Shine an acquisition means for acquiring data including pixel information in which a change in pixel temperature has occurred; and a generation means for executing a predetermined process on the data acquired at a first time to generate a frame; The first time for a first frame and the first time for a second frame generated by the generating means are Some overlap To do so a control means for performing the control; and a calculation means for calculating a motion vector based on a plurality of frames whose start times for the predetermined processing on the data have the first time difference. [Effects of the Invention]

[0006] According to the present invention, it is possible to improve the calculation cycle of motion vectors. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an optical device. [Figure 2] 1A and 1B are diagrams illustrating an example of the configuration of an image stabilization mechanism included in an optical device. [Figure 3] 10A and 10B are diagrams illustrating an example of control of focal length according to a difficult scene. [Figure 4] FIG. 1 is a diagram illustrating a conventional method for calculating a motion vector. [Figure 5] 4A to 4C are diagrams illustrating a method for calculating a motion vector by the optical device of the first embodiment. [Figure 6] 10 is a flowchart illustrating an operation process of the optical device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Example 1 FIG. 1 is a diagram showing an example of the configuration of an optical device according to this embodiment. 1, an interchangeable lens camera (image capture device) will be described as an example of an optical device. Of course, the present invention can also be applied to a camera with an integrated lens. The present invention can also be applied to a motion vector calculation device that calculates a motion vector based on information obtained from an external source.

[0009] 1 includes a main body 100 of the imaging device, and a lens unit 190 that guides incident light to an imaging element 105 in the main body 100. The lens unit 190 is detachable from the main body 100.

[0010] First, the configuration of the main body 100 will be described. The shutter 103 adjusts the amount of light. The shutter control unit 104 controls the shutter 103 in cooperation with the lens control unit 194 in the lens unit 190 based on exposure information from the image processing unit 106. The image sensor 105 is an imaging means that photoelectrically converts subject light. Specifically, an optical image of a subject (not shown) is formed on the image sensor 105 via the lens 195, aperture 193, lens side mount unit 192, main body side mount unit 102, and shutter 103, and the optical image is converted into an electrical signal.

[0011] The image processing unit 106 performs predetermined calculations on the video signal output from the image sensor 105, and performs image processing such as pixel interpolation, color conversion, and white balance based on the calculation results. The display unit 110 displays the image processing results by the image processing unit 106. The image processing unit 106 also has an image compression function such as JPEG.

[0012] The recording circuit 107 is a circuit that reads and writes from and to a removable recording medium such as a semiconductor memory for recording or reading image data. The communication unit 111 is connected wirelessly or via a wired cable and transmits and receives video signals and audio signals. The communication unit 111 can also be connected to a wireless LAN (Local Area Network) or the Internet. The communication unit 111 can transmit through images obtained by imaging or images recorded in the recording circuit 107. The communication unit 111 can also receive image data and various other information from external devices.

[0013] The operation unit 114 receives user operations and inputs various operational instructions corresponding to the user operations to the system control unit 150. The operation unit 114 includes one or a combination of a switch, a dial, a touch panel, a pointing device using line-of-sight detection, a voice recognition device, and the like.

[0014] A system timer 112 measures the time used for various controls and the time of a built-in clock. Constants and variables for the operation of the system control unit 150, programs read from the memory 140, and the like are loaded into the system memory 113. The system memory 113 may be, for example, a RAM (Random Access Memory). The system memory 113 also stores output values ​​of each axis of the three-axis acceleration sensor 130. A power switch 115 can switch the power of the imaging device on and off.

[0015] The shutter button 116 is an operating means for issuing a shooting instruction. The first shutter switch 117 is turned on by half-pressing the shutter button 116 to issue a shooting preparation instruction, and generates a first shutter switch signal (SW1). SW1 starts operations such as autofocus processing, auto-exposure processing, auto-white balance processing, and flash pre-flash processing. The second shutter switch 118 is turned on by fully pressing the shutter button 116 to issue a shooting instruction, and generates a second shutter switch signal (SW2). The system control unit 150 uses SW2 to start a series of shooting processing operations, from reading out signals from the image sensor 105 to writing image data to the recording circuit 107.

[0016] Furthermore, the triaxial gyro sensor 120 detects the angular velocity of the imaging device in three axes. The triaxial acceleration sensor 130 detects the acceleration of the imaging device in three axes. The memory 140 stores constants, programs, etc. for the operation of the system control unit 150. The memory 140 also includes electrically erasable and storable nonvolatile memory, and uses ROM (Read Only Memory).

[0017] The system control unit 150 has at least one processor and controls the operation of the entire image capturing apparatus. The system control unit 150 also controls a lens control unit 194 of the lens unit 190 via connectors 101 and 191. The power supply control unit 160 controls a battery detection circuit, a protection circuit, a DC / DC converter, an LDO (Low Drop Out), The power supply control unit 160 controls the power supply unit 161 based on instructions from the system control unit 150, and supplies the desired power supply voltage to each unit of the imaging device for the desired period of time. The power supply control unit 160 detects whether a battery is installed, the battery type, and the remaining charge, and if it detects an overcurrent, it cuts off the power supply to protect the load circuit connected to the power supply circuit. The power supply unit 161 also includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, etc.

[0018] The body-side mount unit 102 is an interface for connecting the body unit 100 to the lens unit 190. The connector 101 is a connector that electrically connects the body unit 100 to the lens unit 190. The vibration-proof unit 170 is a correction unit that corrects (anti-shakes) blurring that occurs in a captured image. The vibration-proof unit 170 changes the position of the image sensor 105 under the control of the system control unit 150, thereby correcting image blur. In this example, the system control unit 150 functions as a correction control unit that quickly calculates a motion vector based on event data and drives the vibration-proof unit 170 using the calculated motion vector as the amount of movement of the imaging device. Note that, for example, a lens for correcting image blur may be provided on the lens unit 190 side, and the imaging device may perform image blur correction by driving this lens.

[0019] The event sensor 180 is an event-based vision sensor having pixels that detect a change in brightness of a subject image formed through the optical system 181 of the event sensor 180 and generate and output a signal (event signal). The event sensor 180 outputs the event signal to the system control unit 150 via serial communication. As a result, the system control unit 150 acquires data related to the event signal (event data). The event data is data that includes pixel information in which a brightness change has occurred. The event data includes, for example, a timestamp of the brightness change, coordinate information of the pixel in which the brightness change was detected, and information on the type of brightness change.

[0020] The event sensor 180 outputs pixel information where a luminance change has occurred, reducing the redundancy of the output information compared to conventional CMOS sensors. Therefore, the event sensor 180 is characterized by high-speed operation, a wide dynamic range, and low power consumption. Furthermore, conventional CMOS sensors output information periodically, so the frame generation cycle required for calculating motion vectors depends on the frame rate. On the other hand, the event sensor outputs information including the timing of luminance changes at high speed, pixel by pixel. Therefore, the system control unit 150 can generate frames at high speed by mapping event data generated over a predetermined period of time. Mapping is an example of a predetermined process for event data, and is a process of grouping information included in the event data into a single block over a predetermined period of time (mapping time). Furthermore, the optical system 181 of the event sensor 180 has an actuator that changes the focal length. The focal length is controlled by the system control unit 150.

[0021] Next, the configuration of the lens unit 190 will be described. The lens unit 190 is an interchangeable lens type lens unit, and guides subject light from a lens 195 through an aperture 193, a lens-side mount unit 192, a body-side mount unit 102, and a shutter 103, and forms an image on an image sensor 105. The connector 191 is a connector that electrically connects the lens unit 190 to the body 100. The lens-side mount unit 192 is an interface for connecting the lens unit 190 to the body 100. The aperture 193 adjusts the amount of light entering from the lens 195.

[0022] The lens control unit 194 controls the entire lens unit 190. The lens control unit 194 also functions as a memory for storing operational constants, variables, programs, etc., a non-volatile memory for storing identification information such as a number unique to the lens unit 190, management information, functional information such as the maximum aperture value, minimum aperture value, and focal length, and current and past setting values. The lens control unit 194 controls the focusing of the lens 195 in accordance with the focus state of the image measured by the image processing unit 106, and is capable of performing an AF operation by changing the imaging position of the subject image incident on the image sensor 105. The lens control unit 194 also has the function of controlling the aperture 193 and the focal length of the lens 195.

[0023] FIG. 2 is a diagram showing an example of the configuration of the vibration isolation mechanism of the optical device of this embodiment. The system control unit 150 calculates a motion vector based on the output of the event sensor 180, and corrects image blur by driving the image stabilization unit 170 based on the calculated motion vector. As shown in FIG. 2, the system control unit 150 has an image stabilization control unit 300, an optical system control unit 301, a poor scene detection unit 302, and a motion vector calculation unit 303.

[0024] The motion vector calculation unit 303 calculates a motion vector based on the event data acquired from the event sensor 180. The stabilization control unit 300 determines the amount of motion of the imaging device 100 from the value of the motion vector calculated by the calculation unit 303, and drives the stabilization unit 170 based on the determined amount of motion so as to prevent the captured image from blurring.

[0025] The optical system control unit 301 controls the optical system 181. The optical system control unit 301 determines a setting range of the focal length of the optical system 181 according to the output of the difficult scene detection unit 302 and the output of the motion vector calculation unit 303, and controls the focal length of the optical system 181 within the setting range. The difficult scene detection unit 302 performs a difficult scene detection process based on the output of the event sensor 180. When pixel information of a subject included in the output of the event sensor 180 satisfies a predetermined subject condition that makes it impossible to accurately calculate a motion vector, the difficult scene detection unit 302 notifies the optical system control unit 301 of information indicating that a difficult scene has been detected as a detection result. The pixel information of the subject is pixel information of a region corresponding to the subject, among the pixel information in which a change in brightness has occurred and which is output by the event sensor 180.

[0026] FIG. 3 is a diagram illustrating an example of controlling the focal length in accordance with a difficult scene. 3(A) and 3(C) show examples of difficult scenes. In FIG. 3(A), the subject is linear. A linear subject 401 is captured within the angle of view 400. In FIG. 3(C), the contrast of the subject is low and below the threshold. A predetermined region of the angle of view 402 (in this example, the central region 403) has low contrast. In the difficult scenes shown in FIGS. 3(A) and 3(C), when two pairs of frames are compared to calculate a motion vector, the direction of movement cannot be uniquely determined, resulting in reduced calculation accuracy.

[0027] When a poor scene as shown in FIG. 3(A) is detected, the optical system control unit 301 reduces the focal length and widens the angle of view. As a result, as shown in FIG. 3(B), the widened angle of view 404 includes subjects surrounding the subject 400 as shown in FIG. 3(A), improving the accuracy of calculating the motion vector. Similarly, when a poor scene as shown in FIG. 3(C) is detected, the optical system control unit 301 reduces the focal length and widens the angle of view. As a result, as shown in FIG. 3(D), the widened angle of view 405 includes subjects surrounding the low-contrast central region 403 as shown in FIG. 3(C), improving the accuracy of calculating the motion vector.

[0028] FIG. 4 is a diagram illustrating a conventional method for calculating a motion vector. Each of the event data 200 to 207 includes coordinate information of a pixel where a luminance change occurred. The pixel indicated by a black rectangle is the pixel where the luminance change occurred. The times 208 to 215 indicate the time when the luminance change occurred.

[0029] Reference numeral 222 indicates the time when mapping starts (mapping start time). In this example, a mapping start time (tm1) 216 and a mapping start time (tm2) 217 ​​are shown. The mapping start time tm2 is expressed as the sum of the immediately preceding mapping start time tm1 and the mapping time m indicated by reference numeral 220.

[0030] Reference numeral 221 denotes a frame group generated from event data. The frame group 221 includes frame 218 and frame 219. Frame 218 is generated by mapping the coordinates of pixels included in event data 200 to 203 that occurred between mapping start time tm1 and mapping time m. Frame 219 is generated by mapping the coordinates of pixels included in event data 204 to 207 that occurred between mapping start time tm2 and mapping time m. Then, a motion vector is calculated by template matching using adjacent frames 218 and 219.

[0031] In the conventional motion vector calculation method described with reference to Fig. 4, the cycle for generating the frame group 221 depends on the mapping time m. Therefore, the motion vector calculation cycle is determined by the mapping time m, making it difficult to improve the motion vector calculation cycle. According to the optical device of Example 1 described below, it is possible to improve the motion vector calculation cycle.

[0032] FIG. 5 is a diagram for explaining a method for calculating a motion vector by the optical device of the first embodiment. Among the elements indicated by the reference numerals in Fig. 5, elements assigned the same reference numerals as those in Fig. 4 are the same as the elements indicated by the reference numerals in Fig. 4. Reference numerals 223, 224, and 225 indicate mapping start times. Reference numeral 229 indicates a mapping start time difference d. The mapping start time difference d indicates the difference (time difference) between the mapping start time corresponding to each frame and the mapping start time corresponding to the immediately preceding frame. The mapping start time difference d (second time) is set to a time shorter than the mapping time m (first time).

[0033] In the first embodiment, the system control unit 150 partially overlaps the mapping of event data for generating frames for multiple frames. Specifically, the system control unit 150 starts mapping of event data every time a time period corresponding to the mapping start time difference d has elapsed, thereby consecutively generating multiple frames 226, 227, and 228. That is, the system control unit 150 starts mapping of event data corresponding to each frame by a delay of the mapping start time difference d from the mapping start time of the event data corresponding to the immediately preceding frame. In this way, a frame group 230 in FIG. 5 is generated.

[0034] The system control unit 150 generates frame 226 by mapping event data 200-203 that occurred between mapping start time tm1 and mapping time m. The system control unit 150 also generates frame 227 by mapping event data 202-205 that occurred between mapping start time tm2 and mapping time m. The mapping start time tm2 is the time that the mapping start time difference d has elapsed since the mapping start time tm1. The system control unit 150 also generates frame 228 by mapping event data 204-207 that occurred between mapping start time tm3 and mapping time m. The mapping start time tm3 is the time that the mapping start time difference d has elapsed since the mapping start time tm2.

[0035] The mapping start time difference d is determined according to the overlapping degree n of the mapping of the event data. For example, the mapping start time difference d is set to the time obtained by dividing the mapping time m by the overlapping degree n of the mapping of the event data, as shown in the following formula. Mapping start time difference d = mapping time m / n

[0036] 5, the mappings of event data 202 and 203 overlap in two frames (frames 226 and 227). Also, the mappings of event data 204 and 205 overlap in two frames (frames 227 and 228). Therefore, in the example shown in FIG. 5, the overlapping degree of the mappings of the event data is 2.

[0037] The system control unit 150 calculates a motion vector by template matching based on multiple frames whose mapping start times differ by a mapping time m. In the example shown in FIG. 5, a motion vector is calculated based on frame 226 and frame 228. A motion vector is also calculated based on frame 227 and a frame (not shown) whose mapping starts at a time that is the mapping time m after the mapping start time tm2. In this way, a motion vector is calculated for each mapping start time difference d. Therefore, according to this embodiment, the motion vector calculation cycle can be improved compared to the conventional motion vector calculation method described with reference to FIG. 4.

[0038] The calculation accuracy of a motion vector using the motion vector calculation method of this embodiment is primarily determined by the mapping time m. This is because the calculation accuracy varies depending on the number of event data included in the frame group 230, and the number of event data depends on the mapping time m. Furthermore, the number of event data generated per unit time varies depending on the subject conditions and focal length, so the value of the mapping time m required to maintain the calculation accuracy of the motion vector also varies depending on the subject conditions and focal length. For example, the longer the focal length of the optical system 181 of the event sensor, the higher the pixel resolution, and therefore the smaller the value of the mapping time m required to maintain the calculation accuracy of the motion vector.

[0039] On the other hand, the calculation period of the motion vector is determined by the mapping time m and the overlapping degree n. Increasing the overlapping degree n improves the calculation period of the motion vector, but increases the processing load of the optical device. Therefore, in this embodiment, the system control unit 150 reduces the value of the mapping time m and controls the focal length of the optical system 181 set by the optical system control unit 301 to be as long as possible so that a high calculation period can be maintained even when the overlapping degree n is reduced. Then, the system control unit 150 determines the mapping time m based on the set focal length so as to maintain the calculation accuracy of the motion vector. Then, the system control unit 150 sets the overlapping degree n based on the determined mapping time m so as to maintain the calculation period of the motion vector, and determines the calculation period by setting the mapping start time difference d according to the overlapping degree n.

[0040] Example 2 FIG. 6 is a flowchart illustrating the operation process of the optical device according to the second embodiment. The optical device of the second embodiment executes control to change the focal length depending on whether a poor scene is detected or not. In the flowchart shown in Fig. 6, S indicates the step number of each process according to the flowchart.

[0041] 6 is realized by system control unit 150 executing a program deployed in system memory 113. This processing starts when the photographer holds imaging device 100 toward a subject. In S501, the system control unit 150 determines whether an objectionable scene has been detected based on the objectionable scene detection result by the objectionable scene detection unit 302. If an objectionable scene has been detected, the process proceeds to S506. If an objectionable scene has not been detected, the process proceeds to S502.

[0042] In the processing of S502 to S504 described below, the system control unit 150 gradually increases the focal length within a range in which a poor scene is not detected, i.e., within a range in which the pixel information of the subject does not satisfy a predetermined subject condition, in order to reduce the processing load. This causes the system control unit 150 to set the focal length to the maximum value within the range in which a poor scene is not detected. Furthermore, in S506 to S508 described below, the system control unit 150 gradually decreases the focal length to obtain an angle of view in which a poor scene is not detected.

[0043] In S502, the system control unit 150 determines whether the focal length is the maximum value that can be set in the optical system 181. The maximum value of the focal length that can be set in the optical system 181 is determined by a known technique, for example, based on the output of the motion vector calculation unit 303. If the focal length is the maximum value that can be set in the optical system 181, the process proceeds to S509. If the focal length is not the maximum value that can be set in the optical system 181, the process proceeds to S503.

[0044] In S503, the system control unit 150 increases the focal length of the optical system 181 by a fixed value. Subsequently, in S504, the system control unit 150 determines whether an objectionable scene has been detected. If an objectionable scene has not been detected, the process returns to S502. If an objectionable scene has been detected, the process proceeds to S505. In S505, the system control unit 150 decreases the focal length of the optical system 181 by a fixed value. Then, the process proceeds to S509. This allows the focal length of the optical system 181 to be controlled to the maximum value within a range in which an objectionable scene is not detected.

[0045] Furthermore, in S506, the system control unit 150 determines whether the focal length is the minimum value that can be set in the optical system 181. The minimum value of the focal length that can be set in the optical system 181 is determined by a known technique, for example, based on the output of the motion vector calculation unit 303. If the focal length is the minimum value that can be set in the optical system 181, the process proceeds to S509. If the focal length is not the minimum value that can be set in the optical system 181, the process proceeds to S507.

[0046] In S507, the system control unit 150 reduces the focal length of the optical system 181 by a fixed value. Subsequently, in S508, the system control unit 150 determines whether an unsuitable scene has been detected. If an unsuitable scene has been detected, the process returns to S506. If an unsuitable scene has not been detected, the process proceeds to S509. This allows the focal length of the optical system 181 to be controlled to the maximum value within a range in which an unsuitable scene is not detected.

[0047] Next, in S509, the system control unit 150 sets a mapping time m based on the focal length set in the optical system 181. Specifically, the system control unit 150 sets a mapping time m that maintains the calculation accuracy of the motion vector based on the change in the focal length from the start of the processing to the present.

[0048] Next, in S510, the system control unit 150 sets the overlapping degree n based on the mapping time m set in S509. Specifically, the system control unit 150 sets the overlapping degree n at which the motion vector calculation period is maintained based on the change in the mapping time m from the start of processing to the present. The optical device of the present embodiment described above can improve the calculation period while maintaining the accuracy of motion vector calculation based on the output of the event sensor and reducing the load on the optical device. While preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments and various modifications and variations are possible within the spirit and scope of the present invention. For example, the present invention can be applied to devices that do not have an optical system as long as they function as a motion vector calculation device by acquiring necessary information from an external device.

[0049] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0050] 100 Main body 150 System control unit 180 Event Sensors 190 Lens Unit

Claims

1. an acquisition means for acquiring data including pixel information where a luminance change has occurred; generating means for performing a predetermined process on the data acquired at a first time to generate a frame; a control means for controlling the first time period for a first frame and the first time period for a second frame generated by the generating means to overlap each other; a calculation means for calculating a motion vector based on a plurality of frames whose start times of the predetermined processing on the data have the first time difference; A motion vector calculation device comprising:

2. The generating means starts the predetermined processing on the data at every second time determined according to the degree of overlap of the predetermined processing on the data, and generates the plurality of frames consecutively.

2. The motion vector calculation device according to claim 1.

3. The calculation means calculates the motion vector for each second time period.

3. The motion vector calculation device according to claim 2.

4. The control means sets the second time to a time obtained by dividing the first time by the degree of overlap of the predetermined process on the data.

4. The motion vector calculation device according to claim 2 or 3.

5. The predetermined processing for the data is mapping of coordinate information of pixels whose luminance has changed, which are included in the data.

5. The motion vector calculation device according to claim 1, wherein the motion vector is a vector of a moving object.

6. The calculation means calculates the motion vector based on a plurality of frames whose first time periods do not overlap, and calculates the motion vector based on a plurality of frames whose first time periods do not overlap.

6. The motion vector calculation device according to claim 1,

7. the acquiring means acquires the data based on an output of an event-based vision sensor that detects a change in brightness; The control means sets the first time period to be shorter as the focal length of the optical system of the vision sensor is longer.

7. The motion vector calculation device according to claim 1, wherein the motion vector is a vector of a moving object.

8. The control means sets a degree of overlap of the predetermined process for the data based on the set first time period.

8. The motion vector calculation device according to claim 7.

9. an optical system control means for controlling the optical system of the vision sensor; The optical system control means performs control to change the focal length of the optical system of the vision sensor when it is determined that pixel information of the subject satisfies a predetermined condition based on the output of the vision sensor.

9. The motion vector calculation device according to claim 7 or 8.

10. The optical system control means controls the optical system of the vision sensor to reduce the focal length when the subject is a linear subject or when the contrast in a predetermined area of ​​the angle of view is lower than a threshold value.

10. The motion vector calculation device according to claim 9.

11. The optical system control means sets the focal length of the optical system of the vision sensor to the maximum value within a range in which pixel information of the subject does not satisfy the predetermined condition.

11. The motion vector calculation device according to claim 9 or 10.

12. an imaging means for imaging a subject; an acquisition means for acquiring data including pixel information where a luminance change has occurred; generating means for performing a predetermined process on the data acquired at a first time to generate a frame; a control means for controlling the first time period for a first frame and the first time period for a second frame generated by the generating means to overlap each other; a calculation means for calculating a motion vector based on a plurality of frames whose start times of the predetermined processing on the data have the first time difference; and a correction control means for driving a correction means based on the calculated motion vector to correct blur in the captured image. An imaging device characterized by:

13. an acquisition step of acquiring data including pixel information in which a luminance change has occurred; a generating step of performing a predetermined process on the data acquired at a first time to generate a frame; a control step of performing control so that the first time for a first frame generated by the generating step and the first time for a second frame partially overlap each other; a calculation step of calculating a motion vector based on a plurality of frames whose start times of the predetermined processing on the data have the first time difference. A motion vector calculation method comprising:

Images