Motion vector calculation device, imaging device, and motion vector calculation method

Abstract

The present invention relates to a motion vector calculation device, an imaging device, and a motion vector calculation method. An optical device obtains event data based on the output of an event sensor that detects a change in luminance of a subject image, and maps the event data obtained in a mapping time to generate a frame. The optical device performs control such that the mapping of the event data partially overlaps among a plurality of frames, and calculates a motion vector based on a plurality of frames having a mapping time difference at a start time of the mapping.

Description

Technical Field

[0001] The present invention relates to a motion vector calculation device, a imaging device, and a motion vector calculation method. Background Art

[0002] A vision sensor based on events (hereinafter referred to as an "event sensor") has been proposed, which has pixels that detect changes in the luminance of incident subject light and output event signals. An optical device such as an imaging device can acquire event data as data related to the event signal. Compared with a known complementary metal oxide semiconductor (CMOS) sensor, the event sensor has characteristics of high-speed operation, high dynamic range, and low power consumption. Japanese Patent Application Laid-Open No. 2020-522067 discloses a method of generating a frame by mapping event data generated within a predetermined time and calculating a motion vector by comparing the generated frames.

[0003] However, in the technique disclosed in Japanese Patent Application Laid-Open No. 2020-522067, the calculation period of the motion vector can be rate-determined according to a predetermined time (hereinafter referred to as the "mapping time") for mapping event data. Although the mapping time is an arbitrarily set value, the value for accurately calculating the motion vector changes according to the magnitude of the motion vector or the shape of the subject. Therefore, it is not desirable to change the mapping time according to the desired calculation period. Summary of the Invention

[0004] According to the present invention, the calculation period of the motion vector is improved.

[0005] According to an embodiment of the present invention, a vector calculation device includes: an acquisition unit configured to acquire data including pixel information in which a luminance change occurs; a generation unit configured to perform a predetermined process on the data acquired in a first time to generate a frame; a control unit configured to perform control so that the predetermined process on the data partially overlaps among a plurality of frames; and a calculation unit configured to calculate a motion vector based on the plurality of frames in which there is a difference in the first time at the start time of the predetermined process on the data.

[0006] According to an embodiment of the present invention, an imaging device includes: an imaging unit configured to image a subject; an acquisition unit configured to acquire data including pixel information where a luminance change occurs; a generation unit configured to perform a predetermined process on the data acquired at a first time to generate a frame; a control unit configured to perform control such that the predetermined process on the data is partially overlapped among a plurality of frames; a calculation unit configured to calculate a motion vector based on the plurality of frames in which there is a difference in the first time at a start time of the predetermined process on the data; and a correction control unit configured to correct shake of a captured image by driving a corrector based on the calculated motion vector.

[0007] According to an embodiment of the present invention, a motion vector calculation method includes: acquiring data including pixel information where a luminance change occurs; performing a predetermined process on the data acquired at a first time to generate a frame; performing control such that the predetermined process on the data is partially overlapped among a plurality of frames; and calculating a motion vector based on the plurality of frames in which there is a difference in the first time at a start time of the predetermined process on the data.

[0008] Other features of the present invention will become apparent from the description of exemplary embodiments with reference to the following drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a diagram showing an exemplary structure of an optical device.

[0010] Figure 2 is a diagram showing an exemplary structure of an anti-shake mechanism included in the optical device.

[0011] Figures 3A to 3D is a diagram showing an example of focal length control according to a weak scene.

[0012] Figure 4 is a diagram showing a motion vector calculation method of the prior art.

[0013] Figure 5 is a diagram showing a motion vector calculation method using the optical device according to the first embodiment.

[0014] Figure 6 is a flowchart showing an operation process of the optical device according to the second embodiment. DETAILED DESCRIPTION

[0015] (First Embodiment)

[0016] Figure 1 is a diagram showing an exemplary structure of an optical device according to an embodiment. In Figure 1In this case, an interchangeable-lens camera (imaging device) will be exemplified as an example of the optical device to be described. Of course, the present invention can also be applied to an integrated-lens camera. The present invention can also be applied to a motion vector calculation device that calculates a motion vector based on information acquired from the outside.

[0017] Figure 1 The imaging device shown in [the figure] includes a main unit 100 of the imaging device and a lens unit 190 that guides incident light to an image sensor 105 inside the main unit 100. The lens unit 190 can be detachably mounted on the main unit 100.

[0018] First, the structure of the main unit 100 will be described. A shutter 103 adjusts the amount of light. A shutter control unit 104 controls the shutter 103 while cooperating with a lens control unit 194 inside the lens unit 190 based on exposure information from an imaging processing unit 106. The image sensor 105 is an imager that performs photoelectric conversion on subject light. Specifically, an optical image of a subject (not shown) is formed on the image sensor 105 via a lens 195, a diaphragm 193, a lens-side mounting unit 192, a main-body-side mounting unit 102, and the shutter 103, and then this optical image is converted into an electrical signal.

[0019] An image processing unit 106 performs predetermined calculation processing on a video signal output from the image sensor 105, and performs image processing such as pixel interpolation processing, color conversion processing, and white balance processing based on the calculation results. A display unit 110 displays the image processing results of the image processing unit 106. The image processing unit 106 has an image compression function such as JPEG.

[0020] A recording circuit 107 is a circuit that reads and writes on a detachable recording medium such as a semiconductor memory for recording or reading image data. A communication unit 111 is connected to a wireless or wired cable to transmit and receive video signals or sound signals. The communication unit 111 can also be connected to a wireless local area network (LAN) or the Internet. The communication unit 111 can transmit a live image obtained by imaging or recording an image recorded on the recording circuit 107. The communication unit 111 can receive image data or various other types of information from an external device.

[0021] An operation unit 114 receives operations from a user and inputs various operation instructions according to the user's operations to a system control unit 150. The operation unit 114 includes any one or a combination of a switch or dial, a touch panel, a gaze detection indicator, and a voice recognition device.

[0022] The system timer 112 measures the time for various types of control or the time of the embedded clock. In the system memory 113, constants and variables for the operation of the system control unit 150, as well as programs read from the memory 140, etc. are loaded. In the system memory 113, for example, a random access memory (RAM) is used. The system memory 113 accumulates the output values of each axis of the triaxial acceleration sensor 130. The power of the power switch 115 can be set by switching between on (ON) and off (OFF) of the imaging device.

[0023] The shutter button 116 is an actuator for giving a photographing instruction. The first shutter switch 117 is turned on according to a photographing preparation instruction of half-pressing the shutter button 116 to generate a first shutter switch signal (SW1). According to SW1, operations such as autofocus processing, automatic exposure processing, automatic white balance processing, or flash pre-flash processing are started. The second shutter switch 118 is turned on according to a photographing instruction of fully pressing the shutter button 116 to generate a second shutter switch signal (SW2). According to SW2, the system control unit 150 starts a series of photographing processing operations from reading a signal from the image sensor 105 until writing image data on the recording circuit 107.

[0024] The triaxial gyro sensor 120 detects the angular velocity of the imaging device on three axes. The triaxial acceleration sensor 130 detects the acceleration of the imaging device on three axes. The memory 140 stores constants or programs for the operation of the system control unit 150, etc. The memory 140 includes a non-volatile memory capable of electrically erasing and storing data, and a read-only memory (ROM) can be used.

[0025] The system control unit 150 includes at least one processor and controls the operation of the entire imaging device. The system control unit 150 controls the lens control unit 194 included in the lens unit 190 via the connectors 101 and 191. The power control unit 160 includes a battery detection circuit, a protection circuit, a DC / DC converter, and a low dropout (LDO) regulator. The power control unit 160 controls the power supply unit 161 based on an instruction from the system control unit 150 and supplies a desired power voltage to each unit of the imaging device within a desired time period. The power control unit 160 detects whether a battery is installed, the type and remaining power of the battery, and when an overcurrent is detected, protects the load circuit connected to the power circuit by disconnecting the power supply. The power supply unit 161 includes a primary battery such as an alkaline battery and a lithium battery, or a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, and an AC adapter.

[0026] The main body side mounting unit 102 is an interface for connecting the main body unit 100 to the lens unit 190. The connector 101 is a connector for electrically connecting the main body unit 100 to the lens unit 190. The anti-vibration unit 170 is a corrector for correcting (reducing vibration) the shake (camera shake) that occurs in the captured image. The anti-vibration unit 170 changes the position of the image sensor 105 under the control of the system control unit 150. Thus, the camera shake is corrected. In this example, the system control unit 150 serves as a correction controller, calculates the motion vector at high speed based on the event data, and drives the anti-vibration unit 170 using the calculated motion vector as the amount of motion of the imaging device. For example, a lens for correcting camera shake may be provided on the lens unit 190 side, and the imaging device may correct the camera shake by driving this lens.

[0027] The event sensor 180 is an event-based vision sensor that has pixels for detecting a change in the brightness of the subject image formed by the optical system 181 of the event sensor 180 and generating and outputting a signal (event signal). The event sensor 180 outputs the event signal to the system control unit 150 through serial communication. Thus, the system control unit 150 acquires data (event data) related to the event signal. The event data is data including pixel information where the brightness change occurs. The event data includes, for example, the timestamp of the brightness change, the coordinate information of the pixel where the brightness change is detected, and information related to the type of the brightness change.

[0028] Since the event sensor 180 outputs the pixel information where the brightness change occurs, the redundancy of the information to be output is reduced compared with the prior art CMOS. Thus, the event sensor 180 has the characteristics of high-speed operation, high dynamic range, and low power. Since the prior art CMOS sensor outputs information periodically, the frame generation period required for calculating the motion vector depends on the frame rate. On the other hand, the event sensor outputs information including the timing of the brightness change at high speed in units of pixels. Thus, the system control unit 150 can generate a frame at high speed by mapping the event data generated within a predetermined time. Mapping is an example of a predetermined process for the event data and is a process of combining the information included in the event data at a predetermined time (mapping time). The optical system 181 of the event sensor 180 includes an actuator for changing the focal length. The focal length is controlled by the system control unit 150.

[0029] Next, the structure of the lens unit 190 will be described. The lens unit 190 is an interchangeable lens type lens unit, and subject light is guided from the lens 195 to the image sensor 105 through the aperture 193, the lens-side mounting unit 192, the body-side mounting unit 102, and the shutter 103 to form an image. The connector 191 is a connector that electrically connects the lens unit 190 to the body unit 100. The lens-side mounting unit 192 is an interface that connects the lens unit 190 to the body unit 100. The aperture 193 adjusts the amount of light entering the lens 195.

[0030] The lens control unit 194 controls the entire lens unit 190. The lens control unit 194 has: a function of a memory for storing constants, variables, programs, etc. for operations; and a function of a non-volatile memory for holding identification information (such as a unique number of the lens unit 190, etc.), management information, function information (such as an open aperture value, a minimum aperture value, and a focal length, etc.), and various current or past setting values, etc. The lens control unit 194 can control the focusing of the lens 195 according to the focusing state of the image measured by the image processing unit 106, change the formation position of the subject image incident on the image sensor 105, and perform an AF operation. The lens control unit 194 also has a function of controlling the aperture 193 or controlling the focal length of the lens 195.

[0031] Figure 2 FIG. is a diagram showing an exemplary structure of an anti-vibration mechanism included in an optical device according to an embodiment. The system control unit 150 corrects camera shake by calculating a motion vector based on the output of the event sensor 180 and driving the anti-vibration unit 170 based on the calculated motion vector. As Figure 2 shown, the system control unit 150 includes an anti-vibration control unit 300, an optical system control unit 301, a weak scene detection unit 302, and a motion vector calculation unit 303.

[0032] The motion vector calculation unit 303 calculates a motion vector based on the event data acquired from the event sensor 180. The anti-vibration control unit 300 determines the amount of movement of the imaging device 100 from the value of the motion vector calculated by the motion vector calculation unit 303, and drives the anti-vibration unit 170 based on the determined amount of movement so that the captured image does not shake.

[0033] The optical system control unit 301 controls the optical system 181. The optical system control unit 301 determines a setting range of the focal length of the optical system 181 based on the output of the weak scene detection unit 302 or the output of the motion vector calculation unit 303, and controls the focal length of the optical system 181 within the setting range. The weak scene detection unit 302 performs weak scene detection processing based on the output of the event sensor 180. When the pixel information of the subject included in the output of the event sensor 180 satisfies a predetermined subject condition in which the motion vector cannot be correctly calculated, the weak scene detection unit 302 notifies the optical system control unit 301 of information indicating that a weak scene has been detected as a detection result. The pixel information of the subject is the pixel information of the area corresponding to the subject in the pixel information of the pixels where the luminance change has occurred output by the event sensor 180.

[0034] Figures 3A to 3D is a diagram showing an example of focal length control according to a weak scene. Figure 3A and Figure 3C show examples of weak scenes. In Figure 3A the subject has a linear shape. Within the field of view angle 400, a subject 401 having a linear shape is shown. In Figure 3C the contrast of the subject is low and equal to or less than a threshold value. A predetermined area (in this example, the central area 403) of the field of view angle 402 has a low contrast. In Figure 3A and Figure 3C in the weak scenes shown, when comparing two sets of frames for calculating the motion vector, the moving direction is not uniquely determined. Therefore, the calculation accuracy deteriorates.

[0035] When detecting Figure 3A the weak scene shown, the optical system control unit 301 shortens the focal length and widens the field of view angle. Therefore, as Figure 3B shown, the widened field of view angle 404 includes Figure 3A the subjects near the subject 400 shown in Figure 3C Therefore, the calculation accuracy of the motion vector is improved. When detecting Figure 3D the weak scene shown, the optical system control unit 301 shortens the focal length and widens the field of view angle. Therefore, as Figure 3C shown, the widened field of view angle 405 includes

[0036] Figure 4 is a diagram showing a motion vector calculation method of the prior art. The event data 200 to event data 207 include the coordinate information of each pixel where the luminance change has occurred. The pixels indicated by the black rectangles are the pixels where the luminance change has occurred. The times 208 to 215 are the times when the luminance change has occurred.

[0037] Reference numeral 222 indicates the time when mapping starts (mapping start time). In this example, the mapping start time (tm1) 216 and the mapping start time (tm2) 217 are shown. The mapping start time tm2 is expressed as the sum of the immediately preceding mapping start time tm1 and the mapping time m represented by reference numeral 220.

[0038] Reference numeral 221 represents a set of frames generated from event data. The set of frames 221 includes frames 218 and 219. Frame 218 is generated from the mapping coordinates of the pixels included in event data 200 to event data 203 generated from the mapping start time tm1 to the mapping time m. Frame 219 is generated from the mapping coordinates of the pixels included in event data 204 to event data 207 generated from the mapping start time tm2 to the mapping time m. Then, a motion vector is calculated by performing template matching using adjacent frames 218 and 219.

[0039] In the motion vector calculation method of the prior art described in the reference Figure 4 the period for generating the set of frames 221 depends on the mapping time m. Therefore, the motion vector calculation period is determined according to the mapping time m, and thus it is difficult to increase the motion vector calculation period. The optical device according to the first embodiment described below can increase the motion vector calculation period.

[0040] Figure 5 is a diagram showing a motion vector calculation method using the optical device of the first embodiment. Among the elements represented by the reference numerals shown in Figure 5 the elements having the same reference numerals as those shown in Figure 4 are the same as the elements represented by the reference numerals shown in Figure 4 Reference numerals 223, 224, and 225 represent mapping start times. Reference numeral 229 represents the mapping start time difference d. The mapping start time difference d indicates the difference (time difference) between the mapping start time corresponding to each frame and the mapping start time corresponding to the immediately preceding frame. The mapping start time difference d (second time) is set to be shorter than the mapping time m (first time).

[0041] In the first embodiment, the system control unit 150 partially overlaps the mapping of event data for generating frames among a plurality of frames. Specifically, whenever the time corresponding to the mapping start time difference d elapses, the system control unit 150 starts the mapping of event data and continuously generates a plurality of frames 226, 227, and 228. That is, the system control unit 150 starts mapping the event data corresponding to each frame by delaying the mapping start time by the mapping start time difference d from the mapping start time of the event data corresponding to the immediately preceding frame. Then, the set of frames 230 in Figure 5 is generated.

[0042] The system control unit 150 generates frame 226 by mapping event data 200 to event data 203 generated from the mapping start time tm1 to the mapping time m. The system control unit 150 generates frame 227 by mapping event data 202 to event data 205 generated from the mapping start time tm2 to the mapping time m. The mapping start time tm2 is the time after the mapping start time tm1 by the mapping start time difference d. The system control unit 150 generates frame 228 by mapping event data 204 to event data 207 generated from the mapping start time tm3 to the mapping time m. The mapping start time tm3 is the time after the mapping start time tm2 by the mapping start time difference d.

[0043] The mapping start time difference d is determined according to the overlap degree n of the event data. For example, as shown in the following formula, the mapping start time difference d is set to the time obtained by dividing the mapping time m by the overlap degree n of the mapping of the event data.

[0044] Mapping start time difference d = mapping time m / n

[0045] In Figure 5 the example shown, the mapping of event data 202 and event data 203 overlaps between two frames (frame 226 and frame 227). The mapping of event data 204 and event data 205 overlaps between two frames (frame 227 and frame 228). Therefore, in Figure 5 the example shown, the overlap degree of the mapping of the event data is 2.

[0046] The system control unit 150 calculates the motion vector by template matching based on multiple frames with a difference of the mapping time m between the mapping start times. In Figure 5 the example shown, the motion vector is calculated based on frame 226 and frame 228. The motion vector is calculated based on frame 227 and a frame (not shown) that starts mapping at the time after the mapping start time tm2 by the mapping time m. Therefore, the motion vector is calculated for each mapping start time difference d. Therefore, according to the embodiment, compared with the prior art motion vector calculation method described in Figure 4 the calculation period of the motion vector can be further improved.

[0047] In the motion vector calculation method of the embodiment, the calculation accuracy of the motion vector is mainly determined according to the mapping time m. This is because the calculation accuracy varies according to the number of event data included in the frame set 230, and the number of event data depends on the mapping time m. Since the number of event data generated at each time varies according to the subject condition or the focal length, the value of the mapping time m for maintaining the calculation accuracy of the motion vector also varies according to the subject condition or the focal length. For example, as the focal length of the optical system 181 of the event sensor becomes larger, the pixel resolution increases. Therefore, the value of the mapping time m for maintaining the calculation accuracy of the motion vector decreases.

[0048] On the other hand, the calculation period of the motion vector is determined according to the mapping time m and the overlap degree n. When the overlap degree n increases, the calculation period of the motion vector increases. However, the processing load of the optical device may increase. Therefore, in the embodiment, although the value of the mapping time m is small and the overlap degree n is small, the system control unit 150 controls so that the focal length of the optical system 181 set by the optical system control unit 301 is as large as possible to maintain a high calculation period. In addition, the system control unit 150 determines the mapping time m based on the set focal length so as to maintain the calculation accuracy of the motion vector. The system control unit 150 determines the calculation period by the following method: setting the overlap degree n based on the determined mapping time m to maintain the calculation period of the motion vector, and setting the mapping start time difference d according to the overlap degree n.

[0049] (Second Embodiment)

[0050] Figure 6 is a flowchart showing the operation process of the optical device according to the second embodiment. The optical device according to the second embodiment controls to change the focal length according to whether a weak scene is detected. Figure 6 In the flowchart, S represents the step number of each process in the flowchart.

[0051] is realized by allowing the system control unit 150 to execute the program loaded in the system memory 113 Figure 6 of the flowchart. When the photographer sets the imaging device 100 for the subject, the process starts. In S501, the system control unit 150 determines whether a weak scene is detected based on the weak scene detection result of the weak scene detection unit 302. When a weak scene is detected, the process proceeds to S506. When a weak scene is not detected, the process proceeds to S502.

[0052] In the processes of S502 to S504 described below, the system control unit 150 gradually increases the focal length within the range where a weak scene is not detected (i.e., the range where the pixel information of the subject does not satisfy the predetermined subject conditions) to reduce the processing load. Therefore, the system control unit 150 sets the focal length to the maximum value within the range where a weak scene is not detected. In S506 to S508 described below, the system control unit 150 gradually decreases the focal length so that the field of view angle where a weak scene is not detected is achieved.

[0053] In S502, the system control unit 150 determines whether the focal length is the maximum value that can be set in the optical system 181. For example, based on the output of the motion vector calculation unit 303 according to known techniques, the maximum value of the focal length that can be set in the optical system 181 can be determined. When the focal length is the maximum value that can be set in the optical system 181, the process proceeds to S509. When the focal length is not the maximum value that can be set in the optical system 181, the process proceeds to S503.

[0054] In S503, the system control unit 150 increases the focal length of the optical system 181 to a constant value. Then, in S504, the system control unit 150 determines whether a weak scene is detected. When a weak scene is not detected, the process returns to S502. When a weak scene is detected, the process proceeds to S505. In S505, the system control unit 150 decreases the focal length of the optical system 181 to a constant value. Then, the process proceeds to S509. Therefore, the focal length of the optical system 181 can be controlled so that the focal length is the maximum value within the range where a weak scene is not detected.

[0055] In S506, the system control unit 150 determines whether the focal length is the minimum value that can be set in the optical system 181. For example, based on the output of the motion vector calculation unit 303 according to known techniques, the minimum value of the focal length that can be set in the optical system 181 can be determined. When the focal length is the minimum value that can be set in the optical system 181, the process proceeds to S509. When the focal length is not the minimum value that can be set in the optical system 181, the process proceeds to S507.

[0056] In S507, the system control unit 150 decreases the focal length of the optical system 181 to a constant value. Then, in S508, the system control unit 150 determines whether a weak scene is detected. When a weak scene is detected, the process returns to S506. When a weak scene is not detected, the process proceeds to S509. Therefore, the focal length of the optical system 181 can be controlled so that the focal length is the maximum value within the range where a weak scene is not detected.

[0057] Then, in S509, the system control unit 150 sets the mapping time m based on the focal length set in the optical system 181. Specifically, the system control unit 150 sets the mapping time m for maintaining the calculation accuracy of the motion vector based on the change in the focal length from the start time point of the process until the current time.

[0058] Then, in S510, the system control unit 150 sets the overlap degree n based on the mapping time m set in S509. Specifically, the system control unit 150 sets the overlap degree n for maintaining the calculation period of the motion vector based on the change in the mapping time m from the start time point of the process until the current time. In the optical device according to the above embodiment, it is possible to improve the calculation period while maintaining the calculation accuracy of the motion vector based on the output of the event sensor and suppressing the load on the optical device. The preferred embodiments of the present invention have been described above, but the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the gist of the present invention. For example, when the device is used as a motion vector calculation device, even if the device does not include an optical system, the present invention can be applied by obtaining necessary information from the outside.

[0059] Other embodiments

[0060] Embodiments of the present invention can also be implemented by the following method, that is, by providing software (program) that executes the functions of the above embodiments to a system or device through a network or various storage media, and the method by which a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0061] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the following claims will be given the broadest interpretation to cover all such modifications and equivalent structures and functions.

[0062] This application claims the benefit of Japanese Patent Application No. 2021-006856 filed on January 20, 2021, which is incorporated herein by reference in its entirety.

Claims

1. A vector calculation device, comprising: An acquisition unit configured to acquire data including pixel information where brightness changes occur; A generation unit configured to perform a predetermined process on the data acquired in a first time to generate a frame; A control unit configured to perform control such that the predetermined process on the data is partially overlapped among a plurality of frames; And A calculation unit configured to calculate a motion vector based on the plurality of frames in which there is a difference in the first time at the start time of the predetermined process on the data.

2. The vector calculation device according to claim 1, wherein, The generation unit starts the predetermined process on the data for each second time to continuously generate the plurality of frames, and the second time is determined according to the degree of overlap of the predetermined process on the data.

3. The vector calculation device according to claim 2, wherein, The calculation unit calculates the motion vector for each second time.

4. The vector calculation device according to claim 2, wherein, The control unit sets the second time to the time obtained by dividing the first time by the degree of overlap of the predetermined process on the data.

5. The vector calculation device according to claim 1, wherein The predetermined process on the data is a mapping of the coordinate information of the pixels with brightness changes included in the data.

6. The vector calculation device according to claim 1, wherein, The acquisition unit acquires the data based on the output of a detection unit that detects brightness changes.

7. The vector calculation device according to claim 6, Among them, The detection unit is an event-based vision sensor, and wherein the control unit sets the first time according to the focal length of the optical system of the vision sensor.

8. The vector calculation device according to claim 7, wherein, The control unit sets the first time based on the focal length of the optical system of the vision sensor.

9. The vector calculation device according to claim 8, wherein, The control unit sets the degree of overlap of the predetermined process on the data based on the set first time.

10. The vector calculation device according to any one of claims 7 to 9, Among them, The vector calculation device further includes an optical system control unit configured to control the optical system of the vision sensor, and wherein the optical system control unit performs control such that when it is determined that the pixel information of the subject satisfies a predetermined condition based on the output of the detection unit, the focal length of the optical system of the vision sensor is changed.

11. The vector calculation device according to claim 10, wherein, The optical system control unit performs control such that when the subject is a subject having a linear shape or the contrast of a predetermined region of the field of view angle is lower than a threshold value, the focal length of the optical system of the vision sensor is reduced.

12. The vector calculation device according to claim 10, wherein, The optical system control unit sets the focal length of the optical system of the vision sensor to the maximum value within the range where the pixel information of the subject does not satisfy the predetermined condition.

13. A camera device, comprising: A camera unit configured to capture an image of a subject; An acquisition unit configured to acquire data including pixel information where brightness changes occur; A generation unit configured to perform a predetermined process on the data acquired in a first time to generate a frame; A control unit configured to perform control such that the predetermined process on the data is partially overlapped among a plurality of frames; A calculation unit configured to calculate a motion vector based on a plurality of frames having a difference in the first time at a start time of the predetermined processing of the data; and A correction control unit configured to correct jitter of a captured image by driving a corrector based on the calculated motion vector.

14. A motion vector calculation method, comprising: Obtaining data including pixel information where a luminance change occurs; Performing predetermined processing on the obtained data in a first time to generate frames; Performing control such that the predetermined processing of the data partially overlaps in a plurality of frames; and Calculating a motion vector based on the plurality of frames having a difference in the first time at a start time of the predetermined processing of the data.

Images