Imaging device

Abstract

To suppress deterioration of detection accuracy of a motion vector when using an imaging element that can control exposure time for each region.SOLUTION: Included are: an image sensor capable of setting exposure time for each region; detection means for detecting a plurality of motion vectors from an image acquired by the image sensor; weight determination means for determining a weight for each of the plurality of motion vectors detected by the detection means; and vibration component calculation means for calculating a vibration component from the plurality of motion vectors weighted by the weight determination means. The weight determination means lowers the weight of the motion vector that has moved in two or more regions.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an imaging device.

Background Art

[0002] Conventionally, there has been known a technique of providing an image sensor capable of setting an exposure time for each region, imaging with a shorter exposure time in a region with high illuminance, and lengthening the exposure time in a region with low illuminance (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem to be solved by the present invention is to suppress a decrease in detection accuracy of motion vectors when using an image sensor capable of controlling an exposure time for each region.

Means for Solving the Problems

[0005] An imaging device according to an aspect of the present invention includes an image sensor capable of setting an exposure time for each region, a detection unit that detects a plurality of motion vectors from an image acquired by the image sensor, a weight determination unit that determines a weight for each of the plurality of motion vectors detected by the detection unit, and a vibration component calculation unit that calculates a vibration component from the plurality of motion vectors weighted by the weight determination unit. The weight determination unit reduces the weight of a motion vector that has moved across two or more regions. Among them, the motion vector obtained by moving regions with different exposure times characterized by reducing the weight.

Effects of the Invention

[0006] It is possible to suppress a decrease in detection accuracy of motion vectors when using an image sensor capable of controlling an exposure time for each region.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Mode for Carrying Out the Invention

[0008] (First Embodiment) FIG. 1 is a block diagram of the imaging device 100 according to the present embodiment.

[0009] The imaging device 100 includes an image sensor 101 (imaging element) capable of setting an exposure time for each region, a lens 102, and a CPU 103. The imaging device 100 further includes a motion vector detection unit 104 that detects a plurality of motion vectors from the video acquired by the image sensor 101, and a vibration component calculation unit 105 that calculates a vibration component.

[0010] The CPU 103 receives the video from the image sensor 101, performs vibration correction based on the information from the vibration component calculation unit 105, and has a function of distributing the video to the connected network. Also, the CPU 103 receives the size and exposure time information of each area from the image sensor 101. Further, the CPU 103 sets the size and exposure time of each area for the image sensor 101.

[0011] The vibration component calculation unit 105 determines the weight for each motion vector based on the motion vector information received from the motion vector detection unit 104 and the size and exposure time information of each area of the image sensor 101 received from the CPU 103. Then, the vibration component calculation unit 105 calculates the vibration component and transmits the calculation result to the CPU 103.

[0012] Subsequently, the operation of the CPU 103 will be described with reference to FIG. 2.

[0013] When the imaging operation is started by the image sensor 101 (step S200), the CPU 103 acquires the video and area information (the size information of each area and the exposure time information of each area) from the image sensor 101. Then, the CPU 103 transmits the acquired area information to the vibration component calculation unit 105 (step S201).

[0014] In parallel, the motion vector detection unit 104 acquires the video from the image sensor 101, detects the motion vector, and transmits the detection result to the vibration calculation unit 105 (step S202).

[0015] Subsequently, the vibration component calculation unit 105 determines the weight of each motion vector based on the area information obtained in step S201 and the detection result of the motion vector obtained in step S202 (step S203).

[0016] Here, the method by which the vibration component calculation unit 105 determines the weight of each motion vector will be described with reference to FIGS. 3, 4, and 5.

[0017] The upper diagram in Fig. 3(a) shows the state where the imaging device 100 is attached to the attachment part 301. The lower diagram in Fig. 3(a) shows the video in that state.

[0018] The video is captured by the image sensor 101 with the exposure time set for each area. In this embodiment, it will be described assuming that the exposure time is set independently for each of the eight areas A1 to D1 and A2 to D2.

[0019] Specific exposure time setting values are shown in Fig. 4. The exposure time set for area A1 is 1 / 50 second, the exposure time set for area B1 is 1 / 100 second, the exposure time set for area C1 is 1 / 100 second, and the exposure time set for area D1 is 1 / 300 second. It is assumed that the exposure times shown in Fig. 4 are also set for areas A2 to D2.

[0020] This area information (the information that it is divided into eight areas A1 to D1 and A2 to D2, and the exposure time information for each area as shown in Fig. 4) is received from the CPU 103.

[0021] Figs. 3(b) and (c) show the state where the attachment part 301 vibrates due to the influence of wind or the like, and as a result, the imaging device 100 vibrates in the vertical direction.

[0022] Fig. 3(b) shows the state where the imaging device 100 vibrates upward. As a result of vibrating upward, compared with the video in Fig. 3(a), the subject moves downward.

[0023] Fig. 3(c) shows the state where the imaging device 100 vibrates downward. As a result of vibrating downward, compared with the video in Fig. 3(a), the subject moves upward. In this embodiment, it will be described assuming that it vibrates continuously as Fig. 3(a) → Fig. 3(b) → Fig. 3(c) → Fig. 3(a) → ···.

[0024] Figures 3(b) and 3(c) show the motion vectors V1 to V5 detected by the motion vector detection unit 104. The motion vectors are detected by the motion vector detection unit 104 based on the amount of movement of the feature points included in the video.

[0025] Figure 5 shows the existing regions and the determined weights for each of the motion vectors V1 to V5.

[0026] The motion vector V1 exists in regions A1 and A2 in the vibration of Figure 3(a) → Figure 3(b) → Figure 3(c) → Figure 3(a) → ···.

[0027] On the other hand, the motion vector V2 exists only in region B2 in the vibration in this embodiment. The motion vector V3 exists only in region C2 in the vibration in this embodiment. The motion vector V4 exists in regions C1 and C2 in the vibration in this embodiment. The motion vector V5 exists in regions D1 and D2 in the vibration in this embodiment.

[0028] The vibration component calculation unit 105 calculates the vibration components based on the motion vectors V1 to V5. At that time, the vibration components are calculated after reducing the weights of the motion vectors that have moved in two or more regions.

[0029] The reason is that when the feature points reciprocate between a region with an appropriate exposure time and a region with an inappropriate exposure time, in the region with an inappropriate exposure time, the amount of movement of the feature points may not be accurately detected compared to the case where it is appropriate.

[0030] The rightmost column of Figure 5 shows the weights of the motion vectors V1 to V5 determined by the vibration component calculation unit 105.

[0031] Since the motion vectors V2 and V3 do not move in two or more regions, the weights are determined to be 1. Since the motion vectors V1, V4, and V5 move in two or more regions, the weights are determined to be values smaller than 1.

[0032] At this time, the vibration component calculation unit 105 determines the weight in consideration of the difference in exposure time of the moved area. Specifically, the greater the difference in exposure time of the moved area, the smaller the weight of the motion vector. This is because it is more difficult to accurately detect the movement amount of the feature point when moving an area with a large difference in exposure time than when moving an area with a small difference in exposure time.

[0033] The motion vector V1 moves the areas A1 (exposure time 1 / 50 second) and A2 (exposure time 1 / 300 second). The motion vector V4 moves the areas C1 (exposure time 1 / 100 second) and C2 (exposure time 1 / 300 second). The motion vector V5 moves the areas D1 (exposure time 1 / 300 second) and D2 (exposure time 1 / 300 second).

[0034] When arranged in descending order of the difference in exposure time of the moved areas, the motion vectors are V1, V4, and V5. Therefore, the vibration component calculation unit 105 determines the weight of the motion vector V1 to be 0.3, the weight of the motion vector V4 to be 0.5, and the weight of the motion vector V5 to be 0.7.

[0035] The vibration component calculation unit 105 determines the weight as described above, calculates the vibration component, and transmits the calculation result to the CPU 103.

[0036] Finally, the CPU 103 performs vibration correction on the video based on the information from the vibration component calculation unit 105, and then distributes the video to the connected network (S205).

[0037] By implementing the method as described above, it is possible to provide the imaging device 100 that can sufficiently suppress the vibration of the video without reducing the detection accuracy of the motion vector even in the device equipped with the image sensor 101 capable of setting the exposure time for each area.

[0038] (Second Embodiment) The block diagram of the imaging device 100 is the same as that of the first embodiment (FIG. 1). The description is omitted.

[0039] In the first embodiment and the second embodiment, the operations of the CPU 103 are different.

[0040] The operation of the CPU 103 in the second embodiment will be described with reference to FIG. 6.

[0041] Compared with the operation of the CPU 103 (FIG. 2) described in the first embodiment, S207 and S208 are added. Other processes (S200 to S206) are the same.

[0042] The description will be made from the point where the processes of step S201 and step S202 are completed.

[0043] When the processes of step S201 and S202 are completed, in step S207, the vibration component calculation unit 105 calculates the vibration component based on the motion vectors V1 to V5. At this time, as described in the first embodiment, the weight of the motion vector is changed according to whether the motion vector has moved two or more regions.

[0044] However, when most of the detected motion vectors have moved in two or more regions, there is a problem that the vibration of the video cannot be sufficiently suppressed compared to the case where most of the detected motion vectors have moved in one region.

[0045] Therefore, in step S207, it is determined whether the number of motion vectors that have moved in two or more regions is more than half of the total number of motion vectors. If it is more than half of the total number of motion vectors, a part of the motion vectors that have moved in two or more regions is selected, and the region where the selected motion vector is detected is changed to expand in the moving direction (step S208). By this change, the selected motion vector becomes a motion vector that has moved in one region, so that the movement amount of the feature points can be accurately detected. After the process of step S207, the number of motion vectors that have moved in two or more regions becomes less than half of the total number of motion vectors.

[0046] Specifically, it will be described.

[0047] When the processes of step S201 and step S202 are completed, as shown in FIG. 5, the motion vectors V1, V4, and V5 are motion vectors that have moved across two or more regions. Since the total number of motion vectors is five, there are at least half (a predetermined number) or more of the motion vectors that have moved across two or more regions among the total number of motion vectors.

[0048] Therefore, among the motion vectors V1, V4, and V5 that have moved across two or more regions, V1 is selected, and the region where the motion vector V1 is detected is changed to expand in the moving direction. The changed state is shown in FIG. 7.

[0049] When it is changed to the state of FIG. 7, imaging is started again from step S200. Then, after the processes of step S201 and step S202 are completed, the process of step S203 is performed via step S207. The state where the process of step S203 is completed (the state where the vibration component calculation unit has determined the weights of the respective motion vectors) is shown in FIG. 8.

[0050] Compared with FIG. 5, the weight of the motion vector V1 has been changed from 0.3 to 1. An increase in the weight is equivalent to the fact that the movement amount of the feature points can be accurately detected. That is, by changing the region where the motion vector V1 is detected to expand in the moving direction, it becomes possible to suppress the vibration of the video compared with the case where no change is made (the first embodiment).

[0051] Also, as described above, it goes without saying that selecting the motion vector V1 with the smallest weight among the motion vectors V1, V4, and V5 can suppress the vibration of the video more than selecting the motion vectors V4 and V5.

[0052] Also, in the above description, a method in which the number of motion vectors that have moved across two or more regions is less than half of the total number of motion vectors is described, but it is not limited to half.

Explanation of Reference Numerals

[0053] 100 Imaging device 101 Image sensor 102 Lens 103 CPU 104 Motion vector detection unit 105 Vibration component calculation unit

Claims

1. An imaging device capable of setting exposure time for each area, Detection means for detecting a plurality of motion vectors from an image obtained by the imaging device, Weight determination means for determining a weight for each of the plurality of motion vectors detected by the detection means, Vibration component calculation means for calculating a vibration component from the plurality of motion vectors weighted by the weight determination means, and having The weight determination means reduces the weight of a motion vector that has moved across areas with different exposure times among the motion vectors that have moved across two or more areas. The imaging device is characterized by this.

2. The imaging device according to claim 1, wherein the greater the difference in exposure time between the areas that have moved, the lower the weight of the motion vector.

3. An imaging device capable of setting exposure time for each area, Setting means for setting the size of the area, and having The imaging device according to claim 1 or 2, wherein the setting means expands the area in which the motion vector has been detected in the moving direction when there are a predetermined number or more of motion vectors that have moved across two or more areas.

Images