Imaging system and mobile object provided with same

Abstract

This imaging system is provided with: an imaging device disposed on a mobile object; an optical axis modification mechanism that, when the imaging device performs imaging as the mobile object is moving in a first direction, modifies the optical axis of the imaging device from a state of a first optical axis of the imaging device when a first image is taken, to states of k optical axes, k being an integer of two or more, so that the optical axis of the imaging device is modified to a state of a second optical axis displaced in a second direction intersecting the first direction when a second image is taken, thereby modifying the optical axis of the imaging device successively in the second direction; and a control device that activates the optical axis modification mechanism. The control device activates the optical axis modification mechanism between an imaging timing for an rth image and an imaging timing for an r+1th image after the imaging device captured the rth image where the condition 2≤r≤n is satisfied for a total number n of images taken.

Description

[Technical field]

[0001] The present disclosure relates to an imaging system that is fixed to a moving body and captures images while the moving body is moving, and to a moving body including the imaging system. [Background technology]

[0002] As transportation infrastructure ages, there is an increasing demand for infrastructure inspection. Instead of visual inspection by humans, the efficiency of inspection can be dramatically improved by capturing images of infrastructure facilities while moving around using a mobile vehicle and then detecting defects through image processing of the captured images.

[0003] For example, in Patent Document 1, an image of a target area is captured by a camera installed in a vehicle while moving. Also, when the vehicle is traveling at a high speed, camera movement causes blurring, but in Patent Document 1, the motion blurring is corrected by using Saccade mirror technology. The blurring is reduced by irradiating the target with light and reflecting the light reflected by the target on a mirror that rotates for a preset exposure time and then entering the camera. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2015 / 060181 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when capturing a high-resolution image, a lens with a long focal length is used, which reduces the angle of view and therefore the imaging range.

[0006] The present disclosure provides an imaging system capable of expanding an imaging range and a mobile body including the imaging system. [Means for solving the problem]

[0007] The imaging system of the present disclosure includes an imaging device disposed on a moving body, an optical axis changing mechanism for changing the optical axis of the imaging device into k optical axis states, the k being a number equal to or greater than 2, so that when the imaging device captures an image while the moving body is moving in a first direction, the optical axis of the imaging device is changed from a state of a first optical axis of the imaging device when capturing a first image to a state of a second optical axis displaced in a second direction intersecting the first direction when capturing a second image, and a control device for operating the optical axis changing mechanism. The control device operates the optical axis changing mechanism between the imaging timing of the first image and the imaging timing of the second image, and between the imaging timing of the rth image and the imaging timing of the r+1th image after the imaging device captures an rth image that satisfies the condition 2≦r≦n for a total number of images n.

[0008] The moving object of the present disclosure is equipped with the imaging system described above. Effect of the Invention

[0009] According to the imaging system and the moving body including the imaging system of the present disclosure, it is possible to provide an imaging system capable of expanding an imaging range and a moving body including the imaging system. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram for explaining a vehicle equipped with an imaging system according to a first embodiment. [Diagram 2] FIG. 1 is a block diagram showing the internal configuration of an imaging system according to a first embodiment. [Diagram 3] FIG. 1 is an explanatory diagram illustrating the state of each imaging device in the state of two optical axes. [Figure 4] FIG. 1 is an explanatory diagram showing an image captured in a first imaging mode; [Diagram 5] FIG. 11 is an explanatory diagram showing an image captured in a second imaging mode; [Figure 6] FIG. 1 is an explanatory diagram illustrating an enlarged imaging range by an imaging device in the state of two optical axes. [Figure 7] 1 is a flowchart showing an image capture process according to the first embodiment. [Figure 8] Graph showing the relationship between the change in moving speed, the timing of exposure time, and the change in optical axis in the first embodiment. [Figure 9] FIG. 13 is a diagram showing a modification of the imaging mode. [Figure 10] FIG. 13 is a diagram showing a modification of the imaging mode. [Figure 11] FIG. 11 is an explanatory diagram for explaining an enlarged imaging range by an imaging device in the state of two optical axes in the second embodiment. [Figure 12A] FIG. 13 is a diagram for explaining a vehicle equipped with an imaging system according to a third embodiment. [Figure 12B] FIG. 13 is an explanatory diagram for explaining the state of each imaging device in the state of two optical axes in the third embodiment. [Figure 13] FIG. 11 is a block diagram showing the internal configuration of an imaging system according to a third embodiment. [Figure 14] FIG. 1 is an explanatory diagram for explaining blur correction in an imaging system; [Figure 15] 11 is a flowchart showing an image capture process according to the third embodiment. [Figure 16] Graph showing the relationship between the change in moving speed, the timing of exposure time, and the change in optical axis in the third embodiment. [Figure 17] FIG. 13 is a block diagram showing the internal configuration of an imaging system according to a fourth embodiment. [Figure 18] 11 is a flowchart showing an image capture process according to the fourth embodiment. [Figure 19] FIG. 13 is a block diagram showing the internal configuration of an imaging system according to a fourth embodiment. [Figure 20] Flowchart showing imaging processing in the fifth embodiment [Figure 21] 11 is a flowchart showing an image capture process according to a modification of the fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] (Embodiment 1) Hereinafter, the first embodiment will be described with reference to the drawings. In the first embodiment, a case will be described in which the moving body is a vehicle 3 such as an automobile, and the imaging system 1 is attached to the top of the vehicle 3. The imaging system 1 of the first embodiment is arranged to capture an image of a road, as an example.

[0012] [1-1. Imaging system configuration] Please refer to Fig. 1 and Fig. 2. Fig. 1 is a diagram for explaining an imaging system 1. Fig. 2 is a block diagram showing an internal configuration of the imaging system 1. In Fig. 1, a vehicle 3 is traveling on, for example, a road 4. For example, holes 4b and cracks 4c have occurred on the road 4. In addition, potholes, ruts, and the like that have occurred on the road surface can be detected from the captured image by image processing.

[0013] The imaging target of the imaging system 1 is at least a part of the structure around the vehicle 3, and is an target that moves relatively in accordance with the moving speed of the vehicle 3 as the vehicle 3 moves. The imaging target area 9 is an area of ​​this imaging target that is acquired as an image. In addition to the road 4, the imaging target may also be the inner wall of a tunnel, the side or bottom of an overpass, a utility pole, or an electric wire. This makes it possible to detect holes, cracks, lifting, peeling, joints, tilt of a utility pole, and bending of an electric wire in the imaging target by image processing from the acquired image.

[0014] An imaging system 1 is installed on the upper surface of a vehicle 3. The imaging system 1 is fixed so as to capture an image of a road 4 below the vehicle 3 in FIG.

[0015] 1 and 2, the imaging system 1 includes a speed detection device 3a, an imaging device 11, an optical axis changing mechanism 12, and a control device 15. The imaging device 11 captures an image of the surroundings of the vehicle 3, and in the first embodiment, captures an image of the surface of the road 4. The imaging device 11 includes a camera body 21, a lens 23, a shutter 24, an imaging element 25, and a camera control unit 27.

[0016] The speed detection device 3a is disposed on the vehicle 3 and detects the moving speed of the vehicle 3. This makes it possible to detect that the vehicle 3 is moving. In the first embodiment, the speed detection device 3a detects the moving speed based on a vehicle speed pulse signal in which a pulse signal is switched ON / OFF for every certain amount of rotation (rotation angle) of the axle of the vehicle 3. The speed detection device 3a sends the vehicle speed pulse signal together with the detected moving speed to the control device 15. In addition to this form, the speed detection device 3a may be, for example, a vehicle speed sensor that detects the moving speed from the rotation speed of the axle of the vehicle 3. The detection of the moving speed may be performed by the control device 15 based on the vehicle speed pulse signal.

[0017] Camera body 21 has replaceable lens 23 attached thereto, and houses image sensor 25 and camera control unit 27. Image sensor 25 is disposed at a position of focal length F of lens 23. Lens 23 is oriented directly toward road 4, which is the subject of the image. Camera body 21 and lens 23 may be integrated. Image sensor 25 converts received light into an electrical signal according to its intensity, and is, for example, a solid-state image sensor such as a CCD image sensor, a CMOS image sensor, or an infrared image sensor.

[0018] The camera control unit 27 opens the shutter 24 while receiving an exposure instruction signal from the control device 15. The shutter 24 may be configured with a multiple blade diaphragm that opens and closes, or may be an electronic shutter. The camera body 21 is supported by a base 61. The base 61 is supported on the upper surface of the vehicle 3 so as to be rotatable.

[0019] When the imaging device 11 captures an image while the vehicle 3 is moving in a first direction that is the +X-axis direction, the optical axis changing mechanism 12 changes the optical axis of the lens 23 of the imaging device 11 from a state of a first optical axis 23ab that is directed perpendicularly to the road 4 when capturing a first image to a state of a second optical axis 23ac that is inclined in a second direction that is the +Y-axis direction that intersects with the first direction when capturing a second image. The optical axis changing mechanism 12 includes a base 61 and a rotation drive unit 63. Note that, in order to change the optical axis, the optical axis changing mechanism 12 may rotate the imaging device 11 about a point different from the principal point 23b of the lens 23 as a rotation center, or may translate the imaging device 11 in parallel, in addition to rotating the optical path.

[0020] The base 61 supports the camera body 21. The imaging device 11 may be configured such that the camera body 21, the lens 23, and the optical axis changing mechanism 12 are integrated together. For example, the imaging device 11 may have a pan function that rotates the camera body 21 and the lens 23 in the horizontal direction.

[0021] The rotation drive unit 63 drives the base 61 to rotate based on a rotation instruction from the control device 15. The rotation drive unit 63 is composed of, for example, a motor and a gear. The camera body 21 and the mirror 41 also rotate together with the rotation of the base 61. Note that the optical axis changing mechanism 12 may include, for example, a rotation stage, and the imaging device 11 may be rotated by the rotation stage.

[0022] Please refer to FIG. 3. FIG. 3 is an explanatory diagram for explaining the imaging device 11 in each state of two types of optical axes, FIG. 3(a) is an explanatory diagram showing the imaging device 11 in a first state C1 in which the lens 23 is in a state of the first optical axis 23ab, and FIG. 3(b) is an explanatory diagram showing the imaging device 11 in a second state C2 in which the lens 23 is in a state of the second optical axis 23ac. By driving the rotation drive unit 63, the imaging device 11 can be displaced between the first state C1 shown in FIG. 3(a) and the second state C2 shown in FIG. 3(b). The optical axis changing mechanism 12 rotates the imaging device 11, for example, around the principal point 23b of the lens 23 as the rotation center. This allows the imaging range of the imaging device 11 to be expanded in the second direction as shown in FIG. 4.

[0023] For example, when the imaging device 11 is in the first state C1, it captures the first image Im1 (see FIG. 4), and after the imaging of the image Im1 is completed, the imaging device 11 changes to the second state C2 to capture the image Im2. After the imaging of the image Im2 is completed, the optical axis 23a of the lens 23 is moved to the first optical axis 23ab in the third direction, which is the -Y-axis direction, so that the imaging device 11 changes to the first state C1 and captures the image Im3. In this way, the imaging device 11 captures the road 4 while alternately changing between the first state C1 and the second state C2, and captures the fourth image Im4, the fifth image Im5, and the sixth image Im6, so that the second direction of the road 4 can be captured over a wider range. The mode in which the imaging device 11 is changed and captured in this way is called the first imaging mode.

[0024] In the first imaging mode, the end area Im3a on the opposite side to the moving direction in the third image Im3 is imaged so as to overlap with the end area Im1a in the moving direction in the first image Im1. Also, the end area Im1b in the second direction (+Y axis direction) in the first image Im1 is imaged so as to overlap with the end area Im2b in the third direction (-Y axis direction) in the second image Im2. Also, the end area Im2b in the third direction (-Y axis direction) in the second image Im2 is imaged so as to overlap with the end area Im3b in the second direction (+Y axis direction) in the third image Im3. In this way, the first image Im1 and the third image Im3 have a common imaging area, and further, the first image Im1, the second image Im2, and the third image Im3 each have a common imaging area. Similarly, by capturing images Im4, Im5, and Im6 in sequence, adjacent images can have overlapping captured areas. Alternatively, adjacent images can capture the road 4 without any gaps. In either case, it is possible to prevent missing images.

[0025] The imaging system 1 may also have a second imaging mode in which images Im1 to Im6 are captured while maintaining the first state C1 without moving the imaging device 11. In this case, as shown in FIG. 5, captured images can be acquired continuously in a line. Note that in FIG. 5, the images Im1 to Im6 have different widths in the Y-axis direction for ease of understanding, but the actual widths are all the same. In the second imaging mode, for example, the imaging area of ​​image Im2 is entirely included in the imaging areas of images Im1 and Im3, so image Im2 does not need to be captured. In other words, images Im1, Im3, and Im5 may be captured in sequence.

[0026] The amount of expansion of the imaging range in the first imaging mode will be described with reference to Fig. 6. Fig. 6 is an explanatory diagram for explaining the amount of expansion of the imaging range, Fig. 6(a) is an explanatory diagram showing the state before the optical axis is changed (Φ=0°), and Fig. 6(b) is an explanatory diagram showing the state after the optical axis is changed (Φ=5°). In Fig. 6, for ease of explanation, the distance between an extension line LE1 (extension line of the imaging plane) of the principal point 23b of the lens 23 and an extension line LE2 of the imaging target plane will be described. In the first embodiment, the imaging target plane is the surface of the road 4.

[0027] If the horizontal size of the imaging element 25 is 7.03 mm, the focal length F of the lens 23 is 35 mm, and the first subject distance D1, which is the distance to the road 4 when the optical axis of the imaging device 11 is at the initial position (optical axis change angle Φ=0), is 1.7 m, the angle of view a is 11.47 deg, and the horizontal imaging range W1 (=2×WL1) is 0.34 m. The horizontal imaging range W1 is calculated by the following formula (1). W1=2×WL1=2×D1×tan(a / 2)···(1) formula

[0028] The amount of magnification WL2 is calculated by the following formula (2) using the optical axis change angle Φ. The optical axis change angle Φ is the angle rotated from the first optical axis 23ab due to the optical axis change. WL2=D1×tan{(Φ+(a / 2)}-D1×tan(a / 2)···(2) formula When the optical axis change angle Φ is 5[deg], the enlargement amount WL2 is 0.15[m], and the imaging range is enlarged horizontally by approximately 44%.

[0029] Here, the optical axis change angle Φ≦the angle of view a. This allows an image captured with the first optical axis 23ab and an image captured with the second optical axis 23ac to be continuous without any gaps or to have an overlapping area. The optical axis change angle Φ may be a predetermined angle designated by the user through the operation unit 19, or may be determined by the control device 15 based on the distance to the imaging target area 9 and the angle of view a of the imaging device 11.

[0030] Please refer to FIG. 2. The control device 15 controls the exposure time of the imaging device 11 and the drive of the optical axis changing mechanism 12, and operates the optical axis changing mechanism 12 between the timing of capturing the first image and the timing of capturing the second image. The control device 15 is a circuit that can be realized by semiconductor elements, etc. The control device 15 can be configured by, for example, a microcomputer, a CPU, an MPU, a GPU, a DSP, an FPGA, or an ASIC. The functions of the control device 15 may be configured by hardware alone, or may be realized by combining hardware and software. The control device 15 realizes a predetermined function by reading data and programs stored in the storage unit 17 and performing various arithmetic processing.

[0031] The control device 15 has an optical axis change instruction unit 71. The optical axis change instruction unit 71 instructs the rotation drive unit 63 of the optical axis changing mechanism 12 to change the state of the optical axis, for example, in response to the timing of receiving the speed detection of the vehicle 3 by the speed detection device 3a or the imaging timing of the imaging device 11 with a constant frame rate.

[0032] The control device 15 transmits an exposure control signal to the camera control unit 27. The exposure control signal has two types of signals: a Hi signal as an ON signal that instructs exposure, and a Low signal as an OFF signal that does not instruct exposure. When capturing images at a constant frame rate, the camera control unit 27 may control the exposure instead of the control device 15.

[0033] The storage unit 17 is a storage medium that stores programs and data necessary to realize the functions of the control device 15. The storage unit 17 can be realized by, for example, a hard disk drive (HDD), an SSD, a RAM, a DRAM, a ferroelectric memory, a flash memory, a magnetic disk, or a combination of these.

[0034] The operation unit 19 is an input device for a user to give instructions to the control device 15. The operation unit 19 may be an input device dedicated to the imaging system 1, or may be a mobile terminal such as a smartphone. When a mobile terminal is used as the operation unit 19, the operation unit 19 and the control device 15 transmit and receive data by wireless communication. The user may use the operation unit 19 to instruct the control device 15 whether the imaging target area is a dark area indoors such as a tunnel or a bright area outdoors such as a mountain slope or a road, or may instruct the imaging interval Tf. The imaging interval Tf is the time between the timing at which imaging of the current image is completed and the timing at which imaging of the next image is completed, and is the time of one frame in the case of video imaging, and is the time interval between the imaging times of the images to be captured in the case of still image imaging. Note that in FIG. 8, imaging intervals Tf1 and Tf2 for each image are exemplified as the imaging interval Tf, and exposure times Tp1, Tp2, and Tp3 for each image are exemplified as the exposure time Tp. The exposure times Tp1, Tp2, and Tp3 are the times from the imaging start times t1, t3, and t5 to the imaging end times t2, t4, and t6, respectively. In the case of video imaging, the frame rate (the number of images taken per second) may be specified. In addition, the user may use the operation unit 19 to instruct the control device 15 to switch between a first imaging mode in which the optical axis changing mechanism 12 is operated and a second imaging mode in which the optical axis changing mechanism 12 is not operated.

[0035] [1-2. Operation of the imaging system] Next, the operation of the imaging system 1 will be described with reference to Figs. 7 and 8. Fig. 7 is a flowchart showing the imaging process performed by the imaging system 1. Fig. 8 is a graph showing the relationship between the exposure time and the timing of changing the optical axis. Fig. 8(a) is a graph showing the time change in vehicle speed, Fig. 8(b) is a graph showing the output timing of an exposure control signal, and Fig. 8(c) is a graph showing the time change in the optical axis change angle Φ. The imaging process shown in Fig. 7 is started, for example, when an instruction to start imaging is given from the operation unit 7 while the vehicle 3 is moving.

[0036] In step S1, the user measures in advance the subject distance from the imaging element 25 of the camera body 21 to the surface of the road 4 to be imaged, and sets the measured subject distance in the control device 15 using the operation unit 7. In addition, by setting the section of the road to be imaged, the control device 15 can determine, for example, based on GPS information and the traveled distance, whether the vehicle has traveled on the road in the set section.

[0037] In step S2, the vehicle 3 starts traveling, and the speed detection device 3a detects the moving speed of the vehicle 3. The detected moving speed is sent to the control device 15. In the first embodiment, imaging is performed in synchronization with the speed detection device 3a detecting the speed, but imaging may be performed at a predetermined cycle, or the user may set the imaging cycle using the operation unit 19. When the exposure is synchronized with the vehicle speed pulse, for example, if the speed of the vehicle 3 is detected at intervals of 40 cm when the moving distance of the vehicle 3 is 60 km / h, the frame rate will be about 40 fps. In addition, the speed detection device 3a does not need to detect the exact moving speed, and may be configured to detect that the vehicle is moving. Then, the detected moving state is sent to the control device 15. The control device 15 may perform imaging after detecting that the vehicle 3 is in a moving state in addition to the user's instruction to start imaging from the operation unit 19. The distance interval for speed detection may be changed according to the specifications of the speed detection device 3a.

[0038] In step S3, if the number of images to be captured next is the second or more, the optical axis change instruction unit 71 of the control device 15 issues a rotation instruction to the rotation drive unit 63 of the optical axis change mechanism 12. This changes the imaging direction of the imaging device 11. If the image is the first image, the optical axis change instruction unit 71 does not need to issue a rotation instruction to the rotation drive unit 63. In this case, the optical axis of the imaging device 11 is located at the initial position (optical axis change angle Φ=0). In the first embodiment, after capturing the first image Im1, the optical axis of the imaging device 11 is fixed during the first time Tm1, and the optical axis of the imaging device 11 is rotated during the second time Tm2. In FIG. 8, Tm1a and Tm1b are exemplified as the first time Tm1, and Tm2a and Tm2b are exemplified as the second time Tm2. The optical axis of the imaging device 11 starts to be changed at the timing of time t2 when the imaging of the first image Im1 is finished, and ends at time ta. The period from time t2 to ta is a second time Tm2a.

[0039] After the operation of changing the optical axis of the rotation drive unit 63 is completed, the optical axis of the imaging device 11 is fixed during the first time period Tm1a from time ta to time tb which is after the time t4 when the imaging of the second image Im2 is completed. The optical axis change instruction unit 71 controls the optical axis change mechanism 12 such that the exposure time Tp ≤ the first time period Tm1 < the imaging interval Tf and the exposure time Tp is included within the period of the first time period Tm1. According to this control, while the rotation drive unit 63 is changing the optical axis of the imaging device 11, no exposure instruction is sent from the control device 15 to the camera control unit 27, so the influence on image blur due to the optical axis change can be avoided with high precision. On the other hand, the range of possible values for the second time period Tm2 is Tm2 = Tf - Tm1 and Tm2 > 0. When using the exposure time Tp, the range of possible values for the second time period Tm2 is 0 < Tm2 ≤ Tf - Tp. Therefore, when it is necessary to ensure the time of the second time period Tm2 as much as possible considering the responsiveness of the optical axis change mechanism 12, the first time period Tm1 may be made smaller. In this case, it is not necessarily required that the first time period Tm1 satisfies the condition of the exposure time Tp ≤ the first time period Tm1. For example, it is sufficient that the first time period Tm1 satisfies the condition of Tp / 2 ≤ Tm1 < Tf, which is a condition where the stop period of the optical axis change mechanism 12 during exposure is longer than or equal to the period of changing the optical axis by the optical axis change mechanism 12 and the stop period of the optical axis change mechanism 12 during imaging during exposure is dominant. In this way, the first time period Tm1 is determined based on the imaging interval Tf and the exposure time Tp, and the second time period Tm2 is determined by the imaging interval Tf and the first time period Tm1. During this first time period Tm1a, in step S4, the control device 15, for example, continuously sends a Hi signal to the camera control unit 27 of the imaging device 11 for the exposure time Tp2. Thereby, the imaging element 25 images the imaging target during the exposure time Tp2. The image captured by the imaging element 25 is recorded from the camera control unit 27 to the storage unit 17, thereby acquiring the captured image.

[0040] In step S5, the control device 15 judges whether the vehicle 3 has traveled a predetermined section. When the control device 15 judges that the vehicle 3 has completed traveling the predetermined section, the control device 15 has completed acquiring images of the road in this section and therefore terminates the moving image capturing. Alternatively, the control device 15 may terminate the moving image capturing in response to an instruction from the operation unit 7 by the user operating the operation unit 7. When the control device 15 judges that the vehicle 3 has not completed traveling the predetermined section, it returns to step S2 and performs the moving image capturing again. In step S3, after the first time Tm1a has elapsed, the optical axis change instruction unit 71 instructs the rotation drive unit 63 of the optical axis changing mechanism 12 to rotate so as to return the optical axis of the imaging device 11 to the initial position (optical axis change angle Φ=0) during the second time Tm2b from time tb to tc. After the optical axis changing operation of the rotation drive unit 63 is completed, the optical axis of the imaging device 11 is fixed for a first time Tm1b from time tc to time td after time t6 when imaging of the third image Im3 is completed. During this first time Tm1b, in step S4, the control device 15 continues to send, for example, a Hi signal to the camera control unit 27 of the imaging device 11 for an exposure time Tp3. This causes the imaging element 25 to capture an image of the imaging target for the exposure time Tp3.

[0041] A modified example of the first embodiment will be described with reference to Figs. 9 and 10. In the first embodiment, the optical axis of the imaging device 11 has two optical axis states, the first optical axis 23ab and the second optical axis 23ac, but this is not limited thereto. The optical axis changing mechanism 12 may change the optical axis of the imaging device 11 to k optical axis states, which is two or more, and sequentially change the optical axis of the imaging device 11 in the second direction. In this way, the imaging device 11 may have three or more optical axis states, and the optical axis changing mechanism 12 may change the imaging device 11 to each optical axis state and capture an image of the imaging target in each optical axis state. This makes it possible to further expand the imaging range.

[0042] 9 shows the relationship between the order of imaging and the captured images when the imaging device 11 has three optical axis states. The optical axis changing mechanism 12 changes the optical axis to three (k=3) states and sequentially changes the optical axis of the imaging device 11 to the second direction. When the imaging device 11 captures an r-th image that satisfies the condition 2≦r≦n for a total number of images n, if r is not a multiple of k, that is, if r is not a multiple of 3 (r / 3 is not an integer), for example, if r=5, the optical axis of the imaging device 11 is changed to the second direction from the fifth image capture for the sixth image capture. If r is a multiple of k, that is, if r is a multiple of 3 (r / 3 is an integer), for example, if r=6, the optical axis of the imaging device 11 is changed to the third direction opposite to the second direction from the sixth image capture for the seventh image capture. The control device 15 operates the optical axis changing mechanism 12 between the first image capturing timing and the second image capturing timing and between the r-th image capturing timing and the r+1-th image capturing timing, and the r+1-th captured image is an image in the first direction when the r-th captured image is captured. This allows the imaging range to be expanded more than when capturing two images in the second direction. In addition, the control device 15 calculates the imaging interval Tfk between the r-th image capturing and the r+k-th image capturing based on the moving speed of the vehicle 3. If the imaging range of one image in the traveling direction is Lx [m], the overlapping range of the images in the traveling direction is Ly [m], and the moving speed is V [km / h], the time interval Tx [sec] between the r-th image capturing and the r+k-th image capturing satisfies the following formula. Here, the coefficient c is a coefficient for converting the units of distance [m] and [km] and the units of time [hour] and [sec], and is 3.6. Tx≦c×(Lx-Ly) / V If the minimum overlapping range of images in the direction of travel is Lymin [m], then the maximum value Txmax of the time interval Tx is calculated as c × (Lx-Lymin) / V, and the time interval Tx may be set to be equal to or less than Txmax. The minimum overlapping range of images in the direction of travel, Lymin, is the range required to join consecutive images into one image by image recognition. For example, Lymin may be 20% or more of the shooting range Lx. The number k of changeable optical axis states can be calculated from the set time interval Tx and imaging interval Tf. k is calculated as an integer value that satisfies the following formula. k≦Tx / Tf Since the amount of movement of the optical axis from the r+k−1th image to the r+kth image is large, k may not be the maximum integer value that satisfies the above formula. In that case, k is set to the maximum integer value −1. Furthermore, the control device 15 may calculate the imaging interval Tf between the rth image and the r+1th image based on the moving speed and the number k of changeable optical axes. As described above, the imaging interval Tf may be calculated from the time interval Tx set based on the moving speed V and the number k of changeable optical axis states. The imaging interval Tf is set to a value that satisfies the following formula. Tf≦Tx / k In order to prevent the number of captured images from increasing and the storage capacity of image data from increasing, Let Tf = Tx / k. Furthermore, when r / k is not an integer, the optical axis change angle Φ by which the optical axis of the imaging device 11 is changed when capturing the rth image to the r+1th image is equal to or smaller than the angle of view a of the imaging device 11, and the rth captured image and the r+kth captured image have a common imaging area, so that the rth captured image, the r+1th captured image, and the r+kth captured image each have a common imaging area. For example, when r=4, r / k is not an integer, so that the fourth image Im4 and the seventh image Im7 have a common imaging area, so that the fourth image Im4 and the fifth image Im5 have a common imaging area, and the fifth image Im5 and the seventh image Im7 have a common imaging area.

[0043] FIG. 10 shows the relationship between the order of imaging and the captured images when the imaging device 11 has four optical axis states. The optical axis changing mechanism 12 changes the optical axis state to four (k=4) and sequentially changes the optical axis of the imaging device 11 to the second direction. The control device 15 calculates the imaging interval Tf between the rth image and the r+1th image based on the moving speed and the number of optical axes k=4. When the imaging device 11 captures the rth image that satisfies the condition of 2≦r≦n for the total number of images n, if r is not a multiple of 4 (r / 4 is not an integer), for example, if r=5, the optical axis of the imaging device is changed to the second direction from the fifth image capture for the sixth image capture, and if r is a multiple of 4 (r / 4 is an integer), for example, if r=8, the optical axis of the imaging device 11 is changed to the third direction opposite to the second direction from the eighth image capture for the ninth image capture. This makes it possible to expand the imaging range more than when capturing two or three images in the second direction. Furthermore, when r / k is not an integer, for example when r=5, the fifth image Im5 and the ninth image Im9 have a common imaging area, so that the fifth image Im5 and the sixth image Im6 have a common imaging area, and the sixth image Im6 and the ninth image Im9 have a common imaging area.

[0044] [3. Effects, etc.] Thus, the imaging system 1 includes an imaging device 11 disposed in a vehicle 3, an optical axis changing mechanism 12 that changes the optical axis of the imaging device 11 to k optical axis states, where k is an integer of 2 or more, so that when the imaging device 11 captures an image while the vehicle 3 is moving in a first direction that is the +X-axis direction, the optical axis 23a of the imaging device 11 is changed from the state of the first optical axis 23ab of the imaging device 11 at the time of capturing a first image to the state of the second optical axis 23ac displaced in the +Y-axis direction intersecting the +X-axis direction at the time of capturing a second image, and sequentially changes the optical axis of the imaging device 11 to the second direction, and a control device 15 that operates the optical axis changing mechanism 12. The control device 15 operates the optical axis changing mechanism between the imaging timing of the first image and the imaging timing of the second image, and between the imaging timing of the rth image and the imaging timing of the r+1th image after the imaging device 11 captures an rth image that satisfies the condition 2≦r≦n (n is an integer of 2 or more) for the total number of images n.

[0045] In a moving vehicle 3, the range of captured images can be expanded by changing the state of the imaging device 11 to two or more optical axes. In addition, since images are not captured while the optical axis is being changed, blurring of the captured image can be prevented.

[0046] In addition, the control device 15 operates the optical axis changing mechanism 12 to fix the optical axis of the imaging device 11 for the first time Tm1 based on the imaging interval Tf between the rth and r+1th images and the exposure time Tp of the imaging device 11 in the rth and r+1th images, and to change the optical axis of the imaging device 11 for the second time Tm2 based on the imaging interval Tf and the first time Tm1 between the imaging timing of the rth image and the imaging timing of the r+1th image. As a result, the optical axis of the imaging device 11 is solid Since the focal length is set at the same angle as the optical axis, blurring of an image caused by rotation of the optical axis can be prevented.

[0047] Furthermore, when r / k is not an integer, the optical axis change angle Φ by which the optical axis of the imaging device 11 is changed when capturing the rth image to the r+1th image is equal to or smaller than the angle of view a of the imaging device 11. This allows the rth image and the r+1th image to be continuous with no gaps in the second direction or to have an overlapping region.

[0048] In addition, when r is not divisible by k (r / k is not an integer), the optical axis changing mechanism 12 changes the optical axis of the imaging device 11 in a second direction from the time of capturing the r-th image for the r+1th image, and when r / k is an integer, changes the optical axis of the imaging device in a third direction opposite to the second direction from the time of capturing the r-th image for the r+1th image.

[0049] Furthermore, when r / k is an integer, the optical axis changing mechanism 12 changes the optical axis of the imaging device 11 to a state in which the first optical axis is displaced in the third direction when the r+1th image is captured relative to the rth image.

[0050] Moreover, the r+kth captured image is an image of the imaging target area 9 located in the first direction from the imaging target area 9 when the rth captured image was captured, and the end area on the opposite side to the first direction in the r+kth captured image overlaps with the end area in the first direction in the rth captured image. This makes it possible to prevent imaging omissions between images in the first direction.

[0051] The control device 15 also includes a first imaging mode in which the imaging device 11 captures images while operating the optical axis changing mechanism 12 between the imaging timing of the first image and the imaging timing of the second image and between the imaging timing of the rth image and the imaging timing of the r+1th image, and a second imaging mode in which the imaging device 11 captures images continuously without operating the optical axis changing mechanism 12. This makes it possible to select the imaging mode according to the imaging range.

[0052] (Embodiment 2) The imaging system 1A in the second embodiment captures an image of an imaging target by varying the states of two optical axes, and in the first state C1 and the second state C2, each optical axis is inclined with respect to the imaging target. With respect to this point and other points described below, the imaging system 1A in the second embodiment has the same configuration as the imaging system 1 in the first embodiment.

[0053] In the imaging system 1 of the first embodiment, the lens 23 is placed in a position facing the imaging target, that is, the axis perpendicular to the imaging target surface is parallel to the optical axis, so that the subject distance in the imaging plane in the first state C1 becomes uniform and an in-focus image can be captured. On the other hand, when the imaging device 11 is moved to the second state C2 in order to change the imaging orientation of the camera so that a wide range can be captured in one driving image capture, the position of the lens 23 may shift from the facing position, causing variation in the subject distance in the imaging plane and resulting in a slightly blurred image.

[0054] Therefore, in the imaging system 1 of the second embodiment, the installation position of the imaging device 11 is tilted in consideration of the optical axis change. As shown in Fig. 11, the imaging device 11 is installed with an installation tilt so that the imaging direction is tilted around an axis parallel to the moving direction of the vehicle 3 by a predetermined angle smaller than the optical axis change angle Φ of the optical axis change mechanism 12 with respect to the imaging target area of ​​the imaging device 11. For example, in Fig. 11, the moving direction of the vehicle 3 is the X-axis direction perpendicular to the paper surface, and the position where the imaging direction is tilted by half the optical axis change angle Φ / 2 to be set so that the imaging direction is tilted around the X-axis parallel to the moving direction within the YZ plane intersecting the moving direction of the vehicle 3 is set as the position of the first state C1a.

[0055] With this configuration, the subject distance D1a in the first state C1a and the subject distance D2a in the second state C2 can be made the same distance. Since the change in subject distance of the images captured in the first state C1 and the second state C2 at the two positions can be suppressed, the image can be captured symmetrically at the initial position (first state C1a) and the optical axis change position (second state C2). Therefore, it is possible to achieve both uniform imaging accuracy and widening of the angle of view at both positions.

[0056] (Embodiment 3) An imaging system 1B in the third embodiment will be described with reference to Figs. 12 to 15. Fig. 12A is an explanatory diagram for explaining a vehicle 3 equipped with the imaging system 1B in the third embodiment. Fig. 12B is an explanatory diagram for explaining the states of the imaging devices in the states of the two optical axes in the third embodiment. Fig. 13 is a block diagram showing the internal configuration of the imaging system 1B in the third embodiment. Fig. 14 is an explanatory diagram for explaining blur correction in the imaging system 1B.

[0057] The imaging system 1B in the third embodiment is configured by providing the imaging system 1 in the first embodiment with a blur correction mechanism 31. With respect to this point and other points that will be described below, the imaging system 1B in the third embodiment is common to the imaging system 1 in the first embodiment.

[0058] As shown in FIG. 12A, the vehicle 3 is traveling, for example, in a tunnel 5. For example, holes 5b and cracks 5c are generated on a wall surface 5a inside the tunnel 5. Thus, when capturing an image in a dark environment such as inside the tunnel 5, if the exposure time is extended, motion blur occurs in the captured image. In addition, when capturing an image of a subject while the vehicle 3 is traveling at high speed, motion blur occurs in the captured image. The blur correction mechanism 31 corrects the optical path of light incident on the imaging system 1 so that motion blur of the image of the imaging target area 9 is reduced even if the imaging device 11B captures an image while the vehicle 3 is moving.

[0059] The camera body 21 is disposed on the vehicle 3 so that the orientation of the lens 23 is parallel to the direction of movement of the vehicle 3. For example, the camera body 21 is disposed so that the lens 23 faces forward or backward of the vehicle 3.

[0060] The blur correction mechanism 31 corrects the optical path of light L1, which is ambient light reflected by the imaging target area 9, in accordance with the movement of the vehicle 3. The blur correction mechanism 31 aligns the direction of light L1, which is ambient light reflected by the imaging target area 9, with the imaging direction of the imaging element 25. The blur correction mechanism 31 includes, for example, a mirror 41 and a mirror driver 43. The mirror 41 totally reflects the light, which is ambient light reflected by the imaging target, in the direction of the imaging device 11.

[0061] 12B, when the vehicle 3 is moving in the first direction, which is the +X-axis direction, and the imaging device 11 captures an image, the optical axis changing mechanism 12B changes the optical axis 23a of the lens 23 of the imaging device 11 from a first optical axis 23ad that is perpendicular to the ceiling of the tunnel 5 when capturing the first image to a second optical axis 23ae that is inclined and displaced in a second direction, which is the +Y-axis direction and intersects with the first direction, when capturing the second image. The optical axis changing mechanism 12B rotates both the blur correction mechanism 31 supported by the base 61 and the imaging device 11B around the rotation axis 61a. Therefore, the optical axis of the blur correction mechanism 31 is also changed together with the optical axis of the imaging device 11 by driving the optical axis changing mechanism 12B.

[0062] The blur correction mechanism 31 and the optical axis changing mechanism 12B are not limited to this configuration. In the case where the imaging device 11B is configured such that the camera body 21 and the lens 23 are integrated, a pan-tilt rotation mechanism that rotates the camera body 21 and the lens 23 around a rotation axis may be used. In this case, the blur correction mechanism 31 corresponds to a mechanism that rotates in the tilt direction, and the optical axis changing mechanism 12B corresponds to a mechanism that rotates in the pan direction. In addition, when the lens 23 is rotated 90 degrees and installed vertically, the blur correction mechanism 31 becomes a mechanism that rotates in the pan direction, and the optical axis changing mechanism 12B becomes a mechanism that rotates in the tilt direction. In this way, when the lens 23 is directly pointed at the subject, it can rotate in the pan direction corresponding to the traveling direction. As a result, the traveling direction can be set to the long side direction of the imaging element 25, and even if the vehicle 3 is moving at a high speed, it is easy to ensure the overlap of the captured images in the moving direction.

[0063] Furthermore, when the camera body 21 and the lens 23 have a mechanism for rotating in one axis direction, the mechanism for rotating in one axis direction may be used as the optical axis changing mechanism 12B. In this case, the blur correction mechanism 31 is composed of a mirror 41 and a mirror driving unit 43. The blur correction mechanism 31 and the optical axis changing mechanism 12B may also be composed of two mirrors and a motor whose respective rotation axes are orthogonal to each other. The entire imaging device 11B may also be rotated in two directions that are orthogonal to each other. Furthermore, a mechanism for driving the camera body 21 and the lens 23 to rotate in one axis direction may be used as the optical axis changing mechanism 12B, and a mechanism for rotating the entire imaging device 11B in the tilt direction may be used as the blur correction mechanism 31.

[0064] The mirror 41 is rotatably disposed so as to face the lens 23. The mirror 41 can be rotated, for example, in either a forward or reverse clockwise direction, and the rotatable angular range may be less than 360 degrees or may be 360 ​​degrees or more. The mirror 41 totally reflects ambient light reflected by the imaging subject toward the imaging device 11. The mirror driving unit 43 drives the mirror 41 to rotate from an initial angle to a specified angle, and returns the mirror 41 to the initial angle after rotating it to the specified angle. The mirror driving unit 43 is, for example, a motor.

[0065] The rotation angle of the mirror 41 is limited by the mechanical constraints of the mirror drive unit 43, and the mirror 41 can be rotated up to the maximum swing angle of the mirror 41 determined by this constraint. The motion blur correction angle θ at which motion blur correction is possible is equal to or smaller than the maximum swing angle of the mirror 41.

[0066] Blur correction by blur correction mechanism 31 will be described with reference to Fig. 14. For example, it is assumed that imaging system 1 located at position A moves to position B together with vehicle 3 during the exposure time. It is assumed that imaging is started at position A and an image is acquired at this timing. For example, hole 5b in imaging target area 9 is captured in the image acquired at position A, but since the exposure time is insufficient, the image is dark and not clear.

[0067] Therefore, exposure is continued until the vehicle 3 moves to position B. In this case, if no blur correction is performed, the image capture target area 9 moves relatively in the opposite direction to the moving direction of the vehicle 3, resulting in an image in which the hole 5b moves relatively. In the image with continued exposure, the amount of movement of the pixels is detected as the amount of blur. In this way, the image captured by the imaging device 11 while the vehicle 3 is moving will be a blurred image.

[0068] Therefore, by rotating the mirror 41 in a direction in which the end 41a of the mirror 41 in the moving direction cancels the relative movement of the imaging target during the exposure time according to the moving speed of the imaging system 1B and the vehicle 3, the imaging system 1 can capture the same imaging target area 9 in the captured image during the exposure time, and obtain an image with significantly reduced motion blur. In FIG. 14, the mirror 41 is rotated clockwise so that the end 41a of the mirror 41 in the moving direction rotates toward the imaging target during the exposure time. By rotating the mirror 41, the amount of movement of pixels in the captured image is corrected to zero.

[0069] 13, the control device 15B includes an optical axis change instruction unit 71, a correction mechanism swing angle calculation unit 73, and a correction mechanism rotation speed calculation unit 75.

[0070] The correction mechanism swing angle calculation unit 73 calculates the mirror swing angle α of the mirror 41 during imaging based on the moving speed V of the vehicle 3, the set exposure time Tp, the subject magnification M, and the focal length F of the lens 23 in the following manner. The mirror swing angle α corresponds to the correction mechanism swing angle.

[0071] The focal length F is a value determined by the lens 23. The subject magnification M is a value determined by the focal length F and the subject distance D. The subject distance D is the distance from the principal point 23b of the lens 23, which is disposed between the subject to be imaged and the imaging element 25, to the imaging target area 9. The subject distance D may be a known value measured in advance, or a value measured by a rangefinder during imaging may be used.

[0072] The amount of movement L of the vehicle 3 during the exposure time Tp from the start time of imaging to the end time of imaging is calculated from the moving speed V and the exposure time Tp by the following formula (3). L [mm] = V [km / h] x 10 6 ×Tp[ms] / (60 2 ×10 3 )...Equation (3)

[0073] The amount of movement P of the pixels on the imaging element 25 from the start time of imaging to the end time of imaging is calculated from the amount of movement L of the vehicle 3 and the subject magnification M by the following formula (4). P [mm] = L [mm] × M (4)

[0074] This pixel movement amount P causes motion blur, so to prevent motion blur, the optical path of light entering the lens 23 is changed by a motion blur compensation angle θ corresponding to the pixel movement amount P. The motion blur compensation angle θ is calculated from the pixel movement amount P and focal length F by the following formula (5). θ[deg]=arctan(P / F)...Equation (5) As described above, the subject magnification M is calculated from the focal length F [mm] and the subject distance D [m] according to the following formula (6). M = F / (D×10 3 )...Equation (6) From equations (3), (4), (5), and (6), θ = arctan(V×10 6 ×Tp / (60 2 ×10 3 ) / (D×10 3 )) = arctan(V×Tp / (D×60 2 ))...Equation (7) In this way, the motion blur correction angle θ is calculated from the motion speed V, the exposure time Tp, and the subject distance D.

[0075] The mirror swing angle α required for motion blur correction during exposure is half the motion blur correction angle θ, and is calculated by the following formula (8). α=θ / q (8) 13, in the case where light from the subject travels through the mirror 41, the lens 23, and the image sensor 25 in that order, the coefficient q is 2. In the case where the pan-tilt mechanism and the entire camera are driven, the coefficient q is 1.

[0076] In this manner, the correction mechanism swing angle calculation unit 73 calculates the mirror swing angle α of the mirror 41.

[0077] The correction mechanism rotation speed calculation unit 75 calculates the rotation speed Vm of the mirror 41 during the exposure period based on the mirror swing angle α and the exposure time Tp using the following formula. Vm=α / Tp...Equation (9)

[0078] In this manner, it is possible to calculate the rotation speed Vm corresponding to the movement speed V of each vehicle 3. Therefore, by rotating the mirror 41 at the rotation speed Vm in the direction opposite to the movement direction after starting imaging, the imaging device 11B can receive light from the same imaging target area 9 during the exposure time, and motion blurring in the captured image can be suppressed.

[0079] Next, the operation of the imaging system 1B will be described with reference to Fig. 15 and Fig. 16. Fig. 15 is a flowchart showing the imaging process performed by the imaging system 1B. Fig. 16 is a graph showing the relationship between the exposure time, the motion blur correction angle, and the optical axis change angle.

[0080] Steps S1 to S3 are the same as the operations of the imaging system 1 of the first embodiment, so the description will be omitted. In step S11, the correction mechanism swing angle calculation unit 73 calculates the mirror swing angle α as the shake correction amount. In addition, the correction mechanism rotation speed calculation unit 75 calculates the rotation speed Vm of the mirror 41 based on the mirror swing angle α.

[0081] In step S12, the control device 15B causes the mirror driver 43 to rotate the mirror 41 at the calculated rotation speed Vm, so that the mirror 41 starts rotating from a predetermined initial angle, which is the rotation start position. This allows motion blur correction during image capture by the image capture device 11B to be performed. At the same time, the control device 15B continues to send a Hi signal, which instructs exposure, to the camera control unit 27 for the exposure time Tp.

[0082] In the imaging device 11B, the camera control unit 27 opens the shutter 24 to expose the image while receiving the Hi signal, thereby acquiring an image (step S12), and stores the acquired image in the storage unit 17. When the exposure time Tp has elapsed, the control device 15B continues to send a Low signal as an OFF signal that instructs the camera control unit 27 to stop exposure. Note that the Low signal may be used as an ON signal that instructs the camera to start exposure, and the Hi signal may be used as an OFF signal that instructs the camera to stop exposure.

[0083] While the camera control unit 27 is receiving the Low signal, the camera control unit 27 closes the shutter 24, and the control device 15B causes the mirror driving unit 43 to rotate the mirror 41 in the reverse direction to return the mirror 41 to the initial angle. The mirror driving unit 43 may also rotate the mirror 41 in the forward direction to return the mirror 41 to the initial angle. This initial angle varies depending on the moving speed of the vehicle 3.

[0084] When images are captured continuously, the operation of the imaging system 1B after returning to step S1 will be described with reference to Fig. 16. Fig. 16 is a graph showing the relationship between the change in moving speed, the timing of the exposure time, and the motion blur correction angle. Fig. 16(a) is a graph showing the moving speed of the vehicle 3 changing over time. Fig. 16(b) is a graph showing the timing of the exposure time for each frame. Fig. 16(c) is a graph showing the motion blur correction angle calculated for each frame. Fig. 16(d) is a graph showing the optical axis change angle.

[0085] At a certain point in time, a Hi signal indicating an imaging instruction is sent and the first image Im1 is captured. At this time, the speed detection device 3a detects the speed of the vehicle 3 and calculates the motion blur correction angle for the next frame. When the imaging of the first image Im1 is completed, the optical axis change instruction unit 71 of the control device 15B instructs the optical axis changing mechanism 12B to change the optical axis. While the optical axis is being changed, the mirror drive unit 43 drives the mirror 41 to the rotation start angle β1 in the motion blur correction direction. The dashed lines extending diagonally from FIG. 16(a) to (c) indicate that the motion blur correction amount and the rotation start angle β1 in the correction direction of the second captured image are determined based on the speed V0 at the time of exposure of the previous frame, and the motion blur correction amount and the rotation start angle β2 in the correction direction of the third captured image are determined based on the speed V1 at the time of exposure of the second frame. Note that FIG. 16(c) shows the blur correction angle, which is an optical angle, and therefore the mirror mechanical angle when mirror 41 is used is q=2 in equation (8) as described above, and is therefore half the rotation start angles β1 and β2 in the correction direction.

[0086] The mirror driver 43 starts to rotate the mirror 41 in the direction for blur correction at the rotation speed calculated by the correction mechanism rotation speed calculator 75. After the change of the optical axis is completed during the second time Tm2, ​​the control device 15B transmits a Hi signal indicating an image capture instruction to the camera controller 27 during the first time Tm1 during which the optical axis is fixed, and an image is captured. In this manner, during the image capture, while the blur correction mechanism 31 is driving, the optical axis change mechanism 12B is in a state where the change of the optical axis is completed. At this time, the second image is blur corrected at a motion blur correction angle θ1 according to the speed V0 at which the image immediately preceding it was captured, and the third image is blur corrected at a motion blur correction angle θ2 according to the speed V1 at which the second image was captured. The motion blur correction angle for the fourth image may be calculated according to the average speed of the speed V1 at which the second image was captured and the speed V2 at which the third image was captured.

[0087] In this way, the angle of view can be expanded while performing blur correction according to the moving speed of the vehicle 3, so that a high-resolution image can be acquired over a wide range.

[0088] (Embodiment 4) An imaging system 1C in embodiment 4 will be described with reference to Fig. 17, Fig. 12B, and Fig. 6. Fig. 17 is a block diagram showing the internal configuration of imaging system 1C in embodiment 4. Note that, in embodiment 1, Fig. 6 has been described as an example in which the image-capturing target surface is the road surface of road 4, but in embodiment 4, an example will be described in which the image-capturing target surface is the wall surface, particularly the ceiling, of tunnel 5.

[0089] The imaging system 1C in the fourth embodiment has a configuration in which a control device 15B of the imaging system 1B in the third embodiment is provided with a subject distance calculation unit 77. The imaging system 1C in the fourth embodiment calculates a blur correction amount corresponding to the subject distance that varies as the optical axis is varied. Apart from this point and the points described below, the imaging system 1C in the fourth embodiment has the same configuration as the imaging system 1B in the third embodiment.

[0090] The subject distance calculation unit 77 calculates the subject distance on the second optical axis 23ae when the optical axis is shifted from the first optical axis 23ad, which is the initial position, to the second optical axis 23ae. A method for calculating the subject distance on the second optical axis 23ae will be described with reference to Fig. 12B and Fig. 6. A second subject distance D2 to the imaging target in the state of the second optical axis 23ae in which the optical axis of the lens 23 is changed by the optical axis change angle Φ from the state of the first optical axis 23ad perpendicularly facing the imaging target, and a third subject distance D3 on the outer edge of the optical path of the angle of view a on the second optical axis 23ae are calculated as follows. The subject distance D3 is the length of a perpendicular line from the end of the optical path of the angle of view a to the extension of the imaging plane after the optical axis change.

[0091] The second object distance D2 is calculated by the following formula (10). D2=D1 / cos(Φ)...(10) formula For example, under the conditions of the first embodiment, the second object distance D2 is 1.706 [m].

[0092] The third object distance D3 is calculated by the following formula (11). D3=[D1 / cos{Φ+(a / 2)}]×sin(90-Φ)···(11) formula For example, under the conditions of the first embodiment, the third object distance D3 is 1.73 [m].

[0093] Using the second object distance D2 calculated in this way, the object magnification M2 on the second optical axis 23ae can be calculated, and by substituting this into equation (4), the pixel movement amount P2 on the second optical axis 23ae can be calculated. The mirror swing angle α can be calculated using equations (5) and (8) using this pixel movement amount P2.

[0094] Next, the operation of the imaging system 1C in the fourth embodiment will be described with reference to Fig. 18. Fig. 18 is a flowchart showing the imaging process in the fourth embodiment. The operation of the imaging system 1C in the fourth embodiment is the same as the operation of the imaging system 1B in the third embodiment, except that step S21 is added.

[0095] Steps S1 to S3, S5, S11, and S12 are the same as the operations of the imaging system 1C of the third embodiment, so the description will be omitted. After changing the optical axis in step S3, in step S21, the subject distance calculation unit 77 calculates the second subject distance D2 based on the optical axis change angle Φ, and sets the calculated second subject distance D2 as a new subject distance to the imaging target. In this way, the control device 15C calculates the second subject distance D2 from the imaging device 11B to the imaging target area that changes before and after the optical axis change based on the optical axis change angle Φ by the optical axis change mechanism 12B, and sets the mirror swing angle α, which is the blur correction amount for correcting the blur in the first direction, based on the second subject distance D2. This makes it possible to improve the accuracy of the correction mechanism swing angle (blur amount) calculated by the correction mechanism swing angle calculation unit 73 in step S11. This makes it possible to realize more accurate tracking, and blur correction can be performed with high accuracy.

[0096] (Embodiment 5) An image capturing system 1D according to the fifth embodiment will be described with reference to Fig. 19. Fig. 19 is a block diagram showing the internal configuration of the image capturing system 1D according to the fifth embodiment.

[0097] The imaging system 1D in the fifth embodiment has a configuration in which the control device of the imaging system 1C in the fourth embodiment is provided with a subject distance detection device 81. Other than this point and the points described below, the imaging system 1D in the fifth embodiment has the same configuration as the imaging system 1C in the fourth embodiment.

[0098] The subject distance detection device 81 detects the distance from the principal point of the lens 23 to the subject. The subject distance detection device 81 is, for example, a laser measurement device. Information on the subject distance detected by the subject distance detection device 81 is sent to the control device 15D. The correction mechanism swing angle calculation unit 73 of the control device 15D calculates the correction mechanism swing angle on the first optical axis 23ad based on the detected first subject distance D1.

[0099] The subject distance detection device 81 detects the distance from the principal point of the lens 23 to the subject, and can therefore accurately calculate the amount of blur correction even if the distance of the imaging device 11B from the wall surface 5a in the tunnel 5 is changed in accordance with the on-site situation. This makes it possible to easily adjust the angle of view of the imaging device 11B by adjusting the distance of the imaging device 11B to the subject.

[0100] Next, the operation of the imaging system 1D in the fifth embodiment will be described with reference to Fig. 20. Fig. 20 is a flowchart showing the imaging process in the fifth embodiment. The operation of the imaging system 1D in the fifth embodiment is the same as the operation of the imaging system 1C in the fourth embodiment except that step S1 is omitted and steps S31 and S32 are added.

[0101] In the imaging system 1D of the fifth embodiment, instead of measuring the object distance in advance, the object distance detection device 81 measures the first object distance D1 when the optical axis is at the first optical axis 23ad which is the initial position.

[0102] After changing the optical axis in step S3, in step S31, the control device 15D determines whether the optical axis is on the first optical axis 23ad, which is the initial position. If the control device 15D determines that the optical axis is on the first optical axis 23ad (Yes in step S31), in step S32, the subject distance detection device 81 detects the first subject distance D1, and the control device 15D sets this detection value as the subject distance on the first optical axis 23ad. This makes it possible to improve the accuracy of the correction mechanism swing angle calculated by the correction mechanism swing angle calculation unit 73 in step S11.

[0103] Furthermore, when the control device 15D determines that the optical axis is on the second optical axis 23ae (No in step S31), in step S21 the subject distance calculation unit 77 calculates the second subject distance D2 based on the optical axis change angle Φ, and sets the calculated second subject distance D2 as a new subject distance to the imaging target. The control device 15D calculates the mirror swing angle α based on the first subject distance D1 before the optical axis change and the optical axis change angle Φ. This makes it possible to improve the accuracy of the correction mechanism swing angle calculated by the correction mechanism swing angle calculation unit 73 in step S11.

[0104] In this way, the control device 15D detects the subject distance using the subject distance detection device 81 when the optical axis is in the state of the first optical axis 23ad before the optical axis is changed, and does not detect the subject distance using the subject distance detection device 81 after the optical axis is changed to the state of the second optical axis 23ae by the optical axis changing mechanism 12B. Therefore, by detecting the subject distance using the subject distance detection device 81 only when the optical axis is in the initial position, it is possible to prevent the oblique distance from being captured by the wall surface of the tunnel 5 directly above. Upward When the optical axis is changed to the vertical direction, it becomes possible to prevent erroneous detection of distance when, for example, a tall vehicle in the adjacent lane, such as a trailer, enters the subject distance detection range. Also, in the case of road surface imaging, changing the optical axis from directly below road surface imaging makes it possible to prevent erroneous detection of distance when a vehicle in the adjacent lane enters the subject distance detection range. Distance detection is possible by installing a rangefinder at a height higher than that of a typical vehicle with respect to the side wall of tunnel 5, but changing the optical axis downward similarly causes erroneous detection of distance when a vehicle in the adjacent lane enters the subject distance detection range. car Vehicles on the line can cause false detections, but this can be prevented.

[0105] Next, the operation of the imaging system 1D in the modified example of the fifth embodiment will be described with reference to Fig. 21. Fig. 21 is a flowchart showing an imaging process in the modified example of the fifth embodiment. The operation of the imaging system 1D in the modified example of the fifth embodiment is the operation of the imaging system 1D in the fifth embodiment with step S33 added.

[0106] In step S33, the control device 15D sets the optical axis change angle Φ according to the subject distance detected by the subject distance detection device 81. For example, when the subject distance becomes closer, the detection value in step S32 becomes smaller, and with the same optical axis change amount, the overlapping area of ​​the captured images in the second direction disappears, which may result in missed images. To avoid this, the optical axis change amount is optimally set based on the detected subject distance. Therefore, in step S21, the subject distance at the change position is calculated based on the set optical axis change amount. This allows the angle of view to be expanded by an amount that corresponds to the subject distance.

[0107] (Other embodiments) As described above, the above embodiment has been described as an example of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to this, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are appropriately performed. Therefore, other embodiments will be exemplified below.

[0108] In the above embodiment, the information on the moving speed V1 from the speed detection device 3a of the vehicle 3 is used, but this is not limiting. The imaging system 1 may include a speed detection device that detects the moving speed of the imaging system 1. The speed detection device may also use a GPS (Global Positioning System) system.

[0109] In the above embodiment, the imaging system 1 captures images of the walls above and below the vehicle 3. However, this is not limiting. The imaging system 1 may capture images of walls on the sides of the vehicle 3.

[0110] In the above embodiment, the moving body is a vehicle 3 such as an automobile. However, the moving body is not limited to a vehicle 3, and may be a vehicle that runs on the ground such as a train or a motorcycle, a ship that travels on the sea, or an air vehicle such as an airplane or a drone that flies in the sky. When the moving body is a ship, the imaging system 1 images the bottom surfaces of bridge piers and bridge girders, and structures constructed along the shore. When the moving body is a train, the position and wear of the overhead wires can be detected by imaging the overhead wires.

[0111] In the above embodiment, an image is captured using ambient light reflected from the imaging target area 9, but this is not limited to the above. Light may be irradiated from a moving body or an imaging system toward the imaging target area 9, and an image may be captured using reflected light of the irradiated light.

[0112] (Overview of the embodiment) (1) An imaging system of the present disclosure includes an imaging device disposed on a moving body, an optical axis changing mechanism for changing the optical axis of the imaging device into k optical axis states, where k is an integer equal to or greater than 2, so that when the imaging device captures an image while the moving body is moving in a first direction, the optical axis of the imaging device is changed from a state of a first optical axis of the imaging device when capturing a first image to a state of a second optical axis displaced in a second direction intersecting the first direction when capturing a second image, thereby sequentially changing the optical axis of the imaging device to the second direction, and a control device for operating the optical axis changing mechanism. The control device operates the optical axis changing mechanism between the timing of capturing the first image and the timing of capturing the second image, and between the timing of capturing the rth image and the timing of capturing the r+1th image after the imaging device captures an rth image that satisfies the condition 2≦r≦n for a total number of images, n.

[0113] This enables the imaging system to expand the imaging range in a direction intersecting the traveling direction of the moving object, thereby enabling the imaging system to capture images over a wide range.

[0114] (2) In the imaging system of (1), the control device operates the optical axis changing mechanism to fix the optical axis of the imaging device for a first time based on the imaging interval and exposure time between the rth and r+1th images when capturing the rth and r+1th images, and to change the optical axis of the imaging device for a second time based on the imaging interval and the first time between the imaging timing of the rth image and the imaging timing of the r+1th image.

[0115] (3) In the imaging system of (2), when r / k is not an integer, the optical axis of the imaging device is changed in the second direction from the time of taking the r-th image for the (r+1)th image, and when r / k is an integer, the optical axis changing mechanism changes the optical axis of the imaging device in a third direction opposite to the second direction from the time of taking the r-th image for the (r+1)th image.

[0116] (4) In the imaging system of (3), when r / k is an integer, the optical axis changing mechanism changes the optical axis of the imaging device to a state in which the first optical axis is displaced in the third direction when the r+1th image is taken relative to the rth image.

[0117] (5) In the imaging system of (4), the r+kth captured image is an image of an imaging target area located in the first direction from the imaging target area when the rth captured image is captured, and an end area on the opposite side to the first direction in the r+kth captured image overlaps with an end area in the first direction in the rth captured image.

[0118] (6) In the imaging system of (4) or (5), a speed detection device is provided that detects the moving speed of the moving object, and the control device calculates the imaging interval between the rth image and the r+kth image based on the moving speed of the moving object, thereby defining the imaging interval in the first direction.

[0119] In the imaging system of (7)(6), the control device calculates the imaging interval between the rth image and the r+1th image based on the moving speed and the number k of optical axes, thereby defining the imaging interval in the second direction.

[0120] (8) In any of the imaging systems (2) to (7), when r / k is not an integer, the optical axis change angle by which the optical axis of the imaging device is changed when taking the rth image to the r+1th image is smaller than the angle of view of the imaging device.

[0121] (9) In the imaging system of (3), when r / k is not an integer, the optical axis change angle by which the optical axis of the imaging device is changed when capturing the rth image to the r+1th image is smaller than the angle of view of the imaging device, and the rth captured image and the r+kth captured image have a common imaging area, so that the rth captured image, the r+1th captured image, and the r+kth captured image each have a common imaging area.

[0122] (10) In any of the imaging systems (2) to (9), the control device has a first mode in which the imaging device captures images while operating an optical axis changing mechanism between the imaging timing of the first image and the imaging timing of the second image and between the imaging timing of the rth image and the imaging timing of the r+1th image, and a second mode in which the imaging device captures images continuously without operating the optical axis changing mechanism.

[0123] (11) In the imaging system according to any one of (2) to (10), a blur correction mechanism is provided that corrects blur in a first direction when an image is captured by the imaging device while the moving object is moving.

[0124] (12) In the imaging system of (11), during imaging, while the shake correction mechanism is being driven, the optical axis changing mechanism has completed changing the optical axis.

[0125] (13) In the imaging system of (11) or (12), the optical axis of the image blur correction mechanism is changed together with the optical axis of the imaging device by driving the optical axis changing mechanism.

[0126] (14) In any of the imaging systems of (2) to (13), the imaging device is installed with an installation inclination such that the imaging direction of the imaging device is inclined around an axis intersecting the movement direction by a predetermined angle smaller than the optical axis changing angle of the optical axis changing mechanism with respect to the imaging target of the imaging device.

[0127] In the imaging system of (15)(14), the installation inclination is half the optical axis changing angle.

[0128] (16) In the imaging system of (12) or (13), the control device calculates a subject distance from the imaging device to the imaging target area, which changes before and after the optical axis is changed, based on an optical axis changing angle by the optical axis changing mechanism, and sets a blur correction amount for correcting blur in the first direction, based on the subject distance.

[0129] In the imaging system of (17) and (16), the control device calculates the amount of blur correction based on the subject distance before the optical axis is changed and the optical axis change angle.

[0130] (18) The imaging system of (17) further comprises a subject distance measuring device for measuring a distance from the imaging device to an object to be imaged.

[0131] In the imaging system of (19)(18), the control device performs subject distance detection using the subject distance measuring device in the state of the first optical axis before the optical axis is changed, and does not perform subject distance detection using the subject distance measuring device after the optical axis is changed to the state of the second optical axis by the optical axis changing mechanism.

[0132] In the imaging system of (20) or (19), the control device sets the optical axis changing angle in accordance with the subject distance detected by the subject distance measuring device.

[0133] (21) A moving object according to the present disclosure includes an imaging system according to any one of (1) to (20). This allows the imaging system to expand the imaging range while the moving object is moving, and capture images over a wide range. [Industrial Applicability]

[0134] The present disclosure is applicable to an imaging system installed on a moving vehicle. [Explanation of symbols]

[0135] 1, 1A, 1B, 1C, 1D Imaging System 3. Vehicle 3a Speed ​​detection device 4 road 4b Hole 4c Cracks 5. Tunnels 5a Wall 5b hole 5c Cracks 7 Control section 9. Imaging area 11 Imaging device 12, 12B Optical axis changing mechanism 15 Control device 17 Memory section 19 Control section 21 Camera body 23 Lens 23a Optical axis 23ab, 23ad 1st optical axis 23ac, 23ae 2nd optical axis 24 Shutter 25 Image sensor 27 Camera control unit 31 Image stabilization mechanism 41, 41B Mirror 43 Mirror drive unit 45 Arm 51 Maximum exposure time calculation unit 53 Exposure time setting unit 61 Base 63 Rotation drive unit 71 Optical axis change instruction unit 73 Correction mechanism swing angle calculation section 75 Correction mechanism rotation speed calculation unit 81 Subject distance detection device α Mirror swing angle F focal length C1 First state C2 Second state M Subject magnification LE1 Extension of the lens principal point LE2 Extension of the imaging surface Φ Optical axis change angle Tf Imaging interval V1, V2, V3 movement speed

Claims

1. An imaging device disposed on a moving object; an optical axis changing mechanism for changing the optical axis of the imaging device into k optical axis states, k being an integer equal to or greater than 2, so that, when the imaging device captures an image while the moving object is moving in a first direction, the optical axis of the imaging device is changed from a state of a first optical axis of the imaging device when capturing a first image to a state of a second optical axis displaced in a second direction intersecting the first direction when capturing a second image, thereby sequentially changing the optical axis of the imaging device to the second direction; A control device that operates the optical axis changing mechanism, The imaging device is installed with an installation inclination such that a predetermined optical axis among the k optical axes is inclined at a predetermined angle smaller than an optical axis changing angle of the optical axis changing mechanism with respect to a direction from an installation position of the imaging device to an imaging target area in a plane including the k optical axes. Imaging system.

2. The installation inclination is half the angle of the optical axis change angle. The imaging system according to claim 1 .

3. The control device includes: The optical axis changing mechanism is operated between the imaging timing of the first image and the imaging timing of the second image, and between the imaging timing of the rth image and the imaging timing of the r+1th image after the imaging device captures an rth image that satisfies a condition of 2≦r≦n in a total number of images n, In the imaging of the r-th image and the r+1-th image, the optical axis of the imaging device is fixed for a first time based on an imaging interval between the r-th image and the r+1-th image of the imaging device and an exposure time, and the optical axis of the imaging device is changed for a second time based on the imaging interval and the first time between the imaging timing of the r-th image and the imaging timing of the r+1-th image, by operating the optical axis changing mechanism. The imaging system according to claim 1 .

4. The optical axis changing mechanism includes: When r / k is not an integer, the optical axis of the imaging device is changed in the second direction from the time of capturing the r-th image for capturing the r+1-th image; When r / k is an integer, the optical axis of the imaging device is changed in a third direction opposite to the second direction from the time when the r-th image is captured, for the image capture of the r+1-th image, to change the optical axis of the imaging device to the state of the first optical axis. The imaging system according to claim 3 .

5. The optical axis changing mechanism comprises: when (r+k) / k is an integer, changing the optical axis of the imaging device to the first optical axis state so that an end region on an opposite side to the first direction in the r+k+1th captured image overlaps with an end region in the r+1th captured image in the first direction; The r+k+1th captured image is an image of an imaging target area located in the first direction from the imaging target area when the r+1th captured image is captured. The imaging system according to claim 4 .

6. a speed detection device for detecting a moving speed of the moving object, the control device calculates an interval between capturing an r-th image and capturing an r+k-th image based on a moving speed of the moving object. The imaging system according to claim 5 .

7. the control device calculates an interval between the rth image capture and the r+1th image capture based on the moving speed and the number k of the optical axes. The imaging system according to claim 6 .

8. When r / k is not an integer, the optical axis change angle by which the optical axis of the imaging device is changed from the rth image to the r+1th image is equal to or smaller than the angle of view of the imaging device. The imaging system according to claim 3 .

9. When r / k is not an integer, an optical axis change angle by which the optical axis of the imaging device is changed when capturing an r-th image to an r+1-th image is equal to or smaller than an angle of view of the imaging device, and the r-th captured image and the r+k-th captured image have a common imaging area, so that the r-th captured image, the r+1-th captured image, and the r+k-th captured image each have a common imaging area. The imaging system according to claim 4 .

10. The control device includes: a first mode in which the optical axis changing mechanism is operated between the imaging timing of the first image and the imaging timing of the second image and between the imaging timing of the r-th image and the imaging timing of the r+1-th image to cause the imaging device to capture an image; a second mode in which the optical axis changing mechanism is not operated and the imaging device is caused to capture an image; Equipped with The imaging system according to claim 3 .

11. a blur correction mechanism that corrects blur in the first direction when the moving object is moving and the imaging device is used to capture an image; The imaging system according to claim 3 .

12. during image capture, while the image blur correction mechanism is driven, the optical axis changing mechanism is in a state where the changing operation of the optical axis has ended, By driving the optical axis changing mechanism, the optical axis of the image blur correction mechanism is also changed together with the optical axis of the imaging device. The imaging system according to claim 11.

13. the control device calculates a subject distance from the imaging device to an imaging target area that changes before and after the optical axis is changed based on an optical axis change angle by the optical axis changing mechanism, and sets a blur correction amount for correcting blur in the first direction based on the subject distance. The imaging system according to claim 11.

14. the control device calculates a blur correction amount based on the subject distance before the optical axis is changed and the optical axis change angle; The imaging system according to claim 13.

15. a subject distance measuring device for measuring a distance from the imaging device to an imaging object; The imaging system of claim 14.

16. the control device performs subject distance detection by the subject distance measurement device in a state of the first optical axis before the optical axis is changed, and does not perform subject distance detection by the subject distance measurement device after the optical axis is changed to the state of the second optical axis by the optical axis changing mechanism. The imaging system of claim 15.

17. the control device sets an optical axis change angle in accordance with the object distance detected by the object distance measuring device. The imaging system of claim 16.

18. An imaging device disposed on a moving object; an optical axis changing mechanism for changing the optical axis of the imaging device into k optical axis states, k being an integer equal to or greater than 2, so that, when the imaging device captures an image while the moving object is moving in a first direction, the optical axis of the imaging device is changed from a state of a first optical axis of the imaging device when capturing a first image to a state of a second optical axis displaced in a second direction intersecting the first direction when capturing a second image, thereby sequentially changing the optical axis of the imaging device to the second direction; A control device that operates the optical axis changing mechanism, The control device includes: a first mode in which the optical axis changing mechanism is operated between the imaging timing of the first image and the imaging timing of the second image, and between the imaging timing of the rth image and the imaging timing of the r+1th image after the imaging device has captured an rth image that satisfies a condition of 2≦r≦n in a total number of images n, to cause the imaging device to capture an image; a second mode in which the optical axis changing mechanism is not operated and the imaging device is caused to capture an image; Equipped with Imaging system.

19. An imaging device disposed on a moving object; an optical axis changing mechanism for changing the optical axis of the imaging device into k optical axis states, k being an integer equal to or greater than 2, so that, when the imaging device captures an image while the moving object is moving in a first direction, the optical axis of the imaging device is changed from a state of a first optical axis of the imaging device when capturing a first image to a state of a second optical axis displaced in a second direction intersecting the first direction when capturing a second image, thereby sequentially changing the optical axis of the imaging device to the second direction; A control device that operates the optical axis changing mechanism; a blur correction mechanism that corrects blur in the first direction when the moving object is moving and the imaging device is used to capture the image, the control device calculates a subject distance from the imaging device to an imaging target area that changes before and after the optical axis is changed based on an optical axis change angle by the optical axis changing mechanism, and sets a blur correction amount for correcting blur in the first direction based on the subject distance. Imaging system.

20. A moving object comprising an imaging system according to any one of claims 1 to 19.