Imaging system and mobile object provided with same

Abstract

This imaging system is provided with: an imaging device that is disposed on a mobile object to perform imaging in a first imaging position and a second imaging position that have different optical axes; an attitude detection device that detects an amount of change in the attitude of the mobile object; an optical axis modification mechanism that, when the mobile object is moving in the first direction, modifies the optical axis of the imaging device from a state of a first optical axis when the imaging device performs imaging in the first imaging position, to a state of a second optical axis that is inclined in a second direction intersecting the first direction when imaging is performed in the second imaging position; and a control device that, on the basis of the amount of change in attitude, sets an amount of optical axis modification by which the optical axis modification mechanism modifies the optical axis of the imaging device. The optical axis modification mechanism modifies the optical axis of the imaging device on the basis of the amount of optical axis modification that has been set.

Description

[Technical field]

[0001] The present disclosure relates to an imaging system that is fixed to a moving body and captures images while the moving body is moving, and to a moving body including the imaging system. [Background technology]

[0002] As transportation infrastructure ages, there is an increasing demand for infrastructure inspection. Instead of visual inspection by humans, the efficiency of inspection can be improved dramatically by capturing images of infrastructure facilities while moving around using a mobile vehicle and then detecting defects through image processing of the captured images.

[0003] For example, in Patent Document 1, a camera installed in a vehicle captures an image of a target area while moving. In addition, when the vehicle is traveling at a high speed, camera movement causes blurring. In Patent Document 1, however, a saccadic mirror technique is used to correct the blurring. by The camera corrects blurring by shining light onto the subject and reflecting the light off the subject onto a rotating mirror for a preset exposure time, which then enters the camera, reducing blurring. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2015 / 060181 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when capturing high-resolution images, a lens with a long focal length is used, which narrows the angle of view and therefore the range of images captured. If the vehicle itself changes position, the overlapping areas of the captured images may become separated, making it impossible to capture a continuous image.

[0006] The present disclosure provides an imaging system capable of expanding an imaging range and ensuring continuity of captured images, and a mobile body equipped with the imaging system. [Means for solving the problem]

[0007] The imaging system of the present disclosure includes a speed detection device that detects the moving speed of a moving body, an imaging device that is disposed on the moving body and captures images at a first imaging position and a second imaging position having different optical axes, an attitude detection device that detects an attitude change amount of the moving body, an optical axis changing mechanism that changes the optical axis of the imaging device from a state of a first optical axis when the imaging device captures images at the first imaging position to a state of a second optical axis displaced in a second direction intersecting the first direction when imaging images at the second imaging position while the moving body is moving in a first direction, and a control device that sets the optical axis change amount by which the optical axis changing mechanism changes the optical axis of the imaging device based on the amount of attitude change, and the optical axis changing mechanism changes the optical axis of the imaging device based on the set optical axis change amount.

[0008] The moving object of the present disclosure is equipped with the imaging system described above. Effect of the Invention

[0009] According to the imaging system and the moving body equipped with the imaging system of the present disclosure, it is possible to provide an imaging system and a moving body equipped with the imaging system that are capable of expanding the imaging range and ensure the continuity of captured images. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a side view illustrating a vehicle equipped with an imaging system according to a first embodiment. [Diagram 2] FIG. 1 is a front view illustrating a vehicle equipped with an imaging system according to a first embodiment. [Diagram 3] FIG. 1 is a block diagram showing the internal configuration of an imaging system according to a first embodiment. [Figure 4] FIG. 1 is an explanatory diagram for explaining the state of each imaging device in the state of two optical axes according to the first embodiment; [Diagram 5]FIG. 1 is an explanatory diagram showing an image captured in a first imaging mode; [Figure 6] FIG. 11 is an explanatory diagram showing an image captured in a second imaging mode; [Figure 7] FIG. 1 is an explanatory diagram illustrating an enlarged imaging range by an imaging device in the state of two optical axes. [Figure 8] 1 is a flowchart showing an image capture process according to the first embodiment. [Figure 9] Graph showing the relationship between the change in moving speed, the timing of exposure time, and the change in optical axis in the first embodiment. [Figure 10] FIG. 13 is a diagram for explaining a vehicle equipped with an imaging system according to a second embodiment. [Figure 11] FIG. 11 is an explanatory diagram for explaining the state of each imaging device in the state of two optical axes according to the second embodiment; [Figure 12] FIG. 11 is a block diagram showing the internal configuration of an imaging system according to a second embodiment. [Figure 13] FIG. 1 is an explanatory diagram for explaining blur correction in an imaging system; [Figure 14] 11 is a flowchart showing an image capture process according to the second embodiment. [Figure 15] Graph showing the relationship between the change in moving speed, the timing of exposure time, and the change in optical axis in the second embodiment. [Figure 16] FIG. 11 is a block diagram showing the internal configuration of an imaging system according to a third embodiment. [Figure 17] 11 is a flowchart showing an image capture process according to the third embodiment. [Figure 18] FIG. 13 is a block diagram showing the internal configuration of an imaging system according to a fourth embodiment. [Figure 19] FIG. 13 is a block diagram showing the internal configuration of an imaging system according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] (Embodiment 1) Hereinafter, the first embodiment will be described with reference to the drawings. In the first embodiment, a case will be described in which the moving body is a vehicle 3 such as an automobile, and the imaging system 1 is attached to the top of the vehicle 3. The imaging system 1 of the first embodiment is disposed to capture an image of a wall 5 erected beside a road, for example. The wall 5 is, for example, a soundproof wall or a tunnel wall.

[0012] [1-1. Imaging system configuration] Please refer to Figs. 1 to 3. Figs. 1 and 2 are diagrams for explaining an imaging system 1. Fig. 3 is a block diagram showing an internal configuration of the imaging system 1. In Figs. 1 and 2, a vehicle 3 is traveling, for example, on a road 4. A wall 5 erected on the side of the road 4 has, for example, a hole 5b and a crack 5c. These holes 5b and cracks 5c can be detected from a captured image by image processing.

[0013] The imaging target of the imaging system 1 is at least a part of the structure around the vehicle 3, and is an target that moves relatively in accordance with the moving speed of the vehicle 3 as the vehicle 3 moves. The imaging target area 9 is an area of ​​this imaging target that is acquired as an image. In addition to the wall 5, the imaging target may also be a road, the side or bottom of an overpass, a utility pole, or an electric wire. This makes it possible to detect holes, cracks, lifting, peeling, joints, tilt of a utility pole, and bending of an electric wire in the imaging target by image processing from the acquired image.

[0014] An imaging system 1 is installed on the upper surface of a vehicle 3. The imaging system 1 is fixed so as to capture an image of a wall 5 on the side of the vehicle 3 in FIG.

[0015] The imaging system 1 includes a speed detection device 3a, an imaging device 11, an optical axis changing mechanism 12, a control device 15, and an attitude detection device 33. The imaging device 11 captures an image of the surroundings of the vehicle 3, and in the first embodiment, captures an image of a wall surface 5a of a wall 5. The imaging device 11 includes a camera body 21, a lens 23, a shutter 24, an imaging element 25, and a camera control unit 27.

[0016] The speed detection device 3a is disposed on the vehicle 3 and detects the moving speed of the vehicle 3. This makes it possible to detect that the vehicle 3 is moving. In the first embodiment, the speed detection device 3a detects the moving speed based on a vehicle speed pulse signal in which a pulse signal is switched ON / OFF for every certain amount of rotation (rotation angle) of the axle of the vehicle 3. The speed detection device 3a sends the vehicle speed pulse signal together with the detected moving speed to the control device 15. In addition to this form, the speed detection device 3a may be, for example, a vehicle speed sensor that detects the moving speed from the rotation speed of the axle of the vehicle 3. The detection of the moving speed may be performed by the control device 15 based on the vehicle speed pulse signal.

[0017] Camera body 21 has replaceable lens 23 attached thereto, and houses image sensor 25 and camera control unit 27. Image sensor 25 is disposed at a position of focal length F of lens 23. Lens 23 is oriented directly toward wall 5, which is the subject. Camera body 21 and lens 23 may be integrated. Image sensor 25 converts received light into an electrical signal according to its intensity, and is, for example, a solid-state image sensor such as a CCD image sensor, a CMOS image sensor, or an infrared image sensor.

[0018] The camera control unit 27 opens the shutter 24 while receiving an exposure instruction signal from the control device 15. The shutter 24 may be configured with a multiple blade diaphragm that opens and closes, or may be an electronic shutter. The camera body 21 is supported on a base 61. The base 61 is supported on the upper surface of the vehicle 3 so as to be rotatable about the traveling direction of the vehicle 3 as a rotation axis. The control device 15 may generate an exposure instruction signal based on the vehicle speed pulse signal received from the speed detection device 3a and transmit the signal to the camera control unit 27.

[0019] When the imaging device 11 captures an image while the vehicle 3 is moving in a first direction that is the +X-axis direction, the optical axis changing mechanism 12 changes the optical axis 23a of the lens 23 of the imaging device 11 from a state of a first optical axis 23ab that is perpendicular to the wall when capturing a first image to a state of a second optical axis 23ac (see FIG. 4) that is inclined in a second direction that is the +Z-axis direction that intersects with the first direction when capturing a second image. The optical axis changing mechanism 12 is capable of changing the optical axis 23a of the lens 23 to the same direction as the roll direction of the vehicle 3. The optical axis changing mechanism 12 includes a base 61 and a rotation drive unit 63.

[0020] The base 61 supports the camera body 21. The imaging device 11 may have a configuration in which the camera body 21, the lens 23, and the optical axis changing mechanism 12 are integrated together.

[0021] The rotation drive unit 63 drives the base 61 to rotate based on a rotation instruction from the control device 15. The rotation drive unit 63 is composed of, for example, a motor and a gear. The camera body 21 also rotates in conjunction with the rotation of the base 61. Note that the optical axis changing mechanism 12 may include, for example, a rotation stage, and the imaging device 11 may be rotated by the rotation stage.

[0022] Please refer to FIG. 4. FIG. 4 is an explanatory diagram for explaining the imaging device 11 in each state of two types of optical axes, FIG. 4(a) is an explanatory diagram showing the imaging device 11 in a first imaging state C1 in which the lens 23 is in a state of the first optical axis 23ab, and FIG. 4(b) is an explanatory diagram showing the imaging device 11 in a second imaging state C2 in which the lens 23 is in a state of the second optical axis 23ac. By driving the rotation drive unit 63, the imaging device 11 can be displaced to the first imaging state C1 shown in FIG. 4(a) and the second imaging state C2 shown in FIG. 4(b). The optical axis changing mechanism 12 rotates the imaging device 11, for example, with the principal point 23b of the lens 23 as the rotation center. As a result, as shown in FIG. 5, the imaging range of the imaging device 11 can be expanded in the second direction, which is the +Z direction intersecting with the first direction, when capturing the second image.

[0023] For example, when the imaging device 11 is in the first imaging state C1, it captures the first image Im1 (see FIG. 5), and after the imaging of the image Im1 is completed, the imaging device 11 changes to the second imaging state C2 to capture the image Im2. After the imaging of the image Im2 is completed, the optical axis 23a of the lens 23 is tilted to the state of the first optical axis 23ab in the third direction, which is the -Z axis direction, so that the imaging device 11 changes to the first imaging state C1 to capture the image Im3. In this way, the imaging device 11 captures the wall 5 while alternately changing between the first imaging state C1 and the second imaging state C2, and captures the fourth image Im4, the fifth image Im5, and the sixth image Im6, so that the wall 5 can be captured over a wider range. The mode in which the imaging device 11 is changed and captured in this way is called the first imaging mode.

[0024] In the first imaging mode, the end region Im3a on the opposite side to the moving direction in the third captured image Im3 is captured so as to overlap with the end region Im1a in the moving direction in the first captured image Im1. Also, the end region Im1b in the second direction (+Z axis direction) in the first captured image Im1 is captured so as to overlap with the end region Im2b in the third direction (-Z axis direction) in the second captured image Im2. The third direction is the opposite direction to the second direction. Also, the end region Im2b in the third direction (-Z axis direction) in the second captured image Im2 is captured so as to overlap with the end region Im3b in the second direction (+Z axis direction) in the third captured image Im3. In this way, the first captured image Im1 and the second captured image Im2 have a common imaging area, and further, the first captured image Im1, the second captured image Im2, and the third captured image Im3 each have a common imaging area. Similarly, by capturing images Im4, Im5, and Im6 in sequence, adjacent images can have overlapping imaging areas. Also, the wall 5 may be captured without any gaps in the adjacent images. In either case, it is possible to prevent missing images between the images.

[0025] The imaging system 1 may also have a second imaging mode in which images Im1 to Im6 are captured while maintaining the first imaging state C1 without changing the imaging device 11. In this case, as shown in FIG. 6, captured images can be acquired continuously in a line. Note that in FIG. 6, the images Im1 to Im6 have different widths in the Z-axis direction for ease of understanding, but the actual widths are all the same. In the second imaging mode, for example, the imaging area of ​​image Im2 is entirely included in the imaging areas of images Im1 and Im3, so image Im2 does not need to be captured. In other words, images Im1, Im3, and Im5 may be captured in sequence.

[0026] The amount of expansion of the imaging range in the first imaging mode will be described with reference to Fig. 7. Fig. 7 is an explanatory diagram for explaining the amount of expansion of the imaging range, Fig. 7(a) is an explanatory diagram showing the state before the optical axis is changed (Φ=0°), and Fig. 7(b) is an explanatory diagram showing the state after the optical axis is changed (Φ=5°).

[0027] The following describes the distance between an extension line LE1 (extension line of the imaging plane) in the Z-axis direction of the principal point of the lens 23 and an extension line LE2 in the Z-axis direction of the imaging target surface 9a. In the first embodiment, the imaging target surface 9a is the wall surface of the wall 5.

[0028] If the size of the vertical image sensor 25 is 7.03 mm, the focal length F of the lens 23 is 35 mm, and the first object distance D1, which is the distance to the wall 5 when the optical axis of the image capture device 11 is at the initial position (optical axis change angle Φ=0), is 1.7 m, the angle of view a is 11.47 deg, and the vertical shooting range W1 (=2×WL1) is 0.34 m. The vertical shooting range W1 is calculated by the following formula (1). W1=2×WL1=2×D1×tan(a / 2)···(1) formula

[0029] The amount of magnification WL2 is calculated by the following formula (2) using the optical axis change angle Φ. The optical axis change angle Φ is the angle rotated from the first optical axis 23ab due to the optical axis change. WL2=D1×tan{(Φ+(a / 2)}-D1×tan(a / 2)...Equation (2) When the optical axis change angle Φ is 5[deg], the enlargement amount WL2 is 0.15[m], and the imaging range is enlarged vertically by approximately 44%.

[0030] Here, the optical axis change angle Φ≦angle of view a. This allows an image captured with the first optical axis 23ab and an image captured with the second optical axis 23ac to be continuous without a gap or to have an overlapping area. The optical axis change angle used for the first optical axis change to expand the imaging range of the captured image is set to Φ1. The first optical axis change angle Φ1 may be a predetermined angle designated by the user from the operation unit 19, or may be determined by the control device 15 based on the distance to the imaging target area and the angle of view a of the imaging device 11.

[0031] Please refer to Fig. 3. The attitude detection device 33 detects an attitude change amount in the expansion direction of the imaging range with respect to a reference attitude of the vehicle 3. The attitude detection device 33 is, for example, a gyro sensor or an acceleration sensor. The detected attitude change amount is output to the control device 15. In the first embodiment, the attitude detection device 33 detects an attitude change amount γ in at least the roll direction of the vehicle 3 with respect to the horizontal. The attitude detection device 33 may detect an attitude change amount in the yaw direction or pitch direction other than the roll direction.

[0032] The control device 15 controls the exposure time of the imaging device 11 and the drive of the optical axis changing mechanism 12, and operates the optical axis changing mechanism 12 between the timing of capturing the first image and the timing of capturing the second image. In addition, the control device 15 sets the optical axis change amount by which the optical axis changing mechanism 12 changes the optical axis of the imaging device 11 based on the detected amount of change in attitude.

[0033] The control device 15 is a circuit that can be realized by semiconductor elements or the like. The control device 15 can be configured by, for example, a microcomputer, a CPU, an MPU, a GPU, a DSP, an FPGA, or an ASIC. The functions of the control device 15 may be configured by hardware alone, or may be realized by combining hardware and software. The control device 15 realizes a predetermined function by reading data and programs stored in the storage unit 17 and performing various arithmetic processing.

[0034] The control device 15 has an optical axis change instruction unit 71. The optical axis change instruction unit 71 instructs the rotation drive unit 63 of the optical axis change mechanism 12 to change the state of the optical axis in accordance with the amount of change in attitude of the vehicle 3 detected by the attitude detection device 33, for example, at the timing of receiving the speed detection of the vehicle 3 by the speed detection device 3a or at the imaging timing of the imaging device 11 at a constant frame rate. The optical axis change instruction unit 71 instructs two types of optical axis change: a first optical axis change for expanding the shooting range of the captured image, and a second optical axis change for offsetting the influence of the attitude change of the vehicle 3.

[0035] In the first optical axis change, the optical axis change instruction unit 71 instructs the optical axis changing mechanism 12 to change the optical axis in accordance with the imaging cycle. The optical axis change instruction unit 71 always instructs the second optical axis change except while the first optical axis change is being performed. When the attitude change amount in the roll direction of the vehicle 3 is input from the attitude detection device 33 to the control device 15, the optical axis change instruction unit 71 calculates the second optical axis change angle Φ2 that offsets the input attitude change amount in the roll direction, and instructs the optical axis changing mechanism 12 to rotate the optical axis in the roll direction by the second optical axis change angle Φ2. The second optical axis change angle Φ2 is an angle used in the second optical axis change to offset the effect of the attitude change of the vehicle 3. The second optical axis change angle Φ2 is an angle amount with the opposite sign to the attitude change amount. The optical axis change angle Φ is the sum of the first optical axis change angle Φ1 and the second optical axis change angle Φ2 (Φ=Φ1+Φ2). When the attitude change amounts in the yaw and pitch directions are also input to the control device 15, the optical axis change instruction unit 71 may further calculate an optical axis change angle that offsets the attitude change amount in the roll direction affected by the attitude change amounts in the yaw and pitch directions, and may instruct the optical axis changing mechanism 12 to further rotate the optical axis in the roll direction by this optical axis change angle.

[0036] The control device 15 transmits an exposure control signal to the camera control unit 27. The exposure control signal has two types of signals: a Hi signal as an ON signal that instructs exposure, and a Low signal as an OFF signal that does not instruct exposure. When capturing images at a constant frame rate, the camera control unit 27 may control the exposure instead of the control device 15.

[0037] The imaging system 1 also includes a storage unit 17 and an operation unit 19. The storage unit 17 is a storage medium that stores programs and data necessary to realize the functions of the control device 15. The storage unit 17 can be realized by, for example, a hard disk drive (HDD), an SSD, a RAM, a DRAM, a ferroelectric memory, a flash memory, a magnetic disk, or a combination of these.

[0038] The operation unit 19 is an input device for the user to give instructions to the control device 15. The operation unit 19 may be an input device dedicated to the imaging system 1, or may be a mobile terminal such as a smartphone. When a mobile terminal is used as the operation unit 19, the operation unit 19 and the control device 15 transmit and receive data by wireless communication. The user may use the operation unit 19 to instruct the control device 15 whether the imaging target area is a dark area indoors such as a tunnel or a bright area outdoors such as a mountain slope or a road, or may instruct the imaging interval Tf. The imaging interval Tf is the time between the timing when the imaging of the current image is completed and the timing when the imaging of the next image is completed, and is the time of one frame in the case of video imaging, and is the time interval between the imaging times of the images to be captured in the case of still image imaging. In the case of video imaging, the frame rate (the number of images captured per second) may be specified. In addition, the user may use the operation unit 19 to instruct the control device 15 to switch between a first imaging mode in which the optical axis changing mechanism 12 is operated and a second imaging mode in which the optical axis changing mechanism 12 is not operated.

[0039] [1-2. Operation of the imaging system] Next, the operation of the imaging system 1 will be described with reference to FIG. 8 and FIG. 9. FIG. 8 is a flowchart showing the imaging process performed by the imaging system 1. FIG. 9 is a graph showing the relationship between the exposure time and the timing of changing the optical axis. FIG. 9(a) is a graph showing the moving speed of the vehicle 3 that changes over time. The moving speeds V0, V1, V2, and V3 are detected according to the timing of detecting the vehicle speed. FIG. 9(b) is a graph showing the timing of the exposure time for each frame. As the imaging interval Tf, imaging intervals Tf1 and Tf2 for each frame are illustrated, and as the exposure time Tp, exposure times Tp1, Tp2, and Tp3 for each frame are illustrated. The exposure times Tp1, Tp2, and Tp3 are the times from the imaging start times t1, t3, and t5 to the imaging end times t2, t4, and t6, respectively. FIG. 9(c) is a graph showing the amount of change in the attitude of the vehicle 3 detected by the attitude detection device 33. FIG. 9(d) is a graph showing the second optical axis change angle Φ2 calculated by the control device 15. Fig. 9(e) is a graph showing the optical axis change angle Φ that the optical axis change instruction unit 71 commands the rotation drive unit 63. Fig. 9(f) is a graph showing the state of the optical axis of the imaging device 11B rotated by the rotation drive unit 63 as viewed from the imaging target area 9. The imaging process shown in Fig. 8 is started, for example, when an instruction to start imaging is given from the operation unit 19 while the vehicle 3 is moving.

[0040] In step S1, the user measures in advance the subject distance from the imaging element 25 of the camera body 21 to the surface of the wall 5 to be imaged, and sets the measured subject distance in the control device 15 using the operation unit 19. In addition, by setting the section of the road to be imaged, the control device 15 can determine, for example, based on GPS information and the traveled distance, whether the vehicle has traveled on the road in the set section.

[0041] In step S2, the vehicle 3 starts traveling, and the speed detection device 3a detects the moving speed of the vehicle 3. The detected moving speed is sent to the control device 15. In the first embodiment, the speed detection device 3a performs imaging in synchronization with the detection of the speed, but the imaging may be performed at a predetermined cycle, or the user may set the imaging cycle using the operation unit 19. When the exposure is synchronized with the vehicle speed pulse signal, for example, when the speed of the vehicle 3 is detected at intervals of 40 cm for a moving distance of 60 km / h, the frame rate becomes about 40 fps. In addition, the speed detection device 3a does not need to detect the exact moving speed, and may be configured to detect that the vehicle is moving. Then, the detected moving state is sent to the control device 15. The control device 15 may perform imaging after detecting that the vehicle 3 is in a moving state in addition to the instruction to start imaging from the operation unit 19 by the user.

[0042] In step S3, the attitude detection device 33 detects the amount of attitude change. As shown in Fig. 9(c), the attitude detection device 33 may constantly detect the amount of attitude change while the vehicle 3 is traveling. The detected amount of attitude change is transmitted from the attitude detection device 33 to the control device 15. The optical axis change instruction unit 71 of the control device 15 calculates a second optical axis change angle Φ2 that has an opposite sign to and is equal in magnitude to the input amount of attitude change γ (see Fig. 9(d)).

[0043] In step S4, if the number of images to be captured next is the second or more, the optical axis change instruction unit 71 of the control device 15 issues a rotation instruction to the rotation drive unit 63 of the optical axis change mechanism 12 as the first optical axis change. As a result, as shown in FIG. 9(e), the imaging device 11 is rotated by the first optical axis change angle Φ1a from the state of the first optical axis 23ab (Φ1=0[deg]) to the state of the second optical axis 23ac (Φ1=Φ1a[deg]) between time t2 and time ta, and the imaging range of the imaging device 11 is changed. In the case of the first captured image, the optical axis change instruction unit 71 does not need to issue a rotation instruction to the rotation drive unit 63 (Φ1=0[deg]). While the rotation drive unit 63 is changing the optical axis of the imaging device 11 by the first optical axis change (between time t2 and time ta, and between time t4 and time tb), the control device 15 does not issue an exposure instruction to the camera control unit 27.

[0044] Also, the optical axis change instruction unit 71 instructs the rotation drive unit 63 of the optical axis changing mechanism 12 to rotate (second optical axis change) in a direction that offsets the amount of change in attitude while the optical axis change (first optical axis change) for changing the imaging range is not in progress (from time 0 to time t2, from time ta to time t4, and from time tb to time t6). Therefore, the rotation drive unit 63 drives at an optical axis change angle Φ (Φ = Φ1 + Φ2) which is the sum of the first optical axis change angle Φ1 and the second optical axis change angle Φ2. As a result, as shown in FIG. 9(f), the optical axis can be maintained in the state of the first optical axis 23ab (Φ = 0) and the state of the second optical axis 23ac (Φ = Φ1a) even if the attitude of the vehicle 3 changes during the period from time 0 to time t2, from time ta to time t4, and from time tb to time t6 when the optical axis change operation is not in progress in the first optical axis change. Therefore, even if the attitude of the vehicle 3 changes during exposure, the desired changed state of the optical axis can be maintained, and the overlapping area of ​​the captured images can be secured. In this way, the optical axis can be changed in response to the change in the vehicle attitude in each state of the first optical axis 23ab and the second optical axis 23ac.

[0045] After the optical axis changing operation of the rotation drive unit 63 in the first optical axis change is completed at time ta, in step S5, the control device 15 continues to send, for example, a Hi signal to the camera control unit 27 of the imaging device 11 for the exposure time Tp2. As a result, in a state in which the second optical axis change is performed by the optical axis change mechanism 12, the imaging element 25 captures an image of the imaging target for the exposure time Tp2. The image captured by the imaging element 25 is recorded from the camera control unit 27 to the storage unit 17, thereby acquiring a captured image. In this way, the control device 15 does not instruct the second optical axis change while the optical axis change mechanism 12 is performing the first optical axis change (between time t2 and time ta), but instructs the second optical axis change while the imaging device 11 is exposed (from time t1 to time t2, from time t3 to time t4, and from time t5 to time t6).

[0046] In step S6, the control device 15 judges whether the vehicle 3 has traveled a predetermined section. When the control device 15 judges that the vehicle 3 has finished traveling the predetermined section, the control device 15 finishes acquiring images of the road in this section, and ends the moving image capturing. Alternatively, the control device 15 may end the moving image capturing in response to an instruction from the operation unit 19 by the user operating the operation unit 19. When the control device 15 judges that the vehicle 3 has not finished traveling the predetermined section, the control device 15 returns to step S2 and performs the moving image capturing again. As described above, when step S3 is performed and the third image is captured, in step S4, as shown in FIG. 9(e), the imaging device 11 is rotated by the first optical axis change angle -Φ1a from the state of the second optical axis 23ac (Φ1=Φ1a [deg]) to the state of the first optical axis 23ab (Φ1=0 [deg]) during the period from time t4 to time tb, and the imaging range of the imaging device 11 is changed. After that, from time tb to time t6, the first optical axis 23ab is driven at the optical axis change angle Φ (Φ=Φ1+Φ2). This enables the optical axis to be changed in response to changes in the vehicle attitude even in the state of the first optical axis 23ab. After that, steps S5 and S6 are performed in the same manner as described above.

[0047] [1-3. Effects, etc.] In this way, the imaging system 1 includes an imaging device 11 disposed in a vehicle 3, which captures images in a first imaging state C1 and a second imaging state C2 having different optical axes, an attitude detection device 33 which detects the attitude state of the vehicle 3, an optical axis changing mechanism 12 which changes the optical axis of the imaging device 11 from a state of a first optical axis 23ab when the imaging device 11 captures images in the first imaging state C1 to a state of a second optical axis 23ac inclined in a second direction intersecting with the first direction when the imaging device 11 captures images in the second imaging state C2 while the vehicle 3 is moving in a first direction, and a control device 15 which sets an optical axis change amount by which the optical axis changing mechanism 12 changes the optical axis of the imaging device 11 based on an attitude change amount. The optical axis changing mechanism 12 changes the optical axis of the imaging device 11 based on the set optical axis change amount.

[0048] In a moving vehicle 3, by changing the state of the imaging device 11 to two optical axes, the range of the captured image can be expanded in a direction intersecting the traveling direction. By changing the optical axis of the imaging device 11 in response to the change in the attitude state, it is possible to prevent the imaging range from shifting due to the attitude change of the imaging device 11 that changes together with the vehicle 3, and ensure the continuity of the captured images.

[0049] In addition, in the first optical axis change, the control device 15 causes the optical axis changing mechanism 12 to change the optical axis of the imaging device 11 from the state of the first optical axis 23ab to the state of the second optical axis 23ac, or from the state of the second optical axis 23ac to the state of the first optical axis 23ab, between two consecutive imaging timings, for example, between the imaging timing at the exposure time Tp1 and the imaging timing at the exposure time Tp2, or between the imaging timing at the exposure time Tp2 and the imaging timing at the exposure time Tp3. This makes it possible to obtain an image with a wider shooting range.

[0050] In addition, the control device 15 controls the optical axis changing mechanism 12 to change the optical axis of the imaging device 11 between the state of the first optical axis 23ab and the state of the second optical axis 23ac in the first optical axis change by a first optical axis change angle Φ1 as an optical axis change amount for expanding the imaging range of the captured image.

[0051] In addition, the control device 15 does not instruct the optical axis changing mechanism 12 to change the second optical axis while the optical axis changing mechanism 12 is performing the first optical axis change, but instructs the optical axis changing mechanism 12 to change the second optical axis after the optical axis changing mechanism 12 has completed the first optical axis change and before the optical axis changing mechanism 12 next performs the first optical axis change.

[0052] In addition, the attitude detection device 33 detects the amount of attitude change in the roll direction of the vehicle 3, and in the second optical axis change, the control device 15 changes the optical axis change amount to the sum of the second optical axis change angle Φ2 and the first optical axis change angle Φ1, which have the same magnitude and opposite sign as the attitude change amount γ detected by the attitude detection device 33.

[0053] (Embodiment 2) An image capturing system 1A according to the second embodiment will be described with reference to Figs. 10 to 15. Fig. 10 is an explanatory diagram for explaining a vehicle 3 equipped with the image capturing system 1A according to the second embodiment. Fig. 12 is a block diagram showing an internal configuration of the image capturing system 1A according to the second embodiment. Fig. 13 is a block diagram showing the internal configuration of the image capturing system 1A according to the second embodiment. Nobu FIG. 11 is an explanatory diagram for explaining deviation correction.

[0054] The imaging system 1A in the second embodiment is configured by providing the imaging system 1 in the first embodiment with a blur correction mechanism 31. With respect to this point and other points described below, the imaging system 1A in the second embodiment is common to the imaging system 1 in the first embodiment.

[0055] 10, when the vehicle 3 is traveling in a tunnel, for example, holes 5b and cracks 5c are also generated on the wall surface 5a inside the tunnel. When capturing an image in a dark environment such as a tunnel, if the exposure time is extended, the captured image will be blurred. Also, when capturing an image of a subject while the vehicle 3 is traveling at high speed, the captured image will be blurred. The blur correction mechanism 31 corrects the optical path of light incident on the imaging system 1A so that blurring of the image of the imaging target area 9 is reduced even when the imaging device 11A captures an image while the vehicle 3 is moving.

[0056] The camera body 21 is disposed on the vehicle 3 so that the orientation of the lens 23 is parallel to the direction of movement of the vehicle 3. For example, the camera body 21 is disposed so that the lens 23 faces forward or backward of the vehicle 3.

[0057] The blur correction mechanism 31 corrects the optical path of light L1, which is ambient light reflected by the imaging target area 9, in accordance with the movement of the vehicle 3. The blur correction mechanism 31 aligns the direction of light L1, which is ambient light reflected by the imaging target area 9, with the imaging direction of the imaging element 25. The blur correction mechanism 31 includes, for example, a mirror 41 and a mirror driver 43. The mirror 41 totally reflects the light, which is ambient light reflected by the imaging target, in the direction of the imaging device 11.

[0058] As shown in Fig. 11(a), when the imaging device 11 captures an image while the vehicle 3 is moving in a first direction that is the +X-axis direction, the optical axis changing mechanism 12A changes the optical axis 23a of the lens 23 of the imaging device 11 from a first optical axis 23ab that is perpendicular to the tunnel wall 5 when capturing the first image to a second optical axis 23ad that is inclined in a second direction that is the +Z-axis direction that intersects with the first direction when capturing the second image, as shown in Fig. 11(b). The rotation drive unit 63A of the optical axis changing mechanism 12A rotates both the blur correction mechanism 31 supported by the base 61 and the imaging device 11B around the rotation axis 63Aa.

[0059] The blur correction mechanism 31 and the optical axis changing mechanism 12A are not limited to this configuration, and when the imaging device 11A has a configuration in which the camera body 21 and the lens 23 are integrated, a pan-tilt rotation mechanism that rotates the camera body 21 and the lens 23 about a rotation axis may be used. In this case, the blur correction mechanism 31 corresponds to a mechanism that is driven to rotate in the pan direction, and the optical axis changing mechanism 12 corresponds to a mechanism that is driven to rotate in the tilt direction.

[0060] Whether the pan direction drive mechanism or the tilt direction drive mechanism corresponds to the blur correction mechanism 31 and the optical axis changing mechanism 12A may be appropriately changed depending on the orientation of the image sensor 25 and the direction of the image capture target relative to the traveling direction of the vehicle 3. In the second embodiment, when the traveling direction (+X axis direction) of the vehicle 3 on the captured image and the long side of the image sensor 25 are parallel to each other, the image capture area in the traveling direction of the vehicle 3 can be widened by placing importance on the overlapping area between the captured images. When the traveling direction (+X axis direction) of the vehicle 3 on the captured image and the short side of the image sensor 25 are parallel to each other, which is the case when the shooting range in the direction intersecting the traveling direction of the vehicle 3 is emphasized, the pan direction drive mechanism may correspond to the optical axis changing mechanism 12 and the tilt direction drive mechanism may correspond to the blur correction mechanism 31.

[0061] Furthermore, when the camera body 21 and the lens 23 have a mechanism for rotating in one axis direction, the mechanism for rotating in one axis direction may be used as the optical axis changing mechanism 12A. In this case, the blur correction mechanism 31 is composed of a mirror 41 and a mirror driving unit 43. The blur correction mechanism 31 and the optical axis changing mechanism 12A may also be composed of two mirrors and a motor whose respective rotation axes are perpendicular to each other. The entire imaging device 11A may also be rotated in two directions perpendicular to each other. A mechanism for driving the camera body 21 and the lens 23 to rotate in one axis direction may also be used as the optical axis changing mechanism 12A, and a mechanism for rotating the entire imaging device 11A in the pan direction may also be used as the blur correction mechanism 31.

[0062] The mirror 41 is rotatably disposed so as to face the lens 23. The mirror 41 can be rotated, for example, in either a forward or reverse clockwise direction, and the rotatable angular range may be less than 360 degrees or may be 360 ​​degrees or more. The mirror 41 totally reflects ambient light reflected by the imaging subject toward the imaging device 11. The mirror driving unit 43 drives the mirror 41 to rotate from an initial angle to a specified angle, and returns the mirror 41 to the initial angle after rotating it to the specified angle. The mirror driving unit 43 is, for example, a motor.

[0063] The rotation angle of the mirror 41 is limited by the mechanical constraints of the mirror drive unit 43, and the mirror 41 can be rotated up to the maximum swing angle of the mirror 41 determined by this constraint.

[0064] Referring to FIG. 13, the blur correction mechanism 31 Rub The blur correction will now be described. For example, it is assumed that the imaging system 1 located at position A moves to position B together with the vehicle 3 during the exposure time. It is assumed that imaging is started at position A and an image is acquired at this timing. For example, the image acquired at position A includes an image of a hole 5b in the imaging target area 9, but since the exposure time is insufficient, the image is dark and not clear.

[0065] Therefore, exposure is continued until the vehicle 3 moves to position B. In this case, if no blur correction is performed, the image capture target area 9 moves relatively in the opposite direction to the moving direction of the vehicle 3, resulting in an image in which the hole 5b moves relatively. In the image with continued exposure, the amount of movement of the pixels is detected as the amount of blur. In this way, the image captured by the imaging device 11 while the vehicle 3 is moving will be a blurred image.

[0066] Therefore, by rotating the mirror 41 in a direction in which the end 41a of the mirror 41 in the moving direction cancels the relative movement of the imaging target during the exposure time according to the moving speed of the imaging system 1A and the vehicle 3, the imaging system 1 can capture the same imaging target area 9 in the captured image during the exposure time, and obtain an image with significantly reduced blur. In FIG. 13, the mirror 41 is rotated clockwise so that the end 41a of the mirror 41 in the moving direction rotates toward the imaging target during the exposure time. By rotating the mirror 41, the amount of movement of pixels in the captured image is corrected to zero.

[0067] 12, the control device 15A includes an optical axis change instruction unit 71, a swing angle calculation unit 73, and a rotation speed calculation unit 75.

[0068] The swing angle calculation unit 73 calculates the mirror swing angle α of the mirror 41 during imaging based on the moving speed V of the vehicle 3, the set exposure time Tp, the subject magnification M, and the focal length F of the lens 23 in the following manner. The mirror swing angle α corresponds to the correction mechanism swing angle.

[0069] The focal length F is a value determined by the lens 23. The subject magnification M is a value determined by the focal length F and the subject distance. The subject distance is the distance from the principal point 23b of the lens 23, which is disposed between the subject, that is, the imaging target, and the imaging element 25, to the imaging target. The subject distance may be a known value measured in advance, or may be a value measured by a rangefinder during imaging.

[0070] The amount of movement L of the vehicle 3 during the exposure time Tp from the start time of imaging to the end time of imaging is calculated from the moving speed V and the exposure time Tp by the following formula (3). L [mm] = V [km / h] x 10 6 ×Tp[ms] / (60 2 ×10 3 )...Equation (3)

[0071] The amount of movement P of the pixels on the imaging element 25 from the start time of imaging to the end time of imaging is calculated from the amount of movement L of the vehicle 3 and the subject magnification M by the following formula (4). P [mm] = L [mm] × M (4)

[0072] The amount of movement of this pixel P Gobble This can cause , In order to prevent this, the pixel shift amount P is Tebu The optical path of the light incident on the lens 23 is changed by the deflection correction angle θ. . The image distortion correction angle θ is calculated from the pixel movement amount P and the focal length F by the following formula (5). θ[deg]=arctan(P / F)...Equation (5) As described above, the subject magnification M is calculated from the focal length F [mm] and the subject distance D [m] according to the following formula (6). M = F / (D×10 3 )...Equation (6) From equations (3), (4), (5), and (6), θ = arctan(V×10 6 ×Tp / (60 2 ×10 3 ) / (D×10 3 )) = arctan(V×Tp / (D×60 2 ))...Equation (7) Like this , The deviation correction angle θ is calculated from the moving speed V, the exposure time Tp, and the subject distance D.

[0073] During exposure Nobu The mirror angle α required for the correction is ,Since it is half the size of the deflection correction angle θ, it is calculated by the following formula (8). α=θ / k (8) Here, k is the mirror swing angle α as the mechanical swing angle of the drive mechanism and the optical correction angle by which the light incident on the lens 23 is corrected. Nobu k is a conversion coefficient for the deflection correction angle θ. In the case of a configuration in which light from the image capture target travels through mirror 41, lens 23, and image sensor 25 in this order, as in the embodiment of Fig. 12, k = 2. In the case of a configuration in which the pan / tilt mechanism and the entire camera are driven, k = 1.

[0074] In this manner, the swing angle calculation unit 73 calculates the mirror swing angle α of the mirror 41.

[0075] The rotation speed calculation unit 75 calculates the rotation speed Vm of the mirror 41 during the exposure period based on the mirror swing angle α and the exposure time Tp using the following formula. Vm = α / Tp1 (9)

[0076] Therefore, after starting imaging, the mirror 41 is rotated in the opposite direction to the moving direction at a rotation speed Vm, so that the imaging device 11A can receive light from the same imaging target area 9 during the exposure time. Nib The occurrence of leakage can be suppressed.

[0077] Next, the operation of the imaging system 1A will be described with reference to Fig. 14 and Fig. 15. Fig. 14 is a flowchart showing the imaging process performed by the imaging system 1A. Fig. 15 shows the exposure time Fly 15 is a graph showing the relationship between the blur correction angle and the optical axis change angle. FIG. 15(a) is a graph showing the moving speed of the vehicle 3 which changes over time. FIG. 15(b) is a graph showing the timing of the exposure time for each frame. FIG. 15(c) is a graph showing the time when the blur correction mechanism 31 rotates the optical axis for blur correction. Rub15(a) is a graph showing the change in the deviation correction angle over time. Fig. 15(d) is a graph showing the amount of change in attitude of the vehicle 3 detected by the attitude detection device 33. Fig. 15(e) is a graph showing the amount of optical axis change commanded by the optical axis change command unit 71 to the rotation drive unit 63. Fig. 15(f) is a graph showing the position of the optical axis of the imaging device 11A rotated by the rotation drive unit 63. The imaging process shown in Fig. 14 is started, for example, when an instruction to start imaging is given from the operation unit 19 while the vehicle 3 is moving.

[0078] Steps S1 to S4 are the same as the operations of the imaging system 1 of the first embodiment, so the description will be omitted. In step S11, the swing angle calculation unit 73 calculates the mirror swing angle α as a shake correction amount. In addition, the rotation speed calculation unit 75 calculates the rotation speed Vm of the mirror 41 based on the mirror swing angle α.

[0079] In step S12, the control device 15A causes the mirror driver 43 to rotate the mirror 41 at the calculated rotation speed Vm, so that the mirror 41 starts to rotate from a predetermined initial angle. Nobu At the same time, the control device 15A also continues to send an optical axis change signal corresponding to the change in vehicle attitude, so that the change in attitude of the imaging device caused by the change in vehicle attitude is also offset, and the optical axis position is maintained at a constant position. In this state, the control device 15A continues to send a Hi signal instructing exposure to the camera control unit 27 for the exposure time Tp.

[0080] In the imaging device 11A, the camera control unit 27 opens the shutter 24 to expose the camera while receiving the Hi signal, thereby acquiring an image, and stores the acquired image in the storage unit 17. When the exposure time Tp has elapsed, the control device 15A continues to send a Low signal as an OFF signal that instructs the camera to stop exposure to the camera control unit 27. Note that the Low signal may be used as an ON signal that instructs the camera to start exposure, and the Hi signal may be used as an OFF signal that instructs the camera to stop exposure.

[0081] While the camera control unit 27 is receiving the Low signal, the control device 15A closes the shutter 24, and the control device 15A causes the mirror driving unit 43 to rotate the mirror 41 in the reverse direction to return the mirror 41 to the initial angle. Note that the mirror driving unit 43 may also rotate the mirror 41 in the forward direction to return the mirror 41 to the initial angle.

[0082] When images are captured continuously, the operation of the imaging system 1A after returning to step S2 will be described with reference to FIG.

[0083] In synchronization with the timing when the vehicle speed is detected, a Hi signal indicating an image capturing instruction is transmitted and the first image is captured. At this time, the speed of the vehicle 3 detected by the speed detection device 3a is used to determine the next frame. Nobu The blur correction angle is calculated. When the capture of the first image is completed, the optical axis change instruction unit 71 of the control device 15A instructs the optical axis changing mechanism 12A to change the optical axis as the first optical axis change. While the optical axis is being changed, the mirror drive unit 43 drives the mirror 41 to the rotation start angle β1, which is the rotation start position in the blur correction direction. Since the first optical axis change and the drive of the mirror 41 to the rotation start angle only need to be completed before the start of exposure, the timing of the completion of the first optical axis change and the timing of the drive of the mirror 41 to the rotation start angle may be different. Note that FIG. 16(c) shows the blur correction angle, which is an optical angle, and therefore the mirror mechanical angle when the mirror 41 is used is k=2 in the formula (8) as described above, and therefore is half the size of each of the rotation start angles β1 and β2 in the correction direction.

[0084] When the optical axis is driven by the first optical axis change angle Φ1 from the state of the first optical axis 23ab to the state of the second optical axis 23ad, the optical axis change instruction unit 71 instructs the rotation drive unit 63 of the optical axis changing mechanism 12A to rotate (change the second optical axis) in a direction that offsets the amount of change in the attitude of the imaging device caused by the change in the vehicle attitude. This makes it possible to maintain the optical axis in the state of the second optical axis 23ad even if the attitude of the vehicle 3 changes, as shown in FIG. 15(f).

[0085] In this state, the mirror driver 43 starts rotating the mirror 41 at the rotation speed calculated by the rotation speed calculator 75. After the first optical axis change is completed, while the second optical axis change is being performed, the control device 15A transmits a Hi signal indicating an image capture instruction to the camera controller 27, and the image capture device 11A captures an image. At this time, the second image is captured according to the speed V0 at which the previous image was captured. Tabu The third image is corrected by the image blur correction angle θ1, and the third image is taken according to the speed V1 when the second image was taken. Tabu The image blur is corrected by the image blur correction angle θ2. The image blur correction angle θ2 is set according to the average speed of the image blur correction angle V1 when the second image is captured and the image blur correction angle V2 when the third image is captured. Nobu A deviation correction angle may be calculated.

[0086] In this way, the imaging system 1A of the second embodiment includes a speed detection device 3a that detects the moving speed of the vehicle 3, and a first direction that is the moving direction of the vehicle 3 when the vehicle 3 is imaged by the imaging device 11A while moving. Nobu The control device 15A controls the blur correction mechanism 31 based on the moving speed. Rub For correction Nobu Set the correction angle and adjust the amount of change in the optical axis. Tebu Change the correction angle.

[0087] Since the imaging range can be expanded in a direction intersecting the traveling direction while performing blur correction in the moving direction according to the moving speed of the vehicle 3, it is possible to acquire images with high resolution and a wide range. In addition, since the attitude of the imaging device 11A is corrected according to the change in the attitude of the vehicle 3 in the direction in which the imaging range is expanded, continuity between multiple captured images can be ensured.

[0088] (Embodiment 3) An image capturing system 1B according to the third embodiment will be described with reference to Fig. 16. Fig. 16 is a block diagram showing the internal configuration of the image capturing system 1B according to the third embodiment.

[0089] The imaging system 1B in the third embodiment has a configuration in which a control device 15A of the imaging system 1A in the second embodiment is provided with a subject distance calculation unit 77. With respect to this point and other points described below, the imaging system 1B in the third embodiment has the same configuration as the imaging system 1A in the second embodiment.

[0090] The image capture system 1B in the third embodiment calculates the amount of blur correction in accordance with the subject distance that varies with the movement of the optical axis.

[0091] When the optical axis changes from the state of the first optical axis 23ab, which is the initial position, to the state of the second optical axis 23ad, the object distance calculation unit 77 calculates the object distance in the state of the second optical axis 23ad. A method of calculating the object distance in the state of the second optical axis 23ad will be described with reference to FIG. 7. In FIG. 7, the second optical axis indicated by the reference symbol 23ac will be described as the second optical axis 23ad here. A second object distance D2 from the state of the first optical axis 23ab perpendicularly facing the imaging object to the state of the second optical axis 23ad in which the optical axis of the lens 23 is changed by Φ to the imaging object, and a third object distance D3 of the outer edge of the optical path of the angle of view a in the second optical axis 23ad are calculated as follows. The object distance D3 is the length of a perpendicular line from the end of the optical path of the angle of view a to the extension of the imaging plane after the optical axis change.

[0092] The second object distance D2 is calculated by the following formula (10). D2=D1 / cos(Φ)...(10) formula For example, under the conditions of the first embodiment, D2 is 1.706 [m].

[0093] The third object distance D3 is calculated by the following formula (11). D3=D1 / cos{Φ+(a / 2)}×sin(90-Φ)···(11) formula For example, under the conditions of the first embodiment, D3 is 1.73 [m].

[0094] Using the second object distance D2 calculated in this way, the object magnification M2 in the state of the second optical axis 23ad is calculated, and by substituting it into equation (4), the pixel movement amount P2 in the state of the second optical axis 23ad can be calculated. The mirror swing angle α can be calculated using equations (5) and (8) using this pixel movement amount P2.

[0095] Next, the operation of the imaging system 1B in the third embodiment will be described with reference to Fig. 17. Fig. 17 is a flowchart showing the imaging process in the third embodiment. The operation of the imaging system 1B in the third embodiment is the same as the operation of the imaging system 1A in the second embodiment, except that step S21 is added.

[0096] Steps S1 to S4, S11, S12, and S6 are the same as the operations of the imaging system 1A of the second embodiment, so the explanation will be omitted. After changing the optical axis in step S4, in step S21, the subject distance calculation unit 77 calculates the subject distance based on the optical axis change angle Φ, and sets the calculated subject distance as a new value. This makes it possible to improve the accuracy of the swing angle (amount of blur) of the correction mechanism calculated by the swing angle calculation unit 73 in step S11.

[0097] In this way, the control device 15B calculates the subject distance from the imaging device 11A to the imaging target area 9 that changes before and after the optical axis change based on the optical axis change angle changed by the optical axis change mechanism 12, and determines the moving direction of the vehicle 3 based on the subject distance. Nobu To correct this Nobu This allows for more accurate tracking and allows for highly accurate blur correction.

[0098] (Embodiment 4) An image capturing system 1C according to the fourth embodiment will be described with reference to Fig. 18. Fig. 18 is a block diagram showing the internal configuration of the image capturing system 1C according to the fourth embodiment.

[0099] The imaging system 1C in the fourth embodiment is configured by adding a subject distance detection device 81 to the imaging system 1B in the third embodiment. Other than this point and the points described below, the imaging system 1C in the fourth embodiment has the same configuration as the imaging system 1B in the third embodiment.

[0100] The subject distance detection device 81 detects the subject distance from the principal point of the lens 23 to the subject. The subject distance detection device 81 is, for example, a laser measurement device. Information on the subject distance detected by the subject distance detection device 81 is sent to the control device 15C. The swing angle calculation unit 73 of the control device 15C calculates the swing angle of the correction mechanism in the state of the first optical axis 23ab based on the detected first subject distance D1.

[0101] The subject distance detection device 81 detects the subject distance from the principal point of the lens 23 to the subject, and thus can accurately calculate the amount of blur correction even if the height of the imaging device 11A from the surface of the road 4 is changed according to the on-site conditions. This makes it possible to easily adjust the imaging range of the imaging device 11A by adjusting the subject distance of the imaging device 11A relative to the subject. Even if the subject distance changes during driving, such as when imaging the wall surface 5a on the side of a tunnel or when photographing the road surface when the vehicle inclination changes while driving on a slope, the amount of blur correction can be accurately calculated based on the detected subject distance.

[0102] Next, the operation of the imaging system 1C in the fourth embodiment will be described with reference to Fig. 19. Fig. 19 is a flowchart showing the imaging process in the fourth embodiment. The operation of the imaging system 1C in the fourth embodiment is the same as the operation of the imaging system 1B in the third embodiment except that step S1 is omitted and steps S31 and S32 are added.

[0103] In the imaging system 1C in the fourth embodiment, instead of measuring the object distance in advance, the object distance detection device 81 measures the first object distance D1 when the optical axis is in the state of the first optical axis 23ab which is the initial position.

[0104] After changing the optical axis in step S4, in step S31, the control device 15C determines whether the optical axis is in the state of the first optical axis 23ab, which is the initial position. If the control device 15C determines that the optical axis is in the state of the first optical axis 23ab (Yes in step S31), in step S32, the subject distance detection device 81 detects the first subject distance D1, and the control device 15C sets this detection value as the subject distance in the state of the first optical axis 23ab. This makes it possible to improve the accuracy of the swing angle of the correction mechanism calculated by the swing angle calculation unit 73 in step S11.

[0105] Furthermore, when the control device 15C determines that the optical axis is in the state of the second optical axis 23ad (No in step S31), in step S21 the subject distance calculation unit 77 calculates the subject distance based on the first optical axis change angle Φ1 and sets the calculated subject distance as a new value. This makes it possible to improve the accuracy of the swing angle of the correction mechanism calculated by the swing angle calculation unit 73 in step S11.

[0106] In this way, the imaging system 1C includes a subject distance detection device 81 for measuring the subject distance from the imaging device 11A to the imaging target area 9. , The deviation correction angle can be calculated with high accuracy.

[0107] In addition, the control device 15C performs subject distance detection using the subject distance detection device 81 in the state of the first optical axis 23ab before the optical axis is changed, and calculates the subject distance based on the first subject distance D1 detected in the state of the first optical axis 23ab and the first optical axis change angle Φ1, which is the amount of optical axis change, after the optical axis is changed to the state of the second optical axis 23ad by the optical axis changing mechanism 12.

[0108] By detecting the subject distance by the subject distance detection device 81 only when the optical axis is in the initial position, it is possible to prevent erroneous detection of the distance when, for example, a tall vehicle in the adjacent lane, such as a trailer, enters the subject distance detection range when the optical axis is changed from imaging the tunnel wall surface 5a directly above to diagonally upward. Also, in the case of imaging the road surface, it is possible to prevent erroneous detection of the distance when a vehicle in the adjacent lane enters the subject distance detection range by changing the optical axis from imaging the road surface directly below. Distance can be detected by installing a rangefinder at a height higher than the height of a normal vehicle with respect to the tunnel side wall surface 5a, but changing the optical axis downward similarly causes an erroneous detection of a vehicle in the adjacent diagonal line, but this can be prevented.

[0109] (Other embodiments) As described above, the above embodiment has been described as an example of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to this, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are appropriately performed. Therefore, other embodiments will be exemplified below.

[0110] In the above embodiment, the optical axis of the imaging device 11 has two optical axis states, the first optical axis 23ab state and the second optical axis 23ac state, but this is not limited to the above. The imaging device 11 may have three or more optical axis states, and the optical axis changing mechanism 12 may change the imaging device 11 to each optical axis state, and capture an image of the imaging target in each optical axis state.

[0111] In the above embodiment, the information on the moving speed V1 from the speed detection device 3a of the vehicle 3 is used, but this is not limiting. The imaging system 1 may be equipped with a speed detection device that detects the moving speed of the imaging system 1. The speed detection device may also be one that uses a GPS (Global Positioning System) system.

[0112] In the above embodiment, the imaging system 1 captures an image of the wall surface on the side of the vehicle 3, but this is not limited to the above. The imaging system 1 may capture an image of the wall surfaces above and below the vehicle 3.

[0113] In the above embodiment, the moving body is a vehicle 3 such as an automobile. However, the moving body is not limited to a vehicle 3, and may be a vehicle that runs on the ground such as a train or a motorcycle, a ship that travels on the sea, or an air vehicle such as an airplane or a drone that flies in the sky. When the moving body is a ship, the imaging system 1 images the walls of bridge piers and bridge girders, and structures constructed along the shore. When the moving body is a train, the position and wear of the overhead wires can be detected by imaging the overhead wires.

[0114] In the above embodiment, an image is captured using ambient light reflected from the imaging target area 9, but this is not limited to the above. Light may be irradiated from a moving body or an imaging system toward the imaging target area 9, and an image may be captured using reflected light of the irradiated light.

[0115] (Overview of the embodiment) (1) An imaging system according to the present disclosure includes an imaging device disposed on a moving body and capturing images in a first imaging state and a second imaging state having different optical axes, an attitude detection device that detects an attitude change amount of the moving body, an optical axis changing mechanism that changes the optical axis of the imaging device from a state of a first optical axis when the imaging device captures images in the first imaging state to a state of a second optical axis inclined in a second direction intersecting the first direction when capturing images in the second imaging state while the moving body is moving in a first direction, and a control device that sets an optical axis change amount by which the optical axis changing mechanism changes the optical axis of the imaging device based on the attitude change amount. The optical axis changing mechanism changes the optical axis of the imaging device based on the set optical axis change amount.

[0116] This enables the imaging system to expand the imaging range and capture a wide range of images, and since the optical axis of the imaging device is changed in response to changes in the posture of the moving body, the states of the first optical axis and the second optical axis can be stably maintained, ensuring continuity of the captured image.

[0117] (2) In the imaging system of (1), the control device instructs the optical axis changing mechanism to perform two types of optical axis change: a first optical axis change for expanding the imaging range of the captured image, and a second optical axis change for reducing the effect of changes in the posture of the moving body.

[0118] (3) In the imaging system of (2), in the first optical axis change, the control device causes the optical axis changing mechanism to change the optical axis of the imaging device from the first optical axis state to the second optical axis state, or from the second optical axis state to the first optical axis state, between two consecutive imaging timings.

[0119] (4) In the imaging system of (3), the control device causes the optical axis changing mechanism to change the optical axis of the imaging device between the state of the first optical axis and the state of the second optical axis in a first optical axis change by a first optical axis change angle for expanding the imaging range of the captured image as the optical axis change amount.

[0120] (5) In any of the imaging systems (2) to (4), the control device does not instruct the optical axis changing mechanism to change the second optical axis while the optical axis changing mechanism is performing the first optical axis change, but instructs the optical axis changing mechanism to change the second optical axis after the optical axis changing mechanism has completed the first optical axis change and before it next performs the first optical axis change.

[0121] (6) In the imaging system of (4), the attitude detection device detects the amount of attitude change in the roll direction of the moving body, and in the second optical axis change, the control device changes the optical axis to a sum of a second optical axis change angle and a first optical axis change angle, which have the same magnitude and opposite sign as the amount of attitude change detected by the attitude detection device.

[0122] (7) In the imaging system according to any one of (1) to (6), a speed detection device is provided for detecting a moving speed of a moving object, and a camera is provided for correcting blurring in a first direction when an image of the moving object is captured by the imaging device while the moving object is moving. Rub and a correction mechanism, and the control device is Tebu The compensation mechanism Rub For correction Nobu Set the correction angle and adjust the amount of change in the optical axis. Tebu Change the correction angle.

[0123] (8) In the imaging system of (7), the control device calculates a subject distance from the imaging device to an imaging target area that changes before and after an optical axis change based on an optical axis change angle changed by an optical axis change mechanism, and adjusts the first direction based on the subject distance. Nobu The image blur correction angle is set to correct the blur.

[0124] (9) The imaging system of (8) further comprises an object distance measuring device for measuring an object distance from the imaging device to an imaging target area.

[0125] In the imaging system of (10)(9), subject distance detection is performed by a subject distance measuring device in a first optical axis state before the optical axis is changed, and after the optical axis is changed to a second optical axis state by the optical axis changing mechanism, the subject distance is calculated based on the subject distance detected in the first optical axis state and the amount of optical axis change.

[0126] In the imaging system of (11)(4), the control device does not instruct the optical axis changing mechanism to change the second optical axis while the optical axis changing mechanism is changing the first optical axis, but instructs the imaging device to change the second optical axis while the imaging device is exposing.

[0127] (12) A moving body according to the present disclosure includes any one of the imaging systems (1) to (10). This allows the imaging system to expand the imaging range while the moving body is moving, and capture a wide range of images, and also changes the optical axis of the imaging device in response to the change in the posture of the moving body, so that the states of the first optical axis and the second optical axis can be stably maintained, and continuity of captured images can be ensured. [Industrial Applicability]

[0128] The present disclosure is applicable to an imaging system installed on a moving vehicle. [Explanation of symbols]

[0129] 1, 1A, 1B, 1C Imaging System 3. Vehicle 3a Speed ​​detection device 4 road 4b Hole 4c Cracks 5. Wall 5a Wall 5b hole 5c Cracks 9. Imaging area 11 Imaging device 12 Optical axis changing mechanism 15 Control device 17 Memory section 19 Control section 21 Camera body 23 Lens 23a optical axis 23ab 1st optical axis 23ac 2nd optical axis 24 Shutter 25 Image sensor 27 Camera control unit 31 Image stabilization mechanism 33 Attitude detection device 41 Mirror 43 Mirror drive unit 45 Arm 51 Maximum exposure time calculation unit 53 Exposure time setting unit 61 Base 63 Rotation drive unit 71 Optical axis change instruction unit 73 Swing angle calculation section 75 Rotation speed calculation unit 81 Subject distance detection device α Mirror swing angle F focal length C1 First imaging state C2 Second imaging state M Subject magnification LE1 Extension of the lens principal point LE2 Extension of the imaging surface Φ Optical axis change angle Φ1 First optical axis change angle Φ2 Second optical axis change angle Tf1, Tf2 Imaging interval V0, V1, V2, V3 movement speed

Claims

1. an imaging device disposed on a moving body and configured to capture images in a first imaging state and a second imaging state, the first imaging state and the second imaging state having different optical axes; an attitude detection device for detecting an attitude change amount of the moving body; an optical axis changing mechanism that changes an optical axis of the imaging device from a state of a first optical axis when the imaging device captures an image in the first imaging state to a state of a second optical axis inclined in a second direction intersecting the first direction when the imaging device captures an image in the second imaging state while the moving body is moving in a first direction; a control device that sets an optical axis change amount by which the optical axis changing mechanism changes the optical axis of the imaging device based on the posture change amount, The optical axis changing mechanism changes the optical axis of the imaging device based on the set optical axis change amount. Imaging system.

2. The control device instructs the optical axis changing mechanism to perform two types of optical axis changes: a first optical axis change for expanding an imaging range of a captured image, and a second optical axis change for reducing an effect of a posture change of the moving body. The imaging system according to claim 1 .

3. In the first optical axis change, the control device changes the optical axis of the imaging device between two consecutive imaging operations from a state of the first optical axis to a state of the second optical axis, or from a state of the second optical axis to a state of the first optical axis, by the optical axis change mechanism. The imaging system according to claim 2 .

4. The control device causes the optical axis changing mechanism to change the optical axis of the imaging device between a state of the first optical axis and a state of the second optical axis in a first optical axis change by a first optical axis change angle for expanding an imaging range of a captured image as the optical axis change amount. The imaging system according to claim 3 .

5. The control device does not instruct the second optical axis change while the optical axis changing mechanism is performing the first optical axis change, and instructs the second optical axis change after the optical axis changing mechanism completes the first optical axis change and before the optical axis changing mechanism performs the next first optical axis change. The imaging system according to claim 4 .

6. the attitude detection device detects an attitude change amount of the moving body in a roll direction, the control device changes the optical axis change amount to a sum of a second optical axis change angle, the second optical axis change angle having an opposite sign to and equal magnitude to the attitude change amount detected by the attitude detection device, and the first optical axis change angle, The imaging system according to claim 4 .

7. A speed detection device for detecting a moving speed of a moving object; a blur correction mechanism that corrects blurring along the first direction when the moving object is moving and the imaging device is used to capture the image, the control device sets a blur correction angle for blur correction by the blur correction mechanism based on the moving speed, and changes the blur correction angle based on the optical axis change amount. The imaging system according to claim 1 .

8. the control device calculates a subject distance from the imaging device to an imaging target area that changes before and after the optical axis change based on an optical axis change angle changed by the optical axis change mechanism, and sets a blur correction angle for correcting blur along the first direction based on the subject distance. The imaging system according to claim 7 .

9. a subject distance measuring device for measuring the subject distance from the imaging device to the imaging target area, The imaging system of claim 8 .

10. the control device detects a subject distance using the subject distance measuring device in a state of the first optical axis before the optical axis is changed, and calculates the subject distance based on the subject distance detected in the state of the first optical axis and an amount of change of the optical axis after the optical axis is changed to the state of the second optical axis by the optical axis changing mechanism. The imaging system of claim 9.

11. the control device does not instruct the second optical axis change while the optical axis changing mechanism is changing the first optical axis, and instructs the second optical axis change while the imaging device is exposing; The imaging system according to claim 4 .

12. The control device does not instruct the second optical axis change while the optical axis changing mechanism is performing the first optical axis change, and instructs the second optical axis change after the optical axis changing mechanism has completed the first optical axis change and before performing the next first optical axis change. The imaging system according to claim 2 .

13. The control device does not instruct the second optical axis change while the optical axis changing mechanism is performing the first optical axis change, and instructs the second optical axis change after the optical axis changing mechanism has completed the first optical axis change and before performing the next first optical axis change. The imaging system according to claim 3 .

14. A moving object comprising the imaging system according to any one of claims 1 to 13.

Images