Imaging system and mobile body equipped therewith

Abstract

This imaging system provided to a movement body comprises: an imaging device spaced apart from the movement body and having an imaging element for capturing an image of an imaging target which is at least a part of the periphery of the movement body; a blur correction mechanism for correcting blur in the movement direction when the imaging device captures an image during movement of the movement body; and a control unit for controlling an exposure time for the imaging device on the basis of the movement speed of the movement body and a blur amount which can be corrected by the blur correction mechanism.

Description

Technical Field

[0001] The present disclosure relates to an imaging system that corrects blurring in response to the movement of a moving body, and a moving body equipped with the same.

Background Art

[0002] With the aging of traffic infrastructure, the demand for infrastructure inspection is increasing. Instead of visual inspection by humans, imaging the infrastructure while moving with a moving body and detecting defective locations by image processing of the captured images can significantly improve inspection efficiency. However, since imaging is performed while moving, blurring occurs in the captured images.

[0003] For example, Patent Document 1 corrects blurring due to camera movement during exposure using the technology of a saccharide mirror. By irradiating light on an imaging target, reflecting the light reflected by the imaging target on a mirror that rotates during a predetermined exposure time, and making the light incident on a camera, blurring is reduced.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, since the imaging system images an imaging target while moving together with the moving body, it is desired to obtain a clearer image even when the moving speed of the moving body changes.

[0006] The present disclosure provides an imaging system that reduces blurring and performs imaging with an appropriate exposure time, and a moving body equipped with the same.

Means for Solving the Problems

[0007] The imaging system of this disclosure is an imaging system installed on a moving object and comprises: an imaging device having an image sensor for imaging an object that is located away from the moving object and is at least a part of the area surrounding the moving object; a blur correction mechanism for correcting blur in the direction of movement when the imaging device takes an image while the moving object is moving; and a control unit that controls the exposure time of the imaging device based on the moving speed of the moving object and the amount of blur that can be corrected by the blur correction mechanism.

[0008] Furthermore, the mobile body of this disclosure is equipped with the imaging system described above. [Effects of the Invention]

[0009] The imaging system and mobile body equipped therewith of this disclosure can provide an imaging system and mobile body equipped therewith that reduce blur and capture images with an appropriate exposure time. [Brief explanation of the drawing]

[0010] [Figure 1] Diagram illustrating a vehicle equipped with an imaging system. [Figure 2] Block diagram showing the internal configuration of the imaging system in the embodiment. [Figure 3A] Diagram explaining image stabilization in the imaging system. [Figure 3B] Diagram showing the range of rotation of the mirror. [Figure 3C] Diagram illustrating the range that the image sensor can capture. [Figure 3D] An explanatory diagram showing the area that the image sensor can capture, as viewed from the front. [Figure 4A] An explanatory diagram showing an image captured without image stabilization. [Figure 4B] Graph showing the relationship between mirror rotation angle and exposure time during low-speed driving. [Figure 4C] Graph showing the relationship between mirror rotation angle and exposure time during high-speed driving. [Figure 5] Flowchart showing the blur imaging process in the embodiment [Figure 6] Graph showing the relationship between movement speed and maximum exposure time. [Figure 7] Graph showing the relationship between the change in moving speed, the timing of the exposure time, and the movement blur correction angle [Figure 8] Figure showing a modified example of an imaging device and a blur correction mechanism [Embodiment for Carrying out the Invention]

[0011] (Embodiment) Hereinafter, the embodiment will be described with reference to the drawings. In the embodiment, the moving body is a vehicle 3 such as an automobile, and the imaging system 1 is attached to the upper part of the vehicle 3, which will be described as an example.

[0012] [1. Configuration of Imaging System] FIG. 1 is a diagram for explaining the imaging system 1. FIG. 2 is a block diagram showing the internal configuration of the imaging system 1. In FIG. 1, the vehicle 3 is running, for example, inside the tunnel 5. On the wall surface 5a inside the tunnel 5, for example, holes 5b and cracks 5c are generated.

[0013] The imaging target of the imaging system 1 is at least a part of the structures around the vehicle 3, and is a target that moves relatively according to the moving speed of the vehicle 3 as the vehicle 3 moves. The imaging target area 9 is an area to be acquired as an image in this imaging target. In addition to the inner wall of the tunnel 5, the side surface and bottom surface of an overpass, utility poles, and electric wires may also be used as imaging targets. Thereby, holes, cracks, bulges, peeling, joints, the inclination of utility poles, and the deflection of electric wires of the imaging target can be detected by image processing from the acquired images.

[0014] The vehicle 3 includes a speed detection device 3a that detects the moving speed of the vehicle 3. The speed detection device 3a is, for example, a vehicle speed sensor that detects the moving speed from the rotational speed of the vehicle axle of the vehicle 3.

[0015] An imaging system 1 is installed on the top surface of vehicle 3. In Figure 1, the imaging system 1 is fixed to capture an image of the wall surface 5a of the tunnel 5 above vehicle 3, but it may also be installed to capture an image of the wall surface 5a to the side or diagonally to the side of vehicle 3, or the road surface below vehicle 3.

[0016] The imaging system 1 comprises an imaging device 11, a blur correction device 13, and a storage unit 15. The imaging device 11 captures images of the area around the vehicle 3, and when the vehicle 3 is traveling inside, for example, a tunnel 5, it captures images of the tunnel wall 5a. The imaging device 11 comprises a camera body 21, a lens 23, a shutter 24, an image sensor 25, and a camera control unit 27.

[0017] The camera body 21 has a replaceable lens 23 attached and houses an image sensor 25 and a camera control unit 27. The image sensor 25 is positioned at the focal length F of the lens 23. The camera body 21 is positioned on the vehicle 3 such that the orientation of the lens 23 is parallel to the direction of movement of the vehicle 3. For example, the camera body 21 is positioned so that the lens 23 faces forward or backward of the vehicle 3. The camera body 21 and lens 23 may be integrated, in which case the orientation of the lens 23 and the direction of movement are set to be perpendicular. The camera control unit 27 opens the shutter 24 while receiving an exposure instruction signal from the control unit 33. The shutter 24 may be configured with multiple aperture blades opening and closing, or it may be an electronic shutter.

[0018] The image stabilization device 13 corrects the optical path of light incident on the imaging system 1 so that image blur in the target area 9 is reduced even when the imaging device 11 takes images while the vehicle 3 is moving. The image stabilization device 13 comprises an image stabilization mechanism 31 and a control unit 33.

[0019] The image stabilization mechanism 31 corrects the optical path of the reflected light L1, which is reflected from ambient light in the imaging target area 9, in accordance with the movement of the vehicle 3. The image stabilization mechanism 31 aligns the direction of the reflected light L1, which is reflected from ambient light in the imaging target area 9, with the imaging direction of the image sensor 25. The image stabilization mechanism 31 includes, for example, a mirror 41 and a mirror drive unit 43. However, the image stabilization mechanism 31 is not limited to this, and may be a mechanism that rotates a lens barrel, in which the lens 23 and the image sensor 25 are integrated, around a pivot axis, for example, in the pan or tilt direction, or it may be a mechanism that rotates the entire imaging device 11 in an arc shape around the imaging target area 9 in a plane including the direction of travel of the vehicle 3.

[0020] The mirror 41 is rotatably positioned to face the lens 23. The mirror 41 can rotate, for example, in either the forward or counterclockwise direction, and the range of rotational angle may be less than 360 degrees or greater than 360 degrees. The mirror 41 totally reflects the light reflected by the object being imaged from the ambient light towards the imaging device 11. The mirror drive unit 43 drives the mirror 41 to rotate from an initial angle to a specified angle, and after rotating to the specified angle, it returns to the initial angle. The mirror drive unit 43 is, for example, a motor. The rotational angle of the mirror 41 is limited by the mechanical constraints of the mirror drive unit 43, and the mirror 41 can be rotated up to the maximum swing angle of the mirror 41 determined by this limitation. The motion blur correction angle θ for which motion blur correction is possible is less than or equal to the maximum swing angle of the mirror 41. The maximum correctable angle θmax, which is the maximum value of the motion blur correction angle θ, will be described in detail later.

[0021] The image stabilization provided by the image stabilization device 13 will be explained with reference to Figures 3A and 4A. Figure 3A is an explanatory diagram illustrating the image stabilization of the imaging system 1. Figure 4A shows an image captured without image stabilization, with Figure 4A(a) showing the image at the start of imaging and Figure 4A(b) showing the image at the end of imaging.

[0022] For example, suppose the imaging system 1, located at position A, moves to position B along with the vehicle 3 during the exposure time. Image acquisition starts at position A, and the image Im1 acquired at this timing is shown in Figure 4(a). Image Im1 captures, for example, the hole 5b in the imaging target area 9. However, because the exposure time for image Im1 is insufficient, the image is dark and not clear.

[0023] Therefore, exposure continues until vehicle 3 moves to position B. In this case, if no blur correction is performed, the imaging target area 9 moves relative to the direction opposite to the direction of movement of vehicle 3, resulting in image Im2, as shown in Figure 4A(b), where the hole 5b has moved relative to the target area. In image Im2, the amount of pixel movement P is detected as the amount of blur. Thus, the image captured by the imaging device 11 while vehicle 3 is moving is a blurred image.

[0024] Therefore, depending on the movement speed of the imaging system 1 and the vehicle 3, the end of the mirror 41 on the side facing the direction of movement rotates during the exposure time in a direction that cancels out the relative movement of the object being imaged. As a result, the imaging system 1 can capture the same object area 9 in the image during the exposure time, and can obtain an image with significantly reduced blur. In Figure 1, the mirror 41 is rotated clockwise so that the end of the mirror 41 on the side facing the direction of movement rotates around the object being imaged during the exposure time. By rotating the mirror 41, the amount of pixel movement P in the image Im2 is corrected to zero.

[0025] Next, the maximum correctable angle θmax will be explained with reference to Figures 3B to 3D. Figure 3B is an explanatory diagram showing the range in which the mirror 41 can rotate. Figure 3C is an explanatory diagram showing the range in which the image sensor 25 can capture images. Figure 3D is an explanatory diagram showing the range in which the image sensor 25 can capture images from a front view.

[0026] The maximum correctable angle θmax is determined primarily by two factors. The first factor is the maximum swing angle of the mirror 41 itself within the imaging interval, i.e., the maximum mechanical angle Φ1, as shown in Figure 3B. The second factor is the maximum angle Φ2 usable in the pixel range Am used by the image sensor 25, regardless of the rotation angle of the mirror 41, as shown in Figure 3C. Note that this pixel range Am does not have to include all pixels of the image sensor 25 used for inspection; a narrower pixel range than the total pixel range is acceptable, prioritizing the securing of the correction angle. In the case of a mechanism that rotates a lens barrel, in which the lens 23 and image sensor 25 are integrated, around a rotation axis, for example, in the pan or tilt direction, or a mechanism that rotates the entire imaging device 11 in an arc shape around the imaging target area 9 in a plane including the direction of travel of the vehicle 3, the maximum correctable angle θmax is determined by the first factor.

[0027] As shown in Figure 3C, for example, at the rotation start position, the reflective surface of the mirror 41 is nearly horizontal, so the mirror width Amw in the lens diameter direction at the rotation start position is smaller than when the mirror 41 has rotated Φ2. Therefore, as shown in Figure 3D(a), the image circle Cm1 at the rotation start position is smaller than the image circle Cm2 after rotating Φ2. In contrast, as shown in Figure 3D(b), the image circle Cm2 after rotating Φ2 may be designed so that the outer shape of the image sensor 25 is inscribed within it, or the image circle Cm2 may be designed so that the outer shape of the image sensor 25 is inscribed within it when the inclination angle of the mirror 41 with respect to the horizontal plane is 45 degrees. The maximum correctable angle θmax is the smaller of these maximum mechanical angles Φ1 and Φ2.

[0028] Next, the control of exposure time will be explained using Figures 4B and 4C. Figure 4B is a graph showing the relationship between the rotation angle of the mirror 41 and the exposure time during low-speed driving, and Figure 4C is a graph showing the relationship between the rotation angle of the mirror 41 and the exposure time during high-speed driving. The relationship between exposure time and the maximum correctable angle θmax is such that as the speed of the vehicle 3 decreases, the exposure time can be increased as it takes time for the mirror to rotate to the maximum correctable angle θmax.

[0029] As shown in Figure 4B, for example, when imaging with a preset initial exposure time Tp0, when the vehicle 3 is traveling at a low speed, the correction angle θa corresponding to the initial exposure time Tp0 has enough room to increase the rotation angle by (θmax1-θa) up to the maximum correctable angle θmax. Therefore, by extending the exposure time to an exposure time Tpa, which is longer than the initial exposure time Tp0, it is possible to obtain a brighter image with even more light. In contrast, as shown in Figure 4C, for example, when the vehicle 3 is traveling at a high speed, if the correction angle θb corresponding to the initial exposure time Tp0 is greater than the maximum correctable angle θmax, This corresponds to the correction angle portion that exceeds the maximum correctable angle θmax. It becomes impossible to reduce blur. Therefore, by shortening the exposure time to an exposure time Tpb which is shorter than the initial exposure time Tp0, the rotation angle of the mirror 41 can be set to the maximum correctable angle θmax, and the maximum brightness can be obtained while achieving blur correction.

[0030] There exists a correctable pixel movement Pmax corresponding to the maximum correctable angle θmax, and this is the correctable blur amount Sf. In other words, the correctable blur amount Sf is the correctable pixel blur amount in the image sensor 25, and is determined from the maximum correctable angle θmax and the focal length F of the lens 23 by the following equation (1). Sf[mm]=tan(θmax)×F[mm]...Equation (1) Since the focal length F of the lens 23 is constant during imaging, the correctable amount of blur Sf corresponds to the maximum correctable angle θmax. The correctable amount of blur Sf is corrected by the blur correction mechanism 31 controlled by the control unit 33. Thus, the correctable amount of blur Sf is defined by the change angle of the blur correction mechanism 31 and the pixel range Am of the image sensor 25. The blur correction mechanism 31 is equipped with a rotatable blur correction unit, and the change angle is the rotation angle of the blur correction unit. To explain it another way, the change angle is the angle by which the blur correction mechanism 31 changes the direction of light reflected from the image target to the imaging direction of the image sensor. If the blur correction mechanism 31 rotates, for example, the mirror 41, the change angle is the rotation angle of the mirror 41, which corresponds to the blur correction unit. Furthermore, in the case of a mechanism that rotates the lens barrel, which is integrated with the image sensor 25, around a pivot axis, for example in the pan or tilt direction, or a mechanism that rotates the entire imaging device 11 in an arc shape around the imaging target area 9 in a plane including the direction of travel of the vehicle 3, the change angle is the rotation angle of these rotation-driving mechanisms that act as the blur correction unit.

[0031] Next, a configuration in which the maximum exposure time is controlled considering the maximum correctable angle θmax due to the constraints of the image stabilization device 13 and the moving speed V of the vehicle 3 will be described below.

[0032] Refer to Figures 2 and 3A. The control unit 33 controls the image stabilization mechanism 31 to correct for blur. The control unit 33 controls the image stabilization mechanism 31 based on the correctable amount of blur Sf, the movement speed of the vehicle 3, and the exposure time Tp. The control unit 33 includes a maximum exposure time calculation unit 51, an exposure time setting unit 53, a mirror swing angle calculation unit 55, and a mirror rotation speed calculation unit 57.

[0033] The control unit 33 is a circuit that can be implemented using semiconductor elements or the like. The control unit 33 can be composed of, for example, a microcontroller, CPU, MPU, GPU, DSP, FPGA, or ASIC. The functions of the control unit 33 may be implemented using hardware alone, or they may be implemented by combining hardware and software. The control unit 33 reads data and programs stored in the memory unit 15 and performs various arithmetic operations to realize predetermined functions.

[0034] The maximum exposure time calculation unit 51 calculates the maximum exposure time Tmax from the movement speed V1, subject magnification M, focal length F, and maximum correctable angle θmax using the following equation (2). Tmax = F × tan(θmax) × 60 2 / (V1×M×10 3 )...Equation (2)

[0035] The focal length F is a value determined by the lens 23. The subject magnification M is a value determined by the focal length F and the subject distance. The subject distance is the distance from the principal point of the lens 23, which is positioned between the subject (image target 5b) and the image sensor 25, to the image target 5b. The subject distance may be a known value measured in advance, or a value measured during imaging by a rangefinder. The maximum correctable angle θmax is a value determined by the maximum swing angle that the mirror 41 can rotate and the pixel range Am of the image sensor 25, as described above. The maximum correctable angle θmax is also a value determined by the relationship between the diameter of the mirror 41 and the diameter of the lens 23. The maximum correctable angle θmax and the correctable blur amount Sf do not depend on the movement speed, so they are stored in the storage unit 15 as a lookup table linked to the movement speed, for example.

[0036] The exposure time setting unit 53 determines whether the exposure time Tp is less than or equal to the maximum exposure time Tmax. Immediately after the start of image stabilization, the initial exposure time Tp0, which is pre-stored in the storage unit 15, is used as the exposure time Tp. The exposure time setting unit 53 reads the initial exposure time Tp0 from the storage unit 15. The initial exposure time Tp0 is, for example, the standard exposure time when the vehicle 3 is moving at a first moving speed V1. The moving speed V1 is, for example, 80 km / h. After imaging has started, the exposure time Tp is the exposure time of a previously captured image, for example, the exposure time of the previous frame.

[0037] Alternatively, the storage unit 15 may be pre-stored with multiple types of initial exposure times Tp0 corresponding to imaging conditions, and the initial exposure time Tp0 corresponding to the specified conditions may be read from among the multiple initial exposure times Tp0 according to the imaging condition instructions given by the operation unit 7.

[0038] If the exposure time setting unit 53 determines that the exposure time Tp is greater than the maximum exposure time Tmax, it sets the exposure time Tp to the maximum exposure time Tmax. If the exposure time setting unit 53 determines that the exposure time Tp is less than or equal to the maximum exposure time Tmax, it may set the determined exposure time Tp as the first exposure time Tp1 to be imaged next, or it may set it to be greater than or equal to the determined exposure time Tp and less than or equal to the maximum exposure time Tmax. The first exposure time Tp1 set by the exposure time setting unit 53 is transmitted to the mirror swing angle calculation unit 55.

[0039] The mirror swing angle calculation unit 55 calculates the mirror swing angle α of the mirror 41 during imaging in the following sequence, based on the vehicle's moving speed V1, the set first exposure time Tp1, the subject magnification M, and the focal length F of the lens 23.

[0040] The amount of movement L of the vehicle 3 between the start time t1 of imaging and the end time t2 of imaging is calculated from the movement speed V1 and the first exposure time Tp1 using the following equation (3). L[mm] = V1[km / h] × 10 6 ×Tp1[ms] / (60 2 ×10 3)...Equation (3)

[0041] The amount of movement P of the pixels on the image sensor 25 between the time t1 when imaging starts and the time t2 when imaging ends is calculated from the amount of movement L of the vehicle 3 and the subject magnification M using the following equation (4). P[mm] = L[mm] × M···(4)

[0042] Since the amount of pixel movement P causes blurring, the optical path of the light incident on the lens 23 is changed by a motion blur correction angle θ corresponding to the amount of pixel movement P to prevent blurring. The motion blur correction angle θ is calculated from the amount of pixel movement P and the focal length F using the following equation (5). θ[deg]=arctan(P / F)...Equation (5)

[0043] The mirror swing angle α required for blur correction during exposure is half the magnitude of the motion blur correction angle θ, and is calculated by the following equation (6). α = θ / k ... (Equation 6) Here, as in the embodiment shown in Figure 1, when light from the object being imaged travels in the order of mirror 41, lens 23, and image sensor 25, k=2. Also, when a pan-tilt mechanism and overall camera drive are used, k=1.

[0044] In this way, the mirror swing angle calculation unit 55 calculates the mirror swing angle α of the mirror 41.

[0045] The mirror rotation speed calculation unit 57 calculates the rotation speed Vm of the mirror 41 from the mirror swing angle α and the exposure time Tp using the following equation (7). Vm=α / Tp...Equation (7)

[0046] Therefore, by rotating the mirror 41 in the opposite direction to the direction of movement at a rotation speed Vm after the start of imaging, the imaging device 11 can receive light from the same imaging target area 9 during the exposure time, thereby suppressing motion blur in the captured image.

[0047] However, there is a maximum correctable angle θmax for motion blur correction, as mentioned above, and blur cannot be corrected beyond this maximum correctable angle θmax.

[0048] The storage unit 15 is a storage medium that stores the programs and data necessary to realize the functions of the control unit 33. The storage unit 15 can be implemented, for example, by a hard disk (HDD), SSD, RAM, DRAM, ferroelectric memory, flash memory, magnetic disk, or a combination thereof.

[0049] The operation unit 7 is an input device for the user to give instructions to the control unit 33. The operation unit 7 may be an input device dedicated to the imaging system 1, or it may be a mobile terminal such as a smartphone. When a mobile terminal is used as the operation unit 7, the operation unit 7 and the control unit 33 transmit and receive data wirelessly. The user may use the operation unit 7 to indicate whether the area to be imaged is a dark area indoors such as a tunnel, or a bright area outdoors such as a mountain slope, or to indicate the imaging interval Tf. The imaging interval Tf is the time between the current image capture and the capture of the next image; in the case of video imaging, it is the time of one frame, and in the case of still image imaging, it is the time interval between images to be captured.

[0050] [2. Operation of the imaging system] Next, the operation of the imaging system 1 will be explained with reference to Figures 5 and 6. Figure 5 is a flowchart showing the imaging process performed by the imaging system 1. Figure 6 is a graph showing the relationship between the movement speed and the maximum exposure time. The imaging process shown in Figure 5 is started, for example, when the operation unit 7 instructs the system to start imaging while the vehicle 3 is moving.

[0051] The control unit 33 obtains the vehicle's moving speed V1 from the speed detection device 3a (step S1). The control unit 33 calculates the maximum exposure time Tmax from the moving speed V1, the maximum correctable angle θmax, the subject magnification M, and the focal length F using equation (2) (step S2).

[0052] Next, the exposure time setting unit 53 sets the exposure time to be less than or equal to the maximum exposure time Tmax (step S3). Immediately after the start of imaging, the exposure time setting unit 53 reads out the initial exposure time Tp0 stored in the memory unit 15 as the exposure time, and adjusts this initial exposure time Tp0 in relation to the moving speed V1. As shown in Figure 6, the maximum exposure time Tmax decreases as the moving speed of the vehicle 3 increases.

[0053] For example, when vehicle 3 travels on a public road at a speed V2 (e.g., 50 km / h), the difference between the maximum exposure time Tmax and the initial exposure time Tp0 is time Tv2. Therefore, the exposure time setting unit 53 can extend the exposure time from the initial exposure time Tp0 up to a maximum of time Tv2, and this extended exposure time is set as the first exposure time Tp1. This ensures sufficient light during imaging and also reduces blur, resulting in the acquisition of a clear image.

[0054] Furthermore, for example, when vehicle 3 is traveling on a highway at a speed V3 (e.g., 100 km / h), the maximum exposure time Tmax is Tv3 less than the initial exposure time Tp0. Therefore, by setting the first exposure time Tp1 to the maximum exposure time Tmax, which is smaller than the initial exposure time Tp0, the exposure time setting unit 53 can acquire the image with the highest brightness while reducing blur.

[0055] Based on the set first exposure time Tp1, the mirror swing angle calculation unit 55 calculates the motion blur correction angle θ and the mirror swing angle α, and the mirror rotation speed calculation unit 57 calculates the mirror rotation speed. The control unit 33 causes the mirror drive unit 43 to rotate the mirror 41 at the calculated mirror rotation speed, so that the mirror 41 starts rotating from a predetermined initial angle. This performs blur correction during imaging by the imaging device 11 (step S4). At the same time, the control unit 33 also continues to send a Hi signal, for example, as an ON signal to instruct exposure, to the camera control unit 27 for the duration of the first exposure time Tp1.

[0056] In the imaging device 11, the camera control unit 27 acquires an image by opening the shutter 24 and exposing the image while receiving a Hi signal (step S5), and stores the acquired image in the storage unit 15. When the first exposure time Tp1 has elapsed, the control unit 33 continues to send a Low signal to the camera control unit 27 as an OFF signal to instruct the camera to stop exposure. Alternatively, a Low signal may be used as the ON signal to instruct exposure, and a Hi signal may be used as the OFF signal to instruct the camera to stop exposure.

[0057] While the camera control unit 27 receives a Low signal, it closes the shutter 24, and the control unit 33 causes the mirror drive unit 43 to rotate the mirror 41 in the reverse direction to return the mirror 41 to its initial angle. Alternatively, the mirror drive unit 43 may rotate the mirror 41 in the forward direction to return the mirror 41 to its initial angle.

[0058] The control unit 33 determines whether the series of imaging has been completed (step S6). If it determines that imaging has been completed (Yes in step S6), it terminates imaging while moving. If the control unit 33 determines that imaging has not been completed (No in step S6), it returns to step S1 to capture an image in the next frame.

[0059] When images are captured continuously, and therefore the operation of the imaging system 1 after returning to step S1 will be explained with reference to Figure 7. Figure 7 is a graph showing the relationship between the change in moving speed, the timing of the exposure time, and the motion blur correction angle. Figure 7(a) is a graph showing the moving speed of the vehicle 3 as it progresses over time. Figure 7(b) is a graph showing the timing of the exposure time for each frame. Figure 7(c) is a graph showing the motion blur correction angle calculated for each frame. In Figure 7, for example, if the moving speed V1 is 75 km / h, the moving speed V2 is 100 km / h, the imaging interval is 1 / 60 second, and a small change in vehicle speed occurs 1 to 2 times within one frame.

[0060] When images are continuously captured at an imaging interval Tf, in step S1, the movement speed of the vehicle 3 is acquired at each imaging interval Tf. For example, the movement speed V1 is acquired at the start timing of the first frame. In step S2, the control unit 33 calculates the maximum exposure time Tmax from the acquired movement speed V1 and the maximum correctable angle θmax using equation (2).

[0061] Next, step S3 described above is performed in the same manner. The exposure time setting unit 53 sets a first exposure time Tp1 adjusted to be less than or equal to the maximum exposure time Tmax, based on the exposure time of the previous frame. In the first frame, the motion blur correction angle θ1 and the mirror swing angle α1 corresponding to the motion blur correction angle θ1 are calculated. Based on the mirror swing angle α1, the mirror rotation speed calculation unit 57 calculates the mirror rotation speed.

[0062] Next, step S4 described above is performed in the same manner. Simultaneously with step S4, the control unit 33 continues to send a Hi signal to the camera control unit 27 for the first exposure time Tp1. In the imaging device 11, the camera control unit 27 opens the shutter 24 to expose the image while receiving the Hi signal (step S5), and stores the acquired image in the storage unit 15. The start timing of the Hi signal to instruct exposure, which is transmitted from the control unit 33 to the camera control unit 27, is the same timing for each imaging interval Tf.

[0063] After this, steps S6 and No. of step S6 described above are performed in the same manner, and image acquisition in the first frame is completed, and the process returns to step S1 again in order to acquire an image in the next second frame. Back in step S1, for example, at the start of the second frame, a movement speed V2 greater than the movement speed V1 of the first frame is acquired.

[0064] In step S2, the control unit 33 calculates the maximum exposure time Tmax using equation (2) from the acquired movement speed V2 and the maximum correctable angle θmax. Next, step S3 described above is performed in the same manner. The exposure time setting unit 53 sets a second exposure time Tp2 adjusted to be less than or equal to the maximum exposure time Tmax, based on the first exposure time Tp1 of the first frame.

[0065] In Figure 7, since the movement speed V2 of the second frame is greater than the movement speed V1 of the first frame, the movement shake correction angle θ2 and mirror swing angle α2 are calculated to be greater than the movement shake correction angle θ1 and mirror swing angle α1 of the first frame. Hereafter, we will explain assuming that the movement shake correction angle θ2 has reached, for example, the maximum correctable angle θmax. Based on the mirror swing angle α2, the mirror rotation speed calculation unit 57 calculates the mirror rotation speed. Similarly, the processes of steps S4 and S5 are performed to obtain the captured image of the second frame. Next, we return to step S1 to capture an image of the third frame.

[0066] Again in step S1, for example, at the start of the third frame, a movement speed V3 is obtained that is greater than the movement speed V2 of the second frame.

[0067] In step S2, the control unit 33 calculates the maximum exposure time Tmax using equation (2) from the acquired movement speed V3 and the maximum correctable angle θmax. Next, step S3 described above is performed in the same manner. The exposure time setting unit 53 sets a third exposure time Tp3 adjusted to be less than or equal to the maximum exposure time Tmax, based on the second exposure time Tp2 of the second frame. The maximum exposure time Tmax varies depending on the movement speed of the vehicle 3, but if the exposure times Tp2 and Tp3 of the second and third frames are equal to the maximum exposure time Tmax at their respective speeds, then Tp2 > Tp3. In other words, in Figure 7, the movement blur correction angle θ2 of the second frame has reached the maximum correctable angle θmax, and the movement speed in the third frame is faster than in the second frame, so the maximum exposure time for which blur correction is possible in the third frame becomes shorter. Therefore, the third exposure time Tp3 is set to be shorter than the second exposure time Tp2.

[0068] In Figure 7, although the movement speed V3 of the third frame is greater than the movement speed V2 of the second frame, the motion blur correction angle θ2 of the second frame is the maximum correctable angle θmax, so the same motion blur correction angle θ2 and mirror swing angle α2 as the second frame are calculated for the third frame. At this time, since the blur correction angles of the second and third frames are the maximum correctable angle θmax, even if the movement speed V3 of the third frame is greater than the movement speed V2 of the second frame, the motion blur correction angle of the third frame is equal to the motion blur correction angle θ2 of the second frame. Therefore, the mirror rotation speed calculation unit 57 calculates the mirror rotation speed based on the mirror swing angle α2 of the third frame. The captured image of the third frame can then be obtained by performing steps S4 and S5 in the same manner.

[0069] When imaging continuously, although minute changes in movement speed occur within a single frame, the movement speed is acquired at one-frame intervals and the motion blur correction angle is controlled according to the movement speed, thus suppressing image blur caused by changes in movement speed. When the exposure time setting unit 53 always sets the exposure time Tp to the maximum exposure time Tmax, although the brightness changes with each frame, it is possible to acquire the image with the highest brightness while reducing blur. Also, when the exposure time setting unit 53 sets the exposure time Tp to a constant exposure time that is less than the maximum exposure time Tmax but greater than the initial exposure time Tp0, it is possible to acquire an image with reduced blur, higher brightness, and constant brightness between frames, resulting in an easy-to-view image.

[0070] [3. Effects, etc.] Thus, the imaging system 1 is an imaging system installed on the vehicle 3. The imaging system 1 includes an imaging device 11 having an image sensor 25 that images an imaging target area 9 which is at least a part of the area around the vehicle 3, away from the vehicle 3; a blur correction mechanism 31 that corrects blur in the direction of movement when the imaging device 11 takes images while the vehicle 3 is moving; and a control unit 33 that controls the exposure time of the imaging device 11 based on the moving speed of the vehicle 3 and the correctable amount of blur Sf corresponding to the maximum correctable angle θmax by the blur correction mechanism 31.

[0071] The control unit 33 controls the exposure time of the imaging device 11 based on the movement speed of the vehicle 3 and the amount of blur Sf that can be corrected by the blur correction mechanism 31, thereby reducing blur and enabling imaging with an appropriate exposure time. Conventionally, in dark environments, if the exposure time is set to be long, the movement speed of the vehicle 3 is fast, and the exposure time is not controlled based on the amount of blur that can be corrected, blur exceeding the correctable amount may occur, and even if sufficient light is secured, the image may be blurred and the inspection accuracy may decrease. Also, conventionally, in order to reliably achieve blur correction, the exposure time is set to be short, the movement speed of the vehicle 3 is slow, and the exposure time is not controlled based on the amount of blur that can be corrected, but blur correction can be performed at a correction angle smaller than the maximum correctable angle θmax, so a blur-free image can be obtained without problems in terms of blur correction accuracy. However, in dark environments, even if there is no blur, sufficient light may not be secured. The imaging system 1 of this embodiment optimizes the exposure time based on the premise of blur correction, making it possible to fully demonstrate blur correction performance and secure the maximum amount of light, thereby realizing the acquisition of bright and clear images.

[0072] (Other embodiments) As described above, the above embodiments have been explained as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these embodiments and can be applied to embodiments that have been modified, replaced, added, or omitted as appropriate. Therefore, other embodiments will be illustrated below.

[0073] In the above embodiment, the imaging device 11 and the image stabilization mechanism 31 were separate components, but this is not limited to this configuration. As shown in Figure 8, the imaging device 11 may have a housing 22 that connects the lens 23 and the camera body 21, and the image stabilization mechanism 31 may be housed inside the housing 22.

[0074] In the above embodiment, information on the moving speed V1 from the vehicle 3's speed detection device 3a was used, but the embodiment is not limited to this. The imaging system 1 may be equipped with a speed detection device that detects the moving speed of the imaging system 1. Furthermore, the speed detection device may utilize a GPS (Global Positioning System) system.

[0075] In the above embodiment, the imaging system 1 was imaging the upper and side walls of the vehicle 3, but it is not limited to this. The imaging system 1 may also be imaging the road surface below the vehicle 3. From the captured images, potholes, cracks, rutting, etc., that have occurred on the road surface can be detected by image processing.

[0076] In the above embodiment, the case where the moving object is a vehicle 3 such as an automobile was described. However, the moving object is not limited to a vehicle 3, but may be a ground vehicle such as a train or motorcycle, a ship traveling on the sea, or an aircraft such as an airplane or drone. When the moving object is a ship, the imaging system 1 images the bottom surface of bridge piers and bridge girders, or structures built along the shore. When the moving object is a train, the position and wear of the overhead wires can be detected by imaging the overhead wires.

[0077] In the above embodiment, an image was captured using light reflected from ambient light onto the imaging target area 9, but this is not limited to this. Light may also be shone from a moving object or imaging system toward the imaging target area 9, and an image may be captured using the reflected light of the shone object.

[0078] (Summary of the embodiment) (1) The imaging system of the present disclosure is an imaging system installed on a moving body and comprises: an imaging device having an image sensor for imaging an object that is located away from the moving body and is at least a part of the area surrounding the moving body; a blur correction mechanism for correcting blur in the direction of movement when the imaging device takes an image while the moving body is moving; and a control unit that controls the exposure time of the imaging device based on the speed of movement of the moving body and the amount of blur that can be corrected by the blur correction mechanism.

[0079] This reduces blur and allows for the acquisition of images captured with the appropriate exposure time.

[0080] (2) In the imaging system of (1), the image stabilization mechanism is a mechanism that aligns the direction of light reflected by ambient light from the object to be imaged with the imaging direction of the image sensor. The amount of correctable blur is defined by the angle of change of the image stabilization mechanism, and the control unit controls the image stabilization mechanism based on the amount of correctable blur, the movement speed, and the exposure time.

[0081] (3)(2) In the imaging system, the blur correction mechanism includes a mirror that totally reflects the light reflected by the object being imaged from the ambient light towards the imaging device, and a mirror drive unit that rotates the mirror. The amount of correctable blur is determined by the rotation angle of the mirror and the pixel range of the image sensor, and the control unit controls the rotation of the mirror by the mirror drive unit based on the amount of correctable blur, the movement speed and the exposure time.

[0082] (4) In the imaging system of (3), the control unit calculates the maximum exposure time that the imaging device can capture based on the movement speed of the moving object and the amount of correctable blur.

[0083] In the imaging system of (5)(4), the control unit controls the exposure time of the imaging device to be less than or equal to the maximum exposure time.

[0084] In the imaging system of (6)(5), if the rotation angle corresponding to the movement speed of the moving body is smaller than the maximum correctable angle that can be driven by the mirror drive unit, the control unit extends the exposure time, which is set in advance for the movement speed, to a time less than or equal to the maximum exposure time. If the rotation angle corresponding to the movement speed of the moving body is larger than the maximum correctable angle, the control unit sets the exposure time, which is set in advance for the movement speed, to the maximum exposure time.

[0085] (7) In any of the imaging systems described in (4) to (6), the control unit sets the maximum exposure time to be less than or equal to the imaging interval of continuous imaging.

[0086] (8) In any of the imaging systems described in (1) to (7), a speed detection device is provided to detect the movement speed of the imaging system.

[0087] (9) The mobile body of the present disclosure comprises any of the imaging systems (1) to (8). This allows the imaging system to capture images of the area around the mobile body with reduced blur while the mobile body is moving.

[0088] The imaging system described in this disclosure is realized through hardware resources, such as a processor, memory, and cooperation with programs. [Industrial applicability]

[0089] This disclosure is applicable to imaging systems installed on moving objects. [Explanation of Symbols]

[0090] 1. Imaging System 3 vehicles 3a Speed ​​detection device 5 tunnels 5a Wall surface 5b hole 5c cracks 7 Control section 9. Area to be imaged 11 Imaging device 13 Image stabilization device 15 Storage section 17 Control Unit 21 Camera body 23 lenses 24 shutters 25 Image sensor 27 Camera Control Unit 31 Image stabilization mechanism 33 Control Unit 41 Mirror 43 Mirror drive unit 51 Maximum exposure time calculation unit 53 Exposure time setting section 55 Mirror Angle Calculation Unit 57 Mirror rotation speed calculation unit α Mirror Swing Angle f focal length M Subject magnification Tf imaging interval Tmax Maximum exposure time θmax is the maximum possible corrected angle. V1 Movement Speed

Claims

1. An imaging system installed on a mobile device, An imaging device having an image sensor that captures an image target which is at least a part of the area surrounding the moving body, away from the moving body; The system includes a mirror that totally reflects ambient light reflected by the object to be imaged toward the image sensor, a mirror drive unit that rotates the mirror, and a blur correction mechanism that corrects blur in the direction of movement when the imaging device takes an image while the moving object is moving. The system includes a control unit that controls the exposure time of the imaging device based on the movement speed of the moving body and the correctable amount of blur defined by the rotation angle of the mirror of the blur correction mechanism and the pixel range of the image sensor, and controls the rotation of the mirror by the mirror drive unit based on the correctable amount of blur, the movement speed and the exposure time, Imaging system.

2. The aforementioned image stabilization mechanism is a mechanism that aligns the direction of light reflected by the object being imaged from ambient light with the imaging direction of the image sensor. The amount of correctable shake is defined by the maximum adjustable angle of the shake correction mechanism. The control unit controls the image stabilization mechanism based on the correctable amount of blur, the movement speed, and the exposure time. The imaging system according to claim 1.

3. The control unit calculates the maximum exposure time that the imaging device can capture based on the movement speed of the moving body and the amount of correctable blur. The imaging system according to claim 1.

4. The control unit controls the exposure time of the imaging device to be less than or equal to the maximum exposure time. The imaging system according to claim 3.

5. The control unit extends the exposure time, which is set in advance for the moving speed, to a time less than or equal to the maximum exposure time, if the rotation angle corresponding to the moving speed of the moving body is smaller than the maximum correctable angle that can be driven by the mirror drive unit, and sets the exposure time, which is set in advance for the moving speed, to the maximum exposure time, if the rotation angle corresponding to the moving speed of the moving body is larger than the maximum correctable angle. The imaging system according to claim 4.

6. The control unit sets the maximum exposure time to be less than or equal to the imaging interval of continuous imaging. The imaging system according to any one of claims 3 to 5.

7. The system includes a speed detection device for detecting the movement speed of the imaging system. The imaging system according to any one of claims 1 to 6.

8. The control unit sets the exposure time before each image captured by the imaging device. The imaging system according to any one of claims 1 to 7.

9. A mobile body comprising an imaging system as specified in any one of claims 1 to 8.

Images