Imaging system and imaging method

Abstract

To provide an imaging system and an imaging method capable of suppressing image streaming.SOLUTION: An imaging system is mounted on a moving body and captures an image of an object and is equipped with an imaging device that captures the image of the object and a laser Doppler velocimeter that measures the speed of the moving body. The imaging device is equipped with a processor, an image sensor, a mechanism for moving the image sensor, and a motion detection sensor that detects movement of the imaging device. When the speed measured by the laser Doppler velocimeter or the speed calculated on the basis of the acceleration from the motion detection sensor is incorrect, the processor determines that either the speed measured by the laser Doppler velocimeter or the speed calculated on the basis of the acceleration from the motion detection sensor is the target speed, and captures the image of the target object with the image sensor while moving the image sensor in the same direction as the moving object using a mechanism and at a moving speed corresponding to the target speed.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to an imaging system and an imaging method. [Background technology]

[0002] Patent Document 1 discloses an intermittent tracking photographing device that includes an image sensor that photographs a subject at a predetermined frame rate, a change means that changes the relative positional relationship between the subject and the image sensor, and a control unit that controls the change means for each frame so that when the shutter within the frame is opened, the subject is tracked to suppress changes in the relative positional relationship that occur due to movement of the subject or movement of the device, and when the shutter within the frame is closed, the relative positional relationship is returned to a predetermined home position. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2020-527814 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of one embodiment of the technique disclosed herein is to provide an imaging system and imaging method that are mounted on a moving body and are capable of suppressing image streaming when imaging an object. [Means for solving the problem]

[0005] The first embodiment of the photographing system is mounted on a moving body and photographs an object, and comprises an photographing device that photographs the object and a laser Doppler velocimeter that measures the speed of the moving body. The photographing device comprises a processor, an image sensor, a mechanism for moving the image sensor, and a motion detection sensor that detects movement of the photographing device. The processor determines as a target speed either the speed measured by the laser Doppler velocimeter or, if the speed measured by the laser Doppler velocimeter is incorrect, a speed calculated based on the acceleration from the motion detection sensor. The processor photographs the object with the image sensor while moving the image sensor in the same moving direction as the moving body and at a moving speed corresponding to the target speed using the mechanism.

[0006] In the imaging system of the second aspect, in the first aspect, the motion detection sensor includes an acceleration sensor and an angular velocity sensor.

[0007] In the imaging system of the third aspect, in the second aspect, the acceleration is a translational acceleration component obtained by subtracting a gravitational acceleration component and a rotational motion component calculated based on the angular velocity of the angular velocity sensor from the output value of the acceleration sensor.

[0008] In the imaging system of a fourth aspect, in any one of the first to third aspects, the processor causes the image sensor to be positioned at a standby position by a mechanism before imaging the object.

[0009] The imaging system of a fifth aspect is any one of the first to fourth aspects, wherein the moving object is a vehicle.

[0010] A sixth embodiment of the photographing method is a photographing method for photographing an object by mounting an photographing system on a moving object, the photographing system comprising an photographing device for photographing an object and a laser Doppler velocimeter for measuring the speed of the moving object, the photographing device comprising a processor, an image sensor, a mechanism for moving the image sensor, and a motion detection sensor for detecting the movement of the photographing device, the processor determines as a target speed either the speed measured by the laser Doppler velocimeter or, if the speed measured by the laser Doppler velocimeter is incorrect, a speed calculated based on the acceleration from the motion detection sensor, and photographs the object with the image sensor while moving the image sensor in the same moving direction as the moving object and at a moving speed corresponding to the target speed using the mechanism.

[0011] A seventh aspect of the imaging method is the sixth aspect, wherein the motion detection sensor includes an acceleration sensor and an angular velocity sensor.

[0012] In an eighth aspect of the imaging method, in the seventh aspect, the acceleration is a translational acceleration component obtained by subtracting a gravitational acceleration component and a rotational motion component calculated based on the angular velocity of the angular velocity sensor from the output value of the acceleration sensor.

[0013] A ninth aspect of the imaging method is any one of the sixth to eighth aspects, in which the processor positions the image sensor at a standby position using a mechanism before imaging the object.

[0014] The imaging method of a tenth aspect is any one of the sixth to ninth aspects, wherein the moving object is a vehicle. [Brief description of the drawings]

[0015] [Figure 1] 1 is a diagram showing a state in which an inner wall of a tunnel is photographed by a vehicle equipped with an imaging system according to an embodiment. [Diagram 2] 10A and 10B are diagrams for explaining the relationship between the timing of shooting and the position of an image sensor. [Diagram 3] FIG. 2 is a diagram for explaining a laser Doppler speedometer attached to a vehicle. [Figure 4] FIG. 1 is a block diagram showing a schematic configuration of a photographing system. [Diagram 5] 13 is a flowchart showing a processing procedure. [Figure 6] 13 is a timing chart of a processing procedure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, an imaging system and an imaging method according to an embodiment will be described with reference to the drawings.

[0017] [Embodiment] Many social infrastructure facilities are aging. Tunnels are particularly prone to deterioration (cracks in the walls), and in 2019 the Ministry of Land, Infrastructure, Transport and Tourism revised its road tunnel regular inspection guidelines, recommending that visual inspections be replaced with camera image measurement mounted on mobile inspection vehicles, which do not require traffic restrictions, to improve efficiency.

[0018] Inspection methods using camera image measurement include the line scan method and the area scan method.

[0019] The line scan method is a method of photographing an object with an image sensor in which photodiodes are arranged in a single row, and there is no image blur even when photographing while moving at a high speed of 40km / h, and it can achieve the "measurement accuracy" (image quality that allows crack width to be read in 0.1mm units) that the Ministry of Land, Infrastructure, Transport and Tourism requires as a required performance. However, since image processing after photographing cannot be automated and manual work is required, the investment is not sufficient. The vehicle speed of 40km / h is a conventional standard established because it is 80% of the maximum speed on general roads and allows inspections to be performed without disrupting traffic.

[0020] The area scan method is a method of photographing an object using an image sensor with a two-dimensional array of photodiodes, and it is possible to achieve image processing that does not require manual labor. However, depending on the shooting environment, it may be difficult to increase the shutter speed because sufficient illumination light cannot be obtained for a wide shooting range, and image blur may occur when shooting while moving at high speed. Therefore, the area scan method can only be applied to tunnel inspections where low-speed movement is acceptable. However, the shooting conditions are subject to restrictions on the brightness of the lighting so as not to interfere with traffic, the depth of field, and to avoid high-sensitivity noise affecting the measurement accuracy.

[0021] [overview] Fig. 1 is a diagram showing a state in which an inner wall IW of a tunnel TU is photographed by a vehicle 40 equipped with an imaging system 1 of the embodiment. As shown in Fig. 1, the imaging system 1 including a camera unit 10 and a laser Doppler speedometer 30 is mounted on the vehicle 40. The vehicle 40 equipped with the imaging system 1 travels at a constant speed on a road R inside the tunnel TU in a traveling direction A indicated by an arrow.

[0022] The camera unit 10 photographs the inner wall IW from the vehicle 40 traveling at a constant speed with an area scan type image sensor. The camera unit 10 photographs only a limited photographing range SR in one photographing. Therefore, in order to photograph the entire range in the longitudinal direction of the tunnel TU, the camera unit 10 performs photographing SP at regular intervals Le in the traveling direction A while the vehicle 40 is traveling along the road R. Note that the photographing SP may be called exposure. The position where the photographing SP is performed is determined in advance based on the distance of the tunnel TU, the photographing range SR, and the overlapping rate of adjacent photographing ranges SR. The positions where each photographing SP is performed are determined in advance as photographing positions L1s, L2s, ... Lns. L1s, L2s, ... Lns are distances from a reference position. n is a parameter that identifies each photographing position.

[0023] A plurality of images captured by the camera unit 10 are combined in the longitudinal direction of the tunnel TU to generate a combined image CI. The generated combined image CI is used for inspection.

[0024] Next, it will be described how the camera unit 10 of the embodiment can suppress image streaming. Fig. 2 is a diagram for explaining how the camera unit 10 can suppress image streaming. As shown in Fig. 2, the camera unit 10 includes a lens device 100 and an area scan type image sensor 102. Note that the laser Doppler velocimeter 30 is omitted.

[0025] The image sensor 102 is supported by a moving mechanism 114 (see FIG. 4) that moves the image sensor 102 mounted on the camera unit 10. The image sensor 102 can be moved in the forward and backward directions along the traveling direction A between a first position P1 and a second position P2 by the moving mechanism 114. The forward and backward directions along the traveling direction A are also referred to as the translation direction. Here, the forward direction is the same direction as the traveling direction A, and the backward direction is the opposite direction to the traveling direction A.

[0026] As shown in Fig. 2, the camera unit 10 is moved in a traveling direction A by a vehicle 40. Fig. 2 shows the relationship between the position (movement trajectory) of the image sensor 102 and the timing of shooting SP. Here, the first to third states show the position of the camera unit 10 with respect to the inner wall IW over time.

[0027] The first state is a state before the camera unit 10 performs a shooting SP, and the image sensor 102 is located at a first position P1. The first position P1 is an example of a standby position of the present invention. The second position P2 is a position on the opposite side to the first position P1.

[0028] The second state is a state in which the camera unit 10 captures an image SP of the inner wall IW. The image sensor 102 moves from the first position P1 to an intermediate position between the first position P1 and the second position P2. The image sensor 102 moves in the same direction as the traveling direction A at a moving speed corresponding to the speed of the vehicle 40. By moving the image sensor 102, the capturing range SR can be moved in the image flow direction of the inner wall IW.

[0029] Since the image sensor 102 captures an image of the inner wall IW while moving in the direction of the second position P2 at a moving speed corresponding to the speed of the vehicle 40, image streaming can be suppressed.

[0030] The third state is a state in which the image sensor 102 passes the intermediate position and reaches the second position P2 after capturing an image. When the image sensor 102 reaches the second position P2 in the third state, the image sensor 102 starts moving toward the first position.

[0031] Next, the image sensor 102 moves from the second position P2 to the first position P1, and changes from the third state to the first state. By repeating the first state to the third state, the imaging system 1 obtains a plurality of captured images along the longitudinal direction of the tunnel TU, as shown in FIG. 1, while suppressing image flow.

[0032] In the imaging system 1 of the embodiment, since the image sensor 102 is moved at a moving speed corresponding to the speed of the vehicle 40 while imaging, it is important to accurately calculate the speed of the vehicle 40. The imaging system 1 applies a laser Doppler speedometer 30 from the viewpoints of measurement stability, accuracy, and processing time required for imaging inspection.

[0033] Fig. 3 is a diagram for explaining the laser Doppler speedometer 30. 3-1 in Fig. 3 is a diagram for explaining a case where the laser Doppler speedometer 30 calculates the speed (ground speed or road surface speed) of the vehicle 40. 3-2 in Fig. 3 is a diagram for explaining a case where the laser Doppler speedometer 30 cannot calculate the speed.

[0034] As shown in 3-1 of FIG. 3, the laser Doppler speedometer 30 is disposed at a position facing the road R. The laser Doppler speedometer 30 irradiates the road R, which moves relative to the laser Doppler speedometer 30, with irradiation light LD (laser light) from the front and rear sides in the moving direction. The laser Doppler speedometer 30 receives scattered light SL of each irradiation light LD reflected from the road R at a light receiving unit. Here, the scattered light contains frequency components affected by the Doppler effect. The laser Doppler speedometer 30 extracts the Doppler frequency and calculates the speed of the vehicle 40 based on the Doppler frequency.

[0035] The laser Doppler speedometer 30 uses laser light and does not require contact with the road R. The laser Doppler speedometer 30 can measure speed with high accuracy and can process data at high speed, so that the speed can be calculated in real time.

[0036] On the other hand, the laser Doppler speedometer 30 is affected by the surface properties of the road R. For example, as shown in 3-2 of FIG. 3, if there is a step D on the road R, the light receiving unit may not be able to receive the scattered light SL. In addition, if the road R absorbs the irradiated light LD, or if the surface of the road R is wet, the light receiving unit may not be able to receive the scattered light SL. If the laser Doppler speedometer 30 cannot accurately calculate the speed of the vehicle 40, the image sensor 102 cannot be moved at a moving speed corresponding to the speed of the vehicle 40. As a result, image blurring occurs in the captured image.

[0037] Therefore, the imaging system 1 of the embodiment estimates the speed of the vehicle 40 when an error occurs in the calculation of the speed by the laser Doppler speedometer 30. Specifically, the imaging system 1 estimates the speed of the vehicle 40 from the speed of the vehicle 40 immediately before the error occurred, the elapsed time until the error occurred, and the acceleration. The acceleration is acquired from an acceleration sensor, which is one of the motion sensors built into the camera unit 10. Since the estimation of the speed of the vehicle 40 is executed by the CPU 112 (see FIG. 4) provided in the camera unit 10, the speed of the vehicle 40 can be estimated at high speed.

[0038] Next, the photographing system 1 will be described in detail.

[0039] Fig. 4 is a block diagram showing a schematic configuration of the photography system 1. As shown in Fig. 4, the photography system 1 includes a camera unit 10 and a laser Doppler velocimeter 30 (LV velocimeter 30).

[0040] The camera unit 10 includes a lens device 100. The lens device 100 includes a lens group 104 and an aperture 106. The lens device 100 also includes a lens driving unit (not shown) that drives the lens group 104, an aperture driving unit (not shown) that drives the aperture, and the like. The lens device 100 may be detachable from the camera unit 10, or may be integrated with the camera unit 10. The lens group 104 includes at least a focus lens that is movable in the optical axis direction. The aperture 106 is formed of, for example, an iris aperture.

[0041] 4, the camera unit 10 includes an image sensor 102, a shutter 108, and a system control unit 110 including a CPU (Central Processing Unit) 112. The camera unit 10 includes a movement mechanism 114 that moves the image sensor 102, a motion detection sensor 116 that includes an acceleration sensor 118 and an angular velocity sensor 120, and a clock 122.

[0042] The image sensor 102 is, for example, a CMOS (Complementary Metal-Oxide Semiconductor) type image sensor having a predetermined color filter array (for example, a Bayer array, etc.). In the camera unit 10 of the embodiment, the image sensor 102 is configured to include a drive unit, an ADC (Analog to Digital Converter), a signal processing unit, etc. The image sensor 102 can be configured with an organic thin-film imaging element, an XY address type, a CCD (Charged Coupled Device) type image sensor, etc., in addition to the CMOS type image sensor.

[0043] The shutter 108 is disposed between the aperture 106 and the image sensor 102. The shutter 108 is driven by a shutter drive unit (not shown). The shutter drive unit controls the opening and closing of the shutter 108 in response to a trigger signal from an imaging control 134 of the CPU 112 (described later), and controls the exposure time (shutter speed) in the image sensor 102.

[0044] The moving mechanism 114 supports the image sensor 102 and can move it, for example, in a plane perpendicular to the optical axis of the image sensor 102. The moving mechanism 114 can move the image sensor 102 at least in a translation direction along the traveling direction A. The moving mechanism 114 can include an actuator such as a motor (for example, a voice coil motor). The image sensor 102 is moved by applying a driving force generated by the actuator to the image sensor 102.

[0045] The motion detection sensor 116 includes an acceleration sensor 118 and an angular velocity sensor 120. The motion detection sensor 116 only needs to include at least the acceleration sensor 118.

[0046] Furthermore, the moving mechanism 114 may include a Hall element that generates a voltage signal corresponding to the position in order to obtain position information of the image sensor 102 .

[0047] The clock 122 measures the time required to control the image capture system 1, such as the time until the LV speedometer 30 next acquires the speed of the vehicle 40.

[0048] The system control unit 110 performs overall control of the camera unit 10. In addition to a CPU 112, the system control unit 110 may include a ROM (Read-only Memory), a RAM (Random Access Memory), and the like. The system control unit 110 is connected to the LV speedometer 30, the image sensor 102, the shutter 108, the moving mechanism 114, the acceleration sensor 118, the angular velocity sensor 120, and a clock 122. By executing various programs, the CPU 112 can acquire information necessary for controlling the camera unit 10, calculate various physical quantities (command values), and generate and output signals.

[0049] The camera unit 10 is an example of an imaging device of the present invention. The image sensor 102 is an example of an image sensor of the present invention. The laser Doppler speedometer 30 is an example of a laser Doppler speedometer of the present invention. The vehicle 40 is an example of a moving body of the present invention. The CPU 112 is an example of a processor of the present invention. The moving mechanism 114 is an example of a mechanism of the present invention. The motion detection sensor 116 is an example of a motion detection sensor of the present invention. The acceleration sensor 118 is an example of an acceleration sensor of the present invention. The angular velocity sensor 120 is an example of an angular velocity sensor of the present invention.

[0050] Next, the flow of calculation processing by the CPU 112 will be described. As shown in FIG. 4, the CPU 112 executes processing required for control. The LV output value (speed LV) calculated by the LV speedometer 30, the time measured by the clock 122, and the translational acceleration are output to the CPU 112 and processed by the calculation processing 130. The calculation processing 130 determines a target speed Vn and a moving distance Ln from the LV output value (speed), the time, and the translational acceleration. The calculation processing 130 determines a command value 132 from the target speed Vn and the moving distance Ln. The command value 132 is output to the shooting control 134. The shooting control 134 generates a trigger signal that turns the shutter ON. The trigger signal is output to the shutter drive unit, and the shutter 108 is opened and closed to perform shooting SP. The moving distance Ln is a distance from a reference position, and is calculated as L1, L2, ... Ln for each distance. n is a parameter for identifying the moving distance.

[0051] The command value 132 is output to the shift control 136. The shift control 136 also receives a position signal from the Hall element of the movement mechanism 114. The shift control 136 calculates a start time ts and a movement speed VIn at which to start moving the image sensor 102, from the command value 132 and the position signal, in order to suppress image flow. The motor control 138 generates a control signal corresponding to the start time ts and the movement speed VIn. The control signal is output to the movement mechanism 114. The movement mechanism 114 moves the image sensor 102 in a direction (translation direction) and at a movement speed VIn in which image flow is suppressed.

[0052] The acceleration from the acceleration sensor 118 and the angular velocity from the angular velocity sensor 120 are output to the shake processing 140 of the CPU 112. The shake processing 140 calculates the amount of shake EB of the camera unit 10 from the angular velocity and the acceleration. The shake processing 140 determines a command value 142 from the amount of shake EB. In the shake processing 140, a translational component of the image sensor 102 is excluded from the command value 142. The translational component is used to suppress image blur. The command value 142 is output to the camera shake correction 144. The camera shake correction 144 also receives a position signal from a Hall element of the movement mechanism 114. The camera shake correction 144 calculates the movement direction of the image sensor 102, etc., to cancel the camera shake in the direction other than the translational direction, from the command value 142 and the position signal. The motor control 146 generates a control signal corresponding to the movement direction of the image sensor 102, etc. The control signal is output to the movement mechanism 114. The moving mechanism 114 moves the image sensor 102 in a direction in which image stabilization is suppressed.

[0053] Next, a description will be given of a photographing method using the photographing system 1. Fig. 5 is a flow chart showing the processing procedure. Fig. 6 is a timing chart showing the processing procedure.

[0054] First, the speed LV of the vehicle 40 measured by the laser Doppler speedometer 30 is acquired (step S1). Specifically, the ground speed measured by the laser Doppler speedometer 30 is acquired and output to the system control unit 110 as the speed LV (LV output value) of the vehicle 40. As shown in FIG. 6, the laser Doppler speedometer 30 acquires the speed LV of the vehicle 40 while traveling. The system control unit 110 acquires the speed LV from the laser Doppler speedometer 30 in about 2 [ms]. Note that the speed LV is acquired multiple times while traveling through the tunnel TU. A parameter n is added to identify each acquired speed LV. The speed LV acquired in step S1 is processed as a speed LVn. Therefore, the next acquired speed LV is processed as a speed LVn+1.

[0055] Next, in-camera information is acquired (step S2). Specifically, the system control unit 110 acquires time t from the clock 122 and acceleration sensor output value α from the acceleration sensor 118. Similar to step S1, the acquired multiple times t are processed as time tn with parameters added. Also, the acquired multiple acceleration sensor output values ​​α are processed as acceleration sensor output values ​​αn with parameters added.

[0056] 6, time tn is the time from when the laser Doppler velocimeter 30 acquires a velocity LVn-1 to when it acquires a velocity LVn. The acceleration sensor output value αn is the output value from the acceleration sensor 118 from when it acquires the velocity LVn-1 to when it acquires the velocity LVn.

[0057] Next, various information is input to the calculation processing of the CPU 112 (step S3). Specifically, the speed LVn, the time tn, and the acceleration sensor output value αn are input to the calculation processing 130 of the CPU 112.

[0058] Next, it is determined whether the speed LV (=speed LVn) acquired from the laser Doppler velocimeter 30 is correct (step S4). Specifically, in the calculation process 130 of the CPU 112, it is determined whether the speed LVn is correct. For example, the speed LVn can be determined to be incorrect in the following cases. Examples of such cases include when the difference from the previously measured speed LVn-1 is large compared to a threshold value, when the laser Doppler velocimeter 30 cannot receive the scattered light correctly and cannot acquire the speed LVn, and when a malfunction occurs in the laser Doppler velocimeter 30.

[0059] If the velocity LV (=velocity LVn) acquired from the laser Doppler velocimeter 30 is correct (step S4: Yes), the process proceeds to step S5.

[0060] Next, the speed LV (speed LVn) is used to calculate the speed and the travel distance (step S5). Specifically, the calculation process 130 calculates the speed and the travel distance. The calculation process 130 determines the target speed Vn by the following formula (1). Also, the calculation process 130 determines the travel distance Ln by the following formula (2).

[0061]

number

[0062]

number

[0063] If the velocity LV (=velocity LVn) acquired from the laser Doppler velocimeter 30 is incorrect (step S4: No), the process proceeds to step S6.

[0064] Next, the acceleration sensor output value α (acceleration sensor output value αn) is used to calculate the speed and the travel distance (step S6). Specifically, the calculation process 130 calculates the speed and the travel distance. The calculation process 130 determines the target speed Vn by the following equation (3). Also, the calculation process 130 determines the travel distance Ln by the following equation (4).

[0065]

number

[0066]

number

[0067] The target speed Vn is the speed of the vehicle 40, and is a reference speed for determining the moving speed VIn when the image sensor 102 is moved.

[0068] As described above, if the speed LVn measured by the laser Doppler speed meter 30 is accurate, the target speed Vn is the speed LVn acquired by the laser Doppler speed meter 30 (see formula 1), and if the speed LVn measured by the laser Doppler speed meter 30 is not accurate, the target speed Vn is the speed calculated based on the acceleration sensor output value αn (see formula 3). The speed LVn is an example of a speed measured by the laser Doppler speed meter of the present invention. The acceleration sensor output value αn is an example of an acceleration of the present invention. The target speed Vn is an example of a target speed of the present invention. The moving speed VIn is an example of a moving speed of the present invention.

[0069] Although the acceleration sensor output value αn has been described as an example, the acceleration sensor output value αn may contain noise in the traveling direction due to changes in the attitude of the camera unit 10 attached to the tip of the photographing stand of the vehicle 40. The type of noise differs depending on the relative position of the camera unit 10 and the inner wall IW of the tunnel TU. For example, the following can be cited as directions in which attitude changes occur.

[0070] The first is road surface inclination, which causes noise in the direction of rotation with the tunnel width direction as the normal line. The second is meandering, which causes noise in the direction of rotation with the height direction as the normal line and the driving direction as the normal line. The third is step vibration, which causes noise in all directions.

[0071] Therefore, it is preferable to remove these noises by the following procedure.

[0072] First, the change in attitude of the camera unit 10 is calculated from the angular velocity of each axis measured by the angular velocity sensor 120.

[0073] Secondly, the acceleration of each axis measured by the acceleration sensor 118 is subjected to coordinate conversion based on the calculated attitude change. This is to deal with deviation of coordinates from the running direction due to attitude change.

[0074] Third, the change in the gravitational acceleration component is calculated from the calculated attitude change and subtracted from the acceleration of each coordinate-transformed axis. This is to deal with the effect of the gravitational acceleration component due to the attitude change.

[0075] Fourth, the rotational motion components obtained in step 3 are removed from the angular velocity of each axis to leave only the translational acceleration components.

[0076] The acceleration sensor output value, the translational acceleration component, the gravitational acceleration component, and the rotational motion component have the relationship shown in the following formula (5). Therefore, only the translational acceleration component can be extracted.

[0077]

number

[0078] The translational acceleration component is an example of the acceleration of the present invention.

[0079] Next, a moving speed VIn at which the image sensor 102 is moved is specified (step S7). Specifically, the shift control 136 of the CPU 112 calculates the moving speed VIn based on the target speed Vn.

[0080] Here, a method for calculating the moving speed VIn from the target speed Vn will be described. Assuming that the target speed Vn=40 km / h, the subject distance SD from the camera unit 10 to the subject (inner wall IW)=2500 mm, the focal length FL=17 mm, and the shutter speed=500 μs.

[0081] The target speed Vn can be expressed by the following equation (6).

[0082]

number

[0083] The target speed Vn is the relative speed of the subject (inner wall IW) with respect to the camera unit 10. Therefore, during exposure (shutter speed=500 μs), the subject (inner wall IW) advances by a movement distance during exposure La calculated by the following formula (7).

[0084]

number

[0085] The amount of image plane shift during exposure Lb can be calculated from the movement distance during exposure La, the subject distance SD = 2500 mm, and the focal length FL = 17 mm using the following formula (8). The amount of image plane shift during exposure Lb is the distance that the image advances on the image sensor 102.

[0086]

number

[0087] From the amount of image plane shift during exposure Lb and the shutter speed=500 μs, the image plane movement speed during exposure VIm can be calculated by the following formula (9).

[0088]

number

[0089] During exposure, image flow can be suppressed by moving the image sensor 102 in the opposite direction to the direction in which the subject moves at an image plane movement speed VIm during exposure. Therefore, the movement speed VIn of the image sensor 102 is 75.56 mm / s.

[0090] CPU 112 executes steps S2 to S7 in about 1 ms. Therefore, CPU 112 executes steps S1 to S7 in about 3 ms.

[0091] Next, the image sensor 102 starts moving after the start time ts (step S8). Specifically, the shift control 136 calculates the start time ts for moving the image sensor 102. As shown in FIG. 6, the start time ts can be calculated from the photographing position Lns where the exposure (photographing SP) is performed, the moving distance Ln which is the position where the laser Doppler velocimeter 30 measures the speed, the target speed Vn, the moving speed VIn, and the image plane shift amount during exposure Lb by the following formula (10).

[0092]

number

[0093] The motor control 138 generates a control signal corresponding to the start time ts and the moving speed VIn. The motor control 138 controls the moving mechanism 114 to move the image sensor 102 in a direction and at a moving speed VIn such that image streaming is suppressed.

[0094] Next, at the photographing position Lns, the shutter is turned ON to perform the photographing SP (step S9). Specifically, when the moving distance Ln reaches the photographing position Lns, the photographing control 134 generates a trigger signal to turn the shutter ON, and the shutter 108 is opened and closed to perform the photographing SP. The fact that the moving distance Ln reaches the photographing position Lns can be detected by the following formula (11) based on the moving distance Ln, the target speed Vn, and the elapsed time te. The elapsed time te is the time from when the laser Doppler velocimeter 30 measured the speed LV. The acquired multiple elapsed times te are processed as elapsed times ten. n is a parameter that identifies the multiple elapsed times te.

[0095]

number

[0096] Next, when the photographing is completed, the image sensor 102 is moved to the first position P1, which is a standby position (step S10).

[0097] Next, the timing from step S1 to step S10 will be described with reference to Fig. 6. As shown in Fig. 6, step S1 is executed at a position of travel distance Ln, and speed LVn is acquired from the laser Doppler speedometer 30. Then, steps S2 to S7 are executed. After the vehicle 40 reaches the travel distance Ln, steps S1 to S7 are executed in about 3 [ms]. As described above, in step S6, if the speed LV of the laser Doppler speedometer 30 is incorrect, a target speed Vn is calculated from the acceleration.

[0098] Step S8 is executed, and the image sensor 102 waiting at the first position P1 at the start time ts starts to move. In the first 5 ms, the image sensor 102 is accelerated from the first position P1 toward the second position P2.

[0099] Next, step S9 is executed, and the photographing SP is performed during the 10 ms period during which the image sensor 102 is moved at a constant speed VIn. That is, the image sensor 102 is controlled to move at a constant speed when passing the photographing position Lns. The constant speed movement time of the image sensor 102 is set to be longer than the shutter speed so that exposure (photographing SP) can be reliably performed while the image sensor 102 is moving at a constant speed. When the constant speed movement is completed, the image sensor 102 is decelerated and reaches the second position P2. Note that the moving mechanism 114 of the camera unit 10 can correct camera shake in directions other than the translation direction while the photographing SP is being performed.

[0100] Next, step S10 is executed, where the image sensor 102 is accelerated toward the first position P1, and is decelerated when it passes the center between the second position P2 and the first position P1. The image sensor 102 reaches the first position P1.

[0101] The timing at which the laser Doppler speedometer 30 acquires the speed LV is not particularly limited, but can be set to, for example, the timing at which (Lns-Ln) / Vn becomes 200 ms or less. Also, it can be appropriately changed according to the speed of the vehicle 40.

[0102] Next, as shown in Fig. 5, it is determined whether or not the entire range has been photographed (step S11). If it is determined that the entire range has not been photographed (step S11: No), the process proceeds to step S1, and steps S1 to S11 are repeated until the entire range has been photographed. On the other hand, if it is determined that the entire range has been photographed (step S11: Yes), the photographing ends. Whether or not the entire range has been photographed can be determined, for example, by whether the travel distance Ln is greater or smaller than a set threshold value.

[0103] Preferred Embodiments In the embodiment, the photographing system 1 including one camera unit 10 has been described, but multiple camera units 10 can be applied. For example, the multiple camera units 10 can be arranged so that the photographing range SR photographs different areas of the inner wall IW in the circumferential direction of the tunnel TU. More of the inner wall IW of the tunnel TU can be photographed in one photographing.

[0104] 4 illustrates a case where image stabilization is performed by moving mechanism 114 arranged in camera unit 10, but an image stabilization mechanism can be arranged in lens device 100 to correct image stabilization in directions other than the translational direction. That is, image stabilization may be performed by both moving mechanism 114 arranged in camera unit 10 and the image stabilization mechanism arranged in lens device 100.

[0105] Although the vehicle 40 is illustrated as an example of the moving body, the moving body is not particularly limited. The moving body may be a railroad car or a drone (unmanned aerial vehicle).

[0106] Although the present invention has been described above, the present invention is not limited to the above examples, and various improvements and modifications can be made without departing from the gist of the present invention.

[0107] [System Control Unit Configuration] The system control unit 110 is realized by various processors. The various processors include a CPU and / or a GPU (Graphic Processing Unit) which is a general-purpose processor that executes a program and functions as various processing units, a programmable logic device (PLD) such as an FPGA (Field Programmable Gate Array) which is a processor whose circuit configuration can be changed after manufacture, and a dedicated electric circuit such as an ASIC (Application Specific Integrated Circuit) which is a processor having a circuit configuration designed exclusively for executing a specific process. A program is synonymous with software.

[0108] A processing unit may be composed of one of these various processors, or may be composed of two or more processors of the same or different types. For example, a processing unit may be composed of multiple FPGAs, or a combination of a CPU and an FPGA. Also, multiple processing units may be composed of one processor. As an example of multiple processing units being composed of one processor, first, as represented by a computer used for a client or a server, one processor is composed of a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, as represented by a system on chip (SoC), there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units in one IC (Integrated Circuit) chip. In this way, the various processing units are composed of one or more of the above various processors as a hardware structure.

[0109] Although the present invention has been described above, the present invention is not limited to the above examples, and various improvements and modifications may be made without departing from the gist of the present invention. [Explanation of symbols]

[0110] 1. Shooting System 10 Camera unit 30 Laser Doppler Velocimeter 40 vehicles 100 Lens device 102 Image Sensor 104 Lens Group 106 Aperture 108 Shutter 110 System control section 112 CPU 114 Moving mechanism 116 Motion Detection Sensor 118 Acceleration Sensor 120 Angular Rate Sensor 122 Clock 130 Calculation Processing 132 Command value 134 Shooting Control 136 Shift Control 138 Motor Control 140 Runout Processing 142 Command value 144 Image Stabilization 146 Motor Control A Travel direction CI composite image D Step part LD irradiation light SL Scattered light Le interval Ln Distance traveled Lns Shooting position P1 1st position P2 2nd position R road SP shooting SR Shooting Range TU Tunnel IW inner wall

Claims

1. An imaging system mounted on a moving body for imaging an object, comprising: An imaging device for imaging an object; A laser Doppler velocimeter that measures the speed of the moving object; Equipped with The imaging device is a processor, an image sensor, a mechanism for moving the image sensor, and a motion detection sensor for detecting a motion of the photographing device; The processor, determining, as a target speed, either the speed measured by the laser Doppler speedometer or, if the speed measured by the laser Doppler speedometer is incorrect, a speed calculated based on the acceleration from the motion detection sensor; an image of the object is captured by the image sensor while the image sensor is moved by the mechanism in the same direction as the moving object and at a moving speed corresponding to the target speed; Shooting system.

2. The imaging system according to claim 1 , wherein the motion detection sensor includes an acceleration sensor and an angular velocity sensor.

3. The imaging system according to claim 2 , wherein the acceleration is a translational acceleration component obtained by subtracting a gravitational acceleration component and a rotational motion component calculated based on the angular velocity of the angular velocity sensor from an output value of the acceleration sensor.

4. The photographing system according to claim 1 , wherein the processor causes the image sensor to be positioned at a standby position by the mechanism before photographing the object.

5. The imaging system according to claim 1 , wherein the moving object is a vehicle.

6. An imaging device for imaging an object; A laser Doppler speedometer that measures the speed of a moving object; Equipped with The imaging device is A method for photographing an object by mounting an imaging system on a moving body, the imaging system including a processor, an image sensor, a mechanism for moving the image sensor, and a motion detection sensor for detecting motion of the imaging device, the method comprising: The processor, determining, as a target speed, either the speed measured by the laser Doppler speedometer or, if the speed measured by the laser Doppler speedometer is incorrect, a speed calculated based on the acceleration from the motion detection sensor; an image of the object is captured by the image sensor while the image sensor is moved by the mechanism in the same direction as the moving object and at a moving speed corresponding to the target speed; How to shoot.

7. The imaging method according to claim 6 , wherein the motion detection sensor includes an acceleration sensor and an angular velocity sensor.

8. The imaging method according to claim 7 , wherein the acceleration is a translational acceleration component obtained by subtracting a gravitational acceleration component and a rotational motion component calculated based on the angular velocity of the angular velocity sensor from an output value of the acceleration sensor.

9. The photographing method according to claim 6 , wherein the processor causes the image sensor to be positioned at a standby position by the mechanism before photographing the object.

10. The imaging method according to claim 6 or 7, wherein the moving object is a vehicle.

Images