Image sensing device

Abstract

An image sensing device (100) comprising a laser scanner (1), a camera (2), and a synchronization circuit (4). The laser scanner (1) is capable of scanning a laser in a direction parallel to an X-axis or a Y-axis. The camera (2) is capable of scanning an imaging region in a direction parallel to the X-axis or the Y-axis. The synchronization circuit (4) controls the laser scanner (1) and the camera (2) such that the irradiation direction of the laser and the line-of-sight direction of the camera (2) intersect on a two-dimensional surface within a three-dimensional space. The camera (2) images an object (41) intersecting the two-dimensional surface by scanning. The synchronization circuit (4) acquires an image of the object (41) by synchronizing the scanning of the laser scanner (1) and the camera (2).

Description

Image Sensing Device

[0001] The present disclosure relates to image sensing devices.

[0002] In recent years, a sensing method has been proposed that uses an illumination plane created by a line laser and a sensing plane sensed by a line sensor to detect an object (see, for example, Patent Document 1).

[0003] Special Publication No. 2022-530349

[0004] Conventional techniques have a problem in that when capturing an image of an object with low reflectance, the brightness of the image is low.

[0005] An object of the present disclosure is to solve the above-mentioned problems and to provide an image sensing device that can improve the brightness of an image even when capturing an image of an object with low reflectivity.

[0006] An image sensing device according to one aspect of the present disclosure includes: a laser scanner capable of scanning a laser extending in a direction parallel to a Y axis in an XYZ Cartesian coordinate system in a direction parallel to an X axis in the XYZ Cartesian coordinate system; a camera capable of scanning an imaging area extending in a direction parallel to the Y axis in a direction parallel to the X axis; and a synchronization circuit that controls the laser scanner and the camera, wherein the laser scanner and the camera are arranged along the X axis in the XYZ Cartesian coordinate system, the synchronization circuit controls the laser scanner and the camera so that the irradiation direction of the laser emitted from the laser scanner and the line of sight direction of the camera intersect on a two-dimensional plane in three-dimensional space, the camera captures an image of an object intersecting the two-dimensional plane by scanning in a −X direction of the X axis, and the synchronization circuit acquires an image of the object by synchronizing first scans of the laser scanner and the camera in the −X direction, After the first scan, the synchronization circuit executes a first frame step to control the laser scanner and the camera so that the irradiation direction of the laser and the line of sight direction of the camera are in the +X direction of the X axis; after the first frame step, the synchronization circuit acquires an image of the object by synchronizing a second scan of the laser scanner and the camera in the -X direction; after the second scan, the synchronization circuit executes a second frame step to control the laser scanner and the camera so that the irradiation direction of the laser and the line of sight direction of the camera are in the +X direction; the synchronization circuit controls the laser scanner so that the scanning speed of the laser scanner in the second frame step is different from the scanning speed of the laser scanner in the first frame step; and the synchronization circuit controls the camera so that the scanning speed and exposure time of the camera in the second frame step are different from the scanning speed and exposure time of the camera in the first frame step.An image sensing device according to another aspect of the present disclosure includes: a laser scanner capable of scanning a laser extending in a direction parallel to an X-axis in an XYZ Cartesian coordinate system in a direction parallel to a Y-axis in the XYZ Cartesian coordinate system; a camera capable of scanning an imaging area extending in a direction parallel to the X-axis in a direction parallel to the Y-axis; and a synchronization circuit that controls the laser scanner and the camera, wherein the laser scanner and the camera are arranged along the X-axis in the XYZ Cartesian coordinate system, the synchronization circuit controls the laser scanner and the camera so that the irradiation direction of the laser emitted from the laser scanner and the line of sight direction of the camera always face in the same direction within a YZ plane in three-dimensional space, the camera images an object that intersects the two-dimensional plane by scanning in a -Y direction of the Y axis, and the synchronization circuit acquires an image of the object by synchronizing first scans of the laser scanner and the camera in the -Y direction, After the first scan, the synchronization circuit executes a first frame process to control the laser scanner and the camera so that the laser irradiation direction and the camera's line of sight are in the same direction in the +Y direction of the Y axis at the same time; after the first frame process, the synchronization circuit acquires an image of the object by synchronizing a second scan of the laser scanner and the camera in the -Y direction; after the second scan, the synchronization circuit executes a second frame process to control the laser scanner and the camera so that the laser irradiation direction and the camera's line of sight are in the +Y direction; the scanning speed of the laser scanner in the second frame process is different from the scanning speed of the laser scanner in the first frame process; and the scanning speed and exposure time of the camera in the second frame process are different from the scanning speed and exposure time of the camera in the first frame process.

[0007] According to the present disclosure, it is possible to provide an image sensing device that can improve the brightness of an image even when capturing an image of an object with low reflectance.

[0008] 13 is a perspective view schematically illustrating the configuration of an image sensing device (also referred to as a light curtain sensor device) for performing light curtain sensing. FIG. 14 is a diagram illustrating an example of the configuration of a laser scanner. FIG. 15 is a diagram explaining the generation of a light curtain in the XZ plane. (A) and (B) are diagrams explaining the operation of a light curtain on a virtual screen. FIG. 16 is a diagram illustrating the operation of a laser scanner and the operation of each camera. FIG. 17 is a diagram illustrating a geometric arrangement for generating a light curtain with a camera. FIG. 18 is a diagram illustrating the line scanning operation of a camera, the laser irradiation position, and the time. FIG. 19 is a diagram illustrating the line scanning operation of a camera, the laser irradiation position, and the time. FIG. 19 is a diagram illustrating the line scanning operation of a camera, the laser irradiation position, and the time. FIG. 19 is a diagram illustrating the line scanning operation of a camera, the laser irradiation position, and the time. FIG. 19 is a diagram illustrating the line scanning operation of a camera, the laser irradiation position, and the time. FIG. 19 is a diagram illustrating the line scanning operation of a camera, the laser irradiation position, and the time. FIG. 19 is a diagram illustrating the line scanning operation of a camera, the laser irradiation position, and the time. Fig. 15 is a plan view showing the image sensing device shown in Fig. 14. Fig. 16 is a diagram showing an example of the configuration of a laser scanner. Fig. 17 is a diagram showing an example of the configuration of a laser scanner. Fig. 18 is a diagram showing an example of the configuration of a laser scanner. Fig. 19 is a diagram showing the operation of a laser on a virtual screen. Fig. 20 is a diagram showing the operation of a laser on a virtual screen. Fig. 21 is a diagram specifically explaining a method for controlling sensing by an epipolar imaging sensor.

[0009] In the XYZ Cartesian coordinate system shown in each drawing, the Z-axis direction (Z-axis) indicates a direction perpendicular to the virtual screen, the Y-direction (Y) indicates a direction perpendicular to the Z-axis direction and parallel to the imaging area, and the X-direction (X-axis) indicates a direction perpendicular to both the Z-axis direction and the Y-axis direction.

[0010] 1 is a perspective view showing a schematic configuration of an image sensing device 100 (also referred to as a light curtain sensor device) for performing light curtain sensing in the present disclosure. In the example shown in FIG. 1, a light curtain 25 is generated by a combination of a laser scanner 1 and a camera 2.

[0011] The image sensing device 100 includes a laser scanner 1, a camera 2, and a synchronization circuit 4. In the image sensing device 100, the laser scanner 1 and the camera 2 are arranged in parallel along the X-axis direction and are controlled by the synchronization circuit 4. In the example shown in Fig. 1, the laser scanner 1 and the camera 2 are arranged along the X-axis in an XYZ Cartesian coordinate system.

[0012] 2 is a diagram showing an example of the configuration of the laser scanner 1. The laser scanner 1 emits a laser 12. The laser scanner 1 can scan in a direction parallel to the X axis with the laser 12 (also called a line laser) extending in a direction parallel to the Y axis in an XYZ orthogonal coordinate system.

[0013] A laser beam 12 that spreads in the Y-axis direction is emitted from the laser scanner 1. A virtual screen 10, which is a plane for the purpose of explanation, exists perpendicular to the Z-axis direction at a position Z0 away from the laser scanner 1 in the Z-axis direction. The laser beam emitted from the laser scanner 1 appears as a linear laser beam 12 on the virtual screen 10 that extends in a direction parallel to the Y-axis.

[0014] A laser beam with a wavelength λ is emitted from a laser light source 5, and is converted into a laser beam 12 that spreads in the Y direction by a line beam generating element 6. The line beam generating element 6 is, for example, a cylindrical lens or a Powell lens. The laser beam 12 is deflected in the Z direction by a galvanometer mirror 7. The galvanometer mirror 7 can be swung around the Y axis at angles of ±α, and the laser beam 12 is scanned around the Y axis within an angle range of ±2α. On the virtual screen 10, the laser beam 12 is scanned from the +X direction to the −X direction, irradiating the entire area of ​​the scan range 13 in FIG. 1 or 2 .

[0015] <Camera 2> Camera 2 can scan an imaging area 22 (also referred to as a linear imaging area), which extends in a direction parallel to the Y axis of an XYZ Cartesian coordinate system, in a direction parallel to the X axis of the XYZ Cartesian coordinate system. For example, camera 2 captures an image of an object 41 (also referred to as a target object) that intersects with a two-dimensional plane in three-dimensional space by scanning in the +X direction of the X axis.

[0016] In this embodiment, camera 2 is a rolling shutter camera. Camera 2 reads out luminance information of image sensor pixels sequentially, starting from the top row. By shortening the exposure time of camera 2 under the control of synchronization circuit 4, the imaging area extending in the row direction of the image sensor can be scanned in the column direction perpendicular to the row direction. For example, in a 1000-row rolling shutter camera operating at a frame rate of 50 fps, if the exposure time per row is set to te = 20 μs, adjacent rows can be separated in time and sensing can be performed row by row. In actual operation, synchronization circuit 4 controls camera 2 to shift the exposure area in sequence by five rows, for example, with an exposure time of te = 100 μs.

[0017] Normally, a rolling shutter camera is used so that the row direction is horizontal, but in this embodiment, camera 2 is installed rotated -90° around its optical axis. Camera 2 is installed so that the optical axis direction of camera 2 faces slightly inward from the Z-axis direction (+X direction) so that the entire imaging range 23 of camera 2 on virtual screen 10 roughly overlaps with the entire laser scanning range 13. However, even if camera 2 faces in the Z-axis direction, if there is overlap between the entire imaging range 23 and the entire scanning range 13, a light curtain 25 is generated within the overlapping region.

[0018] The camera 2 is also equipped with a bandpass filter 8. The bandpass filter 8 may be installed at the tip of the lens barrel of the camera 2 or inside the camera 2. The bandpass filter 8 is a wavelength filter that allows only light of wavelength λ emitted from the laser light source 5 to pass through.

[0019] <Synchronization Circuit 4> The synchronization circuit 4 controls the laser scanner 1 and the camera 2. The synchronization circuit 4 can synchronize the operation of the laser scanner 1 and the operation of the camera 2. For example, the synchronization circuit 4 controls the laser scanner 1 and the camera 2 so that the irradiation direction of the laser 12 emitted from the laser scanner 1 and the line of sight direction of the camera 2 intersect on a two-dimensional plane in three-dimensional space.

[0020] The synchronization circuit 4 synchronizes the first scan in the −X direction of the laser scanner 1 and the camera 2 to acquire an image of the object 41. After the first scan, the synchronization circuit 4 executes a first frame process to control the laser scanner 1 and the camera 2 to return the irradiation direction of the laser 12 and the line of sight direction of the camera 2 to the +X direction of the X axis.

[0021] After the first frame step, the synchronization circuit 4 synchronizes the second scan of the laser scanner 1 and the camera 2 in the −X direction to acquire an image of the object 41. After the second scan, the synchronization circuit 4 executes a second frame step to control the laser scanner 1 and the camera 2 to return the irradiation direction of the laser 12 and the line of sight direction of the camera 2 to the +X direction.

[0022] The synchronization circuit 4 controls the laser scanner 1 so that the scanning speed of the laser scanner 1 in the second frame step is different from the scanning speed of the laser scanner 1 in the first frame step. The synchronization circuit 4 controls the camera 2 so that the scanning speed and exposure time of the camera 2 in the second frame step are different from the scanning speed and exposure time of the camera 2 in the first frame step.

[0023] If the synchronization circuit 4 detects a new object while scanning is being performed by the laser scanner 1 and the camera 2, the synchronization circuit 4 may slow down the scanning speed of the laser scanner 1 and the camera 2 and increase the exposure time of the camera 2. This allows sensing to be performed at an optimal frame rate and image brightness depending on the object.

[0024] The synchronization circuit 4 may limit the imaging range of the camera 2 and control the laser scanner 1 and the camera 2 so that the laser 12 is irradiated within the imaging range. This makes it possible to increase the brightness only in low image brightness areas and shorten the sensing time.

[0025] <<Example 1 of Scanning Speed ​​and Exposure Time>> The scanning speed of the laser scanner 1 in the second frame step is slower than the scanning speed of the laser scanner 1 in the first frame step. In this case, the scanning speed of the camera 2 in the second frame step is slower than the scanning speed of the camera 2 in the first frame step, and the exposure time of the camera 2 in the second frame step is longer than the exposure time of the camera 2 in the first frame step. This can improve image brightness.

[0026] <<Example 2 of Scanning Speed ​​and Exposure Time>> The scanning speed of the laser scanner 1 in the second frame step is faster than the scanning speed of the laser scanner 1 in the first frame step. In this case, the scanning speed of the camera 2 in the second frame step is faster than the scanning speed of the camera 2 in the first frame step, and the exposure time of the camera 2 in the second frame step is shorter than the exposure time of the camera 2 in the first frame step. This makes it possible to improve the frame rate when the image brightness is sufficient.

[0027] <Light Curtain Sensing> Fig. 3 is a diagram illustrating the generation of a light curtain 25 in the XZ plane. Figs. 4A and 4B are diagrams illustrating the operation of the light curtain 25 on the virtual screen 10. In Fig. 3, the angle θ formed between the irradiation direction 11 of the laser 12 from the Z-axis direction and the Z-axis is taken as θ, where a clockwise direction is taken as positive. In Fig. 3, the angle θa formed between the irradiation direction 11a and the Z-axis at a certain time ta has a negative value. Similarly, the angle θb formed between the irradiation direction 11b and the Z-axis at time tb has a positive value.

[0028] Because the camera 2 is installed rotated -90° around the optical axis, the line of sight 21 of the camera 2 scans clockwise within the XZ plane. The angle with the Z axis is designated as φ clockwise. In FIG. 3, the angle φa with the line of sight 21a at time ta and the angle φb with the line of sight 21b at time tb are both negative values. Here, at time ta, the irradiation direction 11a and the line of sight 21a intersect at an intersection 24a on the virtual screen 10a. Similarly, at time tb, the irradiation direction 11b and the line of sight 21b intersect at an intersection 24b on the virtual screen 10a. FIGS. 4 and 5 show the relationship between the laser 12 and the imaging area 22 on the virtual screen 10a at a distance Za and on the virtual screen 10b at a distance Zb.

[0029] On the virtual screen 10a, scanning is performed so that the laser 12 and the imaging area 22 always overlap. As will be explained later, such an operation is possible by appropriately controlling the angle of the galvanometer mirror 7 with respect to time. At this time, since the virtual screen 10b is installed at a different distance from the virtual screen 10a, as is clear from FIG. 5, the laser 12 and the imaging area 22 never overlap.

[0030] When the light curtain 25 is generated on the virtual screen 10a and the object 41 to be detected (also referred to as the measurement object) intersects with the light curtain 25, an image outside the intersecting area is not captured by the camera 2. This is because the exposure time for each pixel in one frame is extremely short (for example, exposure time te = 60 μs), and the bandpass filter 8 cuts off light rays other than those with the laser wavelength before they reach the image sensor inside the camera 2. For this reason, the background is not captured by light curtain sensing.

[0031] Here, the conditions for scanning the virtual screen 10a so that the laser 12 and the imaging area 22 overlap will be described. First, when the laser 12 is linear and extends in a direction parallel to the Y axis, the imaging area 22 of the camera 2 must extend precisely in a direction parallel to the Y axis. To achieve this, a fine adjustment mechanism is provided for finely adjusting the orientation of the camera 2 around its optical axis. Furthermore, the imaging lens of the camera 2 must be a distortion-corrected lens. Otherwise, the imaging area 22 will be distorted from its linear shape. Furthermore, to scan the virtual screen 10a while keeping the laser 12 and the imaging area 22 overlapping, the imaging start timing of the camera 2 and the scanning start timing of the laser scanner 1 must be synchronized with an appropriate delay. For this purpose, the synchronization circuit 4 is used. Furthermore, the scanning speed of the laser 12 and the scanning speed of the imaging area 22 on the virtual screen 10a must be the same.

[0032] 5 is a diagram illustrating the operation of the laser scanner and the operation of the camera. In FIG. 5, the dashed line represents the angular function φ(t) of the line of sight direction of the camera 2, and the solid line represents the angular function θ(t) of the irradiation direction of the laser 12. A planar light curtain perpendicular to the Z axis is generated at a position at a distance Z0 = 0.5 m in sections P1 and P2, and at a position at a distance Z0 = 1 m in sections P3 and P4. The specific parameters used to calculate the graph shown in FIG. 5 will be described later.

[0033] <Operation in Section P1> First, the operation in section P1 will be described. Figure 6 is a diagram showing the geometric arrangement for generating the light curtain 25. The laser 12 and the imaging area 22 overlap on a virtual screen 10a placed at a distance Z0 in the Z-axis direction. As shown in Figure 6, the angular function θ(t) of the irradiation direction 11 of the laser 12 and the angular function φ(t) of the line of sight of the camera 2 are angles from the Z-axis direction. In the arrangement shown in Figure 6, both the angles θ and φ have negative values. The angle of the galvanometer mirror is half the angle θ (α = θ(t) / 2). The angular function φ(t) of the line of sight of the camera 2 is expressed as in equation (1) using a constant coefficient k1 determined by the frame rate: φ(t) = k1 × t + φmin (1)

[0034] Since it is generally difficult to operate the line scan speed of a rolling shutter camera at anything other than a constant speed, the angular function θ(t) of the irradiation direction 11 of the laser 12 is controlled to match the angular function φ(t) of the line of sight direction of the camera 2. Using Figure 6, the relationship between θ(t) and φ(t) is found. The rotation axis of the laser scanner 1 is set to point Q, the rotation axis of the line of sight direction 21 of the camera 2 is set to point R, and the distance between them is set to b. The foot of the perpendicular line drawn from the intersection point 24 to the line QR is set to point P. Then, the signed distance PR can be written as Z0 × tan φ using φ. Furthermore, since the signed distance PQ can be written as Z0 × tan θ, the following equation (2) holds. Z0 × tan φ = Z0 × tan θ - b (2)

[0035] That is, when the angle φ is fixed, the angle θ of the laser 12 is calculated by the following equation (3): θ=Arctan(tan φ+b / Z0) (3)

[0036] When the virtual screen 10a is a plane perpendicular to the Z axis, the distance Z0 is a constant value. The function that generates the planar light curtain 25 at the distance Z0 is shown in the graph of section P1 in Figure 5. The specific parameters for plotting the angle function for section P1 are as follows:

[0037] <<Parameters for Section P1>> Scanning range of laser scanner 1: -25°≦θ≦25° (θmin = -25°, θmax = 25°) Scanning range of camera 2: -35°≦φ≦15° (φmin = -35°, φmax = 15°) b = 0.15 m Time for camera 2 to scan the entire line T1 = 16 ms Z0 = 0.5 m

[0038] The coefficient k1 in equation (1) is expressed as in equation (4): k1=(φmax-φmin) / T1 (4)

[0039] Here, the camera 2 has a field angle of 50° in the ZX plane, and its optical axis is tilted at −10° with respect to the Z axis.

[0040] The position of the plane with parameters Z0 = 0.5 m in section P1 corresponds to the position marked on the virtual screen 10b in Figure 2. In the graph of section P1 in Figure 5, the line-of-sight function φ(t) of camera 2 is a linear function of time t, as expressed by equation (1), and varies from φmin = -35° to φmax = 15° between time t = 0 and 16 ms. The angle θ of laser 12 during this period is given by equation (3) above, and this is shown as θ(t) in section P1(1). However, because angle θ can only be swung up to a maximum of 25°, the scan for generating light curtain 25 ends before T1 = 16 ms is reached. This corresponds to intersection 26 being located to the left of intersection 27 on the virtual screen 10b in Figure 3. Note that the intersection of the irradiation direction of the laser 12 at the maximum angle θmax and the virtual screen 10b is designated as 26, and the intersection of the line of sight direction 21 of the camera 2 at the maximum angle φmax and the virtual screen 10b is designated as 27. When the scan of the laser 12 in section P1(1) is completed, the angle of the galvanometer mirror 7 must be returned in the negative direction in preparation for the next laser scan. Because the galvanometer mirror 7 is an object with a moment of inertia, a non-small finite time Tback is required to return it to the start position of the next scan. While this differs depending on the product specifications of the galvanometer mirror 7, for example, Tback = 4 ms is required. In section P1(2), the laser angle is swung from +25° to -25° using a sine function.

[0041] <<Operation of Synchronization Circuit>> A synchronization circuit 4 is used to operate the angular function φ(t) of the line of sight direction of the camera 2 and the angular function θ(t) of the irradiation direction 11 of the laser 12 at the precise timing shown in FIG. 5. The synchronization circuit 4 is pre-programmed with a function waveform that provides the angular function θ(t) of the irradiation direction 11 of the laser 12, and the timing for outputting a trigger pulse for the camera 2 shown in FIG. 5 in accordance with the angular function θ(t) of the irradiation direction 11 of the laser 12. The synchronization circuit 4 outputs an analog voltage waveform that provides the angular function θ(t) of the irradiation direction 11 of the laser 12 to the laser scanner 1. A trigger pulse is output to the camera 2. The camera 2 starts capturing one frame of image at the rising edge of the trigger pulse voltage. In other words, it starts scanning the line capture area.

[0042] 5 shows the on / off timing of the laser 12. The synchronization circuit 4 controls the on / off of the laser 12 during the period P1(1) in which the light curtain is generated.

[0043] <Three-dimensional measurement using a light curtain sensor> Figure 12 is a diagram showing an example of sensing an object using a light curtain sensor as a comparative example. Figure 13 is a diagram showing the control timing of the camera and laser scanner for obtaining a three-dimensional depth map using the light curtain sensor shown in Figure 12. The explanation for Figure 13 is the same as for Figure 5. Three-dimensional measurement can be performed by installing multiple light curtains with narrower spacing between them in the Z direction. For example, in the example shown in Figure 12, a light curtain is installed in section Pn at a distance Zn calculated by equation (5): Zn = Z0 + (n-1) ΔZ (5) (n is a natural number)

[0044] That is, the distance between adjacent light curtains is ΔZ. An object 41 is placed within the sensing area of ​​such a light curtain. Images of cross sections of the object 41 where the light curtains 25a, 25b, 25c, ... intersect are captured by the camera 2. The cross-sectional area of ​​the object 41 reflected by the light curtain at a distance Zn is located at a position at a distance Zn from the light curtain sensor. Since many cross-sectional images at different distances Zn are acquired, distance information of the surface of the object 41, i.e., a three-dimensional depth map, can be obtained.

[0045] In the example shown in FIG. 13, planar light curtains are generated at positions Z=0.5 m, 0.7 m, 0.9 m, and 1.1 m in sections P1(1), P2(1), P3(1), and P4(1).

[0046] Here, if the object has a black region or a region where the normal direction of the surface is nearly perpendicular to the line of sight of camera 2, the amount of reflected and scattered light from those regions that enters the pupil of camera 2 will be small. In other words, the brightness of the images in those regions may be significantly reduced, and regions may be missing data when generating a three-dimensional depth map.

[0047] To increase the brightness of an image, it is sufficient to increase the amount of emitted laser light, but there is an upper limit to the amount of laser light that can be generated. Another commonly considered method for increasing image brightness is to increase the shutter open time te of the camera 2. However, a major problem with light curtain sensors is that image brightness cannot be increased simply by increasing the shutter open time te.

[0048] 7 to 11 are diagrams showing the line scanning operation of the camera 2 and the irradiation position and time of the laser 12. FIG. 7 shows, for example, the generation of a light curtain in section P1 in FIG. 5. In FIGS. 7 to 11, the horizontal axis represents the passage of time, and the vertical axis represents the row number of the rolling shutter camera 2. The width of the rectangular hatched area 51 in the diagram represents the time required to read an image for one row of pixels, i.e., the shutter open time te. In the rolling shutter camera 2, pixels are read sequentially, starting from the top row, one row at a time, resulting in a read delay time tc for each row. For example, assuming T = 16 ms and 1,000 rows, tc = 16 μs. In FIGS. 7 to 11, for example, te = 48 μs.

[0049] The hatched areas 52 in Figures 7 to 11 represent the time when the laser 12 irradiates each row. Here, it is assumed that the line width of the laser 12 on the virtual screen 10a is sufficiently smaller than the resolution of the camera 2 on the virtual screen 10a. In this case, when the laser 12 scans the virtual screen 10a, it irradiates each row of the image rows of the camera 2 in sequence. Therefore, the hatched areas 52 representing the irradiation timing of the laser 12 in Figures 7 to 11 do not overlap in the time direction for each row, but shift diagonally downward and to the right. Since the angular function θ(t) of the irradiation direction 11 of the laser 12 is controlled by the function shown in section P1(1) in Figure 5, in Figure 7, the area 52 overlaps with the area 51. Thus, in the time and space where the area 52 and the area 51 overlap, if an object 41 is present, the reflected and scattered light therefrom enters the camera 2 and is sensed. Because scanning 1,000 lines per frame is performed at a high speed of T = 16 ms, it appears as if a light curtain exists in space. Furthermore, a timing chart on the virtual screen 10b at a distance Z0', which is different from the distance Z0 at which the light curtain is generated, is shown, for example, in Figure 10 . That is, regions 52 and 51 each undergo similar transitions over time, but region 52 shifts in the time direction depending on the difference between distance Z0' and distance Z0, so the two do not overlap. Even if an object 41 exists at this distance Z0', an image of the object will not be sensed. In other words, generating a light curtain corresponds to the hatched region 52, representing the irradiation of the laser 12, moving over hatched region 51 over time in the diagram of Figure 7 when a planar object is present as a physical object on the light curtain.

[0050] Figure 8 shows a timing chart for when the shutter open time te is extended on the virtual screen 10a on which the light curtain is generated. Compared to Figure 7, the regions 51 are stretched in the time direction. However, the time width of the overlap with region 52 remains unchanged. In other words, even if the shutter open time te is extended, the amount of light incident on the camera 2 does not change, and the brightness of the image does not improve.

[0051] To make matters worse, extending the shutter open time te causes the problem of the thickness of the light curtain in the Z direction to increase. At a distance (Z0 + δ) slightly different from distance Z0, as explained above using FIG. 10, region 52 shifts to the right, resulting in the chart shown in FIG. 11. Regions 51 and 52 still overlap, indicating that sensing occurs even at distance (Z0 + δ). In other words, extending the shutter open time te increases the thickness of the light curtain.

[0052] <Solution: Operation in Section P2 of Fig. 5> In this first embodiment, the camera 2 and laser scanner 1 are controlled as shown in section P2 of Fig. 5. In section P2, the angular function φ(t) of the line of sight direction of the camera 2 is expressed by equations (6) and (7) which have the same form as equation (1). φ(t) = k2 × (t - t2a) + φmin (6) k2 = (φmax - φmin) / T2 (7)

[0053] The time T2 required for camera 2 to scan the entire line is set to twice the time T1, i.e., T2 = 32 ms. The angular function θ(t) of the irradiation direction of laser 12 is calculated using equation (3). This graph corresponds to section P2(1) in FIG. 5, which is twice the horizontal length of the graph for section P1(1). In section P2(1), the shutter open time te of camera 2 is also doubled. A timing chart for this operation is shown in FIG. 9. Doubling T2 compared to T1 corresponds to doubling the time tc required for one line transition. Because the laser scanning speed is twice as slow, the width of region 51 is doubled, and the width of region 51, which corresponds to the camera shutter open time, is also doubled. In this case, the overlap time between region 51 and region 52 in section P2(1) is doubled compared to section P1(1). In other words, even if the power of laser 12 remains the same, the image brightness is doubled. Furthermore, on the virtual screen 10b, which is at a distance (Z0+δ) slightly different from the distance Z0 at which the light curtain is generated, the region 52 shifts slightly to the side. The amount of shift in Fig. 9 is doubled compared to Figs. 7 and 11, so when the difference in distance δ is the same between Figs. 7 and 9, the overlapping ratio between the region 51 and the region 52 is the same. In other words, the thickness of the light curtain does not change between the section P1(1) and the section P2(1).

[0054] To summarize the above, by slowing down the frame time of the rolling shutter camera 2, slowing down the laser scanning speed accordingly, and extending the camera shutter open time te, it is possible to increase the image brightness without increasing the laser power, while still maintaining the same thickness of the light curtain.

[0055] 5, a planar light curtain is generated at a position Z=1 m, and the time required for camera 2 to perform a full line scan is T1=16 ms, the same as in section P1. In section P4, a planar light curtain is generated at the same position as in section P3, but the time required for a full line scan is twice that of section P3, resulting in twice the brightness of the image.

[0056] As described above, by changing the line feed time tc of camera 2 as a rolling shutter camera, the shutter open time te of camera 2, and the scanning speed of laser 12, it is possible to change the image brightness in the next frame even in the middle of a frame in which multiple light curtains are generated consecutively. Various modifications can be considered for image sensing using a projection light curtain sensor as follows.

[0057] <<Variation 1>> During high-speed light curtain sensing, if it is suspected that there is an unsensing area within the sensed image area, even though an object that should be detected is present, a variation can be considered in which the light curtain sensing speed is slowed down to increase image brightness. For example, if a person wearing a black leather coat is recognized as a person by image recognition technology such as artificial intelligence, but part of the body is not detected, the scanning speed of the projection light curtain can be slowed down to increase image brightness in the next frame. This will enable sensing of the black area. In this way, the angle functions φ(t) and θ(t) can be changed during a series of sensing in consecutive frames to increase image brightness. This makes it possible to detect areas that could not be detected by high-speed sensing, resulting in highly accurate sensing.

[0058] <<Variation 2>> A variation is also possible in which slow sensing is performed to obtain a high-luminance image in the first frame, and the scan speed for the next frame is determined based on that luminance information. That is, if sufficient luminance information is obtained, the luminance is reduced and sensing is performed at a high speed. In this case, the image luminance value of the object sensed in the first frame is compared with the sensing threshold, and the sensing speed for the next frame can be determined from the ratio. For example, if the image luminance value is 30 compared to a threshold value of 5, even if the sensing speed is tripled and the image luminance value is reduced to one-third, a luminance value of 10 can be expected, which is sufficient for object detection.

[0059] <<Variation 3>> When it is impossible to know what kind of object is present in the detection area, a variation can be considered in which high-intensity, slow-motion frames A are randomly inserted between high-speed sensing. If no new object is detected in frame A, high-speed sensing is continued, and if a new object is detected, the frame is switched to a slow-motion frame.

[0060] <<Variation 4>> When the field of view range containing an object with low image brightness is known, a variation can be considered in which the camera's Region of Interest (ROI) function is used to limit the camera's imaging range, thereby slowly scanning only a partial area. The ROI function is a function that changes the camera's control so that, for example, in a camera with 1,000 pixel rows, only 200 rows, from row 301 to row 500, are captured in one frame. The ROI function can reduce the time spent imaging unnecessary areas. In projection light curtain sensing, a light curtain with a limited area can be generated by scanning only the pixels in the rows narrowed down by the ROI function with a laser. This can also save time in light curtain sensing.

[0061] As described above, according to this embodiment, it is possible to provide the image sensing device 100 that can improve the brightness of the image even when capturing an image of an object 41 with low reflectance.

[0062] <Embodiment 2> <Epipolar Imaging> Epipolar imaging is a technology for projecting illumination light and acquiring an image of the projected object. Similar to a light curtain sensor, a laser scanner 1 and a camera 2 are arranged in parallel in the X direction and controlled by a synchronization circuit. The laser scanner 1 scans a linear laser extending in the X direction in the -Y direction, and this movement is repeated with a period T. The camera 2 is a rolling shutter camera, and by shortening the exposure time, it is possible to repeatedly capture images by scanning an imaging area extending in the X direction in the -Y direction. The scanning speed of the camera 2 in the Y direction is set to the same speed as the scanning speed of the laser scanner 1. The synchronization circuit controls the area irradiated by the laser 12 and the imaging area of ​​the camera 2 so that they always coincide.

[0063] The plane formed by the irradiation area of ​​the laser 12 and the imaging area of ​​the camera 2 is the same as the epipolar plane in triangulation. The camera in the device configured as described above always captures only scattered light from objects within the epipolar plane at a given moment, which is why it is called epipolar imaging. Epipolar imaging can suppress scattered stray light even for objects that generate large amounts of reflected and scattered stray light. By utilizing this effect, even shiny metal objects can be measured without stray light and with no errors. Furthermore, since it can efficiently capture low-intensity projected light even in bright outdoor environments, it can also be used for outdoor sensing.

[0064] <Image Sensing Device 101> Fig. 14 is a perspective view showing the configuration of an image sensing device 101 (also referred to as an epipolar imaging sensor) for performing epipolar imaging in the present disclosure. Fig. 15 is a plan view showing the image sensing device 101 shown in Fig. 14. The image sensing device 101 includes a laser scanner 1, a camera 2, and a synchronization circuit 4. The camera 2 and the laser scanner 1 are arranged along the X axis of an XYZ Cartesian coordinate system. A virtual screen 10 is set at a position a distance Z0 from the camera 2 and perpendicular to the optical axis C of the camera. This virtual screen 10 does not exist physically, but represents a plane for purposes of explanation.

[0065] 16 to 18 are diagrams showing an example of the configuration of the laser scanner 1. The laser scanner 1 emits a laser 12. The laser scanner 1 can scan the laser 12, which extends in a direction parallel to the X-axis in an XYZ orthogonal coordinate system, in a direction parallel to the Y-axis in the XYZ orthogonal coordinate system.

[0066] The laser scanner 1 emits a laser beam 12 that spreads in the X-axis direction, and on the virtual screen 10, the laser beam 12 extends in a direction parallel to the X-axis. As shown in Figures 16 to 18, a laser beam is emitted from the laser light source 5, reflected by the mirror 9, and then generated by the line beam generating element 6 as the laser beam 12 that spreads in a direction parallel to the X-axis. The line beam generating element 6 is, for example, a cylindrical lens or a Powell lens. The laser beam 12 is deflected in the Z direction by the galvanometer mirror 7. The galvanometer mirror 7 can be swung around the X-axis within an angle range of ±α, and the laser beam 12 scans around the X-axis within an angle range of ±2α. On the virtual screen 10, the laser beam 12 scans in the Y direction, irradiating the area of ​​the scan range 13 in Figure 14.

[0067] The optical axis L of the laser scanner 1 is defined as the traveling direction of the center of the linear beam when the galvanometer mirror angle α = 0°. Here, the optical axis C of the camera 2 and the optical axis L of the laser scanner are assumed to be parallel to each other. In this case, the optical axis L of the laser scanner 1 is perpendicular to the virtual screen 10.

[0068] 19 to 21 are diagrams showing the operation of the laser 12 on the virtual screen 10. For example, Fig. 20 shows the operation of the laser 12 on the virtual screen 10, and illustrates how the laser 12 repeatedly scans from the top to the bottom of the scan range 13 in the -Y direction at a speed VL. The laser 12 exists at the top of the scan range 13 as laser 12a at time t = ta, is shown as laser 12b at time t = tb, and exists at the bottom of the scan range 13 as laser 12c at time t = tc. When it reaches the bottom, it returns to the top at high speed and repeats the above operation.

[0069] <Camera 2> Camera 2 can scan an imaging area 22 (also referred to as a linear imaging area) extending in a direction parallel to the X axis of an XYZ Cartesian coordinate system in a direction parallel to the Y axis of the XYZ Cartesian coordinate system. For example, camera 2 captures an image of an object (also referred to as a target object) that intersects with a two-dimensional plane by scanning in the −Y direction of the Y axis.

[0070] In this embodiment, the camera 2 is a rolling shutter camera, and by shortening the exposure time, it is possible to repeatedly scan the imaging area 22 extending in a direction parallel to the X axis in the −Y direction.

[0071] On the virtual screen 10, the imaging area 22 of the camera 2 is scanned from the top to the bottom of the entire imaging range 23 of the camera 2. This is shown in Figure 19. The imaging area 22 of the camera 2 is scanned at a speed Vc from imaging area 22a at the top of the entire imaging range 23 to imaging area 22c at the bottom. Here, the device configuration is set so that the imaging area 22a of the camera and the laser 12a overlap in the Y-axis direction on the virtual screen 10. This setting can be made by zooming the camera lens, setting a Region of Interest (ROI) that limits the imaging area of ​​the camera, or setting the scan range of the laser 12 in the Y direction. A mechanism for fine-tuning the installation orientation of the camera 2 or laser scanner 1 is also important.

[0072] <Synchronization Circuit 4> The synchronization circuit 4 controls the laser scanner 1 and the camera 2. The synchronization circuit 4 can synchronize the operation of the laser scanner 1 and the operation of the camera 2. For example, the synchronization circuit 4 controls the laser scanner 1 and the camera 2 so that the irradiation direction of the laser 12 emitted from the laser scanner 1 and the line of sight direction of the camera 2 are the same.

[0073] In this embodiment, the first and second frame steps each include a step of acquiring an image of an object by scanning the laser scanner 1 and camera 2 in the -Y direction, and a step of returning them to the same position in the +Y direction in preparation for acquiring an image in the next frame. In the first frame step, the synchronization circuit 4 acquires an image of the object by scanning in a synchronized manner so that the irradiation direction of the laser 12 and the line of sight direction of the camera 2 always face in the same direction within the YZ plane. After the first scan, the synchronization circuit 4 controls the laser scanner 1 and camera 2 so that the irradiation direction of the laser 12 and the line of sight direction of the camera 2 return to the same direction in the +Y direction of the Y axis at the same time, thereby completing the first frame step.

[0074] In the second frame step, after the first frame step, the synchronization circuit 4 acquires an image of the object by scanning in synchronization so that the irradiation direction of the laser 12 and the line of sight direction of the camera 2 always face in the same direction within the YZ plane. After the second scan, the synchronization circuit 4 controls the laser scanner 1 and the camera 2 to return the irradiation direction of the laser 12 and the line of sight direction of the camera 2 to the +Y direction, thereby completing the first frame step.

[0075] The scanning speed of the laser scanner 1 in the second frame step is different from the scanning speed of the laser scanner 1 in the first frame step. The scanning speed and exposure time of the camera 2 in the second frame step are different from the scanning speed and exposure time of the camera 2 in the first frame step. This allows the image brightness to be changed between frames.

[0076] If the synchronization circuit 4 detects a new object while scanning is being performed by the laser scanner 1 and the camera 2, the synchronization circuit 4 may slow down the scanning speed of the laser scanner 1 and the camera 2 and increase the exposure time of the camera 2. This allows sensing to be performed at an optimal frame rate and image brightness depending on the object.

[0077] The synchronization circuit 4 may limit the imaging range of the camera 2 and control the laser scanner 1 and the camera 2 so that the laser 12 is irradiated within the imaging range. This makes it possible to increase the brightness only in low image brightness areas and shorten the sensing time.

[0078] <<Example 1 of Scanning Speed ​​and Exposure Time>> The scanning speed of the laser scanner 1 in the second frame step is slower than the scanning speed of the laser scanner 1 in the first frame step. In this case, the scanning speed of the camera 2 in the second frame step is slower than the scanning speed of the camera 2 in the first frame step, and the exposure time of the camera 2 in the second frame step is longer than the exposure time of the camera 2 in the first frame step. This can improve image brightness.

[0079] <<Example 2 of Scanning Speed ​​and Exposure Time>> The scanning speed of the laser scanner 1 in the second frame step is faster than the scanning speed of the laser scanner 1 in the first frame step. In this case, the scanning speed of the camera 2 in the second frame step is faster than the scanning speed of the camera 2 in the first frame step, and the exposure time of the camera 2 in the second frame step is shorter than the exposure time of the camera 2 in the first frame step. This makes it possible to improve the frame rate when the image brightness is sufficient.

[0080] The operation of the synchronization circuit 4 will be described in detail below. The synchronization circuit 4 synchronizes the image capture time of the imaging region 22a with the irradiation time of the laser 12a at t = ta. Furthermore, the synchronization circuit 4 synchronizes the scanning speed Vc of the imaging region 22 in the -Y direction with the scanning speed VL of the laser 12 in the -Y direction. As a result, as shown in FIG. 21 , during one cycle from time t = ta to t = tc, the irradiation region of the laser 12 and the imaging region of the camera 2 are scanned from top to bottom while always overlapping in the Y-axis direction. This cycle is repeated. The overlapping range 30 between the imaging region 23 of the camera 2 and the scanning range 13 of the laser scanner 1 is the range in which epipolar imaging is possible.

[0081] Here, it is important that the camera 2 and the laser scanner 1 are aligned in a direction parallel to the X axis, i.e., that they are at the same position coordinates in the Y axis direction and the Z axis direction. With this arrangement, no matter what distance Z the virtual screen 10 is in front of the camera, the irradiation area of ​​the laser 12 and the imaging area 22 of the camera 2 always overlap, as shown in Figure 21.

[0082] 22 is a diagram specifically illustrating a method for controlling sensing by an epipolar imaging sensor. In FIG. 22, the dashed line represents the angular function φ(t) of the line of sight direction of camera 2, and the solid line represents the angular function θ(t) of the irradiation direction of laser 12. The intervals in which epipolar imaging is performed are P1(1), P2(1), P3(1), ..., where the angular function φ(t) and the angular function θ(t) are the same function. The intervals P1(2), P2(2), P3(2), ... are steps for returning galvanometer mirror 7 to the scan start position, and require a finite amount of time.

[0083] <<Operation in Section P1>> In section P1(1) in Fig. 22, the angular function θ(t) of the irradiation direction of laser 12 and the angular function φ(t) of the line of sight direction of camera 2 are expressed by equations (8) and (9). θ(t) = φ(t) = k1 × t + φmin (8) k1 = (φmax - φmin) / T1 (9)

[0084] Here, the scanning range of camera 2 is -25°≦φ≦25° (φmin=-25°, φmax=25°), and the time it takes for the camera to scan the entire line is T1=16 ms. In this way, when the angular function θ(t) of the irradiation direction of laser 12 and the angular function φ(t) of the line of sight direction of camera 2 are controlled by synchronization circuit 4 so that they have the same value at the same time, the irradiation area of ​​laser 12 and the imaging area 22 of camera 2 overlap on virtual screen 10 placed at an arbitrary distance Z from sensor 101, making epipolar imaging possible.

[0085] <<Issues of Epipolar Imaging Sensors>> Epipolar imaging also has the same issues as the light curtain sensor in embodiment 1. That is, if part of object 41 includes a black area or an area where the normal direction of the surface is nearly perpendicular to the line of sight of camera 2, the amount of reflected and scattered light from these areas that enters the pupil of camera 2 is reduced. In other words, there is a problem in that the brightness of the image in these areas is significantly reduced.

[0086] However, epipolar imaging sensors have a problem in that simply increasing the shutter open time te of the camera 2 does not result in an increase in image brightness, even if the shutter open time te is increased in order to increase image brightness, which is similar to the problem with light curtain sensors.

[0087] The diagram showing the line scanning operation of the camera 2 and the irradiation position and time of the laser 12 in the epipolar imaging sensor can also be explained using FIG. 7 , which was used to explain the light curtain sensor. FIG. 7 can be read as showing the operation in section P1(1) in FIG. 22. The meanings indicated by parameters such as area 51, area 52, te, tc, and tp are the same as those already described, so explanations will be omitted here. The difference from the light curtain sensor is as follows: In the first embodiment, area 51 and area 52 overlap only when the image sensing device 100 is at a specific distance Z, and a light curtain is generated. On the other hand, in the second embodiment, area 51 and area 52 overlap at any distance from the image sensing device 101, making epipolar imaging possible.

[0088] Now, the timing chart for when the shutter open time te is extended in epipolar imaging is the same as that for the light curtain sensor, as shown in Figure 8. Although each region 51 is stretched in the time direction, the time width over which it overlaps with region 52 remains unchanged. In other words, even if the shutter open time te is extended, the amount of light entering the camera does not change, and the brightness of the image does not improve.

[0089] To make matters worse, extending the shutter open time te increases the thickness of the epipolar plane in the Y direction in epipolar imaging. Increasing the width of region 51 means that overlap is maintained even if the irradiation position of laser 12 is slightly shifted in the Y direction. Light scattered within this range of shift where overlap is maintained is incident on camera 2, reducing the effect of suppressing scattered light from outside the true epipolar plane. This range in the Y direction is referred to here as the Y-direction thickness of the epipolar plane.

[0090] <Solution: Operation in Section P2 of Fig. 22> In order to solve the above-mentioned problem, in this second embodiment, the camera 2 and the laser scanner 1 are controlled as shown in section P2 of Fig. 22. In section P2, the angular function φ(t) of the line of sight direction of the camera 2 is expressed by equations (8) and (9), which have the same form as equation (1). θ(t) = φ(t) = k2 × (t - t2a) + φmin (10) k2 = (φmax - φmin) / T2 (11)

[0091] Here, the scanning range of camera 2 is -25°≦φ≦25° (φmin=-25°, φmax=25°), and the time T2 for camera 2 to scan the entire line is twice T1, i.e., T2=32 ms. In section P2(1), the shutter open time te of camera 2 is also doubled. The timing chart for such an operation can be shown in FIG. 9, similar to that of the light curtain sensor.

[0092] T2 being twice T1 corresponds to doubling the time tc required for one row transition. Because the scanning speed of laser 12 is twice as slow, the width of region 51 is doubled, and the width of region 51, which corresponds to the shutter open time of camera 2, is also doubled. At this time, the overlap time between region 51 and region 52 in section P2(1) is doubled compared to that in section P1(1). In other words, even if the power of laser 12 is the same, the image brightness is doubled. Similarly to a light curtain sensor, the thickness of the epipolar surface does not change between section P1(1) and section P2(1).

[0093] Section P3 in Figure 22 shows the angular function when the scanning speed is reduced to one-third and the camera shutter open time te is tripled. To summarize the above, by slowing down the frame time of the rolling shutter camera 2, slowing down the laser scanning speed accordingly, and extending the camera shutter open time te, it is possible to increase image brightness without increasing the laser power. Furthermore, the thickness of the epipolar surface can be kept the same.

[0094] Modifications of epipolar sensing can be similar to those of the light curtain sensor. In Modifications 1 to 4 of the first embodiment, modifications can be made by replacing the term "light curtain sensing" with "epipolar sensing," but this description is omitted here to avoid redundancy.

[0095] As described above, according to this embodiment, it is possible to provide the image sensing device 101 that can improve the brightness of an image even when capturing an image of an object with low reflectance.

[0096] The features of the above-described embodiments and modifications can be combined with each other.

[0097] REFERENCE SIGNS LIST 1 Laser scanner, 2 Camera, 4 Synchronization circuit, 6 Line beam generating element, 7 Galvanometer mirror, 10, 10a, 10b Virtual screen, 11 Irradiation direction, 12 Laser, 13 Scan range, 21 Line of sight direction, 22 Imaging area, 25 Light curtain, 41 Object, 100, 101 Image sensing device.

Claims

1. A laser scanner capable of scanning a laser extending in a direction parallel to the Y-axis in the XYZ orthogonal coordinate system in a direction parallel to the X-axis in the XYZ orthogonal coordinate system, A camera capable of scanning an imaging region extending in a direction parallel to the Y-axis in a direction parallel to the X-axis in the XYZ orthogonal coordinate system, A synchronization circuit for controlling the laser scanner and the camera Comprising: The laser scanner and the camera are arranged along the X-axis in the XYZ orthogonal coordinate system, The synchronization circuit controls the laser scanner and the camera so that the irradiation direction of the laser irradiated from the laser scanner and the line-of-sight direction of the camera intersect on a two-dimensional plane in a three-dimensional space, The camera images an object intersecting the two-dimensional plane by scanning in the -X direction of the X-axis, The synchronization circuit acquires an image of the object by synchronizing the first scan of the laser scanner and the camera in the -X direction, After the first scan, the synchronization circuit executes a first frame process of controlling the laser scanner and the camera so that the irradiation direction of the laser and the line-of-sight direction of the camera are in the +X direction of the X-axis, After the first frame process, the synchronization circuit acquires an image of the object by synchronizing the second scan of the laser scanner and the camera in the -X direction, After the second scan, the synchronization circuit executes a second frame process of controlling the laser scanner and the camera so that the irradiation direction of the laser and the line-of-sight direction of the camera are in the +X direction, The synchronization circuit controls the laser scanner so that the scan speed of the laser scanner in the second frame process is different from the scan speed of the laser scanner in the first frame process, The synchronization circuit controls the camera so that the scan speed and exposure time of the camera in the second frame process are different from the scan speed and exposure time of the camera in the first frame process An image sensing device characterized by the above.

2. A laser scanner capable of scanning a laser extending in a direction parallel to the X-axis in the XYZ orthogonal coordinate system in a direction parallel to the Y-axis in the XYZ orthogonal coordinate system, A camera capable of scanning an imaging region extending in a direction parallel to the X-axis in a direction parallel to the Y-axis in the XYZ orthogonal coordinate system, A synchronization circuit for controlling the laser scanner and the camera comprising, the laser scanner and the camera are arranged along the X-axis in the XYZ orthogonal coordinate system, the synchronization circuit controls the laser scanner and the camera such that the irradiation direction of the laser emitted from the laser scanner and the line-of-sight direction of the camera always point in the same direction within the YZ plane in the three-dimensional space, the camera images an object intersecting a two-dimensional plane within the three-dimensional space by scanning in the -Y direction of the Y-axis, the synchronization circuit acquires an image of the object by synchronizing the first scan of the laser scanner and the camera in the -Y direction, after the first scan, the synchronization circuit executes a first frame process of controlling the laser scanner and the camera such that the irradiation direction of the laser and the line-of-sight direction of the camera are in the same direction in the +Y direction of the Y-axis at the same time, after the first frame process, the synchronization circuit acquires an image of the object by synchronizing the second scan of the laser scanner and the camera in the -Y direction, after the second scan, the synchronization circuit executes a second frame process of controlling the laser scanner and the camera such that the irradiation direction of the laser and the line-of-sight direction of the camera are in the +Y direction, the scan speed of the laser scanner in the second frame process is different from the scan speed of the laser scanner in the first frame process, the scan speed and exposure time of the camera in the second frame process are different from the scan speed and exposure time of the camera in the first frame process characterized by an image sensing device.

3. the scan speed of the laser scanner in the second frame process is slower than the scan speed of the laser scanner in the first frame process, the scan speed of the camera in the second frame process is slower than the scan speed of the camera in the first frame process, the exposure time of the camera in the second frame process is longer than the exposure time of the camera in the first frame process characterized by the image sensing device according to claim 1 or 2.

4. The scanning speed of the laser scanner in the second frame process is faster than the scanning speed of the laser scanner in the first frame process. The scanning speed of the camera in the second frame process is faster than the scanning speed of the camera in the first frame process. The exposure time of the camera in the second frame process is shorter than the exposure time of the camera in the first frame process. The image sensing device according to claim 1 or 2, characterized in that.

5. When the synchronization circuit detects a new object while the laser scanner and the camera are performing scanning, the synchronization circuit reduces the scanning speeds of the laser scanner and the camera and increases the exposure time of the camera. The image sensing device according to claim 1 or 2, characterized in that.

6. The synchronization circuit restricts the imaging range of the camera and controls the laser scanner and the camera so that the laser is irradiated within the imaging range. The image sensing device according to claim 1 or 2, characterized in that.