Image sensing device

Abstract

An image sensing device (100) is provided with a laser scanner (1), a camera (2), and a synchronization circuit (4). A laser scanner (1) is capable of scanning a laser beam in a direction parallel to an X-axis or a Y-axis. The camera (2) is capable of scanning an imaging region in a direction parallel to the X-axis or the Y-axis. The synchronization circuit (4) controls the laser scanner (1) and the camera (2) such that the irradiation direction of the laser light and the line-of-sight direction of the camera (2) intersect in a two-dimensional plane in a three-dimensional space. The camera (2) captures an image of an object (41) intersecting a two-dimensional plane by scanning. The synchronization circuit (4) acquires an image of the object (41) by synchronizing the scanning of the laser scanner (1) and the scanning of the camera (2).

Description

Technical Field

[0001] This invention relates to an image sensing device. Background Technology

[0002] In recent years, a sensing method has been proposed that uses an illumination plane established by a line laser and a sensing plane sensed by a line sensor to detect objects (for example, see Patent Document 1).

[0003] Patent Document 1: Japanese Patent Publication No. 2022-530349 Summary of the Invention

[0004] In the prior art, there is a problem that the image brightness is low when photographing objects with low reflectivity.

[0005] The purpose of this invention is to solve the above-mentioned problems and to provide an image sensing device that can improve the brightness of the image even when photographing objects with low reflectivity.

[0006] One aspect of the present invention relates to an image sensing device characterized by having:

[0007] A laser scanner that enables a laser extending in a direction parallel to the Y-axis in an orthogonal XYZ coordinate system to scan in a direction parallel to the X-axis in the same orthogonal XYZ coordinate system.

[0008] A camera capable of scanning an area extending in a direction parallel to the Y-axis along a direction parallel to the X-axis; and

[0009] A synchronization circuit controls the laser scanner and the camera.

[0010] The laser scanner and the camera are configured along the X-axis in the XYZ orthogonal coordinate system.

[0011] The synchronization circuit controls the laser scanner and the camera so that the direction of the laser emitted from the laser scanner intersects the line of sight of the camera in a two-dimensional plane within three-dimensional space.

[0012] The camera captures images of objects intersecting the two-dimensional surface by scanning along the -X direction of the X-axis.

[0013] The synchronization circuit acquires an image of the object by synchronizing the first scan of the laser scanner and the camera in the -X direction.

[0014] After the first scan, the synchronization circuit executes the first frame process, that is, it controls the laser scanner and the camera so that the laser illumination direction and the camera's line of sight are aligned with the +X direction of the X-axis.

[0015] Following the first frame process, the synchronization circuit acquires an image of the object by synchronizing the second scan of the laser scanner and the camera in the -X direction.

[0016] After the second scan, the synchronization circuit executes the second frame process, that is, it controls the laser scanner and the camera so that the laser illumination direction and the camera's line-of-sight direction are aligned with the +X direction.

[0017] The synchronization circuit controls the laser scanner so that the scanning speed of the laser scanner in the second frame process is different from the scanning speed of the laser scanner in the first frame process.

[0018] The synchronization circuit controls the camera so that the scanning speed and exposure time of the camera in the second frame process are different from those of the camera in the first frame process.

[0019] Another aspect of the image sensing device according to the present invention is characterized by having:

[0020] A laser scanner that enables a laser extending in a direction parallel to the X-axis in an orthogonal XYZ coordinate system to scan in a direction parallel to the Y-axis in the same orthogonal XYZ coordinate system.

[0021] A camera capable of scanning an area extending in a direction parallel to the X-axis along a direction parallel to the Y-axis; and

[0022] A synchronization circuit controls the laser scanner and the camera.

[0023] The laser scanner and the camera are configured along the X-axis in the XYZ orthogonal coordinate system.

[0024] The synchronization circuit controls the laser scanner and the camera to ensure that the direction of the laser emitted from the laser scanner and the line of sight of the camera always face the same direction in the YZ plane of three-dimensional space.

[0025] The camera captures images of objects intersecting the two-dimensional surface by scanning along the -Y direction of the Y-axis.

[0026] The synchronization circuit acquires an image of the object by synchronizing the first scan of the laser scanner and the camera in the -Y direction.

[0027] After the first scan, the synchronization circuit executes the first frame process, that is, it controls the laser scanner and the camera so that at the same time, the direction of the laser beam and the direction of the camera's line of sight are the same in the +Y direction of the Y-axis.

[0028] Following the first frame process, the synchronization circuit acquires an image of the object by synchronizing the second scan of the laser scanner and the camera in the -Y direction.

[0029] After the second scan, the synchronization circuit executes the second frame process, that is, it controls the laser scanner and the camera so that the laser illumination direction and the camera's line-of-sight direction are aligned with the +Y direction.

[0030] The scanning speed of the laser scanner in the second frame process is different from the scanning speed of the laser scanner in the first frame process.

[0031] The scanning speed and exposure time of the camera in the second frame process are different from those of the camera in the first frame process.

[0032] The effects of the invention

[0033] According to the present invention, an image sensing device can be provided that can improve the brightness of an image even when photographing an object with low reflectivity. Attached Figure Description

[0034] Figure 1 This is a perspective view that schematically shows the structure of an image sensing device (also known as a light curtain sensor device) used for light curtain sensing.

[0035] Figure 2 This is a diagram illustrating an example of the structure of a laser scanner.

[0036] Figure 3 This diagram illustrates the generation of a light curtain within the XZ plane.

[0037] Figure 4 (A) and (B) are diagrams illustrating the movement of the light screen on the virtual screen.

[0038] Figure 5 This is a diagram illustrating the operation of a laser scanner and the operation of each camera.

[0039] Figure 6 It is a diagram showing the geometric configuration used to generate the light curtain between the camera and the camera.

[0040] Figure 7 It is a diagram that shows the line scanning action of the camera, the position and time of laser illumination.

[0041] Figure 8 It is a diagram that shows the line scanning action of the camera, the position and time of laser illumination.

[0042] Figure 9 It is a diagram that shows the line scanning action of the camera, the position and time of laser illumination.

[0043] Figure 10 It is a diagram that shows the line scanning action of the camera, the position and time of laser illumination.

[0044] Figure 11 It is a diagram that shows the line scanning action of the camera, the position and time of laser illumination.

[0045] Figure 12 This is a diagram illustrating, as a comparative example, how an object is sensed by a light curtain sensor.

[0046] Figure 13 It indicates that it is used for passing through Figure 12 The diagram shown illustrates the timing of the camera and laser scanner controlling the 3D depth map obtained by the light curtain sensor.

[0047] Figure 14 This is a perspective view showing the structure of an image sensing device (also known as a polar imaging sensor) used for polar imaging.

[0048] Figure 15 It means Figure 14 A top view of the image sensing device shown.

[0049] Figure 16 This is a diagram illustrating an example of the structure of a laser scanner.

[0050] Figure 17 This is a diagram illustrating an example of the structure of a laser scanner.

[0051] Figure 18 This is a diagram illustrating an example of the structure of a laser scanner.

[0052] Figure 19 It is a diagram representing the movement of a laser on a virtual screen.

[0053] Figure 20 It is a diagram representing the movement of a laser on a virtual screen.

[0054] Figure 21It is a diagram representing the movement of a laser on a virtual screen.

[0055] Figure 22 This diagram illustrates the control method for sensing based on an epipolar imaging sensor. Detailed Implementation

[0056] In the XYZ orthogonal coordinate system shown in the figures, the Z-axis direction (Z-axis) represents the direction orthogonal to the virtual screen, the Y-axis direction (Y) represents the direction orthogonal to the Z-axis direction and parallel to the shooting area, and the X-axis direction (X-axis) represents the direction orthogonal to both the Z-axis direction and the Y-axis direction.

[0057] <Implementation Method 1>

[0058] Image Sensing Device 100

[0059] Figure 1 This is a perspective view schematically showing the structure of the image sensing device 100 (also called a light curtain sensor device) used for light curtain sensing in this invention.

[0060] exist Figure 1 The example shown illustrates the case where a light screen 25 is generated by a combination of a laser scanner 1 and a camera 2.

[0061] The image sensing device 100 includes a laser scanner 1, a camera 2, and a synchronization circuit 4. In the image sensing device 100, the laser scanner 1 and the camera 2 are arranged along the X-axis, and they are controlled by the synchronization circuit 4. Figure 1 In the example shown, the laser scanner 1 and the camera 2 are configured along the X-axis in the XYZ orthogonal coordinate system.

[0062] Laser Scanner 1

[0063] Figure 2 This is a diagram showing an example of the structure of laser scanner 1.

[0064] Laser scanner 1 emits laser 12. Laser scanner 1 is capable of scanning the laser 12 (also called a line laser) which extends in a direction parallel to the Y-axis in an orthogonal XYZ coordinate system along a direction parallel to the X-axis.

[0065] A laser 12, extending along the Y-axis, is emitted from a laser scanner 1. At a distance Z0 from the laser scanner 1 along the Z-axis, there exists a plane, i.e., a virtual screen 10, perpendicular to the Z-axis direction, used for illustration. The laser emitted from the laser scanner 1 appears as a line 12 extending in a direction parallel to the Y-axis on the virtual screen 10.

[0066] A laser beam with wavelength λ is emitted from laser source 5 and transformed into a laser beam 12 extending along the Y direction by line beam generating element 6. Line beam generating element 6 is, for example, a cylindrical lens or a Powell lens. The laser beam 12 is deflected in the Z direction by galvanometer 7. Galvanometer 7 is capable of oscillating around the Y-axis at angles ±α, and the laser beam 12 scans around the Y-axis within an angle range of ±2α. On virtual screen 10, the laser beam 12 is scanned from the +X direction to the -X direction, illuminating... Figure 1 or Figure 2 The entire area of ​​the scan range 13.

[0067] Camera 2

[0068] Camera 2 is capable of scanning a shooting area 22 (also called a linear shooting area) extending in a direction parallel to the Y-axis in the XYZ orthogonal coordinate system along a direction parallel to the X-axis in the XYZ orthogonal coordinate system. For example, camera 2 can capture images of objects 41 (also called objects) that intersect two-dimensional surfaces in three-dimensional space by scanning along the +X direction of the X-axis.

[0069] In this embodiment, camera 2 is a rolling shutter camera. Camera 2 reads the brightness information of the image sensor pixels sequentially, starting from the top row. By controlling the synchronization circuit 4, the exposure time of camera 2 is shortened, thereby enabling the scanning of the imaging area extending in the row direction of the image sensor along the column direction orthogonal to it. For example, for a 1000-row rolling shutter camera operating at 50fps, if the exposure time for each row is set to te = 20μs, adjacent rows can be separated in time for line-by-line sensing. In actual operation, the synchronization circuit 4 causes camera 2 to shift the exposure area sequentially in five rows, for example, with an exposure time of te = 100μs.

[0070] Typically, rolling shutter cameras are used with the horizontal direction of the line, but in this embodiment, camera 2 is rotated -90° around the optical axis. Camera 2 is positioned such that the optical axis of camera 2 is slightly offset inward (in the +X direction) relative to the Z-axis direction, so that the entire shooting range 23 of camera 2 on virtual screen 10 substantially overlaps with the entire scanning range 13 of the laser. However, if the entire shooting range 23 overlaps with the entire scanning range 13 even when camera 2 is oriented in the Z-axis direction, a light curtain 25 is generated in the overlapping area.

[0071] Additionally, a bandpass filter 8 is provided in the camera 2. This filter can be located at the front of the lens barrel of the camera 2, or it can be located inside the camera 2. The bandpass filter 8 is a wavelength filter that only allows light of wavelength λ emitted from the laser source 5 to pass through.

[0072] Synchronous Circuits 4

[0073] Synchronization circuit 4 controls laser scanner 1 and camera 2. Synchronization circuit 4 enables the operation of laser scanner 1 and camera 2 to be synchronized. For example, synchronization circuit 4 controls laser scanner 1 and camera 2 so that the irradiation direction of laser 12 emitted from laser scanner 1 intersects the line of sight of camera 2 in a two-dimensional plane in three-dimensional space.

[0074] Synchronization circuit 4 acquires an image of object 41 by synchronizing the first scan of laser scanner 1 and camera 2 in the -X direction. After the first scan, synchronization circuit 4 executes the first frame process, that is, it controls laser scanner 1 and camera 2 so that the irradiation direction of laser 12 and the line of sight of camera 2 return to the +X direction of the X axis.

[0075] After the first frame process, the synchronization circuit 4 synchronizes the laser scanner 1 and the camera 2 in the -X direction during the second scan, thereby acquiring an image of the object 41. After the second scan, the synchronization circuit 4 executes the second frame process, that is, it controls the laser scanner 1 and the camera 2 so that the irradiation direction of the laser 12 and the line of sight of the camera 2 return to the +X direction.

[0076] Synchronization circuit 4 controls laser scanner 1 so that the scanning speed of laser scanner 1 in the second frame process is different from that in the first frame process. Synchronization circuit 4 also controls camera 2 so that the scanning speed and exposure time of camera 2 in the second frame process are different from those in the first frame process.

[0077] If a new object is detected by the synchronization circuit 4 during scanning by the laser scanner 1 and camera 2, the synchronization circuit 4 can also reduce the scanning speed of the laser scanner 1 and camera 2, and increase the exposure time of the camera 2. This allows sensing to be performed at the optimal frame rate and image brightness in accordance with the object.

[0078] The synchronization circuit 4 can also control the laser scanner 1 and the camera 2 to limit the shooting range of the camera 2, ensuring that the laser 12 is illuminated within the shooting range. This allows for increased brightness only in areas of low image brightness, thus shortening the sensing time.

[0079] Example 1 of scanning speed and exposure time

[0080] The scanning speed of laser scanner 1 in the second frame process is slower than that in the first frame process. In this case, the scanning speed of camera 2 in the second frame process is also slower than that in the first frame process, and the exposure time of camera 2 in the second frame process is longer than that in the first frame process. This improves image brightness.

[0081] Example 2 of scanning speed and exposure time

[0082] The scanning speed of laser scanner 1 in the second frame process is faster than that in the first frame process. In this case, the scanning speed of camera 2 in the second frame process is also faster than that in the first frame process, and the exposure time of camera 2 in the second frame process is shorter than that in the first frame process. Therefore, the frame rate can be increased when the image brightness is sufficient.

[0083] Light Curtain Sensing

[0084] Figure 3 This diagram illustrates the situation where a light curtain 25 is generated in the XZ plane.

[0085] Figure 4 (A) and Figure 4 (B) is a diagram illustrating the action of the light screen 25 on the virtual screen 10.

[0086] exist Figure 3 In this context, the angle between the illumination direction 11 of the laser 12 from the Z-axis and the Z-axis is defined as θ. However, the clockwise direction is defined as positive. Figure 3 At a certain time ta, the angle θa between the illumination direction 11a at time ta and the Z-axis has a negative value. Similarly, the angle θb between the illumination direction 11b at time tb and the Z-axis has a positive value.

[0087] Camera 2 is positioned with its optical axis rotated -90°, therefore, the line-of-sight direction 21 of camera 2 scans clockwise within the XZ plane. The angle formed clockwise with the Z-axis is defined as φ. Figure 3 In this context, the angles φa and φb formed by the viewing direction 21a at time ta and the viewing direction 21b at time tb are both negative. Here, at time ta, the illumination direction 11a and the viewing direction 21a intersect at point 24a on the virtual screen 10a. Similarly, at time tb, the illumination direction 11b and the viewing direction 21b intersect at point 24b on the virtual screen 10a. Figure 4 and Figure 5 This indicates the relationship between the laser 12 on the virtual screen 10a located at a distance of Za and the virtual screen 10b located at a distance of Zb and the shooting area 22.

[0088] On the virtual screen 10a, scanning is performed with the laser 12 always overlapping the imaging area 22. This can be achieved by appropriately controlling the time-corresponding angle of the galvanometer 7, as explained later. At this time, the virtual screen 10b is positioned at a different distance from the virtual screen 10a, thus, as... Figure 5 As shown, laser 12 does not overlap with the shooting area 22.

[0089] When a light curtain 25 is generated on the virtual screen 10a, if the light curtain 25 intersects with the object 41 (also called the measurement object) being detected, the image outside the intersection area will not be captured by the camera 2. The exposure time of each pixel in a frame is extremely short (e.g., exposure time te = 60 μs), and light outside the laser wavelength is filtered out by the bandpass filter 8 before reaching the image sensor inside the camera 2. Therefore, the background will not be captured during light curtain sensing.

[0090] Here, the conditions for scanning in a manner that causes the laser 12 to overlap with the shooting area 22 on the virtual screen 10a are described. First, when the laser 12 extends in a line along a direction parallel to the Y-axis, the shooting area 22 of the camera 2 needs to extend accurately in a direction parallel to the Y-axis. For this purpose, a fine-tuning mechanism is provided for fine-tuning the orientation of the camera 2 around the optical axis. In addition, the shooting lens of the camera 2 needs to be a lens that has been corrected for distortion. This is because, if this is not done, the shooting area 22 will be distorted from a straight line. Therefore, in order to scan on the virtual screen 10a in a state where the laser 12 overlaps with the shooting area 22, it is necessary to coordinate the timing of the start of the camera 2's shooting with the timing of the start of the laser scanner 1's scanning, and for this purpose, a synchronization circuit 4 is used. Furthermore, on the virtual screen 10, the scanning speed of the laser 12 and the scanning speed of the shooting area 22 need to be the same.

[0091] Figure 5 This is a diagram illustrating the operation of a laser scanner and a camera.

[0092] exist Figure 5 In the diagram, the dashed line represents the angle function φ(t) of the viewing direction of camera 2, and the solid line represents the angle function θ(t) of the illumination direction of laser 12. In intervals P1 and P2, a light curtain perpendicular to the Z-axis is generated at a distance of Z0 = 0.5m; in intervals P3 and P4, a light curtain perpendicular to the Z-axis is generated at a distance of Z0 = 1m. To... Figure 5 The specific parameters used for the calculation of the curve shown will be described later.

[0093] Actions in interval P1

[0094] First, describe the actions in interval P1.

[0095] Figure 6 This is a diagram showing the geometric configuration used to generate the light curtain 25.

[0096] On a virtual screen 10a positioned at a distance Z0 along the Z-axis, laser 12 overlaps with the shooting area 22. The angle function θ(t) of the illumination direction 11 of laser 12 and the angle function φ(t) of the viewing direction of camera 2 are as follows: Figure 6 The figure shows the angle relative to the Z-axis direction. Figure 6 In this configuration, both angles θ and φ have negative values. The angle of the galvanometer is half of θ (α = θ(t) / 2). The angle function φ(t) of the line of sight of camera 2 is expressed by equation (1) using a constant coefficient k1 determined by the frame rate.

[0097] φ(t)=k1×t+φmin (1)

[0098] Typically, it is difficult to make the line scanning speed of a rolling shutter camera operate at a speed other than a constant speed. Therefore, control is performed to match the angle function θ(t) of the illumination direction 11 of the laser 12 with the angle function φ(t) of the viewing direction of the camera 2. Figure 6 Find the relationship between θ(t) and φ(t). Let the rotation axis of laser scanner 1 be point Q, and the rotation axis of the line of sight 21 of camera 2 be point R, with an interval of b. Let the foot of the perpendicular from intersection point 24 to line QR be point P. Thus, the signed distance PR is denoted as Z0×tanφ using φ. Furthermore, since the signed distance PQ is denoted as Z0×tanθ, equation (2) holds.

[0099] Z0×tanφ=Z0×tanθ-b (2)

[0100] That is, given a fixed angle φ, the angle θ of laser 12 is calculated by the following formula (3).

[0101] θ=Arctan(tanφ+b / Z0) (3)

[0102] When the virtual screen 10a is a plane perpendicular to the Z-axis, the distance Z0 is a constant value. Figure 5 The curve for interval P1 illustrates the function of the light curtain 25 that generates the plane at a distance Z0. Specifically, the parameters used to describe the angular function of interval P1 are as follows.

[0103] Parameters of interval P1

[0104] The scanning range of laser scanner 1 is: -25°≤θ≤25° (θmin=-25°, θmax=25°).

[0105] The scanning range of camera 2 is: -35°≤φ≤15° (φmin=-35°, φmax=15°).

[0106] b = 0.15m

[0107] The time for camera 2 to perform the entire line scan is T1 = 16 ms

[0108] Z0 = 0.5m

[0109] Furthermore, the coefficient k1 in equation (1) is expressed as equation (4).

[0110] k1=(φmax-φmin) / T1 (4)

[0111] Here, camera 2 has a field of view of 50° in the ZX plane, and its optical axis is tilted by -10° relative to the Z-axis direction.

[0112] The position of the plane with Z0 = 0.5m in the parameters of interval P1 is equivalent to that of... Figure 2 The location marked on the virtual screen 10b. Figure 5 In the curve of interval P1, the line-of-sight function φ(t) of camera 2, as shown in equation (1), is a first-order function with respect to time t, changing from φmin = -35° to φmax = 15° during the period from t = 0 to 16 ms. The angle θ of laser 12 during this period is given by the above equation (3), and is illustrated by θ(t) in interval P1(1). However, the angle θ can only swing up to 25°, therefore, the scanning used to generate the light curtain 25 will end before reaching T1 = 16 ms. At this time, in Figure 3 On the virtual screen 10b, the intersection point 26 is located to the left of the intersection point 27. Furthermore, the intersection point between the laser 12's illumination direction at its maximum angle θmax and the virtual screen 10b is set to 26, and the intersection point between the camera 2's line-of-sight direction 21 at its maximum angle φmax and the virtual screen 10b is set to 27. If the scanning of laser 12 in interval P1(1) ends, the angle of galvanometer 7 needs to be returned in the negative direction to prepare for the next laser scan. Galvanometer 7 is an object with rotational inertia; therefore, a considerable finite time Tback is required to return to the starting position of the next scan. Although this varies depending on the product specifications of galvanometer 7, for example, Tback = 4 ms is required. In interval P1(2), the laser angle oscillates sinusoidally between +25° and -25°.

[0113] Operation of Synchronous Circuits

[0114] To make the angle function φ(t) of the viewing direction of camera 2 and the angle function θ(t) of the illumination direction 11 of laser 12 equal to... Figure 5 The precise timing shown is achieved using synchronization circuit 4. Synchronization circuit 4 is pre-programmed with a waveform representing the angle function θ(t) of the laser 12's irradiation direction 11, and outputs a waveform that matches the angle function θ(t) of the laser 12's irradiation direction 11. Figure 5 The timing of the trigger pulse for camera 2 is shown. An analog voltage waveform is output from synchronization circuit 4 to laser scanner 1, which gives the angle function θ(t) of the illumination direction 11 of laser 12. A trigger pulse is output to camera 2. Camera 2 is triggered by the rising edge of the trigger pulse voltage to begin capturing one frame. That is, it begins scanning the line-image area.

[0115] Figure 5 The timing of the ON-OFF of laser 12 is shown. Synchronization circuit 4 controls the ON-OFF of laser 12 in the interval P1(1) where the light curtain is generated.

[0116] Three-dimensional measurement achieved by a light curtain sensor

[0117] Figure 12 This is a diagram illustrating, as a comparative example, how an object is sensed by a light curtain sensor.

[0118] Figure 13 It indicates that it is used for passing through Figure 12 The diagram shown illustrates the timing of the camera and laser scanner controlling the 3D depth map obtained by the light curtain sensor. Figure 13 Explanation and Figure 5 same.

[0119] If an action is performed to narrow the spacing of the light curtains in the Z-direction and to arrange a large number of light curtains, then three-dimensional measurement can be performed. For example, in Figure 12 In the example shown, a light curtain is set at the distance Zn calculated by equation (5) in the interval Pn.

[0120] Zn=Z0+(n-1)ΔZ (5)

[0121] (n is a natural number)

[0122] That is, the interval between adjacent light screens is ΔZ. An object 41 is placed in the sensing area of ​​such a light screen. Images of cross-sections intersecting with the object 41, such as light screens 25a, 25b, 25c, ..., are captured by camera 2. The cross-sectional area of ​​the object 41 projected onto the light screen at a distance Zn is located at a distance Zn from the light screen sensor. By acquiring a large number of cross-sectional images at different distances Zn, distance information of the surface of the object 41, i.e., a three-dimensional depth map, can be obtained.

[0123] exist Figure 13 In the example shown, planar light curtains were generated at positions Z = 0.5m, 0.7m, 0.9m, and 1.1m in the intervals P1(1), P2(1), P3(1), and P4(1).

[0124] Here, when there are dark areas in a part of an object, or areas where the surface normal is nearly perpendicular to the line of sight of camera 2, the amount of reflected and scattered light from these areas entering the pupil of camera 2 decreases. That is, the brightness of the image in these areas may be significantly reduced, sometimes resulting in data loss when generating a 3D depth map.

[0125] To improve image brightness, the amount of laser light emitted can be increased, but there is an upper limit to the amount of laser light produced. Another commonly considered method to improve image brightness is to increase the shutter opening time (te) of the camera. However, light curtain sensors present a significant challenge: simply increasing the shutter opening time (te) does not improve image brightness.

[0126] Figures 7 to 11 It is a diagram showing the line scanning action of camera 2, the illumination position and time of laser 12.

[0127] Figure 7 For example Figure 5 The diagram showing the generation of the light curtain in interval P1.

[0128] exist Figures 7 to 11 In the diagram, the horizontal axis represents the passage of time, and the vertical axis represents the row number of the rolling shutter camera 2. The horizontal width of the shaded rectangular area 51 in the diagram represents the time it takes to read the image within one row of pixels, i.e., the shutter opening time te. In the rolling shutter camera 2, each row of pixels is read out sequentially, starting from the top row; therefore, a readout delay time tc is generated for each row. For example, given a specific value, in the case of T = 16 ms and 1000 rows, tc = 16 μs. Figures 7 to 11 For example, te = 48 μs.

[0129] Figures 7 to 11 The shaded area 52 represents the time it takes for the laser 12 to illuminate each row. Here, it is assumed that the line width of the laser 12 on the virtual screen 10a is sufficiently small compared to the resolution of the camera 2 on the virtual screen 10a. In this case, if the laser 12 scans on the virtual screen 10a, it illuminates each row sequentially in the image rows of the camera 2, thus... Figures 7 to 11The shadow region 52, representing the irradiation timing of laser 12, moves diagonally downwards and to the right in each row without overlap in the time direction. The angle function θ(t) of the irradiation direction 11 of laser 12 is given by... Figure 5 The function is controlled as shown in the interval P1(1), therefore, in Figure 7 In this configuration, region 52 overlaps with region 51. Thus, during the time and space of the overlap between regions 52 and 51, if an object 41 is present at that location, the reflected and scattered light from that location will enter the camera 2 and be sensed. Because a high-speed scan of 1000 lines per frame is performed in a short time interval of T = 16 ms, it appears as if a light curtain exists in space. Furthermore, the timing diagram on the virtual screen 10b located at a distance Z0' different from the distance Z0 from which the light curtain is generated is, for example, as shown... Figure 10 As shown. That is, regions 52 and 51 move in the same direction relative to time, but corresponding to the difference between distance Z0' and distance Z0, region 52 is offset in the time direction, and the two do not overlap. Even if object 41 exists at this distance Z0', the image of the object will not be sensed. That is, the action of generating the light curtain is equivalent to: when a planar object, which is a real object, exists on the light curtain, in Figure 7 On the chart, as time passes, it shows that the shadow area 52 illuminated by laser 12 moves over the shadow area 51.

[0130] Figure 8 This shows a timing diagram on a virtual screen 10a where the shutter opening time te is increased, after a light curtain has been generated. Figure 7 In contrast, region 51 extends in the temporal direction. However, the temporal width overlapping with region 52 remains unchanged. That is, even if the shutter opening time te is increased, the amount of light entering camera 2 remains unchanged, and the brightness of the image is not improved.

[0131] Worse still, if the shutter opening time te is increased, the thickness of the light curtain in the Z direction also increases. At a slightly different distance (Z0+δ) from the distance Z0, as described above... Figure 10 As explained, region 52 shifts to the right, becoming Figure 11 The graph shown illustrates this. It reveals that regions 51 and 52 still overlap and are also sensed at a distance of (Z0+δ). That is, if the shutter opening time te is increased, the thickness of the light curtain increases.

[0132] Solution: Figure 5 Actions in interval P2

[0133] In this embodiment 1, the following is performed: Figure 5The interval P2 shows the control of camera 2 and laser scanner 1. In interval P2, the angle function φ(t) of the line of sight of camera 2 is expressed by equations (6) and (7) of the same form as equation (1).

[0134] φ (t)=k2× (t-t2a)+φmin (6)

[0135] k2=(φmax-φmin) / T2 (7)

[0136] The time T2 for camera 2 to perform the entire line scan is set to twice the time T1, T2 = 32 ms. The angle function θ(t) of the irradiation direction of laser 12 is calculated using equation (3). Its curve is... Figure 5 The interval P2(1) becomes the shape of the curve of interval P1(1) stretched horizontally by a factor of 2. In interval P2(1), the shutter opening time te of camera 2 is also doubled. Figure 9 The timing diagram for performing such an action is shown. The fact that T2 becomes twice T1 corresponds to the time tc associated with the movement of one row becoming twice. The laser scanning speed is twice as slow, therefore, the lateral width of region 51 becomes twice, and the lateral width of region 51, which corresponds to the camera shutter opening time, also becomes twice. At this time, compared to region P1(1), the overlap time between region 51 and region 52 in interval P2(1) is also doubled. That is, even if the power of laser 12 is the same, the image brightness becomes twice. Furthermore, on the virtual screen 10b located at a distance (Z0+δ) slightly different from the distance Z0 from the generated light curtain, region 52 is slightly offset laterally. Figure 7 , Figure 11 Compared to the situation, Figure 9 The offset in the middle is increased to twice its original value, therefore, in Figure 7 and Figure 9 Given the same difference in distance δ, regions 51 and 52 overlap by the same proportion. That is, the thickness of the light curtain remains unchanged in intervals P1(1) and P2(1).

[0137] In summary, by slowing down the frame time of the rolling shutter camera 2, correspondingly slowing down the laser scanning speed, and lengthening the shutter opening time te, image brightness can be improved without increasing laser power. Furthermore, the thickness of the light curtain can be kept constant at this time.

[0138] exist Figure 5In interval P3, a planar light curtain is generated at the position Z = 1m. The time for camera 2 to perform the entire line scan is the same as in interval P1, which is T1 = 16ms. In interval P4, a planar light curtain is generated at the same position as in interval P3, but the time for the entire line scan is twice that of interval P3, and the brightness of the image is doubled.

[0139] Variations

[0140] As described above, by changing the line break time tc of camera 2 (which is a rolling shutter camera), the shutter opening time te of camera 2, and the scanning speed of laser 12, the image brightness can be changed in the next frame even in the middle of generating multiple light screen frames in succession. Various variations can be conceived regarding image sensing using a projection-type light screen sensor, as described below.

[0141] Variation Example 1

[0142] Consider the following variation: During high-speed light curtain sensing, if it's suspected that an object exists within the sensed image area but hasn't been detected, the light curtain sensing speed can be slowed down to increase image brightness. For example, consider the scenario where the upper body of a person wearing a black leather jacket cannot be sensed. Even if the person is identified using image recognition technology like AI, but a part of the body is not detected, simply slowing down the scanning speed of the projection-type light curtain in the next frame to increase image brightness allows even black areas to be sensed. Thus, by changing the angle functions φ(t) and θ(t) midway through a series of sensing operations in consecutive frames, image brightness can be increased. This enables the detection of areas undetectable during high-speed sensing, achieving high-precision sensing.

[0143] Variation Example 2

[0144] Another possible variation is to perform slow sensing to obtain a high-brightness image in the first frame, and then determine the scanning speed for the next frame based on the brightness information. That is, if too much brightness information is obtained, the brightness is reduced and sensing is performed at a high speed. In this case, the image brightness value of the object sensed in the first frame can be compared with a sensing threshold, and the sensing speed in the next frame can be determined based on the ratio. For example, if the image brightness value is 30 relative to a threshold of 5, even if the sensing speed is tripled, reducing the image brightness value to 1 / 3, a brightness value of 10 is expected, which is sufficient for object detection.

[0145] Variation Example 3

[0146] Another possible variation is to randomly insert a high-brightness, slowed-down frame A during high-speed sensing when it is impossible to determine what kind of object is present in the detection area. If no new object is detected in frame A, high-speed sensing continues; if a new object is detected, the system switches to the slowed-down frame.

[0147] Variation Example 4

[0148] Another possible variation is to limit the camera's field of view by using the Region of Interest (ROI) function, when the area of ​​view of a low-brightness object is known, thereby slowly scanning only a localized area. The ROI function, for example, refers to the ability to modify camera controls in a 1000-row pixel camera to capture only rows 301 to 500 (200 rows in a single frame). This reduces the time spent capturing unwanted areas. Similarly, in projection-type light curtain sensing, if the laser scans only corresponding to the pixels of the rows selected by the ROI function, a limited light curtain can be generated. This also saves time in light curtain sensing.

[0149] As described above, according to this embodiment, an image sensing device 100 can be provided that can improve the brightness of the image even when photographing an object 41 with low reflectivity.

[0150] <Implementation Method 2>

[0151] Epipolar Imaging

[0152] Polarimetric imaging is a technique that projects illumination light to obtain an image of the projected object. Similar to the light curtain sensor, laser scanner 1 and camera 2 are arranged in the X-direction and controlled by a synchronization circuit. Laser scanner 1 scans a linear laser beam extending in the X-direction along the -Y-direction, its movement repeating at a period T. Camera 2 is a rolling shutter camera, which, by shortening the exposure time, can repeatedly scan the area extending in the X-direction along the -Y-direction. The scanning speed of camera 2 in the Y-direction is set to the same speed as that of laser scanner 1. The synchronization circuit controls the process so that the area illuminated by laser 12 always coincides with the area captured by camera 2.

[0153] The plane formed by the illumination area of ​​laser 12 and the imaging area of ​​camera 2 is the same as the polar plane in triangulation. The camera in this device always captures only the scattered light from the object within the polar plane at a given instant; therefore, it is called polarimetric imaging. Polarimetric imaging can suppress the scattered stray light from objects that generate significant reflected stray light. Utilizing this effect, even for glossy metallic objects, accurate three-dimensional measurements without reflected stray light can be achieved. Furthermore, even in bright outdoor environments, low-light projection light can be efficiently obtained, thus enabling sensing even outdoors.

[0154] Image Sensing Device 101

[0155] Figure 14 This is a perspective view showing the structure of the image sensing device 101 (also called a polar imaging sensor) for performing polar imaging according to the present invention.

[0156] Figure 15 It means Figure 14 A top view of the image sensing device 101 shown.

[0157] The image sensing device 101 includes a laser scanner 1, a camera 2, and a synchronization circuit 4. The camera 2 and the laser scanner 1 are arranged along the X-axis in an XYZ orthogonal coordinate system. A virtual screen 10, perpendicular to the optical axis C of the camera, is set at a position separated from the camera 2 by a distance Z0. This virtual screen 10 does not physically exist; it represents a plane for illustration.

[0158] Laser Scanner 1

[0159] Figures 16 to 18 This is a diagram showing an example of the structure of laser scanner 1.

[0160] Laser scanner 1 emits laser 12. Laser scanner 1 is capable of scanning the laser 12, which extends in a direction parallel to the X-axis in the XYZ orthogonal coordinate system, in a direction parallel to the Y-axis in the XYZ orthogonal coordinate system.

[0161] A laser 12, emitted from laser scanner 1 and extending along the X-axis, appears on virtual screen 10 as a laser 12 extending in a direction parallel to the X-axis. For example... Figures 16 to 18As shown, a laser beam emitted from laser source 5 is reflected by mirror 9 and then transformed into laser beam 12, which extends in a direction parallel to the X-axis, by line beam generating element 6. Line beam generating element 6 is, for example, a cylindrical lens or a Powell lens. This laser beam 12 is deflected in the Z-direction by galvanometer 7. Galvanometer 7 can oscillate around the X-axis within an angle range of ±α, and laser beam 12 scans around the X-axis within an angle range of ±2α. On virtual screen 10, laser beam 12 scans along the Y-direction. Figure 14 The area within the scanning range 13 is irradiated.

[0162] The optical axis L of the laser scanner 1 is set as the direction of travel of the center of the linear beam when the angle α of the galvanometer is 0°. Here, the optical axis C of the camera 2 is set to be parallel to the optical axis L of the laser scanner. At this time, the optical axis L of the laser scanner 1 is perpendicular to the virtual screen 10.

[0163] Figures 19 to 21 This is a diagram showing the movement of laser 12 on virtual screen 10.

[0164] For example, in Figure 20 The image shows the movement of laser 12 on virtual screen 10, illustrating how laser 12 repeatedly scans the scanning range 13 from top to bottom in the -Y direction at a speed VL. For laser 12, at time t = ta, it exists as laser 12a at the upper end of the scanning range 13; at time t = tb, it is shown as laser 12b; and at time t = tc, it exists as laser 12c at the lower end of the scanning range 13. If it reaches the lower end, it returns to the upper end at high speed, repeating the above operation.

[0165] Camera 2

[0166] Camera 2 is capable of scanning a shooting area 22 (also called a linear shooting area) extending in a direction parallel to the X-axis in the XYZ orthogonal coordinate system along a direction parallel to the Y-axis in the XYZ orthogonal coordinate system. For example, camera 2 can capture images of objects (also called objects) that intersect two-dimensional surfaces by scanning along the -Y direction of the Y-axis.

[0167] In this embodiment, the camera 2 is a rolling shutter camera, which, by shortening the exposure time, can repeatedly perform the action of scanning the shooting area 22, which extends in a direction parallel to the X-axis, along the -Y direction.

[0168] On the virtual screen 10, the shooting area 22 of the camera 2 is scanned from the top to the bottom of the entire shooting range 23 of the camera 2. Figure 19The situation is illustrated. The shooting area 22 of camera 2 is scanned at a speed Vc from the shooting area 22a at the upper end of the entire shooting range 23 to the shooting area 22c at the lower end. Here, the device structure is preset so that the shooting area 22a of the camera overlaps with the laser 12a in the Y-axis direction on the virtual screen 10. This setting can be made by zooming the camera lens, setting the region of interest (ROI) that limits the shooting area of ​​the camera, or setting the scanning range of the laser 12 in the Y direction. A mechanism for fine-tuning the setting posture of camera 2 or laser scanner 1 is also important.

[0169] Synchronous Circuits 4

[0170] Synchronization circuit 4 controls laser scanner 1 and camera 2. Synchronization circuit 4 enables the operation of laser scanner 1 and camera 2 to be synchronized. For example, synchronization circuit 4 controls laser scanner 1 and camera 2 so that the direction of the laser 12 emitted from laser scanner 1 is the same as the line of sight of camera 2.

[0171] In this embodiment, the first frame process and the second frame process each consist of the following steps: a process of acquiring an image of the object by scanning in the -Y direction using the laser scanner 1 and the camera 2, and a process of returning to the same position in the +Y direction in preparation for acquiring an image in the next frame. In the first frame process, the synchronization circuit 4 acquires an image of the object by scanning synchronously in a manner that the irradiation direction of the laser 12 and the viewing direction of the camera 2 always face the same direction in the YZ plane. After the first scan, the synchronization circuit 4 controls the laser scanner 1 and the camera 2 to return to the same direction in the +Y direction of the Y axis at the same time, thus ending the first frame process.

[0172] In the second frame process, after the first frame process, the synchronization circuit 4 scans synchronously in a manner that ensures the irradiation direction of the laser 12 and the viewing direction of the camera 2 always face the same direction in the YZ plane, thereby acquiring an image of the object. After the second scan, the synchronization circuit 4 controls the laser scanner 1 and the camera 2 to return to the +Y direction, thus ending the first frame process.

[0173] The scanning speed of laser scanner 1 in the second frame process is different from that in the first frame process. The scanning speed and exposure time of camera 2 in the second frame process are also different from those in the first frame process. Therefore, it is possible to change the image brightness between frames.

[0174] If a new object is detected by the synchronization circuit 4 during scanning by the laser scanner 1 and camera 2, the synchronization circuit 4 can also reduce the scanning speed of the laser scanner 1 and camera 2, and increase the exposure time of the camera 2. This allows sensing to be performed at the optimal frame rate and image brightness in accordance with the object.

[0175] The synchronization circuit 4 can also control the laser scanner 1 and the camera 2 to limit the shooting range of the camera 2, ensuring that the laser 12 is illuminated within the shooting range. This allows for increased brightness only in areas of low image brightness, thus shortening the sensing time.

[0176] Example 1 of scanning speed and exposure time

[0177] The scanning speed of laser scanner 1 in the second frame process is slower than that in the first frame process. In this case, the scanning speed of camera 2 in the second frame process is also slower than that in the first frame process, and the exposure time of camera 2 in the second frame process is longer than that in the first frame process. This improves image brightness.

[0178] Example 2 of scanning speed and exposure time

[0179] The scanning speed of laser scanner 1 in the second frame process is faster than that in the first frame process. In this case, the scanning speed of camera 2 in the second frame process is also faster than that in the first frame process, and the exposure time of camera 2 in the second frame process is shorter than that in the first frame process. Therefore, the frame rate can be increased when the image brightness is sufficient.

[0180] The operation of the synchronization circuit 4 will now be explained in detail. The synchronization circuit 4 ensures that the shooting time of the imaging area 22a coincides with the irradiation time of the laser 12a, both being t = ta. Furthermore, the synchronization circuit 4 ensures that the scanning speed Vc in the -Y direction of the imaging area 22 coincides with the scanning speed VL in the -Y direction of the laser 12. Thus, as... Figure 21 As shown, during one cycle from time t = ta to t = tc, the irradiated area of ​​laser 12 and the imaging area of ​​camera 2 are scanned from top to bottom with their positions always overlapping in the Y-axis direction. This cycle is repeated. The overlap range 30 between the imaging range 23 of camera 2 and the scanning range 13 of laser scanner 1 is the range in which polarimetric imaging can be achieved.

[0181] Here, it is important that the camera 2 and the laser scanner 1 are arranged in a direction parallel to the X-axis, that is, at the same position coordinates in the Y-axis and Z-axis directions. With this configuration, regardless of the distance Z in front of the virtual screen 10, it will become... Figure 21 The configuration shown is such that the irradiation area of ​​the laser 12 always overlaps with the shooting area 22 of the camera 2.

[0182] Figure 22 This diagram illustrates the control method for sensing implemented by a polarimetric imaging sensor.

[0183] exist Figure 22 In the diagram, the dashed line represents the angle function φ(t) of the viewing direction of camera 2, and the solid line represents the angle function θ(t) of the illumination direction of laser 12. The intervals for polarimetric imaging are P1(1), P2(1), P3(1), ..., where the angle function φ(t) and the angle function θ(t) are the same function. The intervals P1(2), P2(2), P3(2), ... are the steps used to return the galvanometer 7 to the scanning start position, which require a finite amount of time.

[0184] Actions in interval P1

[0185] exist Figure 22 In the interval P1(1), the angle function θ(t) of the irradiation direction of laser 12 and the angle function φ(t) of the line of sight of camera 2 are expressed by equations (8) and (9).

[0186] θ(t)=φ (t)=k1×t+φmin (8)

[0187] k1=(φmax-φmin) / T1 (9)

[0188] Here, the scanning range of camera 2 is set to -25°≤φ≤25° (φmin=-25°, φmax=25°), and the time for the camera to perform the entire line scan is T1=16ms. Thus, if the synchronization circuit 4 controls the camera so that the angle function θ(t) of the laser 12's illumination direction and the angle function φ(t) of the camera 2's line-of-sight direction are the same at the same time, then on the virtual screen 10 set at any distance Z from sensor 101, the illumination area of ​​laser 12 overlaps with the shooting area 22 of camera 2, enabling polarimetric imaging.

[0189] The topic of epipolar imaging sensors

[0190] Polar imaging also suffers from the same problem as the light curtain sensor in Embodiment 1. That is, when there are black areas in a part of the object 41, or when there are areas where the normal direction of the surface is close to a right angle with the line of sight of the camera 2, the amount of reflected and scattered light from these areas enters the pupil of the camera 2. In other words, the brightness of the image with these areas is significantly reduced.

[0191] However, polarimetric imaging sensors face the problem that even if the shutter opening time te of camera 2 is increased to improve image brightness, simply increasing the shutter opening time te alone cannot improve image brightness. This is the same problem encountered with light curtain sensors.

[0192] The graphs showing the line scanning action of camera 2, the illumination position and time of laser 12, when using the polarimeter imaging sensor, can also be used in the description of the light curtain sensor. Figure 7 Please provide an explanation. Figure 7 It can be interpreted alternatively as representing Figure 22 The diagram shows the action in interval P1(1). The meanings of parameters such as region 51, region 52, te, tc, and tp are the same as those already explained, so they are omitted here. The following is different from the light curtain sensor. In embodiment 1, region 51 and region 52 overlap to generate a light curtain only when they are at a specific distance Z relative to the image sensing device 100. On the other hand, in embodiment 2, region 51 and region 52 overlap at any distance from the image sensing device 101, enabling polarimetric imaging.

[0193] Here, in polarimetric imaging, the timing diagram for the case where the shutter opening time te is increased is also similar to that of the light curtain sensor. Figure 8 The time sequence diagram shows that regions 51 are stretched along the time direction, but the time width overlapping with region 52 remains unchanged. That is, even if the shutter opening time te is increased, the amount of light entering the camera remains unchanged, and the brightness of the image will not increase.

[0194] Worse still, if the shutter opening time te is increased, the thickness of the polar plane in the Y direction in epipolar imaging also increases. The lateral width expansion of region 51 ensures overlap even if the illumination position of laser 12 is slightly offset in the Y direction. Within this offset range ensuring overlap, scattered light entering camera 2 has a reduced effect on suppressing scattered light from outside the true polar plane. Here, this range in the Y direction is referred to as the thickness of the polar plane in the Y direction.

[0195] Solution: Figure 22 Actions in interval P2

[0196] To address the aforementioned issues, in this second embodiment, the following is performed: Figure 22 The interval P2 shows the control of the camera 2 and the laser scanner 1. In the interval P2, the angle function φ(t) of the line of sight of the camera 2 is expressed by equations (8) and (9), which have the same form as equation (1).

[0197] θ(t)=φ(t)=k2×(t-t2a)+φmin (10)

[0198] k2=(φmax-φmin) / T2 (11)

[0199] Here, the scanning range of camera 2 is: -25°≤φ≤25° (φmin=-25°, φmax=25°), and the time T2 for camera 2 to perform the entire line scan is twice T1, T2=32ms. In interval P2(1), the shutter opening time te of camera 2 is also set to twice. The timing diagram for performing such an action can be obtained from the light curtain sensor in the same way. Figure 9 As shown.

[0200] T2 becoming twice T1 is equivalent to the time tc associated with the movement of line 1 becoming twice. Since the scanning speed of laser 12 is twice as slow, the lateral width of region 51 becomes twice as wide, and the lateral width of region 51 corresponding to the shutter opening time of camera 2 also becomes twice as wide. At this time, compared to region P1(1), the overlap time between region 51 and region 52 in interval P2(1) is increased by twice. That is, even if the power of laser 12 is the same, the image brightness becomes twice as high. The fact that the thickness of the polar plane remains unchanged in intervals P1(1) and P2(1) is the same as that of the light curtain sensor.

[0201] exist Figure 22 The interval P3 shows the angle function when the scanning speed is set to 1 / 3 and the camera shutter opening time te is set to 3 times. In summary, by increasing the time of one frame of the rolling shutter camera 2, correspondingly slowing down the laser scanning speed, and increasing the camera shutter opening time te, image brightness can be increased without increasing laser power. Furthermore, the thickness of the polar plane can be made uniform at this time.

[0202] Regarding variations in polarity sensing, examples similar to those in light curtain sensors are also conceivable. It is conceivable to replace the terminology of light curtain sensing with that of polarity sensing in variations 1 to 4 of Embodiment 1, but since this is repetitive, it is omitted here.

[0203] As described above, according to this embodiment, an image sensing device 101 can be provided that can improve the brightness of the image even when photographing an object with low reflectivity.

[0204] The features in the various embodiments or variations described above can be combined with each other.

[0205] Explanation of the label

[0206] 1 Laser scanner, 2 Camera, 4 Synchronization circuit, 6 Line beam generating element, 7 Galvanometer, 10, 10a, 10b Virtual screen, 11 Illumination direction, 12 Laser, 13 Scanning range, 21 Line of sight, 22 Shooting area, 25 Light curtain, 41 Object, 100, 101 Image sensing device.

Claims

1. An image sensing device, characterized in that, have: A laser scanner that enables a laser extending in a direction parallel to the Y-axis in an orthogonal XYZ coordinate system to scan in a direction parallel to the X-axis in the same orthogonal XYZ coordinate system. A camera that enables a shooting area extending in a direction parallel to the Y-axis to be scanned in a direction parallel to the X-axis; as well as A synchronization circuit controls the laser scanner and the camera. The laser scanner and the camera are configured along the X-axis in the XYZ orthogonal coordinate system. The synchronization circuit controls the laser scanner and the camera so that the direction of the laser emitted from the laser scanner intersects the line of sight of the camera in a two-dimensional plane within three-dimensional space. The camera captures images of objects intersecting the two-dimensional surface by scanning along the -X direction of the X-axis. The synchronization circuit acquires an image of the object by synchronizing the first scan of the laser scanner and the camera in the -X direction. After the first scan, the synchronization circuit executes the first frame process, that is, it controls the laser scanner and the camera so that the laser illumination direction and the camera's line of sight are aligned with the +X direction of the X-axis. Following the first frame process, the synchronization circuit acquires an image of the object by synchronizing the second scan of the laser scanner and the camera in the -X direction. After the second scan, the synchronization circuit executes the second frame process, that is, it controls the laser scanner and the camera so that the laser illumination direction and the camera's line-of-sight direction are aligned with the +X direction. The synchronization circuit controls the laser scanner so that the scanning speed of the laser scanner in the second frame process is different from the scanning speed of the laser scanner in the first frame process. The synchronization circuit controls the camera so that the scanning speed and exposure time of the camera in the second frame process are different from those of the camera in the first frame process.

2. An image sensing device, characterized in that, have: A laser scanner that enables a laser extending in a direction parallel to the X-axis in an orthogonal XYZ coordinate system to scan in a direction parallel to the Y-axis in the same orthogonal XYZ coordinate system. A camera that enables a shooting area extending in a direction parallel to the X-axis to be scanned in a direction parallel to the Y-axis. as well as A synchronization circuit controls the laser scanner and the camera. The laser scanner and the camera are configured along the X-axis in the XYZ orthogonal coordinate system. The synchronization circuit controls the laser scanner and the camera to ensure that the direction of the laser emitted from the laser scanner and the line of sight of the camera always face the same direction in the YZ plane of three-dimensional space. The camera captures images of objects intersecting the two-dimensional surface by scanning along the -Y direction of the Y-axis. The synchronization circuit acquires an image of the object by synchronizing the first scan of the laser scanner and the camera in the -Y direction. After the first scan, the synchronization circuit executes the first frame process, that is, it controls the laser scanner and the camera so that at the same time, the direction of the laser beam and the direction of the camera's line of sight are the same in the +Y direction of the Y-axis. Following the first frame process, the synchronization circuit acquires an image of the object by synchronizing the second scan of the laser scanner and the camera in the -Y direction. After the second scan, the synchronization circuit executes the second frame process, that is, it controls the laser scanner and the camera so that the laser illumination direction and the camera's line-of-sight direction are aligned with the +Y direction. The scanning speed of the laser scanner in the second frame process is different from the scanning speed of the laser scanner in the first frame process. The scanning speed and exposure time of the camera in the second frame process are different from those of the camera in the first frame process.

3. The image sensing device according to claim 1 or 2, characterized in that, The scanning speed of the laser scanner in the second frame process is slower than the scanning speed of the laser scanner in the first frame process. The scanning speed of the camera in the second frame process is slower than the scanning speed of the camera in the first frame process. The exposure time of the camera in the second frame process is longer than the exposure time of the camera in the first frame process.

4. The image sensing device according to claim 1 or 2, characterized in that, The scanning speed of the laser scanner in the second frame process is faster than the scanning speed of the laser scanner in the first frame process. The scanning speed of the camera in the second frame process is faster than the scanning speed of the camera in the first frame process. The exposure time of the camera in the second frame process is shorter than the exposure time of the camera in the first frame process.

5. The image sensing device according to claim 1 or 2, characterized in that, If a new object is detected by the synchronization circuit during scanning by the laser scanner and the camera, the synchronization circuit reduces the scanning speed of the laser scanner and the camera, and increases the exposure time of the camera.

6. The image sensing device according to claim 1 or 2, characterized in that, The synchronization circuit controls the laser scanner and the camera to limit the shooting range of the camera so that the laser is irradiated within the shooting range.

Images