Image Sensing Device

Abstract

An image sensing device (1) comprises: an illumination device (10) including a light source that emits a light beam and an illumination optical system that scans, in a second direction (Y), a linear illumination area (13) extending linearly in a first direction (X) on a virtual reference plane (24), the linear illumination area (13) being an illumination area to which the light beam is projected, the second direction (Y) being orthogonal to the first direction (X); a camera (20) that performs an imaging operation by scanning, in the second direction (Y), a linear imaging area (23) extending linearly in the first direction (X) on the reference plane (24); and a control circuit (30) that controls the operation of the illumination device (10, 50, 60) and the imaging operation of the camera (20) so that the linear illumination area (13) and the linear imaging area (23) keep overlapping on the reference plane (24). The optical axis (11) of the illumination device (10) and the optical axis (21) of the camera (20) are non-parallel to each other, and intersect with each other on the reference plane (24).

Description

[Technical Field]

[0001] The present disclosure relates to image sensing devices. [Background technology]

[0002] There is known an image sensing device that is composed of a projector, a camera, and a synchronization circuit and acquires an image of an object by epipolar imaging (see, for example, Patent Document 1). The projector is an illumination device that horizontally and vertically scans a beam spot, which is an illumination area created by a laser beam. The camera is, for example, a rolling shutter camera, and is an imaging device that scans a capture area horizontally and vertically. The camera and projector are arranged side by side in the X direction, with the optical axis of the camera parallel to the optical axis of the projector. The synchronization circuit controls the operations of the projector and camera so that the illumination area of ​​the projector and the capture area of ​​the camera coincide with each other.

[0003] Epipolar imaging allows capturing images of objects with strong reflection and scattering (e.g., shiny metal objects) while suppressing reflection and scattering light (e.g., reflected stray light). thing This allows for three-dimensional measurements with fewer errors (see, for example, Non-Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent No. 10,359,277 [Non-patent literature]

[0005] [Non-Patent Document 1] Matthew O'Toole et al., “Homogeneous Codes for Energy-Efficient Illumination and Imaging,” ACM SIGGRAPH, 2015 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in the device described in the above document, in order to perform epipolar imaging, the camera and projector must be positioned so that the optical axis of the camera and the optical axis of the projector are parallel, which poses a problem that the sensing area, which is the overlapping area between the projector's illumination range and the camera's capture range, is narrow.

[0007] The present disclosure aims to provide an image sensing device with a wide sensing area. [Means for solving the problem]

[0008] The image sensing device of the present disclosure includes an illumination device including a light source that emits a light beam and an illumination optical system that scans a linear illumination area, which is an illumination area onto which the light beam is projected and extends linearly in a first direction on a virtual reference plane, in a second direction that is a direction perpendicular to the first direction; a camera that performs a photographing operation to scan a linear imaging area, which is an imaging area that extends linearly in the first direction on the reference plane, in the second direction; and a control circuit that controls the operation of the illumination device and the photographing operation of the camera so that the linear illumination area and the linear imaging area continue to overlap on the reference plane, wherein an optical axis of the illumination device and an optical axis of the camera are non-parallel to each other, In front of the lighting device and the camera It is characterized by intersecting. [Effects of the Invention]

[0009] According to the present disclosure, the sensing area can be widened. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a perspective view schematically illustrating a main configuration of an image sensing device according to a first embodiment. [Figure 2]FIG. 2 is a plan view schematically showing the main configuration of the image sensing device of FIG. [Figure 3] FIG. 2 is a perspective view schematically showing the main configuration of the laser scanner of FIG. [Figure 4] FIG. 2 is a plan view schematically showing the main configuration of the laser scanner of FIG. [Figure 5] FIG. 2 is a side view schematically showing the main configuration of the laser scanner of FIG. [Figure 6] 1. (A) is a diagram showing the operation of an image sensing device of a comparative example (not including a keystone distortion generating element), and (B) to (E) are diagrams showing the operation of the image sensing device of FIG. [Figure 7] 2 is a plan view showing the operation of the camera of FIG. 1. [Figure 8] 2 is a plan view showing the operation of the laser scanner of FIG. 1. FIG. [Figure 9] 2 is a plan view showing the operation of the camera and the laser scanner of FIG. 1. FIG. [Figure 10] FIG. 10 is a plan view schematically showing the main configuration of an image sensing device according to a modified example of the first embodiment. [Figure 11] FIG. 10 is a perspective view schematically illustrating the main configuration of an image sensing device of a comparative example (where the optical axis of the camera and the optical axis of the laser scanner are parallel). [Figure 12] FIG. 12 is a plan view schematically showing the main configuration of the image sensing device of FIG. [Figure 13] 12A to 12C are diagrams illustrating the operation of the image sensing device of FIG. [Figure 14] FIG. 10 is a plan view schematically showing the main configuration of an image sensing device of a comparative example (in which the optical axis of the laser scanner is tilted with respect to the optical axis of the camera). [Figure 15] 15(A) to 15(C) are diagrams illustrating the operation of the image sensing device of FIG. 14 (when the optical axis of the laser scanner is tilted). [Figure 16] FIG. 10 is a perspective view schematically showing the main configuration of a laser scanner of an image sensing device according to a second embodiment. [Figure 17]FIG. 17 is a plan view schematically showing the main configuration of the laser scanner of FIG. 16. [Figure 18] FIG. 17 is a side view schematically showing the main configuration of the laser scanner of FIG. 16. [Figure 19] FIG. 10 is a plan view schematically showing the main configuration of an image sensing device according to a second embodiment. [Figure 20] 10A and 10B are diagrams illustrating the operation of the image sensing device according to the second embodiment. [Figure 21] 10A and 10B are diagrams showing the angular function of a galvanometer mirror of an image sensing device according to a second embodiment and the angular function of a galvanometer mirror of an image sensing device according to a second embodiment. [Figure 22] 10A to 10C are diagrams showing the angular functions of a galvanometer mirror for correcting distortion on an illumination reference plane of an image sensing device according to a second embodiment. [Figure 23] 10A to 10C are diagrams showing the angular functions of a galvanometer mirror for correcting distortion on the imaging reference plane of the image sensing device according to the second embodiment. [Figure 24] FIG. 11 is a perspective view schematically showing the main configuration of a laser scanner of an image sensing device according to a third embodiment. [Figure 25] FIG. 25 is a plan view schematically showing the main configuration of the laser scanner of FIG. 24. [Figure 26] FIG. 25 is a side view schematically showing the main configuration of the laser scanner of FIG. 24. [Figure 27] 10A and 10B are diagrams illustrating distortion on an illumination reference plane in an image sensing device of a comparative example (not including a trapezoidal distortion-generating lens). [Figure 28] 10A and 10B are diagrams illustrating distortion on an imaging reference plane in an image sensing device of a comparative example (not including a lens that generates trapezoidal distortion). [Figure 29] 10 is a diagram showing the trajectories of beams on an illumination reference plane of the image sensing device according to the third embodiment. FIG. [Figure 30] 10 is a diagram showing the trajectories of beams on an imaging reference plane of an image sensing device according to a third embodiment. FIG. [Figure 31]FIG. 11 is a perspective view schematically showing the main configuration of a laser scanner including a free-form surface lens of an image sensing device according to a third embodiment. [Figure 32] FIG. 32 is a side view schematically showing the main configuration of the laser scanner of FIG. 31. [Figure 33] FIG. 32 is a plan view schematically showing the main configuration of the laser scanner of FIG. 31. [Figure 34] 10 is a diagram illustrating cross-sectional profiles of a first surface and a second surface of a free-form surface lens of an image sensing device according to a third embodiment. FIG. [Figure 35] 10 is a diagram illustrating cross-sectional profiles of a first surface and a second surface of a free-form surface lens of an image sensing device according to a third embodiment. FIG. [Figure 36] 10 is a diagram showing the trajectory of a beam on an imaging reference plane obtained by controlling the slow axis of a two-dimensional MEMS mirror in an image sensing device according to a third embodiment. FIG. [Figure 37] FIG. 10 is a diagram showing a stripe pattern when a laser beam is turned on and off at equal time intervals. [Figure 38] FIG. 10 shows a vertical stripe pattern created by controlling the on-off time of a laser beam. [Figure 39] FIG. 10 is a plan view schematically showing the main configuration of an image sensing device according to a fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Image sensing devices according to embodiments will be described below with reference to the drawings. The following embodiments are merely examples, and the embodiments can be appropriately combined and modified. In the drawings, components having the same or similar functions are designated by the same reference numerals.

[0012] 1. First embodiment 1-1 Configuration 1 and 2 are a perspective view and a plan view schematically illustrating the main components of an image sensing device 1 according to a first embodiment. The image sensing device 1 is a device that performs epipolar imaging. The image sensing device 1 includes a camera 20 that is an imaging device, a laser scanner 10 that is an illumination device, a control circuit 30 that includes a synchronization circuit, and a keystone distortion generating element 40. The camera 20 and the laser scanner 10 are arranged in parallel along the X direction, and an optical axis 21 of the camera 20 and an optical axis 11 of the laser scanner 10 are non-parallel to each other and intersect in front of the camera 20 and the laser scanner 10. In the first embodiment, the keystone distortion generating element 40 is inserted in front of the laser scanner 10.

[0013] An imaging reference plane 24 (also referred to as an "imaging screen"), which is a planar virtual screen perpendicular to the optical axis 21 of the camera 20, is installed at a position a certain distance Z0 away from the camera 20. In addition, an illumination reference plane 14 (also referred to as an "illumination screen"), which is a planar virtual screen perpendicular to the optical axis of the laser scanner 10 and is a laser projection reference plane inclined at an angle θ with respect to the imaging reference plane 24, is also installed. The imaging reference plane 24 and the illumination reference plane 14 do not physically exist but represent virtual planes for explanation purposes. The optical axis 21 of the camera 20 and the optical axis 11 of the laser scanner 10 intersect on the illumination reference plane 14, which is a virtual reference plane.

[0014] 3 to 5 are a perspective view, a plan view, and a side view schematically showing the configuration of the laser scanner 10 in Fig. 1. A laser beam (also called an "expanded beam") that spreads out like a fan in the X direction, which is a first direction, is emitted from the laser scanner 10. On the illumination reference plane 14, a linear illumination area 13 is formed, which is the expanded laser beam extending in the X direction (i.e., a linear beam with a linear cross section).

[0015] The laser scanner 10 includes a laser light source 110 as a light source that emits a laser beam as a light beam, and an illumination optical system that scans a linear illumination area 13, which is an illumination area onto which the laser beam is projected and extends linearly in the X direction on an illumination reference plane 14, which is a virtual reference plane, in the Y direction, which is a second direction perpendicular to the X direction. The laser beam is emitted from the laser light source 110, reflected by a mirror 111, and then expanded by a beam expansion optical element 112, which is a lens, to form the linear illumination area 13, which is a linear beam that expands in the X direction. The beam expansion optical element 112 is an optical lens such as a cylindrical lens or a Powell lens. As shown in FIG. 5 , the linear illumination area 13 is deflected in the Z direction by a galvanometer mirror 113, which is a scanning optical unit. The galvanometer mirror 113 can swing around the X axis within a predetermined angular range of ±(α / 2), and the linear illumination area 13 is scanned around the X axis within twice the angular range ±α (i.e., scanned within the range of linear illumination areas 13a to 13c in FIG. 5). On the imaging reference plane 24, the linear illumination area 13 is scanned in the Y direction, and the entire laser scan range 12 in FIG. 1 is irradiated.

[0016] The camera 20 performs an imaging operation of scanning in the Y direction a linear imaging area 23, which is an imaging area extending linearly in the X direction on the imaging reference surface 24. The control circuit 30 controls the operation of the laser scanner 10 and the imaging operation of the camera 20 so that the linear illumination area 13 and the linear imaging area 23 continue to overlap on the imaging reference surface 24. The control circuit 30 may be composed of a memory that stores a software program and a processor. In this case, the functions of the control circuit 30 are realized by the processor that executes the software program stored in the memory.

[0017] Fig. 6(A) is a diagram showing the operation of an image sensing device of a comparative example (not including a keystone distortion generating element). Fig. 6(A) shows a linear illumination area 13 on an illumination reference plane 14 of the image sensing device of the comparative example.

[0018] Figures 6(B) to 6(E) are diagrams showing the operation of the image sensing device 1 according to embodiment 1. Figure 6(B) shows the linear illumination area 13 on the illumination reference plane 14, Figure 6(C) shows the linear illumination area 13 on the imaging reference plane 24 in embodiment 1, Figure 6(D) shows the linear imaging area 23 on the imaging reference plane 24, and Figure 6(E) shows the linear illumination area 13 and the linear imaging area 23 on the imaging reference plane 24.

[0019] 6(A) shows how, in the image sensing device of the comparative example, a linear illumination area 13 repeatedly scans from the top to the bottom of the entire laser scan range 12 in the -Y direction at a speed VL on an illumination reference plane 14 when there is no keystone distortion generating element 40. At time t = ta, the linear illumination area 13 exists at the top end of the entire laser scan range 12 as linear illumination area 13a, at time t = tb it is shown as linear illumination area 13b, and at time t = tc it exists at the bottom end of the entire laser scan range 12 as linear illumination area 13c. When it reaches the bottom end, it returns to the top end at high speed and repeats the above operation.

[0020] In the first embodiment, due to the presence of the trapezoidal distortion generating element 40, the entire laser scan range 12 on the illumination reference plane 14 becomes a trapezoid, as shown in FIG. 6B. That is, at time t=ta, the linear illumination area 13a is a straight line rising upward to the right, and as scanning in the -Y direction progresses, it rotates in the XY plane, and the linear illumination area 13c at the bottom becomes a straight line falling downward to the right. The imaging reference plane 24, which is perpendicular to the optical axis 21, is inclined (inclined toward the X direction) with respect to the plane perpendicular to the optical axis 11. The trapezoidal distortion generating element 40 has the function of bringing the extension direction of the linear illumination area 13 on the illumination reference plane 14 closer to the extension direction of the linear imaging area 23. As shown in FIG. 6C, the linear illumination area 13 on the imaging reference plane 24 is scanned as a linear illumination area 13 parallel to the X direction from t=ta to t=tc. The trapezoidal distortion generating element 40 is designed to realize the operation shown in FIG. 6(C), and the specific design of the trapezoidal distortion generating element 40 will be described later.

[0021] The camera 20 is a rolling shutter camera, and by shortening the exposure time, it can repeatedly scan a linear imaging area extending in the X direction in the -Y direction. Scanning of the imaging area of ​​the camera 20 is described, for example, in FIG. 13 and its accompanying text in Non-Patent Document 1. On the imaging reference plane 24, the imaging range 12 of the camera 20 is scanned from the top to the bottom of the entire imaging range 22 of the camera 20. This is shown in FIG. 6(D). Scanning is performed at a speed Vc from a linear imaging area 23a at the top of the entire imaging range 22 to a linear imaging area 23c at the bottom. The device configuration is set so that the linear imaging area 23a of the camera 20 and the linear illumination area 13a overlap in the Y direction on the imaging reference plane 24. This setting can be achieved by zooming the lens of the camera 20, setting a Region of Interest (ROI) that limits the imaging area of ​​the camera 20, and setting the scan range of the linear illumination area 13 in the Y direction. The mechanism for finely adjusting the installation attitude of the camera 20 and the laser scanner 10 is also important.

[0022] The control circuit 30 matches the imaging time of the linear imaging region 23a and the irradiation time of the linear illumination region 13a at t=ta. Furthermore, the scanning speed Vc of the linear imaging region 23 in the -Y direction and the scanning speed VL of the linear illumination region 13 in the -Y direction are matched. Then, as shown in FIG. 6(E), scanning is performed from top to bottom during one cycle from time t=ta to t=tc, with the Y-direction positions always overlapping (preferably always overlapping). This cycle is repeated. The overlapping range of the entire imaging range 22 and the entire laser scanning range 12 of the linear illumination region 13 is the range in which epipolar imaging is possible.

[0023] Here, it is important that the camera 20 and the laser scanner 10 are aligned in the X direction, i.e., that they are located at the same position coordinates in the Y and Z directions (Condition A). With this arrangement, the linear illumination area 13 and the linear imaging area 23 continue to overlap, as shown in FIG. 6(E), regardless of the distance Z in front of the imaging reference plane 24. The reason for this will be explained using FIGS. 7 to 9. FIGS. 7 to 9 are diagrams illustrating the linear imaging area of ​​the image sensing device 1 according to the first embodiment and the operation in the cross-sectional direction (in the YZ plane) of the line laser beam, which is the light beam that forms the linear illumination area 13. That is, FIG. 7 is a plan view showing the operation of the camera in FIG. 1, FIG. 8 is a plan view showing the operation of the laser scanner in FIG. 1, and FIG. 9 is a plan view showing the operation of the camera and laser scanner in FIG. 1.

[0024] FIG. 7 is a diagram showing the range scanned by the linear imaging region 23 from the camera 20 projected onto the YZ plane, and FIG. 8 is a diagram showing the range scanned by the linear illumination region 13 from the laser scanner 10 projected onto the YZ plane. Only when the above-mentioned condition A is satisfied, the scanning ranges in FIGS. 7 and 8 coincide. Furthermore, the trajectories of the linear imaging region 23b and the linear illumination region 13b in the YZ plane at any time t = tb coincide. This is shown in FIG. 9. Therefore, as shown by the hatched area in FIG. 9, the sensing region 25 is wide, enabling epipolar imaging at any position in the Z direction. However, as is clear from FIG. 2, in the range where Z is small (the range very close to the camera), the imaging range of the camera 20 and the scanning range of the linear illumination region 13 do not overlap in the XZ plane, making epipolar imaging impossible. 2 and 9, the far region of the sensing area 25 is separated by the imaging reference plane 24, but in reality, the sensing area 25 extends farther. The actual limit of the far sensing area 25 is determined by the amount of signal that can be detected, because the amount of light received by the camera 20 decreases as the distance increases.

[0025] Specifically, a wedge-shaped prism can be used as the keystone distortion generating element 40. Patent Document 2 shows an example in which a wedge-shaped prism is inserted into the light exit surface of a projector that projects obliquely upward, thereby correcting keystone distortion of a projection pattern on a vertical screen.

[0026] [Patent Document 2] Japanese Patent Application Laid-Open No. 2016-105179

[0027] FIG. 10 is a diagram showing an example of the configuration of an image sensing device according to the first embodiment. By inserting a wedge-shaped prism in front of the laser scanner 10, trapezoidal distortion is generated in the horizontal direction. In FIG. 10, the apex of the wedge-shaped prism 41 is on the right side, and the optical axis 11 is deflected to the left after passing through the wedge-shaped prism 41. Here, the illumination reference surface 14 is perpendicular to the optical axis 11 after exiting the wedge-shaped prism 41. Also, the angle between the normal to the imaging reference surface 24 and the optical axis 11 is θ. The horizontal trapezoidal distortion on the imaging reference surface 24 disappears, and the linear illumination areas 13a to 13b on the imaging reference surface 24 are aligned as shown in FIG. c The shape, material, and installation angle of the wedge prism 41 and the installation angle of the laser scanner 10 may be designed so that all of the beams are parallel to the X axis.

[0028] 1-3 Comparative Example 11 and 12 are perspective and plan views showing the main components of a comparative image sensing device 1a that performs epipolar imaging. The comparative image sensing device 1a comprises a camera 20, a laser scanner 10, and a control circuit 30. The camera 20 and the laser scanner 10 are arranged side by side along the X direction, and the optical axis 21 of the camera 20 and the optical axis 11 of the laser scanner 10 are parallel to each other and point in the Z direction. Assume that an imaging reference plane 24, which is a virtual screen perpendicular to the optical axis 21 of the camera 20, is located at a distance Z0 from the camera 20.

[0029] 13A to 13C are diagrams illustrating the operation of an image sensing device 1a that performs epipolar imaging as a comparative example. Fig. 13A shows how a linear imaging area 23 is scanned by a rolling shutter camera on an imaging reference plane 24. This operation is similar to the operation described in Fig. 6D.

[0030] FIG. 13(B) shows the operation of the linear illumination area 13, and FIG. 13(C) shows the state in which the linear imaging area 23 and the linear illumination area 13 are superimposed.

[0031] Fig. 13(B) shows how the laser scanner 10 scans the linear illumination area 13 on the imaging reference plane 24. In the comparative example configurations of Fig. 11 and Fig. 12, the imaging reference plane 24 is perpendicular to the laser scanner 10, so linear illumination areas 13a to 13c parallel to the X direction are scanned, similar to what has been described with reference to Fig. 6(A).

[0032] As described above, the control circuit 30 is used to synchronize the camera 20 and the laser scanner 10, operating them so that their Y-direction positions on the imaging reference plane 24 continue to overlap (preferably, always overlap). Figure 13(C) shows the overlap between the linear illumination area 13 and the linear imaging area 23 on the imaging reference plane 24. The overlapping area between the two on the imaging reference plane 24 is smaller than that shown in Figure 6(E). Figure 12 shows the overlapping sensing area 25, which is indicated by hatching. Comparing the sensing area 25 in Figure 2 with the sensing area 25 in Figure 12, it is clear that the image sensing device 101 of the comparative example has a problem in that the sensing area 25 capable of epipolar imaging is small. To enlarge the sensing area 25, the distance between the camera 20 and the laser scanner 10 can be reduced while maintaining the parallelism of the optical axis 21 and the optical axis 11. However, this is limited by the size of the device. Furthermore, as will be described in the explanation of configuration example 2 of embodiment 2 and embodiment 3, one major application of epipolar imaging is 3D sensing using stripe pattern projection. Because the stripe pattern projection method uses the principle of triangulation, it is better to increase the distance in the X direction between the laser scanner 10 and the camera 20 to improve the measurement accuracy in the depth direction. However, in conventional epipolar imaging, the optical axis 21 and the optical axis 11 remain parallel, which poses a problem in that the sensing area 25, which is the area where 3D sensing is possible, becomes small.

[0033] 14 and 15(A) to 15(C) are diagrams illustrating the operation of a comparative image sensing device that performs epipolar imaging when the optical axis of the laser scanner 10 is tilted. FIG. 14 is a diagram showing the main configuration, and FIG. 15(A) shows the operation of the linear illumination area 13 on the illumination reference plane 14, and FIG. 15(B) shows the operation of the linear illumination area 13 on the illumination reference plane 14. teeth, Movement of the linear illumination area 13 on the imaging reference plane 24, FIG. 15(C) teeth, 1 shows the operation of overlapping the linear imaging area 23 and the linear illumination area 13 on the imaging reference plane 24.

[0034] Furthermore, if the optical axis 11 of the laser scanner 10 is tilted in the X direction as shown in FIG. 1 to widen the width of the sensing area 25 in the X direction, the optical axis 21 and the optical axis 11 become non-parallel. Then, the imaging reference surface 24, which is perpendicular to the optical axis 21 of the camera 20, is tilted with respect to the optical axis 11. Therefore, the linear illumination area 13 on the imaging reference surface 24 has a right-shoulder-sloping straight line at the top end 13a as shown in FIG. 15(B). As the scan progresses, the linear illumination area 13 rotates in the XY plane, and the bottom end 13c becomes a right-shoulder-sloping straight line. Meanwhile, the linear imaging area on the imaging reference surface 24 is scanned from top to bottom while remaining parallel, as shown in FIG. 13(A). FIG. 15(C) shows how the linear imaging area 23 and the linear illumination area 13 overlap on the imaging reference surface 24 when the control circuit 30 adjusts the Y-direction positions of the linear imaging area 23 and the linear illumination area 13 to coincide on the imaging reference surface 24. 15(C), the inclination of the linear illumination area 13 rotates on the imaging reference plane 24, so scanning cannot be performed while the linear imaging area 23 and the linear illumination area 13 remain overlapped. Although the linear imaging area 23 and the linear illumination area 13 become parallel for just a moment at the midpoint in the Y direction, the sensing area in the Y direction becomes extremely narrow, making it difficult to use as an epipolar imaging sensor.

[0035] Effect 1-4 In the image sensing device 1 according to the first embodiment, by inserting an appropriately designed keystone distortion generating element 40 in front of the laser scanner 10, it becomes possible to scan a line laser beam parallel to the X direction in the Y direction on the imaging reference plane 24 tilted with respect to the optical axis 21. Therefore, epipolar imaging can be performed even if the optical axis 11 is tilted with respect to the optical axis 21, thereby achieving the effect of expanding the sensing area 25.

[0036] In the above explanation, an example was given in which the galvanometer mirror 113 was used as the beam scanning device, but the same effect can be obtained by using a one-dimensional MEMS (Micro Electro Mechanical Systems) mirror that has the function of rotating and oscillating the mirror at high speed. Alternatively, instead of the galvanometer mirror 113, a scanner that rotates a polygon mirror, which is a multifaceted mirror, using a motor may be used.

[0037] 2. Second Embodiment 2-1 Configuration FIG. 16 is a perspective view schematically illustrating the main components of a laser scanner 50 serving as an illumination device of an image sensing device 2 according to a second embodiment. FIGS. 17 and 18 are a plan view and a side view, respectively, schematically illustrating the main components of the laser scanner 50 of FIG. 16. FIG. 19 is a plan view schematically illustrating the main components of an image sensing device 2 according to the second embodiment. Here, the XYZ coordinate axes in FIGS. 16 to 18 are local coordinates of the laser scanner 50, and the Z axis in FIGS. 16 to 18 is aligned with the optical axis 11. In other words, the Z direction in FIG. 19 is different from the Z direction in FIGS. 16 to 18. The second embodiment differs from the first embodiment in that the laser scanner 50 is composed of two galvanometer mirrors 511 and 512. A laser beam 90 emitted from a laser light source 510 travels in the Z-axis direction and is reflected by the galvanometer mirror 511, which serves as a first scanning optical unit. Galvano mirror 511 can change the rotation angle θy of the mirror around its rotation axis at high speed within a range of ±10°. The rotation axis is tilted at an angle θ1 with respect to the Y axis as shown in FIG.

[0038] The laser beam 90 reflected by the galvanometer mirror 511 reaches the galvanometer mirror 512, which serves as a second scanning optical unit. Because the galvanometer mirror 511 reciprocates at high speed, the laser beam 90 that reaches the galvanometer mirror 512 traces an upwardly convex curved trajectory, as indicated by reference numeral 93 in FIG. 16 . The reason the trajectory is curved rather than linear is because light incident on the galvanometer mirror 511 from diagonally above in the Y direction is scanned in the X direction. The galvanometer mirror 512 can change the rotation angle θx around its rotation axis within a range of ±6°, and the rotation axis faces the X-axis direction. The orientation of the galvanometer mirror 512 when θx=0 is determined so that the emission direction of the laser beam 90 when θy=0 and θx=0, i.e., the optical axis 11, faces the Z-axis. When the galvanometer mirror 512 is at -6°, the laser beam becomes the laser beam 90a in FIG. 18, and when the galvanometer mirror 512 is at +6°, the laser beam becomes the laser beam 90c in FIG.

[0039] 《2-2》Operation 20(A), (B) and 21(A), (B) are diagrams illustrating distortion on the screen of the image sensing device 2 according to embodiment 2. Fig. 20(A) shows the trajectory of the laser beam on the illumination reference plane 14, and Fig. 20(B) shows the trajectory of the laser beam on the imaging reference plane 24. Fig. 21(A) shows the angular function θx(t) of the galvanometer mirror 512, and Fig. 21(B) shows the angular function θy(t) of the galvanometer mirror 511.

[0040] When galvanometer mirror 511 performs high-speed scanning back and forth at a constant speed and galvanometer mirror 512 repeats scanning at a constant speed in the direction from +6° to −6°, laser beam 90 traces a trajectory indicated by the upwardly convex curved arrow in FIG. 20(A) on illumination reference plane 14 in FIG. 19. On the outward path, it moves in the direction from −X to +X as indicated by the solid arrow, and on the return path, it moves in the opposite direction as indicated by the dotted arrow. Because galvanometer mirror 512 scans more slowly than galvanometer mirror 511, linear illumination area 13, which is the trajectory of the laser beam in the direction from +Y to −Y, moves across the entire screen. If this one-way trajectory of the laser beam is captured for a time longer than the time required for one-way movement (i.e., if the exposure time of camera 20 is made sufficiently long), it can be considered that a curved laser beam is being irradiated. The dots in FIG. 20(A) represent the points where the laser beam reaches on the illumination reference plane 14 when the rotation angles θx and θy of the galvanometer mirrors 511 and 512 are changed discretely in 1° increments.

[0041] The trajectory of the laser beam on the imaging reference surface 24, which is perpendicular to the optical axis 21 but oblique to the optical axis 11, is shown by the solid or dotted arrow in FIG. 20(B). As in FIG. 20(A), the dots represent the arrival points of the laser beam on the imaging reference surface 24 when the rotation angles θx and θy of the galvanometer mirrors 511 and 512 are discretely changed in 1° increments. On the imaging reference surface 24, the linear illumination area 13, which is a trajectory scanned at high speed in a substantially horizontal direction, is not only convex in the Y direction, but also rotates in its entirety as shown by the linear illumination areas 13a to 13c between times t = ta and t = tc. Since this distorted linear illumination area 13 cannot be superimposed on the linear imaging area 23 of the camera 20, epipolar imaging is not possible.

[0042] Here, FIGS. 21(A) and 21(B) show simplified diagrams of the angular functions of the galvanometer mirrors 511 and 512 that generate the linear illumination region 13 within one frame. In FIGS. 21(A) and 21(B), the step angle of the rotation angle θx of the galvanometer mirror 512 is shown in 2° increments, and the rotation angle θy of the galvanometer mirror 511 changes only in the positive direction from −10° to +10°. In FIG. 21(A), the galvanometer mirror 511 maintains a constant angle during one scan (time Tx). However, in actual operation where the θx steps are smaller, it is acceptable for the angle to change slowly at a constant angular velocity as it moves from −6° to +6°. This is because the angle θx can be considered constant during the short time Tx. Also, in FIG. 21(B), the angle θy changes at a constant velocity during the time Tx.

[0043] The distortion is corrected by adding corrections to the galvanometer mirror angle functions shown in Figures 21(A) and 21(B). That is, a correction function is created that changes the pattern of the laser beam arrival positions at intervals of 1 degree, shown by dots in Figures 20(A) and (B), to a square lattice pattern.

[0044] 22(A) to (C) and 23(A) to (C) are diagrams explaining distortion on the screen of the image sensing device 2 according to embodiment 2. Figs. 22(A) and (B) show the angular function of the galvanometer mirror for correcting distortion on the illumination reference plane 14, and Fig. 22(C) shows the trajectory of the laser beam on the illumination reference plane 14. Figs. 23(A) and (B) show the angular function of the galvanometer mirror for correcting distortion on the imaging reference plane 24, and Fig. 23(C) shows the trajectory of the laser beam on the imaging reference plane 24.

[0045] First, we create angular functions θy(t) and θx(t) that eliminate distortion on the illumination reference plane 14. To find these functions, we need to find a pair of angles θy and θx of two galvanometer mirrors that reach the grid points on the illumination reference plane 14 shown in FIG. 22(C). Since the position coordinates (X, Y) on the illumination reference plane 14 have a one-to-one mapping relationship with the two angles θy and θx, we can numerically find the θy and θx that reach a given (X, Y) coordinate. For example, we create the θy and θx functions shown in FIGS. 22(A) and 22(B). For simplicity, we use a function in which the galvanometer mirror 511 moves from -10° to +10° at a constant angular velocity and then instantly returns to -10°, and θx changes in steps of approximately 2 degrees from approximately -6° to approximately +6°. Each step of the θx function is a downward-convex curve. At this time, the linear illumination area 13a in FIG. 22(C) traces a trajectory of a line segment parallel to the X-axis, as indicated by the rightward arrow. In actual operation, when the galvanometer mirror 511 returns from +10° to -10°, the galvanometer mirror 512 is similarly controlled to scan the laser beam. The angle may also be changed in even finer θx steps. The trajectory of the laser beam on this return path is represented by a leftward dotted arrow in FIG. 22(C). As already explained, when the pair of θy and θx is shifted in 1-degree increments on the illumination reference plane 14, the laser beam arrival position is distorted as shown in FIG. 20(A). However, by using the functions in FIGS. 22(A) and (B), the linear illumination area 13a, which is the trajectory in FIG. 22(C), can be corrected to a straight line. Similarly, a row of pairs of θy and θx can be created that raster scans the dots arranged in a square lattice pattern in FIG. 22(C) from the upper left to the right.

[0046] However, with this alone, on the imaging reference plane 24 that is inclined with respect to the optical axis 11, the trajectory is no longer parallel to the X-axis, as shown by the linear illumination areas 13a and 13c in Fig. 15(B), and epipolar imaging cannot be properly achieved. However, in this case, since the horizontal, straight-line linear illumination area 13 is generated on the illumination reference plane 14 by the functions of Fig. 22(A) and (B), by installing the keystone distortion generating element 40 in front of the laser scanner 50, as described in the first embodiment, it is possible to generate the horizontal, straight-line linear illumination area 13 on the imaging reference plane 24, and epipolar imaging becomes possible.

[0047] However, by appropriately controlling the two galvanometer mirrors 511 and 512, it is possible to generate a linear illumination area 13 extending horizontally on the imaging reference surface 24 without using the keystone distortion generating element 40. As shown in FIG. 23(C), to obtain a linear illumination area 13a by horizontal laser beam scanning on the imaging reference surface 24, the angular functions shown in FIGS. 23(A) and 23(B) can be set. FIG. 23(A) is a graph in which each of the seven downwardly convex curves shown in FIG. 22(A) is rotated differently. Thus, to correct the keystone distortion occurring on the imaging reference surface 24 tilted with respect to the optical axis 11, it is necessary to generate an inverse keystone distortion on the illumination reference surface 14 perpendicular to the optical axis 11. To achieve this, θx must be controlled within the time it takes to draw one linear illumination area 13, and the control function must be changed gradually for each line. Furthermore, in the dot pattern with 1-degree increments shown in FIG. 20(B), the dot spacing in the X direction narrows as the X value increases. To correct the spacing in the X direction, the group of seven straight line segments with an upward slope in Figure 22(B) is changed to a group of curves with a slight downward convexity in Figure 23(B). From the functions in Figures 23(A) and (B), a linear illumination area 13 that is horizontal and straight in the X direction, as shown in Figure 23(C), is generated. In other words, epipolar imaging is possible by operating two galvanometer mirrors 511 and 512 using the functions shown in Figures 23(A) and (B).

[0048] 2-3 Effects In the second embodiment, when epipolar imaging is performed by raster scanning a laser beam using two galvanometer mirrors 511 and 512, sensing that is not possible with epipolar imaging that scans a line laser beam vertically can be performed.

[0049] Furthermore, a vertical stripe pattern can be created by repeatedly turning the laser on and off at high speed. For example, as shown in Figure 23(C), the laser beam is turned on and off 100 times while moving from left to right or right to left once, and the laser is synchronized so that the on and off is performed at the same position in the X direction between each line. This results in 100 vertical stripes extending vertically. Using these vertical stripes for three-dimensional (3D) measurement enables error-free sensing of metal objects. When 3D measurement of a metal object is performed using a stripe pattern projection method other than epipolar imaging, the stripe pattern reflected by the shiny metallic surface becomes a false pattern, resulting in erroneous detection. However, epipolar imaging prevents the reflected false stripe pattern from being captured by the camera 20, enabling 3D measurement without erroneous detection.

[0050] 3. Third Embodiment 3-1. Structure 24 to 26 are perspective, plan, and side views illustrating a schematic configuration of a laser scanner 60 serving as an illumination device of an image sensing device 3 according to a third embodiment. FIGS. 24 to 26 illustrate a laser scanner 60 using a two-dimensional MEMS mirror 620. Similar to a configuration using two galvanometer mirrors, the two-dimensional MEMS mirror 620 can perform raster scanning by deflecting a laser beam in two axial directions. Therefore, it can be used as a device for generating a linear illumination area 13 for epipolar imaging. A laser scanner 60 using a two-dimensional MEMS mirror 620 has the advantage of being more compact and inexpensive than a galvanometer mirror. In FIGS. 24 to 26, a laser beam 90 emitted from a laser light source 610 travels in the Z-axis direction and is reflected by a mirror 611 whose normal is inclined at an angle θ1 with respect to the −Z axis. The reflected light is further reflected by a mirror portion 621 of the two-dimensional MEMS mirror 620.

[0051] 24, two-dimensional MEMS mirror 620 is composed of mirror portion 621 that reflects laser beam 90, hinge 622 that rotates mirror portion 621 by angle θy around the Y axis in the figure, and hinge 623 that rotates mirror portion 621 by angle θx around the X axis. The direction of laser beam 90 when θy = θx = 0 is set to be optical axis 11 of laser scanner 60, and two-dimensional MEMS mirror 620 is installed with its normal tilted at an angle of θ1 with the Z axis so that optical axis 11 is parallel to the +Z axis.

[0052] The two-dimensional MEMS mirror 620 used for raster scanning generally consists of a fast axis, which allows high-speed scanning but does not allow for high-precision angle control, and a slow axis, which allows for low-speed scanning but high-precision angle control. High-precision control of the movement around the fast axis is difficult because the scanning frequency is adjusted to the physical resonance frequency. The scan angle θy around the fast axis cannot be controlled by an arbitrary function and instead repeats reciprocating motion at a constant speed. In Figures 24 to 26, the rotational movement using the hinge 622 corresponds to the rotational scan around the fast axis, and the rotational movement using the hinge 623 corresponds to the rotational scan around the slow axis. The movement around θy scans the laser beam at high speed in the X direction, generating a linear illumination area 13. However, because the laser beam 90 is obliquely incident on the mirror portion 621 in the YZ plane, the linear illumination area 13 obtained by rotating the mirror portion 621 around the fast axis forms an arc in the Y-axis direction.

[0053] 3-2 Configuration example 1 <Suppressing distortion on an oblique screen using a free-form lens> FIG. 27 is a diagram illustrating distortion on the illumination reference plane 14 of an image sensing device of a comparative example (not including a trapezoidal distortion generating lens). FIG. 28 is a diagram illustrating distortion on the imaging reference plane 24 of an image sensing device of a comparative example (not including a trapezoidal distortion generating lens). FIG. 29 is a diagram illustrating the trajectory of a beam on the illumination reference plane of an image sensing device 3 according to embodiment 3 (whole configuration not shown). FIG. 30 is a diagram illustrating the trajectory of a beam on the imaging reference plane of an image sensing device 3 according to embodiment 3. A laser scanner 60 of an image sensing device according to embodiment 3 is shown in FIGS. 24 to 26.

[0054] FIG. 27 shows the linear illumination region 13, which is the trajectory of the laser beam 90 on the illumination reference plane 14 perpendicular to the optical axis 11, when there is no trapezoidal distortion generating element 40. The trajectory on the illumination reference plane 14 when two galvanometer mirrors are used is similar to that shown in FIG. 20(A). However, in FIG. 27, the curvature of the arc varies depending on the position in the Y direction, with the radius of curvature decreasing as one moves downward. That is, the curvature of the arc of the linear illumination region 13c is greater than that of the linear illumination region 13a. The amplitude of the deflection in the X direction also decreases as one moves downward. This is because, in the two-dimensional MEMS mirror 620, the hinge 622 is located inside the hinge 623, and therefore the angle of incidence on the mirror portion 621 in the YZ plane changes to (θ1 + θx) in accordance with the rotation angle of θx caused by the hinge 623. 27 to 30 represent the arrival points of the laser beam 90 on the illumination reference plane 14 when θx and θy are changed in 1-degree increments, similar to the dots in Fig. 20(A). For example, the dot near the linear illumination area 13a, which is the upper end of the linear illumination area 13, represents the arrival point of the laser beam when θy is swung from -5° to +5° while θx is fixed at -3°.

[0055] 20(B), when there is no keystone distortion generating element 40, Fig. 28 shows the laser beam arrival point and linear illumination area 13 on the imaging reference surface 24, which is perpendicular to the optical axis 21 but oblique to the optical axis 11. In Fig. 28, in addition to the arc-shaped distortion shown in Fig. 27, keystone distortion caused by the oblique imaging reference surface 24 is superimposed. Note that the keystone distortion is manifested in the fact that the spacing between dots in the Y direction becomes narrower as one moves in the -X direction.

[0056] In the case of a two-dimensional MEMS mirror, not only is there keystone distortion, but there is also a complex distortion generated by the two-dimensional MEMS mirror itself, so it is difficult to properly correct the distortion on the oblique imaging reference plane 24 with an element of a simple shape such as a wedge-shaped prism, which is an example of the keystone distortion generating element 40 in embodiment 1. Therefore, it is desirable to use a free-form surface lens as the keystone distortion generating element. The following equation (1) is an example of a function shape that represents a free-form surface.

[0057]

number

[0058] Here, Z(x,y) is the displacement of the curved surface at the coordinates (x,y) and represents an Nth-order polynomial of two variables x and y. The variables are a, a normalization parameter, and x i y j Coefficient k of i,j Here, the free-form surface shape was optimized with degree N=6. The positions of the laser beam reaching the illumination reference surface 14 and the imaging reference surface 24 for each degree after optimization, and the linear illumination area 13 are shown in Figures 29 and 30, respectively.

[0059] 31 to 33 are perspective views, side views, and plan views that schematically show the configuration of a laser scanner 60 including a free-form surface lens of the image sensing device 3 according to embodiment 3. Figs. 31 to 33 are views in which a free-form surface lens 70 is inserted after the two-dimensional MEMS mirror 620 in Figs. 24 to 26. The free-form surface lens 70 has a first surface 71 and a second surface 72 that are both free-form surfaces expressed by equation (1).

[0060] 34 and 35 are diagrams showing cross-sectional profiles of the first surface 71 and the second surface 72 of the free-form lens 70 of the image sensing device 3 according to the third embodiment. These diagrams show cross-sectional profiles of the first surface 71 and the second surface 72, respectively, along a plane passing through the optical axis. The solid line represents the sag [mm] in the X direction, and the dashed line represents the sag in the Y direction. The sag refers to the amount of sag in the direction parallel to the optical axis of the lens. Both Figs. 34 and 35 show that the curvature is reversed in the X and Y directions, resulting in a saddle-shaped curve. Furthermore, these graphs are asymmetric in both the X and Y directions. This indicates that correcting the asymmetric distortion shown in Fig. 35 as shown in Fig. 30 requires the use of a free-form lens that is asymmetric in both the X and Y directions. Furthermore, as shown in Fig. 32, the free-form lens 70 is rotated clockwise by an angle φ in the YZ plane. When θy = θx = 0, the incident light beam and the outgoing light beam (i.e., optical axis 11) of laser beam 90 to free-form surface lens 70 are both in the Z-axis direction. The angle between this incident light beam and the normal to the optical axis of first surface 71 is φ = 30°. According to simulations conducted by the inventors of the present application, it was possible to design a free-form surface shape that could cancel out the difference in curvature of linear illumination area 13 on the screen (i.e., illumination reference surface 14 and imaging reference surface 24) shown in FIG. 27 or 28 when φ is within the range of 15° to 45°.

[0061] As described above, by inserting an optimally designed free-form lens 70 that is asymmetric in both the X and Y directions after the two-dimensional MEMS 620, it is possible to generate a linear illumination area 13 parallel to the X direction even on the imaging reference plane 24 that is perpendicular to the optical axis 21 of the camera 20 as shown in FIG. 30, thereby enabling epipolar imaging.

[0062] 3-3 Configuration example 2 <Suppressing distortion by controlling the angle of the slow axis> In Configuration Example 1, the free-form surface lens 70 generates a linear illumination region 13 on the imaging reference plane 24, which is disposed obliquely with respect to the optical axis 11. However, as described in the second embodiment for the case of two galvanometer mirrors, it is also possible to suppress distortion by controlling the rotation angles θx and θy of the mirror portion 621. The control method and angle functions in this case are the same as those described with reference to FIGS. 22 and 23. If the scan angle θx around the slow axis (i.e., around the hinge 623) and the scan angle θy around the fast axis (i.e., around the hinge 622) can be controlled using functions such as those shown in FIGS. 23(A) and 23(B), that is, if the values ​​of θx and θy can be appropriately controlled during the short time required to trace the locus of one linear illumination region 13, the linear illumination region 13 formed by the line beam can be converted into a straight line parallel to the X direction, enabling epipolar imaging.

[0063] However, because the rotation around the fast axis (i.e., around hinge 622) of the two-dimensional MEMS mirror uses the resonance phenomenon as described above, it is difficult to set and control an arbitrary angular function. Even in this case, if the scan angle θx around the slow axis (i.e., around hinge 623) can be controlled, it is possible to make linear illumination area 13, which is the trajectory of the laser beam on imaging reference plane 24, parallel to the X direction.

[0064] FIG. 36 is a diagram showing the trajectory of a laser beam on the imaging reference plane 24 obtained by controlling the slow axis of the two-dimensional MEMS mirror in the image sensing device 3 according to the third embodiment. FIG. 37 shows a stripe pattern 81 obtained when the laser beam is turned on and off at equal time intervals, and FIG. 38 shows an example of a vertical stripe pattern 82 created by controlling the on / off time of the laser beam. In this case, the linear illumination area 13 on the imaging reference plane 24 and the points where the laser beam reaches every 1° are as shown in FIGS. 36, 37, and 38. In the figures, the length of the linear illumination area 13 decreases from top to bottom, but all are parallel to the X direction. Even in this case, epipolar imaging is possible.

[0065] For example, consider using stripe pattern projection for 3D sensing. Stripe pattern projection is a method of projecting a vertical stripe pattern onto a 3D object, photographing the stripe pattern on the 3D object from an oblique angle, and reconstructing the 3D shape from the degree of distortion of the stripe pattern; it is known as one of the active stereo methods. For a trapezoidal illumination area as shown in Figure 36, turning the laser on and off at equal intervals will result in a vertical stripe pattern 81 as shown in Figure 37. This stripe pattern 81 has a pattern in which the angle from the Y axis increases as it moves away in the X direction. Although stripe pattern projection is possible with such a pattern, errors are likely to occur because the projected stripe pattern is not made up of evenly spaced parallel lines.

[0066] To create a stripe pattern in which all lines are vertical, the on / off time interval of the laser can be controlled. If the on / off time interval is controlled so that it becomes longer as the linear illumination area 13 moves from top to bottom, a vertical stripe pattern 82 as shown in Figure 38 can be obtained. Using such a vertical stripe pattern 82 can reduce errors in 3D sensing using the stripe pattern projection method. Furthermore, because epipolar imaging is performed, 3D sensing with less error is possible when photographing shiny metallic surfaces, etc., with less influence from reflected stray light.

[0067] 3-4 Configuration example 3 <Suppressing distortion with free-form lenses and mirror angle control> The above describes the case where distortion is suppressed on the imaging reference surface 24, which is installed at an angle to the optical axis 11, using a free-form lens, and the case where distortion is suppressed by controlling the scan angle θx around the slow axis (i.e., around the hinge 623). However, a hybrid method using both is also conceivable. For example, as described with reference to FIGS. 31 to 35, a free-form lens has a large SAG amount and a complex shape, making it prone to surface shape errors. Furthermore, because the free-form lens is installed at an angle to the optical axis, alignment errors are also prone to occur. If such assembly errors occur, the linear illumination area 13 on the imaging reference surface 24 may be slightly distorted from a straight line. Such slight distortion from a straight line can be corrected by controlling the angle of the mirror. In this case, controlling the angle of the mirror requires a smaller correction amount of the scan angle θx around the slow axis (i.e., around the hinge 623) than when distortion is suppressed by mirror angle control alone. To correct the assembly error, after assembling the laser scanner 60, the irradiation pattern on the screen is measured and the mirror angle is controlled to correct the deviation from the design value.

[0068] Furthermore, when attempting to suppress distortion by mirror angle control alone, even if control is only performed on the scan angle θx around the slow axis (i.e., around hinge 623), a certain degree of large acceleration / deceleration is required, and due to the performance of 2D MEMS, sufficient angle control may not be possible, and distortion may not be suppressed. However, with the hybrid method, the angle of mirror section 621 can be controlled with a smaller force, resulting in higher control accuracy, making it possible to obtain linear illumination area 13 with sufficient parallelism for epipolar imaging.

[0069] 4. Fourth embodiment 39 is a plan view schematically showing the main configuration of an image sensing device 4 according to embodiment 4. In embodiment 1, the keystone distortion generating element 40 is arranged in front of the laser scanner 10 (i.e., on the projection side), but in embodiment 4, the keystone distortion generating element 80 is arranged in front of the camera 20 (i.e., on the imaging side). The keystone distortion generating element 80 has the function of bringing the extension direction of the linear imaging area 23 closer to the extension direction of the linear illumination area 13 on the illumination reference plane 14.

[0070] 39, the optical axis 21 of the camera 20 is inclined at an angle θ with respect to the Z direction. Because a keystone distortion generating element 80 is inserted on the imaging reference plane 24 perpendicular to the optical axis 21, keystone distortion occurs, but the distortion is corrected on the illumination reference plane 14 perpendicular to the laser scanner 10. That is, in the fourth embodiment, the reference plane for aligning the linear illumination area 13 in the X direction is the illumination reference plane 14. In this case, if the linear imaging area 23 of the camera 20 and the linear illumination area 13 are scanned in synchronization with each other, scanning can be performed (i.e., epipolar imaging) with the linear imaging area 23 and the linear illumination area 13 continuously overlapping (preferably always overlapping) on ​​the illumination reference plane 14, similar to the operation on the imaging reference plane 24 in FIG. 6(E).

[0071] By inserting the trapezoidal distortion generating element 80 and making the optical axis 21 intersect with the optical axis 11, the sensing area 25 in Figure 39 has the effect of being wider than the sensing area of ​​the image sensing device of the comparative example shown in Figures 11 and 12.

[0072] In all other respects, the fourth embodiment is the same as the first embodiment. [Explanation of symbols]

[0073] 1 to 4 image sensing device, 10, 50, 60 laser scanner (illumination device), 11 optical axis, 12 total laser scan range, 13 linear illumination area, 14 illumination reference surface (illumination screen), 20 camera, 21 optical axis, 22 total imaging range, 23 linear imaging area, 24 imaging reference surface (imaging screen), 25 sensing area, 30 control circuit, 40 trapezoidal distortion generating element, 70 free-form surface lens, 80 trapezoidal distortion generating element, 90 laser beam (light beam), 110, 510, 610 laser light source (light source), 111, 611 mirror, 113 galvanometer mirror (scanning optical unit), 211 galvanometer mirror (first scanning optical unit), 212 galvanometer mirror (second scanning optical unit), 620 Two-dimensional MEMS mirror, X horizontal (first direction), Y vertical (second direction).

Claims

1. an illumination device including a light source that emits a light beam; and an illumination optical system that scans a linear illumination area onto which the light beam is projected, the linear illumination area extending linearly in a first direction on a virtual reference plane, in a second direction that is a direction perpendicular to the first direction; a camera that performs a photographing operation of scanning a linear imaging area, which is an imaging area that extends linearly in the first direction on the reference plane, in the second direction; a control circuit that controls the operation of the illumination device and the photographing operation of the camera so that the linear illumination area and the linear photographing area continue to overlap on the reference plane; and The optical axis of the lighting device and the optical axis of the camera are non-parallel to each other and intersect in front of the lighting device and the camera. An image sensing device characterized by:

2. The illumination optical system includes: a beam expanding optical element that expands the light beam emitted from the light source in the first direction to generate an expanded beam; a scanning optical unit that scans the linear illumination area formed on the reference surface by the expanded beam in the second direction; have 2. The image sensing device according to claim 1, wherein the image sensing device comprises: a first lens;

3. a keystone distortion generating element disposed in front of the lighting device; The keystone distortion generating element has a function of bringing the extension direction of the linear illumination area closer to the extension direction of the linear imaging area on the reference plane.

3. The image sensing device according to claim 2, wherein the image sensing device comprises: a first lens;

4. The keystone distortion generating element is a free-form lens that is asymmetric in the first direction and asymmetric in the second direction.

4. The image sensing device according to claim 3, wherein the image sensing device comprises:

5. further comprising a keystone distortion generating element disposed in front of the camera; The keystone distortion generating element has a function of bringing the extension direction of the linear imaging area closer to the extension direction of the linear illumination area on the reference plane.

3. The image sensing device according to claim 2, wherein the image sensing device comprises: a first lens;

6. The illumination optical system includes: a first scanning optical unit that forms the linear illumination area by scanning the light beam emitted from the light source in the first direction; a second scanning optical unit that scans the linear illumination area in the second direction; 2. The image sensing device according to claim 1, further comprising:

7. The illumination optical system performs a first scan to form the linear illumination area by scanning the light beam emitted from the light source in the first direction, and a second scan to scan the linear illumination area in the second direction.

2. The image sensing device according to claim 1, wherein the image sensing device comprises: a first lens;

8. The illumination optical system is a two-dimensional MEMS mirror.

8. The image sensing device according to claim 7, wherein the image sensing device comprises:

9. the scanning in the first direction is performed by controlling the two-dimensional MEMS mirror by rotating it around a high speed axis; The scanning in the second direction is performed by controlling the two-dimensional MEMS mirror by rotating it about a slow axis.

9. The image sensing device according to claim 8, wherein the image sensing device comprises:

10. The scan angle control of the scan in the first direction and the scan angle control of the scan in the second direction are performed using a predetermined angle function.

9. The image sensing device according to claim 8, wherein the image sensing device comprises:

11. The scan angle control of the scan in the first direction and the scan angle control of the scan in the second direction are performed using a predetermined angle function.

10. The image sensing device according to claim 9, wherein the image sensing device comprises:

12. The lighting device further includes a free-form lens disposed in front of the lighting device.

12. An image sensing device according to claim 7, wherein the image sensing device comprises: a first lens;

Images