Image sensing device

Abstract

An image sensing device (1) is provided with: an illumination device (10) that includes a light source that emits a light beam, and an illumination optical system that scans in a second direction (Y), which is a direction orthogonal to a first direction (X), a linear illumination region (13) that extends linearly in the first direction (X) on a virtual reference surface (24), and an illumination region on which the light beam is projected; a camera (20) that performs an imaging operation in which a linear imaging region (23), which is a region extending linearly in the first direction (X) on the reference surface (24), is scanned in the second direction (Y); and a control circuit (30) that controls the operation of the illumination devices (10, 50, 60) and the imaging operation of the camera (20) so that the linear illumination region (13) and the linear imaging region (23) continuously overlap on a reference surface (24), and the optical axis (11) of the illumination device (10) and the optical axis (21) of the camera (20) are not parallel to each other and intersect on the reference surface (24).

Description

Technical Field

[0001] The invention relates to an image sensing device. Background Art

[0002] An image sensing device is known that is composed of a projector, a camera, and a synchronization circuit, and that obtains an image of an object through an epipolar line image (for example, refer to Patent Document 1). The projector is an illumination device that scans the illumination area formed by the laser beam, that is, the beam spot, in the horizontal direction and the vertical direction. The camera is, for example, a rolling shutter camera, and is a shooting device that scans the shooting area in the horizontal direction and the vertical direction. The camera and the projector are arranged in the X direction and the optical axis of the camera and the optical axis of the projector are parallel. The synchronization circuit controls the operation of the projector and the camera so that the illumination area of ​​the projector and the shooting area of ​​the camera are consistent.

[0003] If the nuclear line image is used, objects that will produce strong reflection scattering (for example, glossy metal objects) can be photographed while suppressing reflected scattered light (for example, reflected stray light), allowing three-dimensional measurements with fewer errors (for example, refer to non-patent document 1).

[0004] Patent Document 1: U.S. Patent No. 10359277

[0005] Non-patent document 1: Matthew O'Toole et al., "Homogeneous Codes for Energy-Efficient Illumination and Imaging," ACM SIGGRAPH, 2015 Summary of the invention

[0006] However, in the device described in the above document, in order to realize the epipolar line image, the camera and the projector need to be configured so that the optical axis of the camera and the optical axis of the projector become parallel. In this case, there is a problem that the overlapping area of ​​the projector's illuminating range and the camera's shooting range, that is, the sensing area is narrow.

[0007] An object of the present invention is to provide an image sensing device with a wide sensing area.

[0008] The image sensing device of the present invention is characterized in that it has: an illumination device, which includes a light source that emits a light beam, and an illumination optical system that scans an illumination area projected by the light beam and a linear illumination area that extends linearly along a first direction on a virtual reference plane in a direction orthogonal to the first direction, i.e., a second direction; a camera, which performs a photographing action of scanning a photographing area that extends linearly in the first direction on the reference plane, i.e., a linear photographing area, in the second direction; and a control circuit, which controls the action of the illumination device and the photographing action of the camera so that the linear illumination area and the linear photographing area are continuously repeated on the reference plane, and the optical axis of the illumination device and the optical axis of the camera are non-parallel to each other and intersect on the reference plane.

[0009] Effects of the Invention

[0010] According to the present invention, the sensing area can be expanded. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a perspective view schematically showing a main structure of the image sensing device according to the first embodiment.

[0012] Figure 2 It roughly indicates Figure 1 A top view of the main structure of an image sensing device.

[0013] Figure 3 It roughly indicates Figure 1 An oblique view of the main structure of the laser scanner.

[0014] Figure 4 It roughly indicates Figure 1 A top view of the main structure of the laser scanner.

[0015] Figure 5 It roughly indicates Figure 1 A side view of the main structure of a laser scanner.

[0016] Figure 6 (A) is a diagram showing the operation of a comparative image sensing device (without a keystone generating element), and (B) to (E) are diagrams showing Figure 1 FIG. 1 is a diagram of an operation of an image sensing device.

[0017] Figure 7 Yes means Figure 1 An overhead view of the camera's action.

[0018] Figure 8 Yes means Figure 1 An overhead view of the laser scanner in action.

[0019] Fig. 9 Yes means Figure 1 An overhead view of the camera and laser scanner in action.

[0020] Fig.10 This is a plan view schematically showing a main structure of an image sensing device according to a modification of the first embodiment.

[0021] Fig.11 It is a perspective view schematically showing the main structure of an image sensing device of a comparative example (a case where the optical axis of the camera and the optical axis of the laser scanner are parallel).

[0022] Fig.12 It roughly indicates Fig.11 A top view of the main structure of an image sensing device.

[0023] Fig.13 (A) to (C) represent Fig.11 FIG. 1 is a diagram of an operation of an image sensing device.

[0024] Fig.14 It is a plan view schematically showing the main structure of an image sensing device of a comparative example (a case where the optical axis of the laser scanner is tilted with respect to the optical axis of the camera).

[0025] Fig.15 (A) to (C) represent Fig.14 A diagram showing the operation of the image sensing device (when the optical axis of the laser scanner is tilted).

[0026] Fig.16 This is a perspective view schematically showing the main structure of a laser scanner of the image sensing device according to the second embodiment.

[0027] Fig.17 It roughly indicates Fig.16 A top view of the main structure of the laser scanner.

[0028] Fig.18 It roughly indicates Fig.16 A side view of the main structure of a laser scanner.

[0029] Fig.19 This is a plan view schematically showing a main structure of an image sensing device according to Embodiment 2.

[0030] Fig. 20 (A) and (B) are diagrams showing the operation of the image sensing device according to the second embodiment.

[0031] Fig.21 (A) and (B) are graphs showing an angle function of a galvano-mirror and an angle function of a galvano-mirror in an image sensing device according to the second embodiment.

[0032] Fig. 22 (A) to (C) are graphs showing angle functions of the galvano-controlled mirror for correcting distortion on the illumination reference plane of the image sensing device according to the second embodiment.

[0033] Fig.23 (A) to (C) are diagrams showing angle functions of the galvano-mirror for correcting distortion on the imaging reference plane of the image sensing device according to the second embodiment.

[0034] Fig.24 This is a perspective view schematically showing the main structure of a laser scanner of an image sensing device according to a third embodiment.

[0035] Fig.25 It roughly indicates Fig.24 A top view of the main structure of the laser scanner.

[0036] Fig.26 It roughly indicates Fig.24 A side view of the main structure of a laser scanner.

[0037] Fig. 27 This is a diagram for explaining distortion on an illumination reference plane of an image sensing device of a comparative example (a case where a keystone distortion generating lens is not provided).

[0038] Fig.28 This is a diagram for explaining distortion on a photographing reference plane of an image sensing device of a comparative example (a case where a keystone distortion generating lens is not provided).

[0039] Fig.29 FIG. 1 is a diagram showing the trajectory of a light beam on an illumination reference plane of an image sensing device according to the third embodiment.

[0040] Fig.30 FIG. 1 is a diagram showing the trajectory of a light beam on an imaging reference plane of the image sensing device according to the third embodiment.

[0041] Fig.31 This is a perspective view schematically showing the main structure of a laser scanner including a free-form surface lens of an image sensing device according to a third embodiment.

[0042] Fig.32 It roughly indicates Fig.31 A side view of the main structure of a laser scanner.

[0043] Fig.33 It roughly indicates Fig.31 A top view of the main structure of the laser scanner.

[0044] Fig.34It is a diagram showing the cross-sectional profiles of the first surface and the second surface of the free-curved surface lens of the image sensing device according to the third embodiment.

[0045] Fig.35 It is a diagram showing the cross-sectional profiles of the first surface and the second surface of the free-curved surface lens of the image sensing device according to the third embodiment.

[0046] Fig.36 This is a diagram showing the trajectory of a light beam on an imaging reference plane obtained by controlling the low-speed axis of a two-dimensional MEMS mirror in the image sensing device according to the third embodiment.

[0047] Fig.37 This is a diagram showing a fringe pattern when a laser beam is turned on and off at equal time intervals.

[0048] Fig.38 The figure shows a stripe pattern perpendicular to the longitudinal direction created by controlling the on and off timing of the laser beam.

[0049] Fig.39 This is a plan view schematically showing a main structure of an image sensing device according to a fourth embodiment. DETAILED DESCRIPTION

[0050] The image sensing device according to the embodiment is described below with reference to the accompanying drawings. The following embodiments are merely examples, and the embodiments can be appropriately combined and each embodiment can be appropriately changed. In addition, in the drawings, the same reference numerals are used for structures having the same or similar functions.

[0051] 《1》Implementation Method 1

[0052] Structure of 1-1

[0053] Figure 1 and Figure 2 1 is an oblique view and a top view schematically showing the main structure of the image sensing device 1 involved in Embodiment 1. The image sensing device 1 is a device that realizes the epipolar image. The image sensing device 1 has a camera 20 as a shooting device, a laser scanner 10 as an illumination device, a control circuit 30 including a synchronization circuit, and a trapezoidal distortion generating element 40. The camera 20 and the laser scanner 10 are arranged side by side along the X direction, and the optical axis 21 of the camera 20 and the optical axis 11 of the laser scanner 10 are non-parallel to each other and intersect in front of the camera 20 and the laser scanner 10. In Embodiment 1, the trapezoidal distortion generating element 40 is inserted in front of the laser scanner 10.

[0054] A virtual plane 24 (also referred to as a "photographing screen") is provided at a position separated from the camera 20 by a certain distance Z0 and is perpendicular to the optical axis 21 of the camera 20. In addition, a virtual plane 14 (also referred to as a "photographing screen") is provided as a virtual plane 24 perpendicular to the optical axis of the laser scanner 10 and is a laser projection reference plane inclined at an angle θ relative to the photographing reference plane 24. The photographing reference plane 24 and the illumination reference plane 14 are virtual planes for explanation purposes only and do not actually exist physically. In addition, the optical axis 21 of the camera 20 and the optical axis 11 of the laser scanner 10 intersect on the virtual reference plane, the illumination reference plane 14.

[0055] Figures 3 to 5 It roughly indicates Figure 1 The structure of the laser scanner 10 is shown in an oblique view, a top view, and a side view. The laser scanner 10 emits a laser beam (also referred to as an "enlarged beam") that expands in a fan shape in the X direction as the first direction. On the illumination reference plane 14, the enlarged laser beam (i.e., a linear beam having a linear cross section) extending in the X direction becomes a linear illumination area 13.

[0056] The laser scanner 10 includes: a laser light source 110 as a light source, which emits a laser beam as a light beam; and an illumination optical system, which scans the illumination area of ​​the projected laser beam and the linear illumination area 13 extending linearly along the X direction on a virtual reference plane, i.e., an illumination reference plane 14, in the Y direction, which is a second direction orthogonal to the X direction. The laser beam emitted from the laser light source 110, after being reflected by a reflector 111, becomes a linear light beam, i.e., the linear illumination area 13, which is extended in the X direction by a beam magnifying optical element 112 as a lens. The beam magnifying optical element 112 is, for example, an optical lens such as a cylindrical lens or a Powell lens. Figure 5 As shown, the linear illumination area 13 is deflected in the Z direction by the electrically controlled mirror 113 as the scanning optical part. The electrically controlled mirror 113 can swing around the X axis within a predetermined angle range of ±(α / 2), and the linear illumination area 13 scans around the X axis within the angle range of ±α twice as large as the predetermined angle range (i.e., Figure 5 On the imaging reference plane 24, the linear illumination area 13 is scanned in the Y direction. Figure 1 An area within the full laser scanning range of 12 is irradiated.

[0057] The camera 20 performs an imaging operation of scanning the linear imaging area 23, which is an imaging area extending linearly in the X direction on the imaging reference plane 24, in the Y direction. The control circuit 30 controls the operation of the laser scanner 10 and the imaging operation of the camera 20 in such a manner that the linear illumination area 13 and the linear imaging area 23 are continuously repeated on the imaging reference plane 24. In addition, the control circuit 30 may also be composed of a memory storing a software program and a processor. In this case, the function of the control circuit 30 is realized by the processor executing the software program stored in the memory.

[0058] Figure 6 (A) is a diagram showing the operation of a comparative image sensing device (a case where there is no trapezoidal distortion generating element). Figure 6 (A) shows a linear illumination area 13 on an illumination reference plane 14 of an image sensing device of a comparative example.

[0059] Figure 6 (B) to (E) are diagrams showing the operation of the image sensing device 1 according to the first embodiment. Figure 6 (B) shows the linear illumination area 13 on the illumination reference plane 14, Figure 6 (C) shows the linear illumination area 13 on the imaging reference plane 24 in the first embodiment. Figure 6 (D) shows the linear imaging area 23 on the imaging reference plane 24. Figure 6 (E) shows the linear illumination area 13 and the linear imaging area 23 on the imaging reference plane 24 .

[0060] like Figure 6 As shown in (A), in the comparative image sensing device, on the illumination reference plane 14 without the trapezoidal distortion generating element 40, the linear illumination area 13 is repeatedly scanned in the -Y direction from the upper end to the lower end in the entire laser scanning range 12 at a speed VL. The linear illumination area 13 exists as a linear illumination area 13a at the upper end of the entire laser scanning range 12 at time t=ta, is represented as a linear illumination area 13b at time t=tb, and exists as a linear illumination area 13c at the lower end of the entire laser scanning range 12 at time t=tc. If it reaches the lower end, it returns to the upper end at a high speed and repeats the above action.

[0061] In the first embodiment, since the trapezoidal distortion generating element 40 is provided, Figure 6As shown in (B), the entire laser scanning range 12 on the illumination reference plane 14 becomes a trapezoid. That is, at time t=ta, the linear illumination area 13a is a straight line inclined to the upper right, and as the scanning in the -Y direction proceeds and the rotation in the XY plane proceeds, the linear illumination area 13c at the lower end becomes a straight line inclined to the lower right. The shooting reference plane 24 perpendicular to the optical axis 21 is inclined relative to the plane perpendicular to the optical axis 11 (inclined in a manner close to the X direction). The trapezoidal distortion generating element 40 has the function of making the extension direction of the linear illumination area 13 on the illumination reference plane 14 close to the extension direction of the linear shooting area 23. The linear illumination area 13 on the shooting reference plane 24 is as shown. Figure 6 As shown in (C), the linear illumination area 13 parallel to the X direction is scanned from t = ta to t = tc. Figure 6 The trapezoidal distortion generating element 40 is designed in accordance with the operation mode of (C), but the specific design of the trapezoidal distortion generating element 40 will be described later.

[0062] The camera 20 is a rolling shutter camera, and can repeatedly scan a linear imaging region extending in the X direction in the -Y direction by shortening the exposure time. The scanning of the imaging region by the camera 20 is described in, for example, Fig.13 On the photographing reference plane 24, the photographing range 12 of the camera 20 is scanned from the upper end to the lower end of the full photographing range 22 of the camera 20. Figure 6 (D) is shown. Scanning is performed at a speed Vc from the linear shooting area 23a at the upper end of the full shooting range 22 to the linear shooting area 23c at the lower end. Here, on the shooting reference plane 24, the device structure is set in such a way that the linear shooting area 23a of the camera 20 and the linear illumination area 13a overlap in the Y direction. The setting can be performed by zooming the lens of the camera 20, setting the ROI (Region of Interest) that limits the shooting area of ​​the camera 20, and setting the scanning range of the linear illumination area 13 in the Y direction. The mechanism for fine-tuning the setting posture of the camera 20 and the laser scanner 10 is also important.

[0063] The control circuit 30 makes the imaging time of the linear imaging area 23a and the irradiation time of the linear illumination area 13a coincide at t=ta. Also, the scanning speed Vc of the linear imaging area 23 in the -Y direction and the scanning speed VL of the linear illumination area 13 in the -Y direction coincide. Figure 6As shown in (E), from time t = ta to t = tc, during one cycle, the positions in the Y direction are always overlapped (preferably always overlapped) and scanned from top to bottom. This cycle is repeated to perform the operation. The overlapping range of the full shooting range 22 and the full laser scanning range 12 of the linear illumination area 13 is the range in which the epipolar line image can be realized.

[0064] Here, the camera 20 and the laser scanner 10 are arranged side by side in the X direction, that is, they are at the same position coordinates in the Y direction and the Z direction (condition A). With this arrangement, no matter at what distance Z the imaging reference plane 24 is in front of the camera 20, it is arranged as follows. Figure 6 The linear illumination area 13 and the linear imaging area 23 shown in (E) continue to overlap. Figures 7 to 9 The reason is explained. Figures 7 to 9 This is a diagram for explaining the operation of the linear imaging area of ​​the image sensor device 1 according to the first embodiment and the light forming the linear illumination area 13, that is, the linear laser beam in the cross-sectional direction (in the YZ plane). Figure 7 Yes means Figure 1 A top view of the camera's action, Figure 8 Yes means Figure 1 A top view of the laser scanner in action. Fig. 9 Yes means Figure 1 An overhead view of the camera and laser scanner in action.

[0065] Figure 7 This is a diagram showing the range scanned by the linear imaging region 23 from the camera 20 projected on the YZ plane. Figure 8 This is a diagram of the range scanned by the laser scanner 10 and projected on the YZ plane. Figures 7 and 8 The scanning range is consistent. Moreover, the trajectories of the linear imaging area 23b and the linear illumination area 13b in the YZ plane at any time t=tb are also consistent. Fig. 9 Therefore, as shown in Fig. 9 As shown in the middle shaded area, the sensing area 25 is wide, and the epipolar image can be realized at any position in the Z direction. Figure 2 It is clear that in the smallest range Z (the range very close to the camera), the shooting range of the camera 20 and the scanning range of the linear illumination area 13 in the XZ plane do not overlap, so the epipolar line image cannot be realized. Figure 2 , Fig. 9In the figure, the distant area of ​​the sensing area 25 is separated by the imaging reference plane 24, but in reality, the sensing area 25 extends further away. The actual boundary of the distant sensing area 25 is determined by the detectable signal amount because the amount of light received by the camera 20 decreases as it is farther away.

[0066] Specifically, the keystone distortion generating element 40 can function by using a wedge-shaped prism. Patent document 2 shows an example in which a wedge-shaped prism is inserted into the light exit surface of a projector that projects obliquely upward, thereby correcting the keystone distortion of the projection pattern on the screen in the vertical direction.

[0067] Patent Document 2: Japanese Patent Application Publication No. 2016-105179

[0068] Fig.10 1 is a diagram showing a configuration example of an image sensing device according to Embodiment 1. A wedge-shaped prism is inserted into the front surface of the laser scanner 10, thereby causing trapezoidal distortion in the horizontal direction. Fig.10 In the figure, the vertex of the wedge prism 41 is on the right side, and the optical axis 11 deflects to the left when passing through the wedge prism 41. Here, the illumination reference plane 14 is perpendicular to the optical axis 11 after exiting the wedge prism 41. In addition, the angle between the normal line of the photographing reference plane 24 and the optical axis 11 is set to θ. The horizontal trapezoidal distortion on the photographing reference plane 24 disappears, as shown in FIG. Fig. 9 As shown, the shape, material, and installation angle of the wedge prism 41 and the installation angle of the laser scanner 10 are designed so that all the linear illumination areas 13a to 13c on the imaging reference plane 24 are parallel to the X-axis.

[0069] Comparison of 1-3

[0070] Fig.11 and Fig.12 1 is a perspective view and a top view schematically showing the main structure of a comparative image sensing device 1a for realizing an epipolar image. The comparative image sensing device 1a is composed of a camera 20, a laser scanner 10, and a control circuit 30. The camera 20 and the laser scanner 10 are arranged in an X direction, and the optical axis 21 of the camera 20 and the optical axis 11 of the laser scanner 10 are parallel to each other and face the Z direction. It is assumed that there is a virtual screen, i.e., a photographing reference plane 24, which is perpendicular to the optical axis 21 of the camera 20, at a position separated from the camera 20 by a certain distance Z0.

[0071] Fig.13 (A) to (C) are diagrams for explaining the operation of the image sensing device 1a for realizing epipolar line images according to a comparative example. Fig.13(A) shows a situation where a linear imaging area 23 formed by a rolling shutter camera is scanned on the imaging reference plane 24. Figure 6 The action is the same as described in (D).

[0072] exist Fig.13 (B) shows the operation of the linear lighting area 13. Fig.13 (C) shows a state where the linear imaging area 23 and the linear illumination area 13 are overlapped.

[0073] exist Fig.13 (B) shows a situation where the linear illumination area 13 formed by the laser scanner 10 is scanned on the imaging reference plane 24. Fig.11 and Fig.12 In the comparative structure, the photographing reference plane 24 is perpendicular to the laser scanner 10, so Figure 6 Similarly to the case described in (A), scanning is performed from the linear illumination area 13a to 13c parallel to the X direction.

[0074] As described above, the camera 20 and the laser scanner 10 are synchronized using the control circuit 30, and the cameras 20 and the laser scanner 10 are operated so that the positions in the Y direction on the imaged reference surface 24 continuously overlap (preferably always overlap). Fig.13 (C) shows the overlap of the linear illumination area 13 and the linear imaging area 23 on the imaging reference plane 24. The overlap area of ​​the two on the imaging reference plane 24 is greater than Figure 6 The area described by (E) is small. Fig.12 The sensing area 25 as the overlapping area is indicated by shading. Figure 2 The sensing area 25 and Fig.12 Compared with the sensing area 25 in the comparative image sensing device 101, it is known that there is a problem that the sensing area 25 that can realize the epipolar image is small. In order to enlarge the sensing area 25, the distance between the camera 20 and the laser scanner 10 can be made close while maintaining the parallelism of the optical axis 21 and the optical axis 11, but there is a limit to ensure the size of the equipment. In addition, as described in the structural example 2 of the embodiment 2 and the embodiment 3, a large application achieved by epipolar images is 3D sensing through stripe pattern projection. The stripe pattern projection method uses the principle of triangulation, so in order to improve the measurement accuracy in the depth direction, the distance between the laser scanner 10 and the camera 20 in the X direction can be increased. However, in the existing epipolar image, the optical axis 21 and the optical axis 11 remain parallel and unchanged, so there is a problem that the area that can perform 3D sensing, that is, the sensing area 25, becomes smaller.

[0075] Fig.14 and Fig.15(A) to (C) are diagrams for explaining the operation when the optical axis of the laser scanner 10 is tilted in the image sensing device of the comparative example realizing the epipolar line image. Fig.14 It is the main structure diagram. Fig.15 (A) shows the action of the linear illumination area 13 on the illumination reference plane 14, Fig.15 (B) shows the action of the linear illumination area 13 on the photographing reference plane 24. Fig.15 (C) shows an operation of overlapping the linear imaging region 23 and the linear illumination region 13 on the imaging reference plane 24 .

[0076] In addition, in order to expand the width of the sensing area 25 in the X direction, as shown in FIG. Figure 1 As shown in FIG. 1 , if the optical axis 11 of the laser scanner 10 is tilted in the X direction, the optical axis 21 and the optical axis 11 become non-parallel. Then, the imaging reference plane 24 perpendicular to the optical axis 21 of the camera 20 is tilted in the optical axis 11, so that the linear illumination area 13 on the imaging reference plane 24 is as shown in FIG. Fig.15 As shown in (B), the upper end 13a becomes a straight line inclined to the lower right, and as the scanning progresses, it rotates in the XY plane, and the lower end 13c becomes a straight line inclined to the upper right. On the other hand, the linear imaging area on the imaging reference plane 24 is as shown in FIG. Fig.13 As shown in (A), the linear imaging region 23 and the linear illumination region 13 are scanned from the top to the bottom while maintaining parallelism. When the positions of the linear imaging region 23 and the linear illumination region 13 in the Y direction on the imaging reference plane 24 are made consistent by the control circuit 30, the overlapping of the two on the imaging reference plane 24 is shown in FIG. Fig.15 (C) shows. Fig.15 As is clear from (C), the inclination of the linear illumination area 13 rotates on the imaging reference plane 24, so scanning cannot be performed when the linear imaging area 23 and the linear illumination area 13 are overlapped. At the middle position in the Y direction, the linear imaging area 23 and the linear illumination area 13 become parallel for only a moment, but the sensing area in the Y direction becomes extremely narrow, making it difficult to use as a sensor for epipolar line images.

[0077] 《1-4》Effect

[0078] In the image sensing device 1 according to the first embodiment, a properly designed trapezoidal distortion generating element 40 is inserted in front of the laser scanner 10, so that a linear laser beam parallel to the X direction can be scanned in the Y direction on the imaging reference plane 24 tilted relative to the optical axis 21. Therefore, even if the optical axis 11 is tilted relative to the optical axis 21, an epipolar image can be realized, and the effect of enlarging the sensing area 25 can be obtained at this time.

[0079] In addition, in the above description, an example of using the electrically controlled mirror 113 as the light beam scanning device is described, but the same effect can be obtained by using a one-dimensional MEMS (Micro ElectroMechanical Systems) mirror having the function of rotating and vibrating the mirror at high speed. Alternatively, a polygonal mirror, i.e., a polygonal mirror, can be used as a scanner that is rotated by a motor instead of the electrically controlled mirror 113.

[0080] 《2》Implementation Method 2

[0081] Structure of 2-1

[0082] Fig.16 This is a perspective view schematically showing a main structure of a laser scanner 50 serving as an illumination device of the image sensing device 2 according to the second embodiment. Fig.17 and Fig.18 It roughly indicates Fig.16 A top view and a side view of the main structure of the laser scanner 50. Fig.19 2 is a top view schematically showing the main structure of the image sensing device 2 according to the second embodiment. Figures 16 to 18 The XYZ coordinate axes are local coordinates of the laser scanner 50 and exist in the direction of the optical axis 11. Figures 16 to 18 The Z axis. That is, Fig.19 The Z direction and Figures 16 to 18 The difference between the second embodiment and the first embodiment is that the laser scanner 50 is composed of two electrically controlled mirrors 511 and 512. The laser beam 90 emitted from the laser light source 510 travels in the Z-axis direction and is reflected by the electrically controlled mirror 511 as the first scanning optical unit. The electrically controlled mirror 511 changes the rotation angle θy of the mirror at a high speed within the range of ±10° around the rotation axis. The rotation axis is as shown in FIG. Fig.18 As shown, it is inclined at an angle θ1 relative to the Y axis.

[0083] The laser beam 90 reflected by the galvano-mirror 511 reaches the galvano-mirror 512 as the second scanning optical unit. The galvano-mirror 511 reciprocates at a high speed, so the laser beam 90 reaching the galvano-mirror 512 is as follows: Fig.16 As shown in the figure with reference numeral 93, a curve protruding upward is drawn. The reason why the curve is not a straight line is that the light incident on the electric mirror 511 from above in the Y direction is scanned in the X direction. The electric mirror 512 changes the rotation angle θx of the mirror within the range of ±6° around the rotation axis, and the rotation axis is oriented in the X-axis direction. The electric mirror 512 determines the posture when θx=0 so that the emission direction of the laser beam 90 when θy=0 and θx=0, that is, the optical axis 11 is oriented toward the Z axis. When the electric mirror 512 is at -6°, it becomes Fig.18 When the electric mirror 512 is at +6°, the laser beam 90a becomes Fig.18 Laser beam 90c.

[0084] 2-2 Action

[0085] Fig. 20 (A), (B) and Fig.21 (A) and (B) are diagrams for explaining distortion on the screen of the image sensing device 2 according to the second embodiment. Fig. 20 (A) shows the trajectory of the laser beam on the illumination reference surface 14, Fig. 20 (B) shows the trajectory of the laser beam on the imaging reference surface 24 . Fig.21 (A) shows the angle function θx(t) of the electrically controlled mirror 512, Fig.21 (B) shows the angle function θy(t) of the galvano-controlled mirror 511 .

[0086] When the galvano-mirror 511 reciprocates and scans at a constant speed, and the galvano-mirror 512 repeatedly scans from +6° to -6° at a constant speed, Fig.19 On the illumination reference plane 14, the laser beam 90 depicts Fig. 20 The trajectory is shown by the arrow of the upward convex curve in (A). On the outward path, it moves in the direction from -X to +X as shown by the solid arrow, and on the return path, it moves in the opposite direction as shown by the dotted arrow. The electrically controlled reflector 512 scans more slowly than the electrically controlled reflector 511, so the trajectory of the laser beam from the direction of +Y to -Y, that is, the linear illumination area 13, moves across the entire screen. If the trajectory of the unidirectional laser beam is photographed for a time greater than or equal to the time taken for the unidirectional movement (that is, if the exposure time of the camera 20 is made sufficiently long), it can be regarded as being irradiated by the curved laser beam. In addition, Fig. 20 The points in (A) represent the arrival points of the laser beam on the illumination reference surface 14 when the rotation angles θx and θy of the galvano-mirrors 511 and 512 are varied discretely in 1° increments.

[0087] The trajectory of the laser beam on the imaging reference plane 24, which is arranged perpendicular to the optical axis 21 but tilted relative to the optical axis 11, is Fig. 20 The solid or dotted arrows in (B) indicate the points Fig. 20(A) Similarly, it shows the arrival point of the laser beam on the imaging reference plane 24 when the rotation angles θx and θy of the electric mirrors 511 and 512 are discretely changed in 1° increments. On the imaging reference plane 24, the track scanned at high speed in the substantially horizontal direction, i.e., the linear illumination area 13, is not only convex in the Y direction, but also, as shown in the figure, the overall inclination is rotated as shown by the linear illumination areas 13a to 13c from the time t=ta to tc. Since the linear illumination area 13 after the distortion as described above cannot be overlapped with the linear imaging area 23 of the camera 20, the epipolar image cannot be realized.

[0088] Here, in Fig.21 (A) and Fig.21 (B) shows a simplified diagram of the angle function of the galvano-controlled mirror 511 and the galvano-controlled mirror 512 for generating the linear illumination area 13 as described above within one frame. Fig.21 In (A) and (B), it is described that the step angle of the rotation angle θx of the galvano-controlled mirror 512 is displayed in units of 2°, and the rotation angle θy of the galvano-controlled mirror 511 changes only in the positive direction from -10 to +10°. Fig.21 In (A), the angle θx is kept constant during one scan of the electric mirror 511 (set as time Tx), but in actual operation with finer steps of θx, it is also possible to change slowly at a constant angular velocity during the period from -6° to +6°. During the short time Tx, the angle θx is considered constant. In addition, Fig.21 In (B), the angle θy is changed at a constant speed during the time Tx.

[0089] The angle function of the electrically controlled mirror is Fig.21 (A) and Fig.21 Correction is added to the function shown in (B) of , and the distortion is corrected. That is, Fig. 20 The pattern of the laser beam arrival position for each 1 degree angle indicated by dots in (A) and (B) is a square grid correction function.

[0090] Fig. 22 (A) to (C) and Fig.23 (A) to (C) are diagrams for explaining distortion on the screen of the image sensing device 2 according to the second embodiment. Fig. 22 (A) and (B) show the angle function of the electrically controlled reflector for correcting the distortion on the illumination reference plane 14, Fig. 22 (C) shows the trajectory of the laser beam on the illumination reference surface 14 . Fig.23 (A) and (B) show the angle function of the electrically controlled mirror for correcting the distortion on the photographing reference plane 24, Fig.23 (C) shows the trajectory of the laser beam on the imaging reference surface 24 .

[0091] First, on the illumination reference plane 14, create the angle functions θy(t) and θx(t) at which distortion disappears. To find the function, find the angle at which Fig. 22 The angles θy and θx of the two electrically controlled reflectors on the grating points on the illumination reference plane 14 shown in (C) are sufficient. For the two angles θy and θx, the position coordinates (X, Y) on the illumination reference plane 14 are in a one-to-one mapping relationship. Therefore, for a certain (X, Y), θy and θx for reaching the (X, Y) are numerically calculated. For example, Fig. 22 Here, in order to simplify the display of the figure, it is assumed that after the electric mirror 511 moves at a constant angular velocity from -10° to +10°, it returns to -10° instantaneously, and it is assumed that θx changes in steps of about 2 degrees from about -6° to about +6°. Each step of the θx function is a curve that bulges downward. At this time, as Fig. 22 As shown by the arrow to the right of the linear illumination area 13a of (C), the trajectory of the line segment parallel to the X-axis is drawn. In actual operation, when the electrically controlled mirror 511 returns from +10° to -10°, the electrically controlled mirror 512 is also controlled in the same way to scan the laser beam. In addition, the angle can be changed in finer steps of θx. The trajectory of the laser beam on the return path is Fig. 22 As described above, the laser beam reaches the position when the combination of θy and θx is swung at an angle of 1 degree on the illumination reference plane 14. Fig. 20 (A), but by using Fig. 22 The function of (A) and (B) is Fig. 22 The trajectory in (C), that is, the linear illumination area 13a, is corrected to a straight line. Fig. 22 (C) is a column of groups of θy and θx in which the square grid of dots is raster-scanned from the upper left to the right.

[0092] However, only by the above, on the shooting reference plane 24 tilted relative to the optical axis 11, its trajectory is as follows Fig.15 The linear illumination areas 13a and 13c shown in (B) are not parallel to the X-axis, and the epipolar image cannot be properly realized. Fig. 22A linear illumination area 13 is generated horizontally on the illumination reference plane 14 as a function of (A) and (B). Therefore, as described in embodiment 1, a trapezoidal distortion generating element 40 is provided in front of the laser scanner 50, thereby being able to generate a linear illumination area 13 horizontally on the shooting reference plane 24 and realizing a nuclear line image.

[0093] However, if the two electrically controlled mirrors 511 and 512 are appropriately controlled, a linear illumination area 13 extending in the horizontal direction on the imaging reference plane 24 can be generated without using the trapezoidal distortion generating element 40. Fig.23 As shown in (C), in order to obtain the linear illumination area 13a by horizontal laser beam scanning on the shooting reference plane 24,

[0094] Fig.23 You can set it by using the angle function of (A) and (B). Fig.23 (A) is relative to Fig. 22 The seven curve groups protruding downward shown in (A) are assigned different rotated figures. As mentioned above, in order to correct the trapezoidal distortion occurring on the shooting reference plane 24 tilted relative to the optical axis 11, it is necessary to generate trapezoidal distortion in the opposite direction on the illumination reference plane 14 perpendicular to the optical axis 11. For this purpose, θx needs to be controlled within the time of drawing a linear illumination area 13, and its control function needs to be changed slightly for each line. In addition, Fig. 20 In the dot pattern with an angle of 1 degree in (B), as the value of X increases, the dot spacing in the X direction becomes narrower. In order to correct the spacing in the X direction, Fig. 22 In (B), there are 7 straight line segments with an inclination that slopes upward to the right, so in Fig.23 In (B), it becomes a group of curves that are slightly convex downward. Fig.23 The function of (A) and (B) is generated Fig.23 The linear illumination area 13 is a straight line horizontal in the X direction as shown in (C). That is, if Fig.23 By operating the two galvano-mirrors 511 and 512 according to functions such as (A) and (B), epipolar line imaging can be realized.

[0095] 《2-3》Effect

[0096] In the second embodiment, when an epipolar image is realized by raster scanning of a laser beam using two galvano-mirrors 511 and 512, it is possible to perform sensing that is impossible in an epipolar image scanned in the longitudinal direction by a linear laser beam.

[0097] In addition, by repeatedly turning the laser on and off at high speed, a vertical stripe pattern can be created. Fig.23 In (C), the lighting is turned on and off 100 times during one movement from left to right or from right to left, and synchronization control is performed between each line at the same position in the X direction to turn the lighting on and off. As a result, 100 longitudinal stripes extending in the longitudinal direction appear. If the longitudinal stripes are used for 3D measurement, metal objects can be sensed without error. If 3D measurement is performed on a metal object by a stripe pattern projection method that is not a core line image, the stripe pattern reflected by the metal glossy surface becomes a false pattern and causes false detection, but in the core line image, the reflected false stripe pattern will not be introduced through the camera 20, so 3D measurement without false detection can be performed.

[0098] 《3》Implementation Method 3

[0099] Structure of 3-1

[0100] Figure 24 to Figure 26 1 is a perspective view, a top view, and a side view schematically showing the structure of a laser scanner 60 as an illumination device of the image sensing device 3 according to the third embodiment. Figure 24 to Figure 26 , a laser scanner 60 using a two-dimensional MEMS mirror 620 is shown. The two-dimensional MEMS mirror 620 is similar to a structure using two electrically controlled mirrors. By deflecting the laser beam in two-axis directions, it can perform raster scanning. Therefore, it can be used as a device for generating a linear illumination area 13 for realizing a nuclear line image. The laser scanner 60 using a two-dimensional MEMS mirror 620 can be miniaturized compared to an electrically controlled mirror and has the advantage of being low-priced. Figure 24 to Figure 26 In FIG. 6 , the laser beam 90 emitted from the laser light source 610 travels in the Z-axis direction and is reflected by the mirror 611 whose normal line is inclined at an angle θ1 with respect to the −Z-axis. The reflected light is further reflected by the mirror portion 621 of the two-dimensional MEMS mirror 620 .

[0101] 2D MEMS mirror 620 Fig.24 As shown, the 2D MEMS mirror 620 is composed of a reflector portion 621 for reflecting the laser beam 90, a folding portion 622 for rotating the reflector portion 621 at an angle θy around the Y axis of the figure, and a folding portion 623 for rotating around the X axis at an angle θx. The direction of the laser beam 90 when θy=θx=0 is taken as the optical axis 11 of the laser scanner 60, and the 2D MEMS mirror 620 is tilted so that its normal forms an angle θ1 with the Z axis, so that the optical axis 11 becomes parallel to the +Z axis.

[0102] The 2D MEMS mirror 620 used for raster scanning is generally composed of a high-speed axis and a low-speed axis. The high-speed axis can scan at high speed but cannot perform high-precision angle control, while the low-speed axis is at a low speed but can perform high-precision angle control. The reason why it is difficult to perform high-precision control in the movement around the high-speed axis is that the scanning frequency is matched with the physical resonance frequency to perform high-speed movement. The scanning angle θy around the high-speed axis cannot be controlled by an arbitrary function, and the reciprocating motion is repeated at a constant speed. Figure 24 to Figure 26 In the embodiment, the rotational movement using the folding portion 622 is equivalent to the rotational scanning around the high-speed axis, and the rotational movement using the folding portion 623 is equivalent to the rotational scanning around the low-speed axis. By moving around θy, the laser beam scans at high speed in the X direction to generate a linear illumination area 13. However, in the YZ plane, the laser beam 90 is incident on the reflector portion 621 at an angle, so the linear illumination area 13 obtained by the rotation of the reflector portion 621 around the high-speed axis describes an arc in the Y-axis direction.

[0103] 《3-2》Structure Example 1

[0104] 〈Suppressing distortion on an oblique screen using a free-form surface lens〉

[0105] Fig. 27 This is a diagram for explaining the distortion on the illumination reference plane 14 of an image sensing device of a comparative example (a case where a keystone distortion generating lens is not provided). Fig.28 2 is a diagram for explaining the distortion on the photographing reference plane 24 of the image sensing device of the comparative example (the case where there is no keystone distortion generating lens). Fig.29 This is a diagram showing the trajectory of a light beam on an illumination reference plane of an image sensing device 3 (the overall structure is not shown) according to the third embodiment. Fig.30 FIG. 6 is a diagram showing the trajectory of a light beam on the imaging reference plane of the image sensing device 3 according to the third embodiment. The laser scanner 60 of the image sensing device according to the third embodiment is Figure 24 to Figure 26 Shown.

[0106] In the absence of the trapezoidal distortion generating element 40, the trajectory drawn by the laser beam 90 on the illumination reference plane 14 perpendicular to the optical axis 11, that is, the linear illumination area 13 is Fig. 27 The illumination reference surface 14 is shown in FIG. Fig. 20 (A) is similar to the trajectory of Fig. 27In the figure, the degree of curvature of the arc varies depending on the position in the Y direction, and the radius of curvature becomes smaller as it moves downward. That is, the curvature of the arc in the linear illumination area 13c is larger than that in the linear illumination area 13a. In addition, the swing width in the X direction also becomes smaller as it moves downward. The reason is that in the two-dimensional MEMS reflector 620, the folding portion 622 is located on the inner side of the folding portion 623, and the incident angle into the YZ plane of the reflector portion 621 changes with the rotation angle of θx generated by the folding portion 623 to become (θ1+θx). In addition, Figures 27 to 30 The point in Fig. 20 Similarly, the points in (A) represent the arrival points of the laser beam 90 on the illumination reference plane 14 when θx and θy are changed in 1 degree increments. For example, the upper end of the linear illumination area 13, i.e., the point near the linear illumination area 13a, is the arrival point of the laser beam when θx is kept fixed at -3° and θy is swung from -5° to +5°.

[0107] and Fig. 20 Similarly to the case (B), when there is no trapezoidal distortion generating element 40, the arrival point of the laser beam on the shooting reference plane 24 which is perpendicular to the optical axis 21 but inclined to the optical axis 11 and the linear illumination area 13 are Fig.28 Shown. Fig.28 in Fig. 27 In addition to the distortion on the arc shown, there is also a trapezoidal distortion caused by the inclination of the imaging reference plane 24. In addition, the trapezoidal distortion appears as a narrowing of the interval between points in the Y direction as it progresses toward the -X direction.

[0108] In the case of a 2D MEMS mirror, not only trapezoidal distortion but also complex distortion caused by the 2D MEMS mirror itself is added, so it is difficult to properly correct the distortion on the inclined shooting reference plane 24 in an element of a simple shape such as a wedge-shaped prism, which is an example of the trapezoidal distortion generating element 40 in Embodiment 1. Therefore, it is preferable to use a free-form surface lens as a trapezoidal distortion generating element. As an example of a function shape representing a free-form surface, there is the following formula (1).

[0109] [Formula 1]

[0110]

[0111] Here, Z(x, y) is the displacement of the surface in the coordinates (x, y), which is expressed as an N-degree polynomial in two variables x and y. The variables are represented by the standardized parameters a and x. i y j The coefficient k i、jHere, the free-form surface shape was optimized with N=6 times. The optimized illumination reference plane 14, the arrival position of the laser beam per 1 degree on the imaging reference plane 24 and the linear illumination area 13 were Fig.29 and Fig.30 Shown separately.

[0112] Figure 31 to Figure 33 1 and 2 are perspective views, side views, and plan views schematically showing the structure of a laser scanner 60 including a free-form surface lens of an image sensing device 3 according to the third embodiment. Figure 31 to Figure 33 is Figure 24 to Figure 26 In FIG. 6 , a free-form surface lens 70 is inserted after the two-dimensional MEMS mirror 620. The free-form surface lens 70 is a free-form surface whose first surface 71 and second surface 72 are both expressed by equation (1).

[0113] Fig.34 and Fig.35 1 is a diagram showing cross-sectional profiles of a first surface 71 and a second surface 72 of a free-curved surface lens 70 of an image sensing device 3 according to a third embodiment. Fig.34 and Fig.35 The cross-sectional profiles in the planes passing through the optical axes of the first surface 71 and the second surface 72 are shown respectively. The solid line shows the SAG amount [mm] in the X direction, and the dotted line shows the SAG amount in the Y direction. The SAG amount is the amount of cutting in the direction parallel to the optical axis of the lens. Fig.34 and Fig.35 The curvature in the X and Y directions is opposite, indicating that the surface is a saddle shape. In addition, the X and Y directions are asymmetrical, indicating that Fig.35 The asymmetrical distortion shown is Fig.30 When doing correction in that way, both XY need to use asymmetric free-form surface lenses. Fig.32 As shown, the free-form surface lens 70 is installed by rotating clockwise at an angle φ in the YZ plane. When θy=θx=0, the incident light and the outgoing light (i.e., the optical axis 11) of the laser beam 90 to the free-form surface lens 70 are both in the Z-axis direction. The angle formed by the incident light and the normal on the optical axis of the first surface 71 is φ=30°. The inventors of the present application have confirmed through simulation that when φ is in the range of 15° to 45°, the free-form surface shape can be designed so that the Fig. 27 or Fig.28 The difference in curvature of the linear illumination area 13 on the screen (ie, the illumination reference plane 14 and the photographing reference plane 24) is shown.

[0114] As described above, if the asymmetric free-form surface lens 70 is inserted into the 2D MEMS 620, Fig.30As shown, on the optimally designed photographing reference plane 24 in which both X and Y directions are perpendicular to the optical axis 21 of the camera 20, a linear illumination area 13 parallel to the X direction can be generated, and an epipolar line image can be realized.

[0115] 《3-3》Structure Example 2

[0116] <Suppressing distortion caused by angle control of the low-speed axis>

[0117] In the first structural example, the free-form surface lens 70 generates a straight line illumination area 13 on the imaging reference plane 24 which is set obliquely with respect to the optical axis 11. However, in the second embodiment, the rotation angles θx and θy of the mirror unit 621 can be controlled to suppress distortion in the same manner as in the case of two electrically controlled mirrors. The control method and angle function in this case are similar to those used in the embodiment. Fig. 22 and Fig.23 If you can Fig.23 The functions shown in (A) and (B) control the scanning angle θx around the low-speed axis (i.e., around the folding portion 623) and the scanning angle θy around the high-speed axis (i.e., around the folding portion 622). That is, if the values ​​of θx and θy can be properly controlled within a short period of time for depicting the trajectory of a linear illumination area 13, the linear illumination area 13 formed by the linear light beam can be transformed into a straight line parallel to the X direction, thereby realizing a nuclear line image.

[0118] However, since the rotation around the high-speed axis (i.e., around the folding portion 622) in the two-dimensional MEMS mirror uses the resonance phenomenon as described above, it is difficult to set an arbitrary angle function to control it. In this case, if the scanning angle θx around the low-speed axis (i.e., around the folding portion 623) can be controlled, the trajectory of the laser beam on the imaging reference plane 24, that is, the linear illumination area 13, can be set to be parallel to the X direction.

[0119] Fig.36 1 is a diagram showing the trajectory of the laser beam on the imaging reference plane 24 obtained by controlling the low-speed axis of the two-dimensional MEMS mirror of the image sensor device 3 according to the third embodiment. Fig.37 represents a stripe pattern 81 when the laser beam is turned on and off at equal time intervals, Fig.38 An example of a vertical stripe pattern 82 perpendicular to the vertical direction is created by controlling the on and off time of the laser beam. In this case, the linear illumination area 13 on the imaging reference plane 24 and the arrival point of the laser beam every 1° become Fig.36 , Fig.37 , Fig.38 In the figure, the length of the linear illumination region 13 decreases as it goes from the top to the bottom, but all of them are parallel to the X direction. In the above case, the epipolar line image can also be realized.

[0120] For example, as a 3D sensing method, a stripe pattern projection method may be used. The stripe pattern projection method is a method in which a vertical stripe pattern is projected onto a 3D object, the stripe pattern on the 3D object is photographed from an oblique angle, and the 3D shape is restored based on the degree of distortion of the stripe pattern. This method is known as one of the active stereo methods. Fig.36 In the case of the trapezoidal illumination area shown in FIG. 1 , if the laser is turned on and off at equal intervals, the result is Fig.37 The longitudinal stripe pattern 81 shown in FIG. 8 is a pattern in which the angle from the Y axis increases as the distance from the X direction increases. The stripe pattern projection method can also be implemented by the above-mentioned pattern, but since the projected stripe pattern is not parallel lines at equal intervals, errors are easily generated.

[0121] When creating a stripe pattern in which all lines are perpendicular to the longitudinal direction, it is sufficient to control the time interval between the on and off of the laser. If the time interval between the on and off of the linear illumination area 13 is controlled to be longer as it moves from the top to the bottom, the result is Fig.38 The vertical stripe pattern 82 shown in the figure. If the vertical stripe pattern 82 as described above is used, it is possible to reduce errors in 3D sensing performed by the stripe pattern projection method. In addition, since the epipolar line image is realized, when photographing a metallic glossy surface, etc., it is possible to perform 3D sensing with less error due to less influence caused by reflected stray light.

[0122] 《3-4》Structure Example 3

[0123] 〈Suppressing distortion by controlling the angle of free-form lens and reflector〉

[0124] In the above, the case where the distortion is suppressed on the imaging reference plane 24 inclined to the optical axis 11 by using a free-form surface lens and the case where the distortion is suppressed by controlling the scanning angle θx around the low-speed axis (i.e., around the folding portion 623) are described, but the case where both are used (a hybrid method) is also conceivable. For example, if a free-form surface lens is used Figure 31 to Figure 35As described above, increasing the SAG amount and forming a complex shape can easily cause errors in the surface shape. In addition, the free-form surface lens is arranged at an inclination relative to the optical axis, so it is also easy to cause alignment errors. If the above-mentioned assembly errors occur, it is considered that the linear illumination area 13 on the shooting reference plane 24 will be slightly distorted from the straight line. The slight distortion from the straight line as described above can be corrected by controlling the angle of the reflector. The angle control of the reflector part in this case can reduce the correction amount of the scanning angle θx around the low-speed axis (i.e., around the folding part 623) compared to the case where the distortion is suppressed only by controlling the reflector angle. In order to correct the assembly error, after assembling the laser scanner 60, the irradiation pattern on the screen is measured, and the reflector angle is controlled so that the deviation from the design value is corrected.

[0125] In addition, when attempting to suppress distortion only by controlling the mirror angle, even if the scanning angle θx is controlled around the low-speed axis (i.e., around the folding portion 623), a certain degree of large acceleration and deceleration is required, and in terms of the performance of the 2D MEMS, sufficient angle control may not be performed and distortion cannot be suppressed. However, in the hybrid method, the angle of the mirror portion 621 can be controlled by a smaller force, and the control accuracy is improved, so that a linear illumination area 13 with sufficient parallelism in the epipolar line image can be obtained.

[0126] 《4》Implementation Method 4

[0127] Fig.39 1 is a top view schematically showing the main structure of the image sensing device 4 according to Embodiment 4. In Embodiment 1, the trapezoidal distortion generating element 40 is arranged in front of the laser scanner 10 (i.e., on the projection side), but in Embodiment 4, the trapezoidal distortion generating element 80 is arranged in front of the camera 20 (i.e., on the imaging side). The trapezoidal distortion generating element 80 has a function of making the extension direction of the linear imaging area 23 and the extension direction of the linear illumination area 13 on the illumination reference plane 14 close to each other.

[0128] exist Fig.39 In the embodiment 4, the optical axis 21 of the camera 20 is inclined at an angle θ relative to the Z direction. A trapezoidal distortion generating element 80 is inserted into the imaging reference plane 24 perpendicular to the optical axis 21, so that trapezoidal distortion occurs, but the distortion is corrected on the illumination reference plane 14 perpendicular to the laser scanner 10. That is, in the fourth embodiment, the reference plane for aligning the linear illumination area 13 in the X direction is the illumination reference plane 14. In the above case, if the linear imaging area 23 of the camera 20 and the linear illumination area 13 are synchronized and scanned, the linear illumination area 13 of the camera 20 is synchronized with the linear illumination area 13. Figure 6Similarly to the operation on the imaging reference plane 24 in (E), the illumination reference plane 14 can be scanned in a state where the linear imaging region 23 and the linear illumination region 13 continuously overlap (preferably always overlap) (ie, to realize an epipolar line image).

[0129] By inserting the trapezoidal distortion generating element 80, the optical axis 21 intersects the optical axis 11, thereby obtaining Fig.39 The sensing area 25 and Fig.11 and Fig.12 This effect is expanded compared to the sensing area of ​​the image sensing device of the comparative example shown.

[0130] In addition, regarding the contents other than the above, the fourth embodiment is the same as the first embodiment.

[0131] Description of the label

[0132] 1 to 4 image sensing devices, 10, 50, 60 laser scanner (illumination device), 11 optical axis, 12 full laser scanning range, 13 linear illumination area, 14 illumination reference plane (illumination screen), 20 camera, 21 optical axis, 22 full shooting range, 23 linear shooting area, 24 shooting reference plane (shooting screen), 25 sensing area, 30 control circuit, 40 trapezoidal distortion generating element, 70 free-form surface lens, 80 trapezoidal distortion generating element, 90 laser beam (light beam), 110, 510, 610 laser light source (light source), 111, 611 reflector, 113 electrically controlled reflector (scanning optical part), 211 electrically controlled reflector (first scanning optical part), 212 electrically controlled reflector (second scanning optical part), 620 2D MEMS reflector, X horizontal direction (first direction), Y vertical direction (second direction).

Claims

1. An image sensing device, characterized in that: have: An illumination device comprising a light source for emitting a light beam, and an illumination optical system for scanning an illumination area projected by the light beam and a linear illumination area extending linearly in a first direction on a virtual reference plane in a second direction orthogonal to the first direction; a camera that performs an imaging operation of scanning an imaging area, that is, a linear imaging area, which extends linearly in the first direction on the reference plane, in the second direction; as well as a control circuit for controlling the action of the lighting device and the shooting action of the camera so that the linear lighting area and the linear shooting area are continuously repeated on the reference plane, The optical axis of the illumination device and the optical axis of the camera are non-parallel to each other and intersect on the reference plane.

2. The image sensing device according to claim 1, wherein: The illumination optical system comprises: a beam magnifying optical element for generating an amplified beam obtained by expanding the light beam emitted from the light source in the first direction; and The scanning optical unit scans the linear illumination area formed on the reference surface by the amplified light beam in the second direction.

3. The image sensing device according to claim 2, characterized in that: It also has a trapezoidal distortion generating element, which is arranged in front of the lighting device. The trapezoidal distortion generating element has a function of bringing the extending direction of the linear illumination area closer to the extending direction of the linear imaging area on the reference plane.

4. The image sensing device according to claim 3, characterized in that: The trapezoidal distortion generating element is a free-form surface lens which is asymmetric in the first direction and asymmetric in the second direction.

5. The image sensing device according to claim 2, wherein: It also has a trapezoidal distortion generating element, which is arranged in front of the camera. The trapezoidal distortion generating element has a function of bringing the extending direction of the linear imaging area closer to the extending direction of the linear illumination area on the reference plane.

6. The image sensing device according to claim 1, wherein: The illumination optical system comprises: a first scanning optical unit for forming the linear illumination area by scanning the light beam emitted from the light source in the first direction; as well as The second scanning optical unit scans the linear illumination area in the second direction.

7. The image sensing device according to claim 1, wherein: The illumination optical system performs a first scan for forming the linear illumination area by scanning the light beam emitted from the light source in the first direction, and a second scan for drawing the linear illumination area in the second direction.

8. The image sensing device according to claim 7, characterized in that: The illumination optical system is a 2-dimensional MEMS mirror.

9. The image sensing device according to claim 8, characterized in that: The scanning in the first direction is performed by controlling the two-dimensional MEMS mirror which is rotated around a high-speed axis. The scanning in the second direction is performed by controlling the two-dimensional MEMS mirror to rotate around the low-speed axis.

10. The image sensing device according to claim 8 or 9, characterized in that: The scanning angle control of the scanning in the first direction and the scanning angle control of the scanning in the second direction are performed using a pre-calculated angle function.

11. The image sensing device according to any one of claims 7 to 10, characterized in that: A free-form surface lens is also provided, and the free-form surface lens is arranged in front of the lighting device.

Images