Laser scanning method and information processing system

Abstract

This invention provides a method for scanning a laser to generate three-dimensional information of an object without requiring high-precision timing control. [Solution] The scanning mechanism 10 scans the object with a laser according to pattern information indicating a projection pattern. The projection pattern indicated by the pattern information includes a clock line in which bright areas irradiated by the laser and dark areas not irradiated by the laser are arranged alternately, and a data line adjacent to the clock line in which data patterns representing numerical values ​​are arranged according to the arrangement of bright / dark areas. The data line has a sequence of data patterns in which, when a predetermined number of consecutive data patterns are viewed as one data, the data are different from each other. In the projection pattern, all bright areas are connected.

Description

Technical Field

[0001] The present disclosure relates to a technique for scanning a laser to generate three-dimensional information of an object.

Background Art

[0002] One method of obtaining three-dimensional information of an object is the structured light method. The structured light method irradiates an object with stripe or grid pattern light and three-dimensionally images the object based on the reflected light.

[0003] Patent Document 1 discloses an apparatus for measuring the height of a measurement object using laser spot light. In this apparatus, slit light is projected in a time division manner, imaged in one frame, and the height of the measurement object is calculated based on the principle of triangulation.

[0004] Non-Patent Document 1 discloses a technique for projecting pattern light in which a clock line is sandwiched by data lines onto an object using a projector for high-speed 3D shape sensing.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Non-Patent Documents

[0006]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] By irradiating an object with a laser and capturing the image with a non-coaxial camera, the three-dimensional position of the laser spot can be measured using triangulation. However, only one point of the laser spot can be measured. Therefore, while the camera is exposed for a long period of time, the laser is scanned across the object. This allows for the projection of a pattern of light from the laser's scanning trajectory, enabling the measurement of the three-dimensional shape of the entire object.

[0008] To measure three-dimensional position using the structured light method, parallax information between the laser and the camera is required, and laser scanning angle information must be embedded in the projection pattern. For example, one could consider using the projection pattern disclosed in Non-Patent Document 1, in which the clock line is sandwiched between data lines. However, to realize such a projection pattern using a pulse irradiation method with raster scanning, highly accurate timing control of the laser pulses is necessary.

[0009] In view of the foregoing, this disclosure provides a method for scanning a laser without requiring high-precision timing control to generate three-dimensional information of an object. [Means for solving the problem]

[0010] A method for scanning a laser over an object to generate three-dimensional information of the object according to one aspect of the present disclosure, wherein a scanning mechanism for scanning the laser receives pattern information indicating a projection pattern, and the scanning mechanism scans the object with the laser according to the pattern information, the projection pattern indicated by the pattern information includes a clock line in which bright areas irradiated with the laser and dark areas not irradiated with the laser are alternately arranged along a first direction, and a data line adjacent to the clock line in a second direction perpendicular to the first direction, and in which data patterns representing numerical values ​​are arranged along the first direction, the arrangement of the bright areas and dark areas comprises a data pattern sequence in which, when a predetermined number of consecutive data patterns in the first direction are viewed as one data, the data are different from each other, and in the projection pattern, all the bright areas are connected. [Effects of the Invention]

[0011] This disclosure enables scanning of a laser without requiring high-precision timing control to generate three-dimensional information of an object. [Brief explanation of the drawing]

[0012] [Figure 1] Example configuration of the information processing system according to this embodiment [Figure 2] Conceptual diagram of three-dimensional position measurement [Figure 3] Diagram illustrating geometric information in three-dimensional position measurement. [Figure 4] (a) and (b) are diagrams illustrating the method for creating projection patterns in this embodiment. [Figure 5] (a) is an example of a projection pattern in this embodiment, and (b) is an example of a camera image. [Figure 6] Example of the operation flow of the information processing system according to this embodiment [Figure 7] Example of the parallax calculation flow in this embodiment [Figure 8] Examples of projection patterns and scanning trajectories in this embodiment [Figure 9] Example of Projection Pattern and Scanning Trajectory in this Embodiment [Figure 10] Example of Projection Pattern and Scanning Trajectory in this Embodiment

Mode for Carrying Out the Invention

[0013] (Overview) In the method for scanning a target object with a laser to generate three-dimensional information of the target object according to an aspect of the present disclosure, a scanning mechanism for scanning the laser scans the target object according to pattern information indicating a projection pattern. The projection pattern indicated by the pattern information includes a clock line in which bright portions irradiated with the laser and dark portions not irradiated with the laser are alternately arranged along a first direction, and a data line adjacent to the clock line in a second direction perpendicular to the first direction, and along the first direction, data patterns representing numerical values are arranged according to the arrangement of the bright portions and the dark portions. The data line includes a data pattern row in which different data are arranged when a predetermined number of the data patterns continuous in the first direction are regarded as one data. In the projection pattern, all the bright portions are connected.

[0014] According to this configuration, the scanning mechanism scans the target object with the laser according to the pattern information. The projection pattern indicated by the pattern information includes a clock line in which bright portions irradiated with the laser and dark portions not irradiated with the laser are alternately arranged along a first direction, and a data line adjacent to the clock line in a second direction perpendicular to the first direction, and along the first direction, data patterns representing numerical values are arranged according to the arrangement of the bright portions and the dark portions. In the projection pattern, all the bright portions are connected. Therefore, the scanning mechanism can scan the laser so as to trace the bright portions in the projection pattern in one stroke while the laser is on. Therefore, laser scanning for generating three-dimensional information of the target object can be realized without requiring high-precision timing control of the laser.

[0015] In the laser scanning method according to the above aspect, the data line may include a first data line adjacent to one side of the clock line in the second direction and a second data line adjacent to the other side of the clock line in the second direction.

[0016] In the laser scanning method according to the above aspect, the scanning mechanism may operate such that the laser continuously traces the bright portion in the projection pattern.

[0017] Thereby, the scanning mechanism does not need to perform timing control of the laser with high precision in order to irradiate the projection pattern.

[0018] In the laser scanning method according to the above aspect, the laser may be a CW (Continuous Wave) laser.

[0019] An information processing system for generating three-dimensional information of an object according to an aspect of the present disclosure includes a scanning mechanism that scans the object with a laser, a control unit that controls the scanning of the laser by the scanning mechanism according to pattern information indicating a projection pattern, and a generation unit that generates three-dimensional information of the object based on a laser scanning trajectory image obtained by an imaging device. The projection pattern indicated by the pattern information includes a clock line in which bright portions irradiated with the laser and dark portions not irradiated with the laser are alternately arranged along a first direction, and a data line adjacent to the clock line in a second direction perpendicular to the first direction, and in which data patterns representing numerical values are arranged along the first direction according to the arrangement of the bright portions and the dark portions. The data line includes a data pattern column in which different data are arranged when a predetermined number of the data patterns continuous in the first direction are regarded as one data. In the projection pattern, all the bright portions are connected.

[0020] In this configuration, the control unit controls the scanning of the laser by the scanning mechanism according to the pattern information. The generation unit generates three-dimensional information of the object based on the laser scanning trajectory image obtained by the imaging device. The projection pattern indicated by the pattern information includes a clock line along a first direction, where bright areas irradiated by the laser and dark areas not irradiated by the laser are arranged alternately, and a data line adjacent to the clock line in a second direction perpendicular to the first direction, where data patterns representing numerical values ​​are arranged along the first direction, with all bright areas connected in the projection pattern. Therefore, the control unit can control the scanning mechanism to scan the laser so that it traces the bright areas in the projection pattern in a single continuous line while the laser is turned on. Thus, three-dimensional information of the object can be generated without requiring high-precision timing control of the laser.

[0021] In the information processing system according to the above embodiment, the imaging device may be provided, and the central optical axis in the scanning of the laser by the scanning mechanism and the optical axis of the imaging device may be non-coaxial.

[0022] (Embodiment) The embodiments will be described in detail below with reference to the drawings.

[0023] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, arrangement and connection configurations of components, steps, and the order of steps shown in the following embodiments are examples only and are not intended to limit this disclosure. Furthermore, among the components in the following embodiments, those not described in the independent claim representing the highest-level concept will be described as optional components.

[0024] Figure 1 shows an example of the configuration of the information processing system according to this embodiment. The information processing system according to this embodiment generates three-dimensional information of an object. The information processing system in Figure 1 comprises a scanning mechanism 10 that scans a laser, a pattern generation unit 20 that generates pattern information showing a projection pattern, an imaging device (camera) 30, a control unit 40 that controls the operation of the scanning mechanism 10 and the imaging device 30, a storage unit 50 that stores geometric information, which will be described later, and a three-dimensional information generation unit 60 that generates three-dimensional information from the output of the imaging device 30.

[0025] Furthermore, the imaging device 30 may be configured separately from the information processing system. In other words, the information processing system may not include the imaging device 30 and may be configured to operate in connection with an external imaging device.

[0026] The scanning mechanism 10 comprises a laser irradiation device 11, a first mirror 12, and a second mirror 13. The control unit 40 controls the laser output of the laser irradiation device 11 and the angles of the first and second mirrors 12 and 13.

[0027] Figure 2 shows a conceptual diagram of three-dimensional position measurement. As shown in Figure 2, the scanning mechanism 10 irradiates the object with laser light. The camera captures an image of the object irradiated with laser light. The optical axis of the camera is not coaxial with the optical axis of the laser light. When the position of the object in the height direction changes (solid line and dashed line), the irradiation position of the laser light in the captured image changes, as shown in the image captured in Figure 2. From this change in irradiation position, the position of the object in the height direction can be calculated.

[0028] Figure 3 is a diagram illustrating geometric information in three-dimensional position measurement. In Figure 3, α is the angle determined by the mirror angle control of the scanning mechanism 10, and β is the angle obtained from the image captured by the camera. b is the distance between the scanning mechanism 10 and the camera 30. From α, β, and b, the distance z to the object can be calculated. Geometric information such as the known length b and the relationship between angle α and the angles of mirrors 12 and 13 is stored in the memory unit 50.

[0029] In this embodiment, three-dimensional position measurement is performed with reference to the technology disclosed in Non-Patent Document 1. Specifically, a projection pattern including a clock line in which bright and dark areas are arranged alternately, and data lines in which data patterns representing numerical values ​​based on the arrangement of bright and dark areas are arranged in synchronization with the clock line, is projected onto the object by laser scanning.

[0030] However, unlike projecting patterned light with a projector, raster scanning with a laser requires switching the laser on and off according to the arrangement of bright and dark areas in the projection pattern. Therefore, high-precision timing control is necessary.

[0031] Therefore, in this embodiment, the projection pattern is generated by continuously scanning the bright areas in a so-called single-stroke manner with the laser turned on. In this case, the scanning mechanism 10 only needs to control the angles of the mirrors 12 and 13, and high-precision timing control of the laser output is unnecessary. For example, a CW (Continuous Wave) laser may be used.

[0032] Figure 4 shows the method for generating the projection pattern in this embodiment. In this embodiment, the data pattern represents a 4-digit binary number by the arrangement of bright and dark areas in a 2x2 grid. The bottom right grid corresponds to the LSB, the bottom left grid to the second bit from the bottom, the top right grid to the third bit from the bottom, and the top left grid to the MSB. In the example in Figure 4(a), the data patterns "0001" (1 in decimal, and so on) "0011" (3) "0101" (5) and "1011" (11) are arranged on the data line from left to right, synchronized with the brightness and darkness of the clock line.

[0033] Furthermore, instead of placing arbitrary data patterns on the data lines, constraints are imposed on the data patterns to be placed. That is, the data patterns are placed so that bright areas are continuous on the data line from one bright area to the next on the clock line. As a result, as shown in Figure 4(b), bright areas can be made continuous on the clock line via the data lines from one bright area to the next.

[0034] Here, for example, data patterns are placed on the data lines according to the following rules: The columns corresponding to the odd-numbered bright areas of the clock line, that is, the columns where the laser is scanned in the direction from the clock line toward the data line, are designated as the "left columns". The columns corresponding to the even-numbered bright areas of the clock line, that is, the columns where the laser is scanned in the direction from the data line toward the clock line, are designated as the "right columns".

[0035] The data patterns that can be placed in the left column are those in which the lower right square adjacent to the bright area of ​​the clock line is a bright area, and which can be continuously scanned with a laser in a single stroke from the lower right square to the data pattern in the right column adjacent to it. There are three types of data patterns that satisfy this condition: "0001" (1), "0101" (5), and "1111" (15).

[0036] Furthermore, the data patterns that can be placed in the right column are those in which the lower right square adjacent to the bright area of ​​the clock line is a bright area, and which can be continuously scanned with a laser in a single stroke from the data pattern in the left column adjacent to it to the lower right square. The data patterns that meet these conditions are as follows:

[0037] The left column contains "0001" (1), "0011" (3), and "1111" (15). For the left column, "0101" (5), there are "1011" (11) and "1101" (13). The left column contains "1111" (15), "1011" (11), and "1101" (13). By using these six types of data pattern combinations, uniqueness can be ensured for any two consecutive data patterns in a data pattern sequence.

[0038] Figure 5(a) shows an example of a projection pattern in this embodiment. In Figure 5(a), there is a clock line with bright and dark areas arranged horizontally in the drawing direction, and data lines are arranged adjacent to both sides of the clock line in the vertical direction of the drawing. As explained in Figure 4, by imposing constraints on the arrangement of data patterns in the data lines, the bright areas are continuous from one bright area of ​​the clock line to the next, via the data lines. Therefore, by scanning the laser according to the arrows shown in the figure, the projection pattern can be projected onto the object.

[0039] T1 represents the period in which bright and dark areas alternate on the clock line. On the upper data line of the clock line diagram, the data pattern sequence with period T2 contains the six combinations of data patterns mentioned above. Specifically, from left to right, they are arranged as follows: "0001" (1), "0011" (3), "0001" (1), "1111" (15), "0101" (5), "1011" (11), "0101" (5), "1101" (13), "1111" (15), "1011" (11), "1111" (15), and "1101" (13). When these are represented in decimal for every two consecutive data patterns, they become 1-3, 3-1, 1-15, 15-5, 5-11, 11-5, 5-13, 13-15, 15-11, 11-15, and 15-13. In other words, when two consecutive data patterns are viewed as a single data point, in a data pattern sequence with period T2, the data points are distinct from each other. To put it another way, uniqueness is ensured for any two consecutive data patterns in the data pattern sequence.

[0040] Next, we focus on specific data within the data pattern sequence, for example, the first two data patterns (1-3). As shown in Figure 5(b), parallax can be detected by detecting the data patterns (1-3) in focus from a camera image of an object onto which the projection pattern is projected. In the example above, uniqueness is ensured for two consecutive data patterns in the data pattern sequence. However, this is not the only method for ensuring uniqueness for two consecutive data patterns. For example, we can add the condition that data patterns that can be placed in the left column are excluded from the data patterns that can be placed in the right column. In the example above, since "1111" (15) can be placed in the left column, we exclude "1111" (15) from the right column, and from the six possible combinations, we exclude the combination of "0001" (1) from the left column and "1111" (15) from the right column, leaving us with the remaining five possible combinations. In this case as well, uniqueness can be ensured for two consecutive data patterns in the data pattern sequence.

[0041] Figure 6 shows an example of the operation flow of the information processing system according to this embodiment. As shown in Figure 6, when the information processing system starts operation, the pattern generation unit 20 first generates a projection pattern (S11). This projection pattern is, for example, like the one shown in Figure 5(a). The pattern generation unit 20 sends pattern information indicating the generated projection pattern to the control unit 40. The control unit 40 receives the pattern information and sets the angles of the first and second mirrors 12 and 13 to angles corresponding to the initial position of the projection pattern indicated by the pattern information (S12).

[0042] The control unit 40 sets the imaging device 30 to exposure mode (S13) and controls the laser irradiation device 11 to irradiate the laser (S14). Then, the control unit 40 executes a sequence to change the angles of the first and second mirrors 12 and 13 according to the projection pattern (S15). As a result, the laser scans the object according to the projection pattern. When the angle change sequence is completed, the control unit 40 stops the exposure of the imaging device 30 (S16) and stops the laser irradiation (S17).

[0043] The imaging device 30 outputs the captured image data to the three-dimensional information generation unit 60 (S18). The pattern generation unit 20 outputs the pattern information to the three-dimensional information generation unit 60 (S19). The three-dimensional information generation unit 60 acquires geometric information from the storage unit 50 (S1A).

[0044] The three-dimensional information generation unit 60 calculates disparity information from the projection pattern indicated by the pattern information and the image data acquired by the imaging device 30 (S1B), and generates three-dimensional information of the object from the disparity information and geometric information (S1C).

[0045] Figure 7 shows an example of the disparity calculation flow in this embodiment. As described above, disparity calculation is performed in the three-dimensional information generation unit 60. As shown in Figure 7, the captured image data is scanned sequentially (S21). From the scanned image data, data of the region of interest in the projection pattern is read out (S22). As shown in Figure 5, the region of interest includes clock line data and data line data.

[0046] The three-dimensional information generation unit 60 determines where the data of the region of interest corresponds to in the projection pattern (S23). Here, in the data pattern sequence with period T2 shown in Figure 5, different data are arranged in a sequence, so if the parallax does not exceed period T2, the position of the data of the region of interest in the projection pattern can be identified.

[0047] Once the position of the data in the region of interest within the projection pattern is identified, the three-dimensional information generation unit 60 calculates the parallax (S24). Subsequently, the scanning of the captured image data is terminated (S25).

[0048] <Examples of projection patterns and scan paths> Figures 8 to 10 show examples of other projection patterns and scanning trajectories. Figure 8 is an example of scanning in the horizontal direction of the drawing, while Figures 9 and 10 are examples of scanning in the vertical direction of the drawing.

[0049] In Figure 8, the data pattern sequence PT11 on the data line is arranged from left to right as follows: "0001" (1), "0011" (3), "0001" (1), "1111" (15), "0101" (5), "1011" (11), "0101" (5), "1101" (13), "1111" (15), "1011" (11), "1111" (15), "1101" (13), "0001" (1). When we consider two consecutive data patterns as one data, and express them in decimal, we get 1-3, 3-1, 1-15, 15-5, 5-11, 11-5, 5-13, 13-15, 15-11, 11-15, 15-13, 13-1. In other words, in the data pattern sequence PT11, different data are arranged. Furthermore, it is preferable that the length of the data pattern column PT11 is, for example, greater than or equal to the parallax range corresponding to the measurement distance range.

[0050] Furthermore, Figure 8 shows data pattern sequences PT12 and PT13, which are constrained for folding the laser scan. In the projection pattern shown in Figure 8, all bright areas are connected.

[0051] In Figure 9, the data pattern sequence PT21 on the data line is arranged from left to right as follows: "0111" (7), "1110" (14), "1111" (15), "0101" (5), "0101" (5), "1111" (15), "0111" (7), "0101" (5), "0111" (7). When two consecutive data patterns are considered as one data, and expressed in decimal, this becomes 7-4, 14-15, 15-5, 5-5, 5-15, 15-7, 7-5, 5-7. In other words, the data pattern sequence PT21 contains distinct data. It is preferable that the length of the data pattern sequence PT21 is, for example, greater than or equal to the parallax range corresponding to the measurement distance range.

[0052] Furthermore, Figure 9 shows data pattern sequences PT22 and PT23, which are constrained for folding the laser scan. In the projection pattern shown in Figure 9, all bright areas are connected.

[0053] In Figure 10, data pattern sequences PT31 and PT32 are arranged for reversing the laser scan. However, data pattern sequences PT31 and PT32 are not used for distance measurement. In the projection pattern shown in Figure 10, all bright areas are connected.

[0054] The projection patterns in Figures 8 to 10 each show the scanning trajectory of the laser. As can be seen from the illustrated scanning trajectory, the scanning mechanism 10 only needs to operate so that the laser continuously follows the bright areas in the projection pattern.

[0055] As described above, according to this embodiment, in order to generate three-dimensional information of an object, the scanning mechanism 10 scans the object with a laser according to the pattern information. The projection pattern shown by the pattern information includes a clock line in which bright areas irradiated by the laser and dark areas not irradiated by the laser are alternately arranged along a first direction, and a data line adjacent to the clock line in a second direction perpendicular to the first direction, in which a data pattern representing numerical values ​​is arranged along the first direction by the arrangement of bright and dark areas. In the projection pattern, all bright areas are connected. Therefore, with the laser turned on, the scanning mechanism can scan the laser in a manner that traces the bright areas in the projection pattern in a single stroke. Thus, laser scanning for generating three-dimensional information of an object can be realized without requiring high-precision timing control of the laser.

[0056] In the above embodiment, the data pattern arranged on the data line was assumed to be a 2x2 grid of light / dark areas, but the method of representing the data pattern is not limited to this.

[0057] Furthermore, in the above-described embodiment, in the data line, when two consecutive data patterns are viewed as one data item in the data pattern sequence, the data items are considered to be different from each other. However, the number of data patterns that can be viewed as one data item is not limited to two; for example, there may be three or more.

[0058] The functions realized by the components described herein may be implemented in a circuit or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (Central Processing Units), conventional circuits, and / or combinations thereof, programmed to realize the functions described herein. A processor is considered a circuit or processing circuitry, including transistors and other circuits. A processor may be a programmed processor that executes a program stored in memory.

[0059] In this specification, circuitry, unit, and means are hardware programmed to perform or execute the functions described herein. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to perform or execute the functions described herein.

[0060] If the hardware is a processor that is considered to be a type of circuitry, then the circuitry, means, or unit is a combination of hardware and software used to constitute the hardware and / or processor. [Industrial applicability]

[0061] This disclosure allows for scanning of a laser without requiring high-precision timing control to generate three-dimensional information of an object, and is therefore useful, for example, in high-speed 3D shape sensing. [Explanation of symbols]

[0062] 10 Scanning mechanism 30 Imaging device 40 Control Unit 60. Three-Dimensional Information Generation Department

Claims

1. A method for scanning a laser over an object in order to generate three-dimensional information of the object, A scanning mechanism that scans the laser scans the object according to pattern information indicating a projection pattern. The projection pattern indicated by the aforementioned pattern information is Along the first direction, a clock line is formed in which bright areas illuminated by the laser and dark areas not illuminated by the laser are arranged alternately. The system includes a data line adjacent to the clock line in a second direction perpendicular to the first direction, and in which a data pattern representing a numerical value is arranged along the first direction by the arrangement of the bright and dark parts, The data line comprises a data pattern sequence in which, when a predetermined number of consecutive data patterns in the first direction are viewed as a single data, the data are all different from each other. In the projection pattern, all the bright areas are connected. Laser scanning method.

2. In the laser scanning method according to claim 1, The aforementioned data line is, In the second direction, the first data line adjacent to one side of the clock line, In the second direction, the second data line includes the other side of the clock line adjacent to the second data line. Laser scanning method.

3. In the laser scanning method according to claim 1, The scanning mechanism operates such that the laser continuously traces the bright areas in the projection pattern. Laser scanning method.

4. In the laser scanning method according to claim 1, The aforementioned laser is a CW (Continuous Wave) laser. Laser scanning method.

5. An information processing system that generates three-dimensional information of an object, A scanning mechanism for scanning the aforementioned object with a laser, A control unit controls the scanning of the laser by the scanning mechanism according to pattern information indicating a projection pattern, The system comprises a generation unit that generates three-dimensional information of the object based on a laser scanning trajectory image obtained by an imaging device, The projection pattern indicated by the aforementioned pattern information is Along the first direction, a clock line is formed in which bright areas illuminated by the laser and dark areas not illuminated by the laser are arranged alternately. The system includes a data line adjacent to the clock line in a second direction perpendicular to the first direction, and in which a data pattern representing a numerical value is arranged along the first direction by the arrangement of the bright and dark parts, The data line comprises a data pattern sequence in which, when a predetermined number of consecutive data patterns in the first direction are viewed as a single data, the data are all different from each other. In the projection pattern, all the bright areas are connected. Information processing system.

6. In the information processing system described in claim 5, The imaging device is provided as described above, The central optical axis in the laser scanning by the scanning mechanism and the optical axis of the imaging device are non-coaxial. Information processing system.

Images