Projection device, detection device, and random dot pattern design method

The projection device and method address the challenges of accurate position identification and computational load in structured light methods by projecting a random dot pattern with minimum dot distance and using a triangular polygon mesh, enhancing accuracy and efficiency.

JP7593131B2Active Publication Date: 2024-12-03DAI NIPPON PRINTING CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2021008071
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-01-21
Publication Date
2024-12-03
Estimated Expiration
2041-01-21

AI Technical Summary

Technical Problem

Existing structured light methods using random dot patterns struggle with accurate position identification and high computational load due to overlapping autocorrelation values and excessive processing requirements.

Method used

A projection device and method that projects a random dot pattern with predetermined size centered at square lattice intersections, ensuring a minimum distance between dots and using a triangular polygon mesh to facilitate accurate position identification and reduce computational load.

Benefits of technology

Enables accurate position identification and reduces calculation load by ensuring unique autocorrelation values and efficient processing through the use of a triangular polygon mesh and random dot pattern design.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007593131000002
    Figure 0007593131000002
  • Figure 0007593131000003
    Figure 0007593131000003
  • Figure 0007593131000004
    Figure 0007593131000004
Patent Text Reader

Abstract

To provide a projection device, a detection device, and a method for designing a random dot pattern capable of specifying an accurate position and reducing an arithmetic load.SOLUTION: A projection part 10 is a projection device which projects a random dot pattern. The random dot pattern is a pattern in which one dot is randomly arranged in each arrangement area having a predetermined size centered at an intersection point position of a square lattice arranged at regular intervals n. The shortest distance between dots is arranged so as to be equal to or more than a predetermined interference distance d.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a projection device, a detection device, and a method for designing a random dot pattern. [Background technology]

[0002] There is a known technology in which a detection light is projected onto an object, an image is taken, and distance information to each position on the object is obtained from the obtained image data, thereby creating distance distribution data, also known as shape information or a three-dimensional map (see Patent Document 1). The method disclosed in Patent Document 1 uses a structured light method in which a predetermined pattern of detection light is projected from a light-projecting unit and distance is detected from changes in the position and shape of the pattern on the target object.

[0003] In the structured light method, for example, a random dot pattern is projected, and distance distribution data is calculated using the positional deviation of each dot. Here, it may seem difficult to identify the position of these random dots because they are randomly arranged, but the position of the random dots can be identified with high accuracy by calculating the autocorrelation value.

[0004] However, even if the dot pattern is random, it is not possible to specify the position with high accuracy regardless of the dot pattern. For example, if a specific randomly arranged dot pattern is repeatedly arranged, there will be multiple positions where the same autocorrelation value is obtained, making it impossible to specify the position accurately.

[0005] Furthermore, if the number of dots in a random dot pattern is large, the computation load becomes excessive, which may make high-speed processing difficult. [Prior art documents] [Patent documents]

[0006] Patent Document 1 Japanese Patent Application Laid-Open No. 2010-504522 SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] An object of the present invention is to provide a projection device, a detection device, and a method for designing a random dot pattern that can specify an accurate position and reduce a calculation load. MEANS FOR SOLVING THE PROBLEMS

[0008] The present invention solves the above problems by the following means. For ease of understanding, reference numerals corresponding to embodiments of the present invention are attached for explanation, but the present invention is not limited thereto.

[0009] A first invention is a projection device (10) that projects a random dot pattern, wherein the random dot pattern is a pattern in which one dot is randomly arranged in each of arrangement regions of a predetermined size centered on the intersection positions of a square lattice arranged at equal intervals with an interval n, and the shortest distance between each dot is arranged to be equal to or greater than a predetermined interference distance d. The projection device (10) is characterized by this.

[0010] A second invention is the projection device (10) according to the first invention, wherein the arrangement region is a square region of 2·m×2·m centered on the intersection position, with a predetermined value indicating a distance being m, and the relationship between the interval n and the predetermined value m satisfies m < n / 2. The projection device (10) is characterized by this.

[0011] A third invention is the projection device (10) according to the first invention or the second invention, wherein the random dot pattern is arranged such that adjacent dots can be continuously connected by a straight line with the vertex positions of a triangle, the straight lines forming the sides of the triangle do not intersect in the middle of the sides, and the triangles can be arranged in close contact. The projection device (10) is characterized by this.

[0012] The fourth invention is a detection device (1) comprising a projection device (10) described in any one of the first to third inventions, and an imaging unit (20) that images a random dot pattern projected by the projection device (10).

[0013] The fifth invention is the detection device (1) according to the fourth invention, further comprising an arithmetic unit that uses an index list of polygon mesh data configured with the intersection positions of the square lattice as the vertex positions of a triangle as the index list of each dot of the random dot pattern.

[0014] The sixth invention is a method for designing a random dot pattern projected onto an object using a projection device (10), the method comprising: an arranging step of randomly arranging one dot in each of arrangement regions of a predetermined size centered on the intersection positions of a square lattice arranged at equal intervals with an interval n; a confirming step of confirming whether the shortest distance between the dots arranged in the arranging step is separated by a predetermined interference distance d or more; and a rearranging step of moving the positions of the dots whose shortest distance between the dots is not separated by the predetermined interference distance d or more according to the result of the confirming step.

[0015] The seventh invention is the method for designing a random dot pattern according to the sixth invention, wherein the arrangement region is a square region of 2·m×2·m centered on the intersection position, with a predetermined value indicating distance being m, and the relationship between the interval n and the predetermined value m satisfies m < n / 2.

Advantages of the Invention

[0016] According to the present invention, it is possible to provide a projection device, a detection device, and a method for designing a random dot pattern capable of specifying an accurate position.

Brief Description of the Drawings

[0017] [Figure 1] 1 is a diagram illustrating an outline of a configuration of an embodiment of a three-dimensional measurement apparatus 1 according to the present invention and distance calculation. [Diagram 2] FIG. 13 is a diagram illustrating an example of a dot pattern. [Diagram 3] 11A and 11B are diagrams illustrating a method for detecting an area that matches a dot pattern in a specific rectangular area. [Figure 4] FIG. 13 is a diagram illustrating a method for generating a dot pattern. [Diagram 5] FIG. 13 is a diagram showing a triangular polygon mesh set in a dot pattern before being randomly arranged. [Figure 6] FIG. 6 is a diagram showing a triangular polygon mesh set in a case where the dot pattern in FIG. 5 is rearranged to create a random dot pattern. [Figure 7] FIG. 13 is a diagram that allows visual confirmation of a triangular polygon mesh when a randomly arranged dot pattern of the present invention is projected onto a hand. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings.

[0019] (Embodiment) FIG. 1 is a diagram for explaining the configuration of an embodiment of a three-dimensional measurement device 1 according to the present invention and an overview of distance calculation. Note that the figures shown below, including FIG. 1, are schematic diagrams, and the size and shape of each part are exaggerated or omitted as appropriate to facilitate understanding. In the following description, specific values, shapes, materials, etc. are given, but these can be changed as appropriate. The three-dimensional measuring device 1 of this embodiment includes a light projecting unit (projection device) 10, a camera 20, and a calculation unit (not shown). The camera 20 includes an image sensor unit 21 and a photographing lens 22. The three-dimensional measuring device 1 of this embodiment has a single-wavelength laser light source in any band of ultraviolet light, visible light, near-infrared light, or far-infrared light, and employs a structured light method in which a pattern light is projected by projecting light pulses in a circular or rectangular shape, the pattern light projected onto an object is optically observed, and a distance is calculated from the observed parallax of the pattern light using trigonometry.

[0020] The light projection unit 10 includes a light projection lens 10a, a single-wavelength laser light source, and a pattern generating unit for projecting a dot pattern as described above, and projects a randomly arranged dot pattern (detection light). The pattern generating unit may be, for example, a diffractive optical element (DOE). The image sensor unit 21 is a camera equipped with an image pickup element that receives the dot pattern projected by the light projecting unit 10 onto the detection target.

[0021] In the state shown in Figure 1, the following relationships hold due to the similarity of triangles. Here, the distance between the light projecting lens 10a and the photographing lens 22 is the base length d, the distance from the light projecting lens 10a to the subject is Z, the focal length of the light receiving lens is f, and the position of the subject projected on the image sensor unit is x. Z=d f / x Z min =d f / x max In addition, the number of horizontal pixels of the image sensor is n x The horizontal field of view (FoV) of the camera is defined as θ x If so, the following relationship holds: x max =n x / 2 tan(θ x / 2)=x max / f As described above, the distance calculation unit 34 can perform distance calculations by optical trigonometry.

[0022] A pattern projector can be used as the light projecting unit that projects the random dot pattern. By arranging the pattern projector and the imaging camera at horizontally offset positions as shown in Figure 1, the random dot pattern projected onto the measurement target is always observed at a horizontally offset position according to the distance due to the triangular similarity relationship.

[0023] FIG. 2 is a diagram showing an example of a dot pattern. On the dot pattern shown in Fig. 2, there are only one area that matches a dot pattern in a specific rectangular area, not multiple areas. The method of creating the dot pattern will be described later. The autocorrelation characteristic of the random dot pattern of this embodiment is sharp, and it overlaps with itself only in the case of a 0 shift, and has the property that the correlation value drops sharply if it is shifted even by one pixel. The cross-correlation between the partial pattern of the dot pattern and the original pattern also has a sharp correlation, and overlaps only in the case of a 0 shift, but otherwise the cross-correlation value shows a low value. The dot pattern projected onto the measurement target is always observed at a horizontally shifted position depending on the distance due to the triangular similarity relationship.

[0024] FIG. 3 is a diagram illustrating a method for detecting an area that matches a dot pattern in a specific rectangular area. To detect an area that matches a dot pattern in a specific rectangular area, first quantization is performed by setting the dots in the dot pattern to +1 and the non-dots to -1. Then, the pixel values ​​in the specific rectangular area are multiplied and accumulated using the following formula. Here, the correlation value representing the similarity is φ j Let us assume that. The pixels of the template image in the specific rectangular area to be extracted are quantized according to the presence or absence of a dot pattern, and the vector value Q i Let us assume that. The pixels of the image of a specific rectangular area to be compared and the correlation to be determined are quantized based on the presence or absence of a dot pattern, and a vector value P i+j where n is the number of pixels in the rectangular area.

[0025]

number

[0026] By calculating the cross-correlation between a specific rectangular area pattern and the captured horizontal dot pattern, the pixel phase position for distance measurement using the optical triangulation method shown in Figure 1 can be obtained from the peak position of the correlation value.

[0027] Next, the dot pattern projected by light projector 10 of three-dimensional measuring device 1 of this embodiment will be described in more detail. The dot pattern projected by the light-projecting unit 10 in this embodiment is generated by generating random screen coordinates using random numbers, and if the dot does not overlap within a certain distance with an already placed dot, the dot is placed as is, and if the dot overlaps with an already placed dot, the next random number is obtained and the dot placement is repeated.

[0028] FIG. 4 is a diagram for explaining a method for generating a dot pattern. In the example of FIG. 4, only nine dots, dots D1 to D9, are shown, but in reality many more dots are arranged. First, as shown in the figure, dots D1 to D9 are arranged in a square with a pitch of n. Next, each dot is rearranged (moved) in sequence, but here, an example will be described in which dot D5 is moved. Dot D5 is located at the intersection of a square lattice that is equally spaced at intervals n.

[0029] Centering on the position of dot D5 before movement, the position after movement is determined by a random number. At this time, the area where movement can be made is called the placement area. The hatched placement area PA in FIG. 4 is the placement area of dot D5. The placement area PA has a predetermined size centered on dot D5 (the intersection position of the square lattice). Specifically, the placement area PA is a 2·m×2·m square area centered on dot D5 (the intersection position), with a predetermined value indicating the distance being m, and the relationship between the interval n and the predetermined value m satisfies m < n / 2.

[0030] Under the restriction of movement within this placement area PA, the movement destination (relocation destination) is determined randomly to tentatively determine the movement destination (placement process). For example, dot D5 in FIG. 4 is relocated to the position of dot D5B. For the relocated dot D5B, it is determined (confirmed) whether the shortest distance to other dots is equal to or greater than a predetermined interference distance d (confirmation process). If the distance between the relocated dot D5B and another dot (dot D4 in the example of FIG. 4 is the closest) is smaller than the interference distance d, it is determined that the relocation position of dot D5B is inappropriate, and this is cancelled, and a new operation of randomly determining the movement destination (relocation destination) under the restriction of movement within the placement area PA (relocation process) is performed. Also, if the distance between the relocated dot D5B and another dot (dot D4 in the example of FIG. 4 is the closest) is equal to or greater than the interference distance d, it is determined that the relocated dot D5B is appropriate, and this position is determined, and the relocation operation of the next dot is performed in the same manner as above. By repeating the above operations, if finally the shortest distance between all dots becomes equal to or greater than the interference distance d, the random dot pattern arrangement in this embodiment is completed. Note that when using a diffractive optical element for the projection of the dot pattern, for the dot positions by the zero-order light, since they cannot be moved, it is advisable to fix them.

[0031] FIG. 5 is a diagram in which a triangular polygon mesh is set in the dot pattern before random arrangement. FIG. 6 is a diagram showing a triangular polygon mesh set in the case where the dot pattern in FIG. 5 is rearranged to create a random dot pattern. With the above-mentioned light spot position arrangement method, even after all the arrangements are completed, there is no change in the vertex numbers and the vertex relationships of the triangular polygons that make up the mesh, as shown in Fig. 6. It is only necessary to modify the x and y coordinates of the vertices. Therefore, in this embodiment, a triangular polygon mesh is constructed as shown in Fig. 5 with dots arranged on a square lattice, and after the dots are rearranged, the coordinates of the vertices are modified, so that a dot arrangement in which the straight lines that form the sides of the triangles do not intersect in the middle of the sides and the triangles can be arranged closely can be obtained simply and reliably, and a triangular polygon mesh without gaps or overlaps can be constructed into a randomly arranged dot pattern. Therefore, in the calculation unit 30 of this embodiment, an index list of polygon mesh data configured with the intersection positions of a square lattice as the vertex positions of triangles is used as an index list for each dot of the random dot pattern created by rearrangement.

[0032] FIG. 7 is a diagram that allows visual confirmation of a triangular polygon mesh when the randomly arranged dot pattern of the present invention is projected onto a hand. In the dot pattern projected by the light-projecting unit 10 of this embodiment, a mesh pattern such as that shown in FIG. 7 is not projected. However, in the processing after photographing the dot pattern, since each dot is related to each other by a mesh, in terms of data processing, it is possible to treat it as a mesh model such as that shown in FIG. 7.

[0033] As shown in Figure 7, the depth value (distance) of each light point of a random dot pattern projected onto an object can be determined using trigonometry through cross-correlation processing between a partial pattern, which is a small rectangular area containing the surrounding light points, and the original pattern. In addition, the depth values ​​within each triangle are converted to the 3D world coordinate system for each dot based on the screen coordinates (x, y) and depth value z, and the depth values ​​corresponding to all pixels within the triangle can be calculated at high speed using the 3D graphics polygon fill (triangle filling) function as mesh data. Because it is triangular polygon mesh data, the 3D graphics pipeline can be used, making the calculation load very light. Depth data is obtained by adding the depth value z to the screen coordinates (x,y) of the mesh data, making each vertex data (x,y,z), and filling the mesh data with polygon fill.

[0034] As described above, according to this embodiment, the computation load associated with the cross-correlation calculation can be reduced by the ratio of the number of light points of the random dot pattern to the total number of pixels that make up the screen. In addition, 3D mesh data, point cloud data, and depth values ​​can be obtained simultaneously. Furthermore, the filling process of the 3D mesh data can be performed in a 3D graphics pipeline, for example, by using the GPU built into a smartphone. The present invention is not limited to the above-described embodiment. [Explanation of symbols]

[0035] 1. 3D measuring device 10 Light projector 10a Projection lens 20 Camera 21 Image sensor section 22 Photographing lenses 30 Arithmetic section

Claims

1. A projection device for projecting a random dot pattern, comprising: The random dot pattern is a pattern in which one dot is randomly rearranged in each of arrangement areas of a predetermined size centered on dots before rearrangement that are arranged in a square at equal intervals n, The dots are arranged so that the shortest distance between each other is equal to or greater than a predetermined interference distance d. the arrangement area is a 2 m × 2 m square area centered on the dot position before the rearrangement, where m is a predetermined value indicating a distance, The relationship between the interval n and the predetermined value m is as follows: m<n / 2 To satisfy the following: A projection device comprising:

2. 2. The projection device according to claim 1, The random dot pattern is arranged so that adjacent dots can be connected by straight lines continuously with each dot being the vertex of a triangle, and the straight lines forming the sides of the triangles do not cross midway, and the triangles can be arranged in close contact with each other. A projection device comprising:

3. A projection device according to claim 1 or 2; an imaging unit that images the random dot pattern projected by the projection device; A detection device comprising:

4. 4. The detection device according to claim 3, a calculation unit that uses an index list of polygon mesh data in which the positions of the dots before rearrangement, which are arranged in a square at equal intervals of n, are configured as vertex positions of a triangle, as an index list of each dot of the random dot pattern; A detection device comprising:

5. A method for designing a random dot pattern to be projected onto an object using a projection device, comprising the steps of: an arrangement step of randomly arranging one dot in each of arrangement areas of a predetermined size centered on the dots before rearrangement that are arranged in a square at equal intervals of n; a confirmation step of confirming whether or not the shortest distance between the dots arranged in the arrangement step is equal to or greater than a predetermined interference distance d; a rearrangement step of moving the positions of dots whose shortest distance between dots is not equal to or greater than a predetermined interference distance d based on the result of the confirmation step; A method for designing a random dot pattern comprising: the placement area is a 2 m × 2 m square area centered on the dot position before rearrangement, where m is a predetermined value indicating a distance, The relationship between the interval n and the predetermined value m is as follows: m<n / 2 To satisfy the following: A method for designing a random dot pattern comprising:

Citation Information

Patent Citations

  • Methods and systems for acquiring 3D models

    JP2010504522A

  • Measurement system and measurement processing method

    JP2011007576A

  • Diffractive-optical element and measuring device

    JP2012058729A

  • Image process device and image process method

    JP2014002033A

  • Control apparatus, robot, and control method

    JP2015222220A