Method and system for encoding structured light for reconstructing three-dimensional information

Abstract

A method and system for encoding structured light for reconstructing three-dimensional information is provided. The method includes forming at least three projection images, and all the projection images are aligned to form an encoding pattern, which includes a first type of sub-pattern and a second type of sub-pattern, the first type of sub-pattern and the second type of sub-pattern appear alternately, the first type of sub-pattern appears as stripes in only one projection image, the second type of sub-pattern appears as stripes in at least two projection images, and any two adjacent first type sub-patterns appear as stripes in different projection images (S100). The method can determine the correspondence between the imaging position and the projection position with a smaller number of projection images at high resolution, improve the measurement efficiency of the system, and accurately match the imaging position and the projection position for three-dimensional points with a large depth of field, and the matching robustness is good.

Description

[Technical field]

[0001] The present invention relates to the field of reconstructing three-dimensional information, in particular to a method and system of coding structured light for reconstructing three-dimensional information. [Background technology]

[0002] There are many techniques for reconstructing 3D information of object surfaces in industrial control, medicine, etc. Among them, the lattice fringe projection method based on structured light has been widely studied and applied due to its advantages such as high accuracy, high efficiency and high robustness.

[0003] Such a method mainly involves projecting a specific coded lattice pattern onto the surface of the measured object as a structured light projection image by a projection device, then capturing the structured light modulated and reflected by the object surface with an image sensor, such as a camera, to obtain an imaged image, decoding the imaged image, and finding the correspondence between each imaged position in the decoded pattern and the projected position. Since the positions where the coded pattern is projected onto the object are known, the three-dimensional information of the object's surface can be obtained by triangulation based on the correspondence.

[0004] A commonly used method of encoding structured light is time encoding. In the time encoding method, multiple patterns are projected into the measured space in a chronological order, and an image is formed by a camera for each projection. As shown in Figure 3, four images are assumed, and the code value of the shaded area is set to 1, and the code value of the uncovered area is set to 0. Each projection generates one code value for each image point, and after four projections, a code word (a binary code word with a length of 4 in Figure 3) that corresponds one-to-one with each image point is generated, and the correspondence between the image point and the projection point can be determined based on the binary code word of the image point.

[0005] Gray code is an improvement of binary code and has higher robustness than binary code. To increase the resolution, the number of projection positions is relatively large. Whether it is Gray code or binary code, if the number of projection positions is 2N, at least N projection images are required to ensure the decoding of the image formation.

[0006] Although a large number of projection images contributes to improving the measurement accuracy of the system, a large number of projection images affects the measurement efficiency of the system and complicates the 3D reconstruction process. Therefore, it is necessary to reduce the number of images to be acquired while maintaining the measurement accuracy. Summary of the Invention [Problem to be solved by the invention]

[0007] One of the objects of the present invention is to provide a method and system for coding of structured light for reconstructing three-dimensional information, which is able to overcome the problem of the large number of projected images required by currently adopted coding schemes. [Means for solving the problem]

[0008] The technical solution of the present invention is as follows:

[0009] A method for coding structured light for reconstructing three-dimensional information, comprising the steps of: forming at least three projected images, and a coding pattern formed by aligning all the projected images includes a first type of sub-pattern and a second type of sub-pattern, and the sub-patterns are patterns in which all the projected images are formed at the same projection position; the first type sub-pattern and the second type sub-pattern appear alternately, the first type sub-pattern appears as stripes in only one projected image, the second type sub-pattern appears as stripes in at least two projected images, and any two adjacent first type sub-patterns appear as stripes in different projected images; A method for encoding structured light to reconstruct three-dimensional information.

[0010] In some embodiments, the appearance of stripes in different projection images of any two adjacent first type sub-patterns includes the appearance of stripes in all projection images of the first type sub-patterns in sequence in the coding pattern.

[0011] In some embodiments, the patterns of any two adjacent second type sub-patterns are different.

[0012] In some embodiments, the different patterns of any two adjacent second type sub-patterns include the adjacent second type sub-patterns having different positions at which stripes appear in the projected image or different numbers at which stripes appear in the projected image.

[0013] In some embodiments, the second type of sub-pattern is provided with a third type of sub-pattern, the third type of sub-pattern appearing as stripes in all projected images.

[0014] In some embodiments, the sub-pattern of the first piece is set to the third type of sub-pattern.

[0015] In some embodiments, a basic pattern is determined, and each sub-pattern in the basic pattern has a different pattern, and each sub-pattern of the basic pattern is repeated with the same predetermined period.

[0016] In some embodiments, the coding pattern further comprises a fourth type of sub-pattern, and one fourth type of sub-pattern is inserted between adjacent first type and second type sub-patterns.

[0017] The present invention further provides a system for reconstructing three-dimensional information, the system including: a projection device that generates a plurality of projection images according to the above-mentioned method for encoding structured light for reconstructing three-dimensional information and projects the projection images in chronological order onto a surface of a measured object; an image sensor that focuses the projection images projected onto the surface of the measured object to obtain focused images; and a reconstruction unit that decodes the focused images, determines a correspondence between each imaging position and a projection position, and reconstructs three-dimensional information of the surface of the measured object based on the correspondence. Effect of the Invention

[0018] The method and system of structured light encoding for reconstructing three-dimensional information provided by the present invention provides at least the following beneficial effects:

[0019] 1. The present invention provides a coding pattern with a first type of sub-pattern and a second type of sub-pattern, and has these two types of sub-patterns appear alternately. Compared with the conventional binary code coding / Gray code coding method, the present invention determines the correspondence between the imaging position and the projection position with a higher resolution and a smaller number of projection images, thereby improving the measurement efficiency of the system.

[0020] 2. By sequentially making the first type of sub-pattern appear as stripes in all projected images, and / or by introducing a fourth type of sub-pattern, and / or by appropriately increasing the number of projected images, the present invention can obtain a larger depth of field, and can accurately match the imaging position and the projection position for three-dimensional points with a large depth of field, and the matching robustness is good. [Brief description of the drawings]

[0021] Hereinafter, preferred embodiments will be described clearly and easily with reference to the drawings, to further explain the above-mentioned characteristics, technical features, advantages and embodiments of the method and system for encoding structured light for reconstructing three-dimensional information.

[0022] [Figure 1] 1 is a flowchart of an embodiment of a method for encoding structured light for reconstructing three-dimensional information according to the present invention.

[0023] [Diagram 2] FIG. 1 is a structural schematic diagram of a system for reconstructing three-dimensional information according to the present invention.

[0024] [Diagram 3] FIG. 1 is a logic schematic diagram of a conventional binary code encoding pattern.

[0025] [Figure 4] FIG. 2 is a logic schematic diagram of a structured light coding pattern of the present invention.

[0026] [Diagram 5] FIG. 2 is another logic schematic diagram of a structured light coding pattern of the present invention.

[0027] [Figure 6] FIG. 2 is another logic schematic diagram of a structured light coding pattern of the present invention.

[0028] [Figure 7] FIG. 2 is another logic schematic diagram of a structured light coding pattern of the present invention.

[0029] [Figure 8] FIG. 2 is a schematic diagram of a structured light coding pattern of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] In order to more clearly describe the embodiments of the present invention or the technical solutions in the prior art, the following describes specific embodiments of the present invention with reference to the drawings. It is obvious that the drawings in the following description are only some embodiments of the present invention, and those skilled in the art can obtain other drawings and obtain other embodiments based on these drawings without creative efforts.

[0031] In order to simplify the drawings, each drawing shows only the parts related to the present invention in a schematic manner, and they are not shown as the actual structure of the product. In order to simplify the drawings and make them easy to understand, only one of the components having the same configuration or function is depicted in the drawings in a schematic manner or only one of the components is described. In the text, "one" can represent not only "only one" but also "more than one".

[0032] One embodiment of the present invention, as shown in FIG. 1, is a method for encoding structured light for reconstructing three-dimensional information, comprising: forming N projection images, where N≧3; a coding pattern formed by aligning all the projection images includes a first type of sub-pattern A and a second type of sub-pattern B, and the sub-patterns are patterns in which all the projection images are formed at the same projection position; 1. A method for encoding structured light for reconstructing three-dimensional information, comprising: a step S100 in which a first type of subpattern A and a second type of subpattern B appear alternately, the first type of subpattern A appears as stripes in only one projected image, the second type of subpattern B appears as stripes in at least two projected images, and any two adjacent first type subpatterns appear as stripes in different projected images.

[0033] Specifically, this embodiment is a method for time-encoding structured light. The time-encoding method involves projecting a number of patterns onto the surface of the object to be measured in a chronological order. Each pattern, also called a projected image, is composed of a row of structured light with different light and dark intensities.

[0034] A projection device generally includes a light source and a grating. A grating is an optical device that consists of many parallel slits of equal width. Where there is a slit, the light source passes through the slit to form a single light beam, called a fringe. Where there is no slit, the light source does not pass through and a dark beam is formed. A projection image is recorded on the grating, and a light source projects the projection image on the grating onto the object surface.

[0035] All the projected images are aligned to form a coding pattern, and the coding pattern is divided for each projection position. A pattern formed by all the projected images at elements of the same projection position is called a sub-pattern. In other words, a sub-pattern is a pattern of a coding pattern at the same projection position. As shown in Figure 8, in a coding pattern consisting of three projected images, each horizontal column is a projection image, and each vertical column is a sub-pattern.

[0036] The sub-patterns include a first type of sub-pattern A which is a sub-pattern in which stripes appear in only one projected image, and a second type of sub-pattern B which is a sub-pattern in which stripes appear in at least two projected images.

[0037] In the coding pattern, the first type of sub-pattern A and the second type of sub-pattern B appear alternately.

[0038] For example, as shown in Figure 4, there are six projection points (i.e., projection positions) and three projection images, among which the gray cells represent stripes. As shown in projection image 1, stripes appear at projection points 1-3, but do not appear at other projection points.

[0039] All three projection images have stripes at projection point 1, so the subpattern at projection point 1 is the second type of subpattern B, only projection image 1 has stripes at projection point 2, so the subpattern at projection point 2 is the first type of subpattern A, and projection images 1 and 2 have stripes at projection point 3, so the subpattern at projection point 3 is the second type of subpattern B, and so on. The coding pattern consisting of projection images 1-3 is a pattern in which second type subpatterns B and A appear alternately, and in reality, the coding pattern may be formed by the first type subpatterns A and B appearing alternately, but this embodiment is not limited to this.

[0040] Adjacent first-type subpatterns will have stripes in different projection images. As shown in Fig. 4, the subpatterns at projection points 2, 4, and 6 are all first-type subpattern A, and the subpatterns at projection points 2 and 4 or projection points 4 and 6 are adjacent first-type subpatterns, and it can be seen that the patterns of any two adjacent first-type subpatterns are different, and stripes will appear in different projection images.

[0041] When decoding the above-mentioned coding pattern, in one embodiment, the first type of subpattern A and the second type of subpattern B are distinguished based on the difference between the first type of subpattern A and the second type of subpattern B, and further, based on the difference between adjacent first type of subpatterns A and the appearance rule of the first type of subpattern A, the correspondence relationship between the imaging points (i.e., imaging positions) and the projection points (i.e., projection positions) in all the first type of subpatterns A is determined, and then the second type of subpattern B is identified and the correspondence relationship between the imaging points and the projection points is determined.

[0042] The patterns of the adjacent second type sub-patterns may be the same or different. Being different helps to better determine the position of the second type sub-pattern. For example, taking FIG. 4 as an example, after determining the first type sub-pattern of the projection point 2, it is found that the sub-patterns before and after it are the second type. If the patterns of both are the same, it is not easy to determine whether the number is 1 or 3, but if the patterns of both are different, it is easier to determine the number.

[0043] The difference in the patterns of the adjacent second type sub-patterns can be expressed as the difference in the positions at which stripes appear in the projected image or the difference in the number of stripes appearing in the projected image in the adjacent second type sub-patterns. In short, the greater the difference between the two patterns, the more advantageous it is for identification.

[0044] The second type of subpattern can be further divided into a third type of subpattern and a non-third type of subpattern. A subpattern in which stripes appear in all projected images is called a third type of subpattern. The third type of subpattern is easier to identify when decoding because stripes appear in all projected images.

[0045] The coding pattern obtained by the above method is called the base pattern. To facilitate decoding, a constraint can be added to the base pattern, requiring that the patterns of each sub-pattern in the base pattern are different.

[0046] Assuming that the pixel value of the imaging point with stripes is 1, and the pixel value of the imaging point without stripes is 0, the code words corresponding to each sub-pattern in Fig. 4 are different, so they can be decoded according to a normal method, for example, the code words of the imaging points can determine the corresponding relationship between the imaging points and the projection points. Of course, other decoding methods can be adopted, and this embodiment is not limited thereto.

[0047] Due to the requirement of high resolution, a large number of sub-patterns may be required. The basic pattern is made to appear in a repeating cycle to form a new coding pattern, i.e., each sub-pattern of the basic pattern appears repeatedly in the same predetermined cycle. In this way, there is no need to add new projection images. For 2N sub-patterns, only three projection images can be used to meet the decoding requirements.

[0048] In one embodiment, the sub-pattern of the first piece of the basic pattern is a third type of sub-pattern C. Because the third type of sub-pattern C is easy to identify, the boundary of the basic pattern can be more easily identified.

[0049] As shown in Fig. 5, Fig. 5 is a pattern formed by repeatedly making the pattern in Fig. 4 appear based on the pattern in Fig. 4. For such an encoding method, when decoding, the boundary of the basic pattern can be determined first, and the basic pattern can be decoded using the above method. The boundary of the basic pattern may be specified by the rule of repeated appearance of sub-patterns. For example, as shown in Fig. 5, the boundary of the basic pattern is specified based on the rule of repeated appearance of sub-patterns at projection point 1.

[0050] In one embodiment, the appearance of stripes in adjacent first type sub-patterns in different projection images includes the appearance of stripes in the first type sub-patterns in sequence in all projection images in a coding pattern formed by aligning all projection images, where the appearance of stripes in sequence means that stripes appear in one sequence of all projection images or that stripes appear repeatedly with one sequence of all projection images as a period.

[0051] 5, the first type of sub-pattern makes stripes appear at projection points 2, 4, and 6 in the order of projection images 1, 2, and 3, and then repeats the pattern periodically at projection points 8, 10, and 12. Of course, the first type of sub-pattern may make stripes appear at each projection point in the order of projection images 1, 3, and 2, or in other orders such as 2, 3, and 1.

[0052] In this way, the interval between adjacent first-type sub-patterns in the same projection image can be made larger, thereby improving the accuracy of decoding the first-type sub-patterns, obtaining a larger depth of field, and being applicable to the reconstruction of 3D information of the object surface with large surface depth changes. Otherwise, it is only applicable to application scenes with small depth of field. The depth of field corresponds to the depth change of the object surface, and when the depth change of the object surface is not large, the requirement of the depth of field is low, and conversely, when the depth change of the object surface is large, the requirement of the depth of field is high.

[0053] In order to further increase the depth of field, in addition to the above embodiment, a fourth type of sub-pattern D may be introduced into the coding pattern. The fourth type of sub-pattern D has no stripes in all projected images. It is advisable to insert one fourth type of sub-pattern between adjacent first and second type sub-patterns.

[0054] As shown in Fig. 6, the subpatterns of the projection points 1, 5, 9, and 13 are second-type subpatterns, the subpatterns of the projection points 2, 4, 6, 8, 10, 12, 14, and 16 are fourth-type subpatterns, and the subpatterns of the projection points 3, 7, 11, and 15 are first-type subpatterns. There is one fourth-type subpattern between the adjacent second-type subpattern and first-type subpattern.

[0055] For the encoding method shown in FIG. 6, after the second type subpattern is determined by the decoding method, all the fourth type subpatterns can be determined based on the fact that "there is one fourth type subpattern between adjacent second type subpatterns and first type subpatterns."

[0056] In Fig. 5, the interval between adjacent first-type sub-patterns in the same projection image is 6. In Fig. 6, the interval between adjacent first-type sub-patterns in the same projection image is 12, which is much larger than that in Fig. 5, and can be applied to application scenes with a larger depth of field.

[0057] In order to meet the requirement of a larger depth of field, the number of projected images may be increased. As shown in FIG. 7, when four projected images are adopted, the interval between adjacent first-type sub-patterns in the same projected image is 8, which is larger than that in FIG. 5. In addition to increasing the number of projected images, by further increasing "inserting one fourth-type sub-pattern between adjacent first-type sub-patterns and second-type sub-patterns", the interval can be further increased to meet the requirement of a larger depth of field.

[0058] In this embodiment, the encoding method provides a first type of sub-pattern and a second type of sub-pattern, and these two types of sub-patterns are alternately generated, so that the correspondence between the imaging points and the projection points can be determined with a smaller number of projection images at a higher resolution than the binary code encoding / gray code encoding method, and the measurement efficiency of the system is improved. The first type of sub-pattern is sequentially generated as stripes in all projection images, and a fourth type of sub-pattern is introduced, so that the imaging points and the projection points can be matched more accurately than the three-dimensional points with a large depth of field, and the matching robustness is good.

[0059] As shown in FIG. 2, an embodiment of the present invention provides a system for reconstructing three-dimensional information, the system comprising: A projection device 100 that generates a plurality of projection images according to the method of encoding structured light for reconstructing three-dimensional information described in the above embodiment and projects the projection images onto a surface of a measurement object in chronological order; an image sensor 200 for forming an image projected on a surface of a measurement object and acquiring the formed image; and a reconstruction unit 300 that decodes the imaged image to determine a correspondence between each imaged position and a projection position, and reconstructs three-dimensional information of the surface of the measured object based on the correspondence.

[0060] Specifically, the projection device 100 generates multiple projection images according to the structured light encoding method for reconstructing three-dimensional information described in the above embodiment. The reconstruction unit 300 decodes the imaging images according to the structured light decoding method for reconstructing three-dimensional information described in the above embodiment, and determines the correspondence between each imaging position and the projection position.

[0061] The coding pattern is composed of a number of projected images, and the decoding pattern is composed of a number of focused images.

[0062] The above examples can be freely combined as necessary. The above are merely preferred embodiments of the present invention, and those skilled in the art can make some improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered to fall within the scope of protection of the present invention.

Claims

1. A method for encoding structured light for reconstructing three-dimensional information, comprising the steps of: forming at least three projected images; a coding pattern formed by aligning all the projection images includes a first type of sub-pattern and a second type of sub-pattern, and the sub-patterns are patterns in which all the projection images are formed at the same projection position; the first type of sub-pattern and the second type of sub-pattern appear alternately, the first type of sub-pattern appears as stripes in only one projected image, and the second type of sub-pattern appears as stripes in at least two projected images; Any two adjacent first type sub-patterns are characterized by stripes appearing in different projected images. A method for encoding structured light to reconstruct three-dimensional information.

2. Any two adjacent first type sub-patterns have different stripes in the projected image.

2. The method of encoding structured light for reconstructing three-dimensional information according to claim 1, characterized in that in the encoding pattern, the first type of sub-pattern includes stripes appearing in all projected images in sequence.

3. 2. The method of encoding structured light for reconstructing three-dimensional information according to claim 1, wherein the patterns of any two adjacent sub-patterns of the second type are different.

4. The fact that the patterns of any two adjacent second-type sub-patterns are different is The method of encoding structured light for reconstructing three-dimensional information according to claim 3, characterized in that the patterns of adjacent second type sub-patterns include different positions at which stripes appear in the projected image, or different numbers at which stripes appear in the projected image.

5. 5. The method of encoding structured light for reconstructing three-dimensional information according to claim 4, characterized in that the second type of sub-pattern is provided with a third type of sub-pattern, and the third type of sub-pattern appears as stripes in all projected images.

6. 6. The method of encoding structured light for reconstructing three-dimensional information according to claim 5, wherein a sub-pattern of a first piece is set to the third type of sub-pattern.

7. A basic pattern is determined, and each of the sub-patterns in the basic pattern is not identical to each other; 6. The method of encoding structured light for reconstructing three-dimensional information according to claim 5, wherein each sub-pattern of said basic pattern is repeatedly generated at the same predetermined period.

8. The method of encoding structured light for reconstructing three-dimensional information according to any one of claims 1 to 7, characterized in that the coding pattern further includes a fourth type of subpattern, and one fourth type of subpattern is inserted between adjacent first type and second type subpatterns.

9. 1. A system for reconstructing three-dimensional information, comprising: A projection device that generates a plurality of projection images according to the method for encoding structured light for reconstructing three-dimensional information according to any one of claims 1 to 8 and sequentially projects the projection images onto a surface of a measurement object in chronological order; an image sensor that forms an image of the projected image projected onto a surface of the object to be measured and acquires the formed image; a reconstruction unit that decodes the imaged image to determine a correspondence between each imaged position and a projection position, and reconstructs three-dimensional information of the surface of the measured object based on the correspondence; A system comprising:

Images