A three-dimensional scanning system

Abstract

This application provides a three-dimensional scanning system, comprising: a transmitter for emitting a composite patterned light beam toward a scanned object, wherein the composite pattern includes a multi-line pattern and an coded pattern, the multi-line pattern containing multiple lines, and the multiple lines being uniquely encoded by the coded pattern; a receiver for acquiring the composite patterned light beam reflected by the scanned object and generating a composite image, the composite image including a multi-line image and an coded image; and a processor for decoding the multi-line image based on the coded image to identify the multiple lines, and using the identified multiple lines and the principle of line laser scanning to calculate the depth information of the scanned object. Compared with the prior art, the three-dimensional scanning system provided by this application has high accuracy and low power consumption.

Description

Technical Field

[0001] This application relates to the field of three-dimensional imaging technology, and more particularly to a three-dimensional scanning system. Background Technology

[0002] Existing 3D scanning systems typically use DLP (Digital Light Probe) at the transmitter to project a pre-set multi-frame fringe pattern (such as Gray code or phase-shifted fringes) onto the object's surface, easily achieving high measurement accuracy. However, DLP is costly, structurally complex, and projecting multiple frames of fringes is not conducive to dynamic 3D measurement. While using speckle-structured light technology (such as Kinect, RealSense D435 / D455) to project a single-frame speckle pattern onto the object's surface and calculate the depth of the scanned object's points using a matching algorithm can measure dynamic objects, its accuracy is limited.

[0003] To address the aforementioned issues, existing technologies have proposed line laser scanning, where the scanner's transmitter projects one or more laser lines onto the object's surface, which are then captured by the receiver. Based on the principle of line laser scanning, each laser line corresponds to a light plane equation. A ray can be determined by the optical center of the receiver and a point on the laser line in the image. The three-dimensional coordinates of the scanned object's point can be determined by the intersection of this ray and the light plane equation. However, when the transmitter emits multiple laser lines, the receiver captures these multiple laser lines, resulting in a multi-line image. Consequently, a ray originating from any point on any laser line in the multi-line image will intersect multiple light planes, resulting in multiple intersection points. This makes it impossible to uniquely determine the light plane equation corresponding to each laser line.

[0004] To uniquely determine the plane equation corresponding to a laser line, existing methods propose using laser lines of multiple colors, encoding the laser lines by combining the different colors of adjacent laser lines in space. This method requires a color camera to capture the laser line image; however, the Bayer filter in the color camera reduces the camera's sensitivity. For example, when using blue or red laser lines, only 1 / 4 of the pixels in the color camera are sensitive, while the remaining 3 / 4 are almost unresponsive; similarly, when using green laser lines, only half of the pixels in the color camera are sensitive. Furthermore, encoding laser lines using color combinations is unsuitable for scanning objects with rich colors. Summary of the Invention

[0005] This application provides a three-dimensional scanning system, which aims to solve the problems of low accuracy and high power consumption when scanning objects in related technologies.

[0006] To address the aforementioned technical problems, this application provides a three-dimensional scanning system, comprising: a transmitter for emitting a composite patterned beam toward a scanned object, wherein the composite patterned beam includes a multi-line pattern and an coded pattern, the multi-line pattern containing multiple lines, and the multiple lines being uniquely encoded by the coded pattern; a receiver for acquiring the composite patterned beam reflected by the scanned object and generating a composite image, the composite image including a multi-line image and an coded image; and a processor for decoding the multi-line image according to the coded image to identify the multiple lines, and using the identified multiple lines to calculate the depth information of the scanned object based on the principle of line laser scanning.

[0007] The beneficial effects of this application are as follows: Compared with the prior art, this application provides a three-dimensional scanning system that combines coded patterns and multi-line patterns. It can obtain high-precision three-dimensional scanning information using only a few light sources. While ensuring accuracy, it not only reduces costs and power consumption, but also eliminates the need to attach markers to the scanned object, reducing labor costs and increasing the scanning speed, thereby improving the speed of three-dimensional reconstruction. Attached Figure Description

[0008] To more clearly illustrate the related technologies or the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the related technologies or the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application, and not all embodiments. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This application provides a schematic diagram of a three-dimensional scanning system structure. Figure 2 A schematic diagram of a multi-line pattern structure in a composite patterned beam provided in an embodiment of this application; Figure 3 This is a schematic diagram of the coded pattern structure in a composite patterned beam provided in an embodiment of this application; Figures 4(a) to 4(e) are schematic diagrams of the encoding principle structure of the composite patterned beam provided in the embodiments of this application; Figure 5 A schematic diagram illustrating the encoding principle structure of a composite patterned beam provided in an embodiment of this application; Figure 6 A schematic diagram illustrating the encoding principle structure of another composite patterned beam provided in an embodiment of this application; Figure 7-8 This is a schematic diagram illustrating the encoding principle structure of another composite patterned beam provided in an embodiment of this application; Figure 9A schematic diagram illustrating the encoding principle structure of another composite patterned beam provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structure of a pattern modulation element provided in an embodiment of this application; Figure 11 This is a schematic diagram of the structure of another pattern modulation element provided in an embodiment of this application; Figure 12 This is a schematic diagram of the structure of the line laser scanning principle provided in the embodiments of this application; Figures 13(a) and 13(b) are schematic diagrams of an optical system structure provided in an embodiment of this application; Figure 14 This is another schematic diagram of an optical system structure provided in an embodiment of this application; Figure 15 This is a schematic diagram of another optical system structure provided in an embodiment of this application; Figure 16 This is another schematic diagram of an optical system structure provided in an embodiment of this application. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of this application more apparent and understandable, this application will be clearly and completely described below in conjunction with its embodiments and corresponding drawings. Throughout, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions. It should be understood that the various embodiments of this application described below are merely illustrative of this application and are not intended to limit this application. That is, all other embodiments obtained by those skilled in the art based on the various embodiments of this application without creative effort are within the scope of protection of this application. Furthermore, the technical features involved in the various embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0011] Figure 1 This is a schematic diagram of the system structure of a three-dimensional scanning system according to this application. The system includes a transmitter 10, a receiver 11, and a processor (not shown). The transmitter 10 is used to emit a composite patterned beam 12 towards the object being scanned. The composite patterned beam includes a multi-line pattern 120 and an encoded pattern 121, whereby the encoded pattern 121 uniquely encodes the multiple lines contained in the multi-line pattern. The receiver 11 is used to acquire the composite patterned beam 12 reflected by the object being scanned and generate a composite image, which is transmitted to the processor. The composite image includes a multi-line image and an encoded image. The processor is used to decode the multi-line image based on the encoded image to identify multiple lines, and to calculate the depth information of the object being scanned based on the principle of line laser scanning using the identified multiple lines.

[0012] Furthermore, after obtaining the depth information of the scanned object, the processor can, on the one hand, acquire the intrinsic and extrinsic parameters of the transmitter 10 and receiver 11 in the 3D scanning system, and combine the intrinsic and extrinsic parameters with the depth information of the scanned object to obtain the point cloud map of the scanned object; on the other hand, the processor can also be used to perform 3D reconstruction on the obtained point cloud map of the scanned object to obtain the 3D model of the scanned object.

[0013] In one embodiment, the multi-line pattern 120 includes multiple lines, which are laser lines. The multiple laser lines projected by the emitting end 10 include those in a vertical or near-vertical direction, and the multiple laser lines are parallel to each other. Figure 2 As shown. It should be noted that the vertical laser line described in this embodiment is defined as the laser line extending in a direction perpendicular to the baseline between the transmitting end 10 and the receiving end 11. That is, when the transmitting end 10 emits a horizontal laser line, if it is perpendicular to the baseline between the transmitting end 10 and the receiving end 11, it also belongs to the vertical laser line described in this embodiment.

[0014] In one embodiment, the coding pattern 121 is a spot, with at least one spot arranged near or along each laser line. Multiple lines are encoded based on the uniqueness of the shape or size of a single spot or the uniqueness of the distribution combination of multiple spots within the coding pattern, thereby achieving unique encoding of each line in the multi-line pattern. It should be noted that, in addition to spots, the coding pattern in this embodiment can also be a polygon such as a triangle, quadrilateral, or pentagon, or other regular or irregular two-dimensional pattern. The coding pattern on a single line can be a two-dimensional pattern of one shape or a combination of two-dimensional patterns of multiple shapes, such as... Figure 3 As shown, no restrictions are imposed here.

[0015] Figure 4 is a schematic diagram of a composite patterned beam emitted by a transmitter according to the present application. Taking laser lines as an example, it illustrates the principle of uniquely encoding each laser line in a multi-line pattern using the distribution position of the encoded pattern. In some embodiments, multiple encoded patterns are distributed near or along each laser line in the multi-line pattern, thereby achieving unique encoding of each laser line in the multi-line pattern based on the distribution position of the encoded patterns. The distribution position of the encoded patterns can be regular or random, as long as it ensures that each laser line can be uniquely encoded; in this embodiment, there is no limitation on the size or shape of the encoded patterns, and the size or shape of each encoded pattern can be the same or different. It should be noted that Figures 4(a)-(d) are only illustrated with the example of the encoded pattern distributed along the laser line. When the encoded pattern is arranged near the laser line, as shown in Figure 4(e), the encoding principle is the same as the encoding principle of the encoded pattern distributed along the laser line.

[0016] In one embodiment, as shown in Figure 4, when the transmitter projects a composite patterned beam, uniquely encoding each laser line of the multi-line pattern using coded patterns includes: in Figure 4(a), at least one coded pattern is projected at the position of each laser line (vertical solid line) contained in the multi-line pattern, so that all coded patterns are distributed on the same horizontal line (horizontal dashed line). However, such coded patterns cannot encode each line individually. Therefore, based on Figure 4(a), the position of the coded pattern projection is slightly offset vertically, such as moving the coded patterns at certain positions upward by a fixed offset or moving the coded patterns at certain positions downward by a fixed offset, so that they are located on different horizontal lines, as shown in Figure 4(b). Assuming that Figure 4(a) is a preset reference coded pattern, compared with the preset reference coded pattern, each coded pattern has at least two distribution states in the vertical direction, namely, the position of the coded pattern changes or remains unchanged, thereby realizing the unique encoding of each laser line using the distribution states of at least two coded patterns adjacent to each laser line in the horizontal direction. Based on this, unique encoding of at least four laser lines can be achieved.

[0017] In Figure 4(b), each coding pattern has three distribution states in the vertical direction, denoted as high (H), medium (M), and low (L). Taking the distribution states of the three coding patterns adjacent to each laser line in the horizontal direction to achieve unique coding for each laser line as an example, the coding for laser line 1 is MLH, the coding for laser line 2 is LHM, and the coding for laser line 3 is HMM. For the coding patterns of laser line 0 and the last laser line which are missing, in this embodiment, N (None) is preferably used to represent them, such as the coding for laser line 0 being NML. Thus, for each coding pattern having at least three distribution states and utilizing at least three coding patterns adjacent to the current laser line in the horizontal direction, at least 27 codes can be achieved according to the above coding principle, that is, at least 27 unique codes for laser lines can be achieved, and the code for each laser line is different.

[0018] It should be noted that the basic principle of this embodiment for uniquely encoding each laser line using the distribution position of the encoded pattern is as follows: when the X-axis (i.e., the horizontal direction) of the transmitting end is parallel to the baseline of both the transmitting and receiving ends, it can be ensured that the encoded pattern originally on the same horizontal line at the transmitting end is also on the same horizontal line when imaged at the receiving end. This ensures that the vertical position of the encoded pattern in the encoded image generated at the receiving end does not change. Therefore, the processor can use the encoded image and decode the resulting multi-line image according to the above encoding principle to identify each laser line, thereby achieving unique encoding of each laser line. Furthermore, the multi-line pattern in Figure 4 is the vertical solid line portion; the horizontal line, i.e., the dashed line portion, does not actually exist in use and is only used here to better describe the encoding principle.

[0019] In another embodiment, taking 27 laser lines as an example, since the multiple laser lines projected by the transmitter have a certain length, taking the arrangement of 3 coding patterns on each laser line as an example, in order to achieve coding of all laser lines, the coding pattern shown in Figure 4(b) is copied multiple times along the vertical direction. Taking 3 copies as an example, as shown in Figure 4(c), according to the above coding principle, the coding of the current laser line is achieved by using at least 3 adjacent coding patterns in the horizontal direction near each laser line. At this time, the coding for laser line No. 1 is MLH. Therefore, when the transmitter projects the composite patterned beam as shown in Figure 4(c) and the receiver acquires it to generate a composite image containing multi-line images and coded images, which is then transmitted to the processor, the processor can use the coded image in the composite image to decode the multi-line image to uniquely determine each laser line in the multi-line pattern emitted by the transmitter as shown in Figure 4(d). Thus, each laser line in the multi-line pattern emitted by the transmitter and each laser line in the multi-line image acquired by the receiver form a one-to-one correspondence, allowing the processor to use the laser lines with a one-to-one correspondence and perform depth calculation based on the line laser scanning principle to obtain the depth information of the scanned object. The process of decoding the multi-line image using the coded image in the composite image is the reverse process of encoding the multi-line pattern using the coded pattern.

[0020] The decoding process is illustrated by taking the unique encoding of any laser line by utilizing the distribution position of the encoding pattern as an example. Preferably, each laser line in the multi-line pattern is pre-assigned a corresponding number, and the correspondence between the unique encoding of each laser line and the number of each laser line is stored (e.g., stored as a lookup table). The processor identifies the position of at least two encoding patterns adjacent to the current laser line based on the pixel value difference of the neighboring pixels of the pixel region corresponding to the current laser line in the composite image, and obtains the unique code of the current laser line based on the relative position of the at least two encoding patterns adjacent to the current laser line and the preset reference encoding map. Thus, the laser line number corresponding to the code is searched from the correspondence to identify the current laser line, thereby realizing the decoding of each laser line. The preset reference encoding map is where the encoding patterns on each laser line are distributed on the same horizontal line, as shown in Figure 4(a).

[0021] In other words, for a coding pattern distributed near a certain laser line, the code obtained by the change of its relative position compared with the preset reference coding pattern is MLH. As shown in Figure 4(b), this code corresponds to laser line 1, thereby realizing the decoding of the laser line to uniquely determine the current laser line.

[0022] In one embodiment, the processor identifies the position of the coded pattern based on the pixel value differences of neighboring pixels in the pixel region where the current laser line is located in the composite image, and obtains the unique code of the current laser line based on the positions of at least three coded patterns adjacent to the current laser line. This includes: when the receiving end acquires the composite patterned beam reflected back from the scanned object, some pixels in the receiving end can respond to the reflected composite patterned beam to generate a composite image and transmit it to the processor, that is, some pixels in the receiving end do not respond; wherein, the pixel value corresponding to the responding pixel has a numerical difference compared with the pixel value corresponding to the non-responding pixel, and the pixel value corresponding to the non-responding pixel is 0 or a constant value. Thus, a pixel value threshold can be preset, the composite line image is detected pixel by pixel, and the pixel value of each pixel in the composite line image is compared with the pixel value threshold. Pixels greater than and / or equal to the pixel value threshold are defined as responding pixels. The processor searches for responding pixels in the neighboring pixels of the pixel region corresponding to the current laser line, thereby identifying the position of the coded pattern and decoding the current laser line according to the position of the coded pattern corresponding to the current laser line to uniquely determine the current laser line.

[0023] It should be noted that the above embodiments are illustrated using the case where the coded image and the multi-line image contained in the composite image can be a single frame. In another embodiment, the coded image and the multi-line image contained in the composite image can be two independent frames, and at least some pixels in the two frames have a one-to-one alignment relationship. Therefore, when it is necessary to decode a certain laser line in the multi-line pattern, the coded pattern adjacent to the current laser line can be obtained according to the coordinate information of the current laser line and the alignment relationship between the images to achieve decoding.

[0024] Figure 5 This is a schematic diagram illustrating the encoding principle of a composite patterned beam provided in an embodiment of this application. In one embodiment, when multiple laser lines projected by the transmitter have a preset length, multiple encoding patterns can be distributed along the extension direction of each laser line. These multiple encoding patterns along the extension direction are used to uniquely encode each line. Taking three encoding patterns arranged on each laser line as an example, this embodiment can also utilize at least three encoding patterns in the vertical direction near each laser line to encode the current laser line. Compared to the preset reference encoding diagram, the encoding of laser line 0 is HHH, the encoding of laser line 1 is HHM, and so on, thus enabling unique encoding of 27 laser lines. It should be understood that, generally, the field of view of the transmitter 10 is larger than that of the receiver 11. Therefore, the patterns at the beginning and end of the composite patterned beam projected by the transmitter 10 may not be acquired by the receiver 11. In practical applications, the encoding and decoding calculations of the laser lines at both ends of the multi-line pattern can be ignored.

[0025] In summary, Figure 4- Figure 5The illustrated embodiment only uses the arrangement of 3 coded patterns on each laser line as an example. When each coded pattern has 3 offset positions in the vertical direction, the coded patterns can also be arranged near the laser line, and the encoding principle is the same as in the above embodiment, which will not be repeated here. In addition, according to the encoding principle of this application, when each laser line has 4 offset positions in the vertical direction, 64 ( A unique encoding is used to encode each of the 64 laser lines. Therefore, when n encoded patterns are arranged on or near each laser line, and each pattern has m possible offset positions in the vertical direction, the following can be obtained: This encoding can achieve The unique code of each laser line.

[0026] Figure 6 This is a schematic diagram illustrating the encoding principle structure of another composite patterned beam provided in this application. In one embodiment, the encoding pattern is a two-dimensional pattern of various shapes, which can be arranged along each laser line. The shape of the two-dimensional pattern uniquely encodes each laser line of the multi-line pattern. Specifically, when the encoding pattern projected by the transmitter is a two-dimensional pattern of various shapes, the multi-line pattern projected by the transmitter can be horizontal or vertical. In this case, the two-dimensional pattern projected by the transmitter needs to be arranged along each laser line in the multi-line pattern. Since the shapes of the two-dimensional patterns are different, a laser line can be encoded using a two-dimensional pattern of a certain shape, establishing a correspondence between the shape of the two-dimensional pattern and the number of each laser line. This achieves unique encoding of each laser line in the multi-line pattern using two-dimensional patterns of different shapes.

[0027] Figures 7 to 8 This is a schematic diagram illustrating the encoding principle structure of another composite patterned beam according to this application. In one embodiment, such as... Figure 7 As shown, the coding pattern includes multiple coding patterns of different sizes, each arranged along multiple laser lines to uniquely encode each laser line in the multi-line pattern based on the size of each coding pattern. It should be noted that this embodiment does not restrict the position of the coding pattern on the laser lines or the direction of the laser lines; it can be located anywhere on the laser lines. Furthermore, each laser line in this embodiment may have one or more coding patterns of the same size, as long as the sizes of the coding patterns differ between the laser lines.

[0028] Specifically, the transmitter needs to include at least two light sources of different wavelengths. One light source is used to project the coded pattern, and the other light source is used to project the multi-line pattern. The coded pattern projected by the transmitter needs to be arranged along each laser line in the multi-line pattern. Since the size of each coded pattern in the coded pattern is different, one size of coded pattern corresponds to one laser line. The correspondence between the size of the coded pattern and the number of each laser line is established. Thus, each laser line in the multi-line pattern can be uniquely encoded according to the coded patterns of different sizes.

[0029] In another embodiment, the coded pattern is an arrangement of at least two coded patterns of different sizes, such as... Figure 8 As shown, the arrangement of each coded pattern can be arranged along each laser line or near each laser line, so as to encode each laser line based on the uniqueness of the arrangement of each coded pattern in the multi-line pattern. It should be noted that this embodiment does not restrict the position of the coded pattern on the laser line or the direction of the laser line; it can be located at any position on the laser line.

[0030] Specifically, the transmitting end needs to include at least two light sources of different wavelengths. One light source projects a coded pattern, and the other projects a multi-line pattern. The coded patterns projected by the transmitting end are arranged along or near each laser line in the multi-line pattern. Since the projected coded patterns have different arrangements, a correspondence is established between different coded pattern sizes and the numbers of each laser line. Therefore, each laser line of the multi-line pattern can be encoded based on the uniqueness of the coded pattern arrangements. For example, if the coded pattern projected by the transmitting end includes three coded patterns of different sizes... Figure 8 As shown, they are denoted as large (L), medium (M), and small (S). Each laser line is encoded using at least three coded patterns. For example, the code for laser line 1 is SML, and the code for laser line 2 is etc., which can result in 27 possible permutations and combinations, thus achieving a unique code for each of the 27 laser lines.

[0031] It should be noted that the principle of using the arrangement and combination of coding patterns of different sizes is similar to that of using the position offset of coding patterns to encode each laser line. That is, one or a combination of the position, size, and shape characteristics of the coding pattern can encode each laser line in the multi-line image. The only difference is in the form of the coding, but in essence, they are both about uniquely encoding each laser line in the multi-line pattern.

[0032] Figures 2-8 The embodiments described all use lines in a multi-line pattern as laser lines. Figure 9This is a schematic diagram of the encoding principle structure of a composite patterned beam according to this application. In one embodiment, multiple lines in a multi-line pattern are composed of encoded patterns. Specifically, each line is formed by arranging multiple encoded pattern sets, and each encoded pattern set consists of one or more tightly arranged encoded patterns with a preset gap between each encoded pattern set. Since the arrangement of the encoded pattern sets in each line is different, the encoded pattern in this embodiment corresponds to the arrangement of multiple encoded pattern sets in each line. A correspondence is established between the arrangement of the encoded pattern sets on each line and the line number. Based on the arrangement of the encoded pattern sets on each line, the line obtained based on that encoded pattern set is unique. Further, the arrangement of the encoded pattern sets characterizes the arrangement order of the encoded pattern sets in each line. The arrangement order of each encoded pattern set and the number of encoded patterns in each encoded pattern set can be used to obtain the arrangement method of that encoded pattern set. Figure 9 Taking the projection of a vertical multi-line pattern by the transmitter as an example, for line 0, it includes 5 coded pattern sets, which are arranged in the pattern 23523. The order of each number indicates the arrangement order of each coded pattern set, and each number indicates the number of coded patterns included in each coded pattern set.

[0033] It should be noted that, in addition to using the numbers and letters mentioned above, the encoding representation of the coded pattern in this application may also be represented by text or any combination of the numbers, letters and text mentioned above, without any restrictions.

[0034] based on Figures 3-9 The corresponding encoding / decoding principles, combined with this application Figure 1 , Figure 10 and Figure 11 An exemplary description of the system design of a 3D scanning system is provided.

[0035] In some embodiments, such as Figure 1 As shown, the transmitting end 10 includes a light source 101 and a pattern modulation element 102. The light source 101 is used to emit a light beam to the pattern modulation element 102. The pattern modulation element 102 modulates the light beam emitted by the light source 101 and projects a composite patterned light beam 12, including an coded pattern 121 and a multi-line pattern 120, onto the scanned object.

[0036] In some embodiments, the light source 101 may be a light-emitting diode (LED), an edge-emitting laser (EEL), a vertical-cavity surface-emitting laser (VCSEL), or a light source array composed of multiple light sources, and the emitted light may be any one or more combinations of visible light, infrared light, blue light, green light, or ultraviolet light.

[0037] In one embodiment, when the light source 101 in the transmitter 10 is a single light source, the light source 101 emits a light beam to the pattern modulation element 102, and the pattern modulation element 102 modulates the light beam emitted by the light source 101 and projects a composite patterned light beam 12 onto the scanned object. Preferably, when the light source 101 in the transmitter 10 is a single light source, its corresponding pattern modulation element 102 includes at least one mask or diffractive optical element (DOE).

[0038] Specifically, when the pattern modulation element 102 is a single Mask or DOE, the Mask or DOE can be photolithographically etched with patterns that allow only multi-line patterns and coded patterns to pass through, so that the light beam emitted by the light source 101 passes through a single Mask or DOE to obtain a composite patterned light beam 12; when the pattern modulation element 102 includes multiple Masks or DOEs, each Mask or DOE can be photolithographically etched with patterns that allow only partial multi-line patterns or coded patterns to pass through, so that the light beam emitted by the light source 101 passes through each Mask or each DOE to obtain a composite patterned light beam 12. Taking the pattern modulation element 102 as an example consisting of only one Mask, it includes: designing a pattern that meets preset requirements according to preset input beams and preset output beams, and fabricating the pattern on a mask using photolithography to form the pattern modulation element Mask required in this embodiment, such as... Figure 10 As shown. It should be noted that in this embodiment, the pattern that meets the preset requirements must satisfy the condition that only multi-line patterns and coded patterns are allowed to pass through when the light beam emitted by the light source passes through the pattern modulation element, so as to realize the unique encoding of each line in the multi-line pattern using the coded pattern.

[0039] In another embodiment, when the light source 101 in the transmitter 10 includes at least two sub-light sources, the at least two sub-light sources are used to emit light beams to a pattern modulation element respectively, and the pattern modulation element is used to modulate the light beams emitted by the at least two sub-light sources and project a multi-line pattern and a coded pattern onto the scanned object; wherein, the wavelengths of the at least two sub-light sources may be the same or different, each sub-light source may share a pattern modulation element or a pattern modulation element may be provided on the light-emitting side of each sub-light source, and the pattern modulation element includes a Mask or a DOE.

[0040] Specifically, when at least two sub-light sources have the same wavelength, the light-emitting sides of the at least two sub-light sources must be respectively provided with pattern modulation elements and emit beams in a time-division manner to obtain coded patterns and multi-line patterns, so as to avoid the coded patterns and multi-line patterns overlapping when imaging at the receiving end 11, thus making it impossible to distinguish between the coded patterns and multi-line patterns; when at least two light sources have different wavelengths, the at least two sub-light sources can share a pattern modulation element or be respectively provided with pattern modulation elements, and they can emit beams simultaneously or in a time-division manner, without any restrictions here.

[0041] Furthermore, if pattern modulation elements are respectively set on the light-emitting side of each sub-light source, taking the pattern modulation element as a mask as an example, the mask design corresponding to at least two sub-light sources is as follows: Figure 11 As shown, one mask allows only multi-line patterns to pass through, while the other allows only coded patterns to pass through. Each sub-light source in the transmitter emits a beam to a different pattern modulation element. The different pattern modulation elements modulate the beam emitted by each sub-light source to obtain multi-line patterns and coded patterns, which are then projected onto the object being scanned, so as to form a composite patterned beam 12 containing multi-line patterns and coded patterns when it reaches the surface of the object being scanned.

[0042] In some embodiments, the receiver 11 is used to receive the beam reflected back from the scanned object and generate a composite image to be transmitted to the processor. Specifically, when the transmitter 10 transmits an coded pattern and a multi-line pattern to the scanned object in stages, the coded pattern and the multi-line pattern are reflected back to the receiver 11 in sequence from the scanned object. The receiver 11 sequentially acquires the coded pattern and the multi-line pattern and generates a composite image containing the coded image and the multi-line image accordingly. That is, the coded image and the multi-line image at this time are two independent frames. When the transmitter 10 simultaneously transmits a composite patterned beam containing the coded pattern and the multi-line pattern to the scanned object, the composite patterned beam is reflected back to the receiver 11 from the scanned object. The receiver 11 simultaneously acquires the composite patterned beam and generates a composite image, which includes the coded image and the multi-line image. That is, the coded image and the multi-line image at this time are one frame.

[0043] In one embodiment, the receiver 11 includes a monochrome image sensor and a filter. The monochrome image sensor is used to acquire a composite patterned light beam reflected back from the scanned object and generate a composite image for transmission to the processor, or it is used only to acquire a multi-line pattern in the composite patterned light beam reflected back from the scanned object and generate a multi-line image for transmission to the processor. The monochrome image sensor can be any one or more of a charge-coupled device (CCD), complementary metal-oxide-semiconductor (CMOS), avalanche diode (AD), or single-photon avalanche diode (SPAD). The filter is disposed on the light-incident side of the monochrome image sensor and is preferably a narrow-band filter that matches the wavelength of the emitted light beam from the transmitter. It is used to suppress background light noise in other bands so that only the composite patterned light beam or the multi-line pattern in the composite patterned light beam passes through the filter and is acquired by the monochrome image sensor.

[0044] In another embodiment, the receiver 11 further includes a color image sensor, which includes an image sensor and a Bayer filter disposed on the light-incident side of the image sensor, for acquiring the texture of the scanned object and transmitting it to the processor. The processor renders the point cloud module of the scanned object according to the texture of the scanned object to obtain a textured point cloud map or performs texture mapping on the three-dimensional model of the scanned object to obtain a textured three-dimensional model.

[0045] Furthermore, when the black and white image sensor only acquires the multi-line pattern in the composite patterned beam and generates a multi-line image, the light-incident side of the color image sensor is also provided with a narrow-band filter that matches the wavelength of the beam that emits the coded pattern, so that the color image sensor is only used to acquire the coded pattern in the composite patterned beam and generate a coded image that is transmitted to the processor, so that the processor can identify each line in the multi-line image based on the coded image.

[0046] It should be understood that, compared to color image sensors, monochrome image sensors have better sensitivity and higher signal-to-noise ratio in the acquired images. Using monochrome sensors to acquire composite patterned beams or multi-line patterns in composite patterned beams is beneficial for high-precision measurement of the scanned object. In this embodiment, color image sensors are only used to acquire coded patterns for encoding or the texture of the scanned object, and are not directly used for the measurement of the scanned object, so they do not affect the measurement accuracy.

[0047] In one embodiment, the receiver 12 may further include an imaging lens for receiving and transmitting a composite patterned light beam reflected back from an object, allowing the composite patterned light beam to propagate to and pass through a filter to image onto a corresponding pixel of the image sensor. Preferably, the imaging lens comprises a single lens or a lens group consisting of multiple lenses.

[0048] In some embodiments, the processor can be a dedicated circuit, such as a dedicated SOC chip, FPGA chip, ASIC chip, etc., which includes a CPU, memory, bus, etc. It can also include general-purpose processing circuits. For example, when the 3D scanning system is integrated into a smart terminal such as a mobile phone, television, computer, scanner, etc., the processing circuit in the terminal can serve as at least part of the processor.

[0049] In some embodiments, the processor decodes the multi-line image based on the encoded image to identify multiple lines in the multi-line image, and uses the identified multiple lines to calculate the depth information of the scanned object based on the principle of line laser scanning. Specifically, as shown... Figure 12 As shown, based on the principle of line laser scanning, a line emitted by the transmitter towards the scanned object can form a laser scalpel plane. Each laser scalpel plane corresponds to a laser scalpel plane equation, which can be obtained through calibration. The receiver acquires the beam reflected from the scanned object, forming a multi-line image containing a single line on the camera's imaging plane. The processor extracts any point on any line in the multi-line image and, starting from the optical center at the receiver, forms a ray with any point on the line (denoted as p). By finding the intersection of this ray with the laser scalpel plane, the three-dimensional coordinates of p can be determined, thus obtaining the depth information of the scanned object.

[0050] Furthermore, the processor can extract the center lines of the lines in the multi-line image using a center line extraction algorithm. Starting from the optical center at the receiving end, a ray is formed with any center point of the line. The intersection of this ray with the optical blade plane determines the three-dimensional coordinates of the corresponding center point, thus obtaining the depth information of the scanned object. Compared to directly calculating the depth information of the scanned object from any point on the line, this embodiment obtains sub-pixel coordinates for the corresponding center point by calculating the center line on the line, and improves the accuracy of the depth information by using the sub-pixel center point for depth information.

[0051] However, if a 3D scanning system emits only a single line to scan the object, the resulting data will be sparse. To obtain denser data, multiple lines need to be emitted. When the transmitter emits multiple lines, the receiver collects the multiple lines reflected back from the object being measured, thus obtaining a multi-line image. A ray originating from any point on any line in the multi-line image and forming a ray with the optical center of the receiver will intersect with the light planes corresponding to the multiple lines emitted by the transmitter, resulting in multiple intersection points. This makes it impossible to uniquely determine the equation of the light plane corresponding to the current line in the multi-line image.

[0052] To uniquely determine the laser beam plane equation corresponding to the current line in the multi-line image, this embodiment controls the transmitter to emit a composite patterned beam including an coded pattern and a multi-line pattern. The coded pattern is used to uniquely encode multiple lines in the multi-line pattern. The receiver then collects the reflected composite patterned beam and generates a composite image containing the coded image and the multi-line image. The processor can then use the coded image in the composite image to decode the multi-line image to identify each line, thereby uniquely determining the laser beam plane equation corresponding to the current line. This allows for further obtaining the depth information of the scanned object based on the aforementioned line laser scanning principle.

[0053] Based on the system design of the three-dimensional scanning system provided in the embodiments of this application, Figures 13 to 14 show the following: Figure 16 An exemplary optical system architecture for a 3D scanning system is shown.

[0054] Figure 13 is a schematic diagram of the optical system structure of a three-dimensional scanning system according to this application. Specifically, the optical system includes a first transmitting end 20, a second transmitting end 21, and a receiving end 22. The first transmitting end 20 and the second transmitting end are used to emit composite patterned beams including multi-line patterns and coded patterns to the scanned object, respectively. The receiving end 22 is used to collect the composite patterned beams reflected back by the scanned object and generate a composite image including a coded image and a multi-line image. It should be noted that, for better illustration of the optical system provided in this embodiment, this embodiment only uses the first transmitting end 20 and the second transmitting end 21 as examples. However, in actual applications, the transmitting ends can be integrated or set independently, and no limitation is made here.

[0055] In some embodiments, each transmitting end includes a light source and a pattern modulation element. As shown in FIG13, the first transmitting end 20 includes a first light source 200 and a first pattern modulation element 202, and the second transmitting end 21 includes a second light source 210 and a second pattern modulation element 212. The first light source 200 and the second light source 210 are used to emit light beams to the first pattern modulation element 202 and the second pattern modulation element 212, respectively. The light beam emitted by the first light source 200 is modulated by the first pattern modulation element 202 to form a multi-line pattern and is projected onto the scanned object 26, as shown in FIG13(a). The light beam emitted by the second light source 210 is modulated by the second pattern modulation element 212 to form an encoded pattern and is projected onto the scanned object 26, as shown in FIG13(b).

[0056] Furthermore, the wavelengths of the light beams emitted by the first transmitting end 20 and the second transmitting end 21 may be the same or different. In one embodiment, when the wavelengths of the light beams emitted by the transmitting ends are different, the first light source 200 and the second light source 210 are preferably a blue laser light source and a green laser light source, respectively, for emitting blue light and green light to the corresponding pattern modulation elements. The blue laser light source, after being modulated by the first pattern modulation element 202, forms a blue multi-line pattern, and the green laser light source, after being modulated by the second pattern modulation element 212, forms a green coded pattern. The multi-line patterns and coded patterns obtained using different wavelengths, projected onto the scanned object in this embodiment, can be used to scan objects prone to scattering, such as translucent teeth.

[0057] In other embodiments, when the wavelengths of the light beams emitted by the transmitting end are the same, the first light source 200 and the second light source 210 are preferably near-infrared laser light sources, used to emit light beams to the pattern modulation element respectively or simultaneously, and after being modulated by the pattern modulation element, multi-line patterns and coded patterns are formed respectively for scanning general objects.

[0058] Furthermore, when the beam emitted by the transmitting end is a near-infrared laser beam, in one embodiment, if the coded pattern in the projected composite patterned beam is a coded pattern pattern, the transmitting end includes at least two light sources and at least one pattern modulation element. The at least two light sources are preferably a VCSEL array. One light source emits a beam to the pattern modulation element to form a multi-line pattern, while the other light source can be designed to directly emit the coded pattern pattern without passing through the pattern modulation element. In another embodiment, if all composite patterned beams are coded patterns, the transmitting end may also include at least one light source, preferably a VCSEL array, designed to directly emit the coded pattern pattern.

[0059] It should be noted that the wavelength selected for the emitting end of the optical system depends on the properties of the object being scanned. For example, as mentioned above, if the object being scanned is a translucent tooth or other object that is prone to scattering light, then blue laser, green laser, or other lasers with shorter wavelengths can be used. If it is a general object, then near-infrared laser is generally used. This application does not limit this.

[0060] In one embodiment, the emitting end further includes a collimating element disposed between the light source and the pattern modulation element, used to collimate the light beam emitted by the light source to the pattern modulation element, such as collimating element 201 disposed between the first light source 200 and the first pattern modulation element 202, and collimating element 211 disposed between the second light source 210 and the second pattern modulation element 212. Preferably, the collimating element 201 includes a lens or a lens group composed of multiple lenses. It should be noted that, in addition to collimating the light beam emitted by the light source, the collimating element 201 can also be used to homogenize the light beam emitted by the light source.

[0061] In some embodiments, the receiver 22 includes a monochrome image sensor 220, a color image sensor 221, a beam splitter 222, and an imaging lens 223. The monochrome image sensor and the color image sensor are disposed on both sides of the beam splitter 222, preferably in an "L" shape, so as to share a single imaging lens 223. Preferably, a filter is disposed between the monochrome image sensor 220 and the color image sensor 221 and the beam splitter 222. The composite patterned beam reflected back from the scanned object is collimated and focused by the imaging lens 223 onto the beam splitter 222. The beam splitter 222 splits the reflected beam into two beams. One beam, after passing through the filter, only has the blue multi-line pattern in the composite patterned beam, which is then captured by the monochrome image sensor 220. The other beam, after passing through the filter, only has the green coded pattern in the composite patterned beam, which is then captured by the color image sensor. It should be noted that the filter in this embodiment can be a single device, or it can be integrated into the beam splitter 222 or integrated into the black and white image sensor 220 and the color image sensor 221 respectively. No limitation is made here.

[0062] In one embodiment, the imaging lens 223 is a lens group consisting of one or more lenses, used to focus the composite patterned beam reflected back from the scanned object so that the composite patterned beam reflected back from the scanned object is correspondingly imaged onto the corresponding pixel in the image sensor.

[0063] In another embodiment, when it is necessary to acquire the texture information of the scanned object 26, the color image sensor 221 in the receiver 22 is used to acquire the texture information of the scanned object 26, and the black and white image sensor 220 is used to acquire the composite patterned light beam reflected back by the scanned object. To improve the imaging clarity of the color image sensor 221, the optical system in this embodiment also includes a coaxial illumination unit 23. Preferably, the coaxial illumination unit 23 is disposed on the light-incident side of the receiver 22 and is used to provide an ambient light-like light signal to the color image sensor 221. The coaxial illumination unit 23 includes an illumination source 230, a collimating lens 231, and a semi-transparent and semi-reflective optical element 232. The illumination source 230 emits a light beam to the collimating lens 231. After being collimated by the collimating lens 231, the light beam reaches the semi-transparent and semi-reflective optical element 232. Part of the light beam is deflected by the semi-transparent and semi-reflective optical element 232 to the scanned object 26 to supplement the light to the scanned object 26, thereby enabling the color image sensor 221 to clearly acquire the texture information of the scanned object 26. It should be noted that the light source 230 of the coaxial illumination section 23 is preferably a white light source, such as an LED.

[0064] Specifically, when the composite patterned beam reflected back from the scanned object and the beam emitted by the coaxial illumination unit 23 simultaneously enter the receiving end 22, the beam splitting element 222 in the receiving end 22 splits the beam received by the receiving end into two beams. One beam enters the color image sensor 221 to obtain the texture image of the scanned object, and the other beam enters the black and white image sensor 220. A narrow-band filter is provided in front of the black and white image sensor 220 to filter ambient light, allowing only the composite patterned beam emitted by the transmitting end to pass through so as to form a composite image on the black and white image sensor 220. It should be noted that when the color image sensor 221 is used to acquire the texture information of the scanned object 26, no filter is required on the light-incident side of the color image sensor 221.

[0065] In one embodiment, a collimating lens 24 is further provided on the light-incident side of the coaxial illumination section 23. The collimating lens 24 is used to collimate the composite patterned light beam reflected back from the scanned object to the coaxial illumination section 23 and allow it to pass through the coaxial illumination section 23. Preferably, the collimating lens 24 can be a single lens or a lens group composed of multiple lenses.

[0066] In one embodiment, the 3D scanning system further includes a deflecting optical element 25, used to receive a light beam emitted from the transmitting end and reflect it to the object being scanned 26 to achieve scanning of the object 26, and also used to receive the light beam reflected back from the object being scanned 26 and deflect it to the receiving end 22, thereby changing the transmitting and receiving optical paths of the 3D scanning system, making the transmitting and receiving optical paths partially coaxial, reducing the size, and realizing the miniaturization of the 3D scanning system. Preferably, the deflecting optical element 25 may include any one of a rotating mirror, a reflecting mirror, a prism, or a MEMS, without limitation herein.

[0067] Figure 14 This is a schematic diagram of the optical system structure of another three-dimensional scanning system according to this application. Compared with the optical system structure shown in Figure 13, the optical system provided in this embodiment differs in the transmitting end, while the receiving end is the same as described above, and will not be repeated here.

[0068] In one embodiment, the transmitting end 30 includes at least a first light source 300, a second light source 301, a beam combining element 302, and a pattern modulation element 303. The first light source 300 and the second light source 301 are respectively disposed on different light-incident sides of the beam combining element 302, and the pattern modulation element 303 is disposed on the light-outceasing side of the beam combining element 302. Specifically, the first light source 300 and the second light source 301 are used to simultaneously or sequentially emit light beams of different wavelengths to the beam combining element 302. The beam combining element 302 is used to homogenize the two different wavelength light beams and integrate them into the same optical path for propagation to the pattern modulation element 303. The pattern modulation element 303 is used to modulate the received light beams to obtain multi-line patterns and coded patterns, which are then projected onto the scanned object 26.

[0069] In one embodiment, the homogenizing and beam combining element 302 includes at least two homogenizing elements and at least one beam combining element 3022. At least one homogenizing element is provided on the light-emitting side of at least two different wavelength light sources, such as a first homogenizing element 3020 corresponding to the first light source 300 and a second homogenizing element 3021 corresponding to the second light source 301. This homogenizes the light beam emitted by the light source, thereby homogenizing the light beam at the energy level and collimating it to the beam combining element 3022, thereby improving light energy utilization and avoiding coherent light. The beam combining element 3022 is used to integrate the light beams emitted by different light sources after collimation by the collimating element onto the same optical path and project them onto the pattern modulation element 303. It should be noted that the homogenizing element in this embodiment can be a single lens or a combination of multiple lenses. The lens can be one or more combinations of compound eye lenses, diffusers, microlenses, etc. The beam combining element preferably includes one or more combinations of dichroic mirrors, prisms, etc., but this is not limited here.

[0070] Furthermore, since the beams emitted from different wavelength light sources are integrated into the same optical path and projected onto the pattern modulation element by the beam combining element 302 in this embodiment, in one embodiment, the pattern modulation element 303 includes at least one DOE or at least one Mask for modulating the beams projected by the beam combining element 302 to generate a multi-line pattern and a coded pattern. For example, when the coded pattern is a coded pattern pattern, in order to achieve the simultaneous generation of the multi-line pattern and the coded pattern by only one pattern modulation element, lines that transmit only the wavelength of the light source corresponding to the emitted multi-line pattern and coded patterns that transmit only the emitted coded pattern pattern can be photolithographically etched on the same DOE or the same Mask.

[0071] In one embodiment, the transmitting end further includes a projection lens 304 for collimating the light beam projected by the collimating pattern modulation element 303 onto the object being scanned. It should be noted that the projection lens may include a single lens or a combination of multiple lenses, and no limitation is made here.

[0072] Figure 15 This is a schematic diagram of the optical system structure of another three-dimensional scanning system according to this application. The optical system includes a transmitter 40 and a receiver 22. It should be noted that the optical elements with the same names in the transmitter and receiver in this embodiment have the same function as described above, and will not be repeated here.

[0073] In some embodiments, the emitting end 40 includes at least one light source 400, a collimating and homogenizing section 401, and a pattern modulation element 402. The collimating and homogenizing section 401 is disposed between the light source 400 and the pattern modulation element 402. The light source 400 emits a light beam to the collimating and homogenizing section 401, which collimates and homogenizes the light beam before projecting it onto the pattern modulation element 402. The pattern modulation element 402 has a shape that allows only multi-line patterns and coded patterns to pass through. When the collimated and homogenized light beam is projected onto the pattern modulation element 402, the pattern modulation element 402 can modulate the light beam to generate multi-line patterns and coded patterns to be projected onto the scanned object 26.

[0074] In one embodiment, the collimation and homogenization section 401 includes a collimating element 4010 and a homogenizing element 4011. The collimating element 4010 collimates the light beam emitted by the light source 400 to the homogenizing element 4011, and the homogenizing element 4011 homogenizes the light beam collimated by the collimating element 4010 and projects it onto the pattern modulation element 402, thereby homogenizing the energy of the light beam emitted by the light source 400 and improving the light energy utilization rate when the light beam emitted by the light source passes through the pattern modulation element 402. It should be noted that the collimating element 4010 is preferably a single lens or a group of multiple lenses, and the homogenizing element 4011 is a lens that can be one or more combinations of compound eye lenses, diffusers, microlenses, etc. It should be noted that the collimating element 4010 and the homogenizing element 4011 can be integrated or independently arranged, and this application does not impose any limitations on this.

[0075] In one embodiment, the transmitter 40 further includes a projection lens 403 for collimating the light beam projected by the pattern modulation element 401 onto the scanned object 26. It should be noted that the projection lens 403 may include a single lens or a combination of multiple lenses, used only to collimate the light beam; no limitation is made here.

[0076] In one embodiment, the receiver 22 includes at least one monochrome sensor 220 and at least one imaging lens 223. The imaging lens 223 is used to focus the composite patterned light beam reflected back from the scanned object, such that the composite patterned light beam reflected back from the scanned object is correspondingly imaged onto the corresponding pixels of the monochrome sensor 220. The monochrome sensor 220 is used to receive the multi-line pattern and coded pattern reflected back from the scanned object 26 and generate a composite image including the multi-line image and the coded image, which is transmitted to the processor to acquire the depth information of the scanned object. Further, the receiver may include two monochrome sensors to form a binocular stereo vision for acquiring the depth information of the scanned object; this is not limited here.

[0077] In another embodiment, the receiver may be the same as the receiver shown in FIG13, which will not be described again here.

[0078] Figure 16 This is a schematic diagram of the optical system structure of another three-dimensional scanning system provided in this application. The optical system includes a transmitter 50, a receiver 22, and a deflection optical element 25. It should be noted that the function and composition of the optical element with the same name in the transmitter in this embodiment are the same as those described above, and will not be repeated here.

[0079] In one embodiment, the emitting end 50 includes a light source, a collimating and homogenizing section, a pattern modulation element, a reflective element, a semi-transparent and semi-reflective element, a beam combining element, and a collimating element. The first light source 500 and the second light source 501 emit light beams of different wavelengths to the first collimating and homogenizing section 502 and the second collimating and homogenizing section 503, respectively. The first collimating and homogenizing section 502 collimates the light beam emitted by the first light source 500 to the corresponding first pattern modulation element 504, and the second collimating and homogenizing section 503 collimates the light beam emitted by the second light source 501 to the reflective element 507, and the reflective element 507 reflects the light beam to the second pattern modulation element 505. The first pattern modulation element 504 and the second pattern modulation element 505 modulate the light beam to correspondingly form multi-line patterns and coded patterns, which are then projected onto the semi-transparent and semi-reflective element 506. It should be noted that the reflective element reflects the collimated and homogenized light beam to the pattern modulation element to change the emitted light path and achieve miniaturization. It may include a mirror, a prism, etc., as long as it can reflect the light beam; no limitation is imposed here.

[0080] Furthermore, the beam emitted by the first light source 500 is modulated by the first pattern modulation element 504 to form a multi-line pattern and propagates through the semi-transparent and semi-reflective element 506 to the beam combining element 508. The beam emitted by the second light source 501 is modulated by the second pattern modulation element 505 to form a coded pattern and is reflected by the semi-transparent and semi-reflective element 506 to the beam combining element 508. The beam combining element 508 is used to combine the multi-line pattern and the coded pattern into a composite patterned beam and collimated by the collimating element 509 before projecting it onto the deflecting optical element 25. The deflecting optical element 25 is used to change the scanning direction of the composite patterned beam so that the composite patterned beam scans the scanned object 26, and to receive the composite patterned beam reflected back by the scanned object 26 and deflect it to the receiving end 22.

[0081] In one embodiment, the transmitter 50 further includes a projection lens 509, which is disposed on the light-emitting side of the light combining element 508, and its image plane is located precisely on the light-emitting surfaces of the first image modulation element 504 and the second pattern modulation element 504, so that the multi-line pattern and the coded pattern can be highly aligned when projected onto the scanned object 26, that is, the coded pattern can be precisely distributed and arranged along the multi-line pattern. Preferably, the projection lens 509 is used to collimate the light beam integrated by the light combining element 508 and project it onto the deflecting optical element 25.

[0082] In one embodiment, the transmitter 50 further includes an illumination unit 510 for providing ambient light-like illumination to the scanned object 26. Specifically, when the scanned object 25 is in a dark environment (such as teeth or gums), the illumination unit 510 can be turned on to provide supplementary light to the scanned object 25 in order to obtain the texture information of the scanned object 26. Preferably, the illumination unit 510 includes an illumination source and a collimation element. The illumination source is a white light source (such as an LED) to mimic ambient light. In this case, the illumination source emits a light beam to the collimation element and, after being collimated by the collimation element, emits it towards the deflection optical element 25. Under the deflection of the deflection optical element 25, the light is projected onto the scanned object 26 to provide supplementary light.

[0083] In one embodiment, the receiver 22 in this implementation is the same as the receiver shown in FIG13, and will not be described again here.

[0084] Specifically, the first light source 500 uses a blue laser light source with a wavelength of 450nm, and the second light source 501 uses a blue laser light source with a wavelength of 405nm. The beam emitted by the first light source 500 is modulated by the first pattern modulation element 504 to form a multi-line pattern, and the beam emitted by the second light source 501 is modulated by the second pattern modulation element 505 to form a coded pattern. The multi-line pattern and the coded pattern are integrated by the beam combining element 508 to form a composite patterned beam, which is then collimated by the projection lens 509 and propagated to the deflection optical element 25. The deflection optical element 25 is used to change the propagation direction of the composite patterned beam to complete the scanning of the object 26 being scanned.

[0085] Furthermore, the transmitter 50 emits a white light beam and blue laser beams of different wavelengths to the object being scanned 26. The beams emitted by the transmitter 50 to the object being scanned 26 are reflected back to the deflecting optical element 25, which deflects the reflected beams to the receiver 22. The imaging lens 223 in the receiver 22 focuses the reflected beams to the beam splitter 222. The beam splitter 222 splits the beams received by the receiver to obtain two beams. One white light beam enters the color image sensor 221 to obtain the texture image of the object being scanned, and the other blue light beam (i.e., beams in the 450nm and 405nm bands) enters the monochrome image sensor 220. A blue light filter is provided in front of the monochrome image sensor 220, which only allows the composite patterned beam emitted by the transmitter to pass through so as to image a composite image including coded images and multi-line images on the monochrome image sensor 220, thereby filtering stray light.

[0086] It should be noted that the various embodiments in this application are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0087] It should also be noted that, in this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0088] The above description of the disclosed embodiments enables those skilled in the art to implement or use the content of this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in this application may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional scanning system, characterized in that, include: The transmitting end is used to emit a composite patterned beam toward the scanned object, wherein the composite pattern includes a multi-line pattern and an coded pattern, and the coded pattern uniquely encodes the multiple lines contained in the multi-line pattern; The receiving end is used to acquire the composite patterned beam reflected by the scanned object and generate a composite image, the composite image including a multi-line image and an coded image; The processor is configured to decode the multi-line image based on the encoded image to identify the multiple lines, and to calculate the depth information of the scanned object based on the identified multiple lines and the principle of line laser scanning.

2. The three-dimensional scanning system as described in claim 1, characterized in that, The processor is also used to acquire the intrinsic and extrinsic parameters of the transmitter and the receiver, and combine the intrinsic and extrinsic parameters with the depth information of the scanned object to obtain a point cloud map of the scanned object.

3. The three-dimensional scanning system as described in claim 2, characterized in that, The processor is also used to perform three-dimensional reconstruction of the point cloud map to obtain a three-dimensional model of the scanned object.

4. The three-dimensional scanning system according to any one of claims 1 to 3, characterized in that, The step of calculating the depth information of the scanned object using the identified multiple lines and based on the principle of line laser scanning includes: The equation of the light blade plane corresponding to each line in the multi-line pattern emitted by the transmitter is pre-calibrated; Each line in the multi-line image is identified to uniquely determine the light scalpel plane equation corresponding to each line. Any point on any line is extracted and a ray is formed from the optical center of the receiving end and any point of the currently extracted line. The intersection point of the ray and the light scalpel plane equation corresponding to the current line is calculated to obtain the depth information of the scanned object.

5. The three-dimensional scanning system according to any one of claims 1 to 3, characterized in that, The multiple lines projected by the transmitting end include those in a vertical or near-vertical direction; wherein, the vertical lines are defined as those whose extension direction is perpendicular to the baseline between the transmitting end and the receiving end.

6. The three-dimensional scanning system as described in claim 5, characterized in that, The encoding pattern uniquely encodes the multiple lines contained in the multi-line pattern by using the size or shape of the encoding pattern to uniquely encode each line.

7. The three-dimensional scanning system as described in claim 5, characterized in that, Each line in the multi-line pattern is formed by arranging multiple sets of coded patterns. The coded pattern is equivalent to the arrangement of the multiple sets of coded patterns in each line. The multiple sets of coded patterns are composed of one or more closely arranged coded patterns with a preset gap between each set. Different arrangement of coded pattern sets makes each line unique based on the coded pattern sets of different arrangement methods.

8. The three-dimensional scanning system as described in claim 5, characterized in that, At least one of the coded patterns is distributed near or along each line, wherein the unique encoding of the multiple lines contained in the multi-line pattern by the lines includes: uniquely encoding each line in the multi-line pattern according to the distribution position of the coded pattern.

9. The three-dimensional scanning system as described in claim 8, characterized in that, The step of uniquely encoding each line in the multi-line pattern according to the distribution position of the encoded pattern includes: Using a preset reference coding diagram where the coding patterns are distributed on the same horizontal line as a reference, the position of some coding patterns projected onto each line is adjusted vertically so that the adjusted coding pattern distribution position has at least two distribution states in the vertical direction. Each line is uniquely encoded by utilizing the distribution of at least two coded patterns that are horizontally adjacent to each line.

10. The three-dimensional scanning system as described in claim 9, characterized in that, Decoding the multi-line image based on the encoded image to identify the multiple lines includes: Each line in the multi-line pattern is pre-assigned a corresponding number, and the correspondence between the unique code of each line in the coded pattern and the number of each line is stored. The distribution position of the neighboring coded patterns of the current line is identified based on the pixel value difference of the neighboring pixels of the pixel region corresponding to the current line in the composite image, and the unique code of the current line is obtained based on the relative position of the coded patterns adjacent to the current line and the preset reference coded image. The unique code of the current line is searched from the correspondence to identify the multiple lines.

11. The three-dimensional scanning system as described in claim 8, characterized in that, When the line has a preset length, multiple coding patterns can be distributed along the extension direction of the line, and each line is uniquely coded by the distribution position of the coding patterns along the extension direction of the line.

12. The three-dimensional scanning system according to any one of claims 1 to 3, characterized in that, The transmitting end includes a light source and a pattern modulation element, wherein the light source is used to emit a light beam to the pattern modulation element, the pattern modulation element modulates the light beam emitted by the light source and projects the composite patterned light beam, including coded patterns and multi-line patterns, onto the scanned object.

13. The three-dimensional scanning system as described in claim 12, characterized in that, The emitting end also includes a collimating and homogenizing section disposed between the light source and the pattern modulation element, for collimating and homogenizing the light beam emitted by the light source and projecting it onto the pattern modulation element.

14. The three-dimensional scanning system as described in claim 13, characterized in that, The collimation and homogenization section includes a collimating element and a homogenizing element. The collimating element is used to collimate the light beam emitted by the light source to the homogenizing element. The homogenizing element is used to homogenize the light beam collimated by the collimating element and project it onto the pattern modulation element. The collimating element and the homogenizing element are either integrated or independently designed.

15. The three-dimensional scanning system as described in claim 14, characterized in that, The light-diffusing element includes any one or more combinations of compound eye lenses, diffusers, or microlenses.

16. The three-dimensional scanning system as described in claim 12, characterized in that, The transmitting end also includes a collimating element disposed between the light source and the pattern modulation element, used to collimate the light beam emitted by the light source to the pattern modulation element.

17. The three-dimensional scanning system according to any one of claims 13 to 16, characterized in that, The transmitter also includes a projection lens, which is used to collimate the light beam projected by the pattern modulation element onto the scanned object.

18. The three-dimensional scanning system according to any one of claims 13 to 16, characterized in that, When the light source is a single light source, the pattern modulation element includes at least one Mask or at least one DOE.

19. The three-dimensional scanning system as described in claim 18, characterized in that, When the pattern modulation element is a Mask or DOE, the Mask or DOE is photolithographically etched with a pattern that allows only the multi-line pattern and the coded pattern to pass through, so that the light beam emitted by the light source passes through the Mask or the DOE to obtain the composite patterned light beam.

20. The three-dimensional scanning system as described in claim 18, characterized in that, When the pattern modulation element includes multiple Masks or DOEs, each Mask or DOE can be photolithographically patterned with a pattern that allows only multi-line patterns or coded patterns to pass through, so that the light beam emitted by the light source passes through each Mask or DOE to obtain a composite patterned light beam.

21. The three-dimensional scanning system according to any one of claims 14 to 16, characterized in that, When the light source includes at least two sub-light sources, the at least two sub-light sources are used to emit light beams to the pattern modulation element respectively, and the pattern modulation element is used to modulate the light beams emitted by the at least two sub-light sources and project the multi-line pattern and the coded pattern onto the scanned object.

22. The three-dimensional scanning system as described in claim 21, characterized in that, The wavelengths of the at least two sub-light sources may be the same or different, and each sub-light source may share a pattern modulation element or have a pattern modulation element set on the light-emitting side of each sub-light source.

23. The three-dimensional scanning system as described in claim 22, characterized in that, When the light beams emitted by the at least two sub-light sources have different wavelengths, the emitting end further includes a beam homogenizing and combining element, which is disposed between the light source and the pattern modulation element, for homogenizing the light beams emitted with different wavelengths and integrating them into the same optical path to propagate to the pattern modulation element.

24. The three-dimensional scanning system as described in claim 23, characterized in that, The beam-splitting element includes a beam-splitting element and a beam-combining element. The beam-splitting element is used to homogenize and collimate the light beam emitted by the light source to the beam-combining element. The beam-combining element is used to integrate the light beams emitted by different light sources after being collimated by the collimating element into the same optical path and project them onto the pattern modulation element.

25. The three-dimensional scanning system as described in claim 24, characterized in that, The light-diffusing element includes any one of a compound eye lens, a diffuser, or a microlens, and the beam-combining element includes a dichroic mirror or a prism.

26. The three-dimensional scanning system according to any one of claims 22 to 25, characterized in that, When the wavelengths of the at least two sub-light sources are different, the at least two sub-light sources are a blue laser light source and a green laser light source, wherein the blue laser light source is used to form a blue multi-line pattern, and the green laser light source is used to form a green coded pattern.

27. The three-dimensional scanning system as described in claim 22, characterized in that, When the light beams emitted by the at least two sub-light sources have different wavelengths, the emitting end further includes a collimating and homogenizing section for collimating and homogenizing the light beams emitted by the light sources and propagating them to the pattern modulation element.

28. The three-dimensional scanning system as described in claim 27, characterized in that, The emitting end also includes a reflective element disposed in the optical path between the collimating and homogenizing section and the pattern modulation element, for reflecting the collimated and homogenized light beam to the pattern modulation element, thereby changing the emission optical path to achieve miniaturization.

29. The three-dimensional scanning system as described in claim 28, characterized in that, The transmitting end also includes a beam combining element, used to receive the multi-line pattern and coded pattern modulated by the pattern modulation element, and to synthesize a composite patterned beam and project it onto the scanned object.

30. The three-dimensional scanning system as described in claim 29, characterized in that, The transmitting end also includes a semi-transparent and semi-reflective element, disposed between the pattern modulation element and the light combining element, for transmitting or reflecting the multi-line pattern or coded pattern obtained by the pattern modulation element to the light combining element.

31. The three-dimensional scanning system according to any one of claims 27 to 30, characterized in that, When the wavelengths of the at least two sub-light sources are different, the at least two sub-light sources are blue laser light sources with different wavelengths.

32. The three-dimensional scanning system according to any one of claims 29 to 30, characterized in that, The transmitter also includes a projection lens, which is located on the light-emitting side of the light combining element. The image plane of the projection lens is located exactly on the light-emitting surface of the pattern modulation element, so that the coded pattern is exactly distributed and arranged along the multi-line pattern.

33. The three-dimensional scanning system as described in claim 22, characterized in that, When the wavelengths of the at least two sub-light sources are the same, the at least two sub-light sources are blue laser sources or near-infrared laser sources.

34. The three-dimensional scanning system according to any one of claims 1 to 3, characterized in that, The receiving end includes a black and white image sensor, which is used to acquire the composite patterned beam reflected back by the scanned object and generate the composite image, or only to acquire the multi-line pattern in the composite patterned beam reflected back by the scanned object and generate the multi-line image.

35. The three-dimensional scanning system as described in claim 34, characterized in that, The black and white image sensor has a filter on the light-incident side to suppress background light noise, so that only the composite patterned beam or the multi-line pattern in the composite patterned beam passes through the filter and is acquired by the black and white image sensor.

36. The three-dimensional scanning system as described in claim 34, characterized in that, The receiving end also includes a color image sensor, which is used to acquire the texture of the scanned object.

37. The three-dimensional scanning system as described in claim 36, characterized in that, When the black-and-white image sensor is used only to acquire the multi-line pattern in the composite patterned beam reflected back by the scanned object and generate a multi-line image, the color image sensor is also provided with a filter that matches the wavelength of the beam emitting the coded pattern, so that the color image sensor is used only to acquire the coded pattern in the composite patterned beam.

38. The three-dimensional scanning system as described in claim 37, characterized in that, The receiver also includes a beam splitter for splitting the reflected composite patterned beam so that the multi-line pattern and the coded pattern are acquired by the black-and-white image sensor and the color image sensor, respectively.

39. The three-dimensional scanning system according to any one of claims 35 to 38, characterized in that, The receiving end also includes an imaging lens for focusing the composite patterned beam reflected back from the scanned object onto the corresponding pixel in the image sensor.

40. The three-dimensional scanning system according to any one of claims 36 to 38, characterized in that, The three-dimensional scanning system further includes an illumination unit, which includes an illumination source and a collimation element. The illumination source emits a light beam to the collimation element, and the light beam collimated by the collimation element provides ambient light for the scanned object.

41. The three-dimensional scanning system according to any one of claims 36 to 38, characterized in that, The three-dimensional scanning system also includes a coaxial illumination unit, which is located on the light-incident side of the receiving end and is used to provide ambient light to the color image sensor so as to obtain a clear texture of the scanned object.

42. The three-dimensional scanning system as described in claim 41, characterized in that, The coaxial illumination unit includes an illumination source, a collimating lens, and a semi-transparent and semi-reflective optical element. The illumination source emits a light beam to the collimating lens, which then collimates the beam before it reaches the semi-transparent and semi-reflective optical element. A portion of the light beam is deflected by the semi-transparent and semi-reflective optical element to the object being scanned to provide supplemental illumination to the object.

43. The three-dimensional scanning system according to any one of claims 1 to 3, characterized in that, The three-dimensional scanning system also includes a deflecting optical element, which is used to receive a light beam emitted by the transmitting end and reflect it to the object being scanned to achieve scanning of the object being scanned, and is also used to receive a light beam reflected back by the object being scanned and deflect it to the receiving end.

44. The three-dimensional scanning system as described in claim 43, characterized in that, The deflecting optical element includes any one of a rotating mirror, a reflecting mirror, a prism, or a MEMS.

Images