Three-dimensional scanning system, three-dimensional scanning method and computer readable storage medium

Abstract

The invention provides a three-dimensional scanning system, a three-dimensional scanning method and a computer readable storage medium. The system comprises a transmitting end, a receiving end and a processor, a transmitting end transmits a composite line pattern light beam comprising a dense line pattern and a sparse line pattern to a scanned object, and each coding line in the sparse line pattern is at least aligned with a part of measurement lines in the dense line pattern to realize unique coding; a receiving end collects the composite line pattern light beam reflected by the scanned object and generates left and right composite line images, wherein the left and right composite line images comprise dense line images and sparse line images; and the processor decodes the dense line images in the left and right composite line images according to the arrangement numbers of the coding lines in the sparse line images in the left and right composite line images so as to identify a plurality of measurement lines, and performs stereo matching by using the identified measurement lines in the left and right dense line images so as to obtain depth information of the scanned object. According to the system provided by the invention, high-precision three-dimensional scanning information can be obtained while the cost is reduced.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional imaging technology, and in particular to a three-dimensional scanning system, method and computer-readable storage medium. Background Art

[0002] Existing three-dimensional scanning systems for teeth in the mouth generally use digital light processing (DLP) at the transmitting end to project preset multi-frame stripe patterns (such as Gray code or phase-shifted stripes) onto the surface of the object, which can easily obtain high measurement accuracy. However, the DLP hardware cost is high, the structure is complex, and the projection of multi-frame stripes is not conducive to dynamic three-dimensional measurement.

[0003] Based on the technology of speckle structured light (such as Kinect, Intel RealSense D435 / D455), a single frame of speckle pattern is projected onto the scene, and the depth of the target point is calculated through a matching algorithm. Its advantages are low cost and the ability to measure dynamic objects, but the accuracy is limited. There is a problem with scanning teeth with this solution: if the speckles are too dense, the speckles will be blurred due to the scattering caused by the translucent nature of the tooth surface, making matching difficult; if the speckles are too sparse, it will also cause difficulties in point correspondence. In order to achieve reliable point correspondence, some solutions use multiple cameras (such as 3 or more).

[0004] The key to using three-dimensional scanning technology with laser lines is to project a single or multiple laser lines onto the scene. Because each laser line corresponds to a light plane, once the number of a laser line in the camera image is identified, the corresponding light plane equation is determined. A ray can be determined based on the coordinates of the camera center point and a pixel point on the laser line in the image. The three-dimensional coordinates of the target point can be determined by the intersection of this ray and the light plane equation. The point cloud obtained by this method is usually sparse (for example, a single-frame scan can only obtain the point cloud of the area illuminated by a few or a dozen laser lines). In order to achieve the splicing of multi-frame scan point clouds, this method usually requires artificial marker points to be attached to the surface of the object. However, it is inconvenient to attach marker points on the surface of some targets, such as teeth in the mouth.

[0005] In order to achieve point cloud stitching without artificial markers, it is necessary to project more dense laser lines. In order to identify the number of laser lines, some schemes project laser lines of multiple colors and use the combination of different colors of adjacent laser lines in space to encode the laser line numbers. This scheme requires a color camera to capture the image of the laser line, and color cameras usually use a Bayer filter, which reduces the sensitivity of the camera. For example, when using blue or red laser lines, only 1 / 4 of all the pixels of the color camera have a sensitive response, and the other 3 / 4 of the pixels have almost no response. The color camera also has only half of the pixels that respond sensitively to the green laser line. Summary of the invention

[0006] The present application provides a three-dimensional scanning system, method and computer-readable storage medium, which do not require external marker points to be attached to the scanned object, and can reduce costs and power consumption while ensuring high-precision measurement results.

[0007] In a first aspect, a three-dimensional scanning system is provided, the system comprising a transmitting end, a receiving end and a processor, wherein the transmitting end is used to transmit a composite line pattern light beam to a scanned object, the composite line pattern comprising a dense line pattern consisting of M measuring lines and a sparse line pattern consisting of N coding lines, M and N are positive integers and M is greater than N, each coding line in the sparse line pattern is aligned with at least part of the measuring lines in the dense line pattern to achieve unique coding; the receiving end comprises a first camera and a second camera, the first camera is used to collect the composite line pattern light beam reflected by the scanned object and generate a first composite line image, the first composite line image comprises a first dense line image and a first sparse line image; The two cameras are used to collect the composite line pattern light beam reflected by the scanned object and generate a second composite line image, which includes a second dense line image and a second sparse line image; the processor is used to decode the first dense line image according to the arrangement number of each encoded line of the first sparse line image to identify multiple measurement lines of the first dense line image, and decode the second dense line image according to the arrangement number of each encoded line of the second sparse line image to identify multiple measurement lines of the second dense line image; the processor is also used to perform stereo matching using the identified multiple measurement lines of the first dense line image and the multiple measurement lines of the second dense line image to obtain depth information of the scanned object.

[0008] In a second aspect, a three-dimensional scanning method is provided, the method comprising: emitting a composite line pattern beam to a scanned object, the composite line pattern comprising a dense line pattern consisting of M measuring lines and a sparse line pattern consisting of N coding lines, M and N being positive integers and M being greater than N, and each coding line in the sparse line pattern being aligned with at least part of the measuring lines in the dense line pattern to achieve unique coding; collecting the composite line pattern beam reflected by the scanned object and generating a first composite line image and a second composite line image, the first composite line image comprising a first dense line image and a first sparse line image, and the second composite line image comprising a second dense line image and a second sparse line image; decoding the first dense line image according to the arrangement number of each coding line of the first sparse line image to identify multiple measuring lines of the first dense line image; decoding the second dense line image according to the arrangement number of each coding line of the second sparse line image to identify multiple measuring lines of the second dense line image; and performing stereo matching using the identified multiple measuring lines of the first dense line image and the multiple measuring lines of the second dense line image to obtain depth information of the scanned object.

[0009] In a third aspect, an electronic device is provided, comprising a processor and a memory, wherein the processor and the memory are connected, wherein the memory is used to store program code, and the processor is used to call the program code to execute a method in any possible implementation of the method design of the second aspect above.

[0010] In a fourth aspect, a computer-readable storage medium is provided, storing a computer program or instructions, wherein the computer program or instructions are used to implement the method in any possible implementation manner in the method design of the second aspect.

[0011] In a fifth aspect, a computer program product is provided, comprising instructions (or programs), which, when executed by a processor, enable the method in any possible implementation of the method design of the second aspect to be executed.

[0012] In the technical solution of the present application, high-precision three-dimensional scanning information can be obtained by using only a few light sources. While ensuring accuracy, it not only reduces costs and power consumption, but also eliminates the need to attach external marker points to the scanned object, thereby reducing labor costs and increasing the scanning rate, thereby improving the rate of three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a structural schematic diagram of a three-dimensional scanning system provided in an embodiment of the present application.

[0014] Figure 2 It is a schematic diagram of a multi-frequency multi-line pattern provided in an embodiment of the present application.

[0015] Figure 3It is a schematic diagram of line pattern number recognition provided in an embodiment of the present application.

[0016] Figure 4 It is a schematic diagram of a composite line pattern light beam provided in an embodiment of the present application.

[0017] Figure 5 It is a schematic diagram of the optical system structure of a three-dimensional scanning system provided in an embodiment of the present application.

[0018] Figure 6 It is a schematic diagram of the acquisition field of view of a three-dimensional scanning system provided in an embodiment of the present application.

[0019] Figure 7 This is a schematic diagram of the distribution positions of various structural components of a three-dimensional scanning system provided in an embodiment of the present application.

[0020] Figure 8 It is a flow chart of a three-dimensional scanning method provided in an embodiment of the present application.

[0021] Fig. 9 It is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0023] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In the present application, "at least one" means one or more, and "more" means two or more.

[0024] The prefixes such as "first" and "second" used in the embodiments of the present application are only used to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefixes such as ordinal numbers used to distinguish description objects in the embodiments of the present application does not constitute a limitation on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and the use of such prefixes should not constitute an unnecessary limitation.

[0025] Figure 1 It is a schematic diagram of the system structure of a three-dimensional scanning system provided according to the present application.

[0026] like Figure 1As shown, the three-dimensional scanning system 10 includes a transmitting end 110, a receiving end 120 and a processor (not shown in the figure), wherein the transmitting end 110 is used to transmit a composite line pattern light beam 111 to the scanned object, and the composite line pattern light beam 111 includes a dense line pattern 140 composed of M measuring lines and a sparse line pattern 130 composed of N coding lines, M and N are positive integers and M is greater than N, and multiple measuring lines in the dense line pattern 140 are uniquely encoded by the arrangement number of each coding line in the sparse line pattern 130, that is, each coding line in the sparse line pattern 130 is aligned with at least part of the measuring lines in the dense line pattern 140 to achieve unique encoding.

[0027] The receiving end 120 may include a left camera 121 and a right camera 122. The left and right cameras of the receiving end 120 are used to collect the composite line pattern beam 111 reflected by the scanned object, generate left and right composite line images, and transmit the generated left and right composite line images to the processor; the left and right composite line images both include dense line images and sparse line images. It should be noted that the left and right cameras in the present application include the left camera 121 and the right camera 122, the left camera 121 may also be referred to as the first camera, and the right camera 122 may also be referred to as the second camera; the left and right composite line images include the left composite line image and the right composite line image, the left composite line image may also be referred to as the first composite line image, and the right composite line image may also be referred to as the second composite line image. In the present application, the first camera is used to collect the composite line pattern beam 111 reflected by the scanned object and generate the first composite line image, the first composite line image includes the first dense line image and the first sparse line image; the second camera is used to collect the composite line pattern beam 111 reflected by the scanned object and generate the second composite line image, the second composite line image includes the second dense line image and the second sparse line image.

[0028] The processor is used to decode the dense line images in the left and right composite line images according to the arrangement numbers of the coded lines in the sparse line images in the left and right composite line images to identify multiple measurement lines, and to perform stereo matching using the identified measurement lines in the left and right dense line images to obtain depth information of the scanned object. Specifically, the processor is used to decode the first dense line image according to the arrangement numbers of the coded lines in the first sparse line image to identify multiple measurement lines of the first dense line image, and to decode the second dense line image according to the arrangement numbers of the coded lines in the second sparse line image to identify multiple measurement lines of the second dense line image; the processor is also used to perform stereo matching using the identified multiple measurement lines in the first dense line image and the multiple measurement lines in the second dense line image to obtain depth information of the scanned object.

[0029] Furthermore, after obtaining the depth information of the scanned object, the processor can, on the one hand, obtain the internal and external parameters of the transmitting end 110 and the receiving end 120 in the three-dimensional scanning system 10, and obtain the point cloud of the scanned object by combining the internal and external parameters and the depth information of the scanned object; on the other hand, the processor can also perform three-dimensional reconstruction on the obtained point cloud map of the scanned object to obtain a three-dimensional model of the scanned object.

[0030] In the present application, the transmitting end 110 can project multi-line laser images of multiple frequencies, and the receiving end 120 can use a binocular camera (including the left camera 121 and the right camera 122 mentioned above) to capture the image of the laser line. The image obtained by the binocular camera is used to decode the low-frequency laser line number. The low-frequency laser line has fewer line bundles and is easier to decode; on this basis, the high-frequency laser line number is decoded, so that a denser three-dimensional point cloud can be obtained, and a three-dimensional model of the scanned object can be obtained.

[0031] In some embodiments, Figure 1 As shown, the dense line pattern 140 includes a plurality of measuring lines 141, and the sparse line pattern 130 includes a plurality of encoding lines. Exemplarily, the measuring lines in the dense line pattern 140 and the encoding lines in the sparse line pattern 130 can be laser lines, and the plurality of laser lines are parallel to each other. Furthermore, the laser lines projected by the transmitting end 110 include a vertical direction, or a direction close to the vertical direction. It should be noted that the vertical laser line described in the embodiment of the present application is defined as the extension direction of the laser line being perpendicular to the baseline between the transmitting end 110 and the receiving end 120, that is, when the transmitting end 110 emits a horizontal laser line, when it is perpendicular to the baseline between the transmitting end 110 and the receiving end 120, it also belongs to the vertical laser line described in this embodiment.

[0032] At the transmitting end 110, a laser line can form a light plane (the light plane equation needs to be obtained through calibration), and the laser line is reflected by the surface of the object, forming an image of the laser line on the camera imaging plane of the receiving end 120. The intersection of a ray formed by a point (denoted as p) at the center of the laser line in the image starting from the optical center of the camera and the light plane can determine the three-dimensional coordinates of p. This is the simplest case with only one laser line.

[0033] However, when a single-line laser is used for 3D scanning, only one linear point cloud can be obtained per frame, which is too inefficient. In order to improve efficiency, multiple laser lines need to be used for scanning. When multiple laser lines are used, multiple light planes will be formed in space. The ray starting from a point on a laser line on the camera image will intersect with multiple light planes, thereby obtaining multiple intersection points. In order to determine the position of the target in space, it is necessary to identify the number of the laser line in the image, so as to uniquely determine the light plane corresponding to the laser line.

[0034] From the above analysis, it can be seen that for multi-line laser 3D scanning, the key is to identify the number of laser lines in the image. Therefore, the encoding of laser line numbers is very important.

[0035] Combine the following Figure 2 It is described how to use the encoding lines of the sparse line pattern 130 to uniquely encode multiple measurement lines of the dense line pattern 140. Figure 2 is a schematic diagram of a multi-frequency multi-line pattern provided by the present application. Figure 2 (a) is a sparse line pattern 130 projected by the transmitting end 110 at time t, Figure 2 (b) is a dense line pattern 140 projected by the transmitting end 110 at time t+1.

[0036] like Figure 2 As shown in (a), at a certain moment (such as moment t), a low-frequency multi-line pattern (such as a sparse line pattern 130) is projected through the transmitting end 110. The sparse line pattern 130 is Figure 2 As indicated by hollow lines, the sparse line pattern 130 may include 6 coding lines. At the same time, the left and right cameras of the receiving end 120 are controlled to respectively collect the left and right sparse line images pair_L.

[0037] like Figure 2 As shown in (b), at the next moment (such as moment t+1), a high-frequency multi-line pattern (such as a dense line pattern 140) is projected through the transmitting end 110. The dense line pattern 140 is Figure 2 As indicated by black lines in FIG. 1 , the dense line pattern 140 may include 26 measurement lines. At the same time, the left and right cameras of the receiving end 120 are controlled to respectively collect the left and right dense line images pair_H.

[0038] The measurement lines in the dense line pattern 140 are aligned and overlapped with at least one encoding line in the sparse line pattern 130, that is, each measurement line in the dense line pattern 140 and the encoding line aligned therewith are projected at the same position in space. Figure 2 As shown, the first coding line is aligned with the first measuring line, the second coding line is aligned with the sixth measuring line, etc. Thus, the unique coding of multiple measuring lines in the dense line pattern 140 can be determined by the arrangement number of the coding lines in the sparse line pattern 130 .

[0039] The coded lines in the sparse line image pair_L have a lower frequency, that is, there are fewer line bundles, and it is easier to obtain their numbers based on the acquired images. In some examples, if there is only a single line, then as long as the line is detected in the sparse line image pair_L, its number can be immediately determined (only one number). Once the number of the low-frequency line is determined, the number of the high-frequency line between any two low-frequency lines can also be determined. Therefore, the number of each coded line in the sparse line image should be determined first, and then the number of each measurement line in the dense line image can be determined based on the alignment relationship between the coded line in the sparse line image and the measurement line of the dense line image.

[0040] Combine the following Figure 3 The invention describes how to use the arrangement number of each coded line in the sparse line image to identify each measurement line in the dense line image. Figure 3 It is a schematic diagram of line pattern number recognition provided by this application.

[0041] like Figure 3 As shown, the working principle of the present application scheme is explained by taking the transmitting end (such as a multi-line pattern projector) projecting a sparse line pattern beam of 3 coded lines as an example. When the transmitting end projects the sparse line pattern beam to the surface of the object, the three-dimensional point P on the surface of the object is located on one of the coded lines, and its projection point in the first sparse line image of the left camera 121 is p, and its projection point in the second sparse line image of the right camera 122 is p′. However, in actual measurement, the sparse line pattern beam of 3 coded lines is first projected to the surface of the object by the transmitting end, and the image including the three-dimensional point P on the surface of the object is collected by the receiving end, and then the world three-dimensional coordinates of the point P on the surface of the object are estimated inversely based on the coordinate information in the image. That is, in the actual measurement process, the world three-dimensional coordinates of the three-dimensional point P on the surface of the object are unknown, and can only be calculated by collecting the sparse line image and the dense line image by the left camera 121 or the right camera 122.

[0042] Specifically, any point on each coding line in the first sparse line image or the second sparse line image is connected to the optical center of the left camera or the right camera to form a ray, and a ray equation corresponding to the ray and a light plane equation corresponding to a plurality of coding lines projected by the transmitting end are solved to obtain a plurality of intersection points, and the plurality of intersection points are reversely projected to the second sparse line image or the first sparse line image based on the projection constraint to determine the intersection points that just fall on the corresponding coding lines in the second sparse line image or the first sparse line image, so as to obtain the matching relationship between each coding line in the first sparse line image and the second sparse line image; and according to the matching relationship between each coding line in the first sparse line image and the second sparse line image, The number of each coded line in the first sparse line image and the second sparse line image is obtained based on the alignment relationship, so as to further identify the number of the measurement line of the first dense line image and the second dense line image based on the alignment relationship between each coded line in the first sparse line image and the second sparse line image and each measurement line in the first dense line image and the second dense line image, thereby completing the decoding process of identifying the measurement line of the dense line image using the coded line arrangement number of the sparse line image; the measurement lines of the first dense line image and the measurement lines of the second dense line image with the same number are used for stereo matching to obtain the depth information of the object surface, and then obtain the world three-dimensional coordinates of the point P on the object surface.

[0043] Taking the image coordinate p in the sparse line image captured by the left camera 121 as an example, the ray formed by the line connecting it with the optical center of the left camera forms three intersection points with the three light planes corresponding to the three coding lines respectively. The coordinates of the intersection points can be solved by the ray equation and the light plane equation, among which the light plane equation formed by the three coding lines is known and can be obtained by pre-calibration.

[0044] Specifically, based on the principle of line laser scanning, a coding line emitted by the transmitting end 110 to the scanned object can form a light plane, one light plane corresponds to a light plane equation, each light plane corresponds to a number, and the light plane equation can be obtained by calibration; the receiving end 120 collects the light beam reflected by the scanned object to form a sparse line image containing multiple coding lines on the camera imaging plane. The processor extracts any point on any coding line in the sparse line image and forms a ray from the optical center of the receiving end 120 and any point on the coding line, and the solution of the ray and the light plane equation is the intersection point.

[0045] Furthermore, the processor can also extract the center line on the coding line in the sparse line image through the center line extraction algorithm, and form a ray starting from the optical center of the receiving end 120 and any center point of the coding line. The intersection of the ray and the light plane can be obtained to determine the three-dimensional coordinates of the corresponding center point to obtain the intersection point. Compared with directly calculating the intersection point through any point on the coding line, the present application calculates the center line on the coding line to obtain the corresponding center point with sub-pixel coordinates, and solves the intersection point through the sub-pixel center point, which can improve the calculation accuracy.

[0046] As described in the above embodiment, taking the ray formed by any point on a coding line in the sparse line image collected from the left camera 121 and the optical center of the left camera as an example, it will have intersections with multiple light planes, but only one point among the multiple intersections is a 3D point belonging to the surface of the object. This real 3D point has a strong constraint: its projection point p′ on the sparse line image of the right camera 122 also falls on a line. According to the geometric constraint, multiple three-dimensional intersections are projected onto the sparse line image obtained by the right camera 122. If one of the multiple intersections happens to fall on a coding line on the sparse line image of the right camera 122, then this coding line and the coding line where point p is located in the sparse line image of the left camera 121 are formed by collecting the same line beam, thereby determining the matching relationship between the coding lines in the sparse line images of the left and right cameras; then according to the number of the light plane corresponding to the intersection, the number of the coding line can be confirmed. It should be noted that, when any point on a coded line in the sparse line image captured by the right camera 122 is connected to the optical center of the right camera to form a ray, the intersection with the light plane is projected onto the sparse line image captured by the left camera. The method for confirming the coded line number is similar to the above embodiment, and this application will not repeat it here.

[0047] Specifically, the formula for projecting the three-dimensional intersection of the space onto the camera plane is: c x=K[R|T]X, where Z c is the depth distance between the spatial point and the camera, x is the homogeneous image coordinate, R|T represents the rotation matrix and translation matrix between the camera and the world coordinate system of the scanned object, and K represents the 3×3 camera intrinsic parameter.

[0048] After determining the number of each coded line in the sparse line image, each measured line in the dense line image is further decoded based on the alignment relationship between the coded lines in the sparse line image and the measured lines in the dense line image to determine the number of each measured line in the dense line image.

[0049] After decoding each measurement line in the dense line images of the left and right cameras based on the above encoding and decoding principle, the measurement lines with the same number in the dense line images of the left and right cameras can be subjected to stereo matching calculation to obtain the depth information of the scanned object. Furthermore, after obtaining the depth information of the scanned object, the processor can also obtain a point cloud image of the scanned object based on the internal and external parameters of the transmitting end and the receiving end and the depth information of the scanned object; in addition, the processor can also perform three-dimensional reconstruction on the obtained point cloud image of the scanned object, and finally obtain a three-dimensional model of the scanned object.

[0050] Combination of the above Figures 1 to 3 The system design of the three-dimensional scanning system provided in this application is exemplified.

[0051] like Figure 1The three-dimensional scanning system 10 shown includes a transmitting end 110, a receiving end 120 and a processor; wherein the transmitting end 110 is used to transmit a composite line pattern light beam 111 to the scanned object; the receiving end 120 is used to collect the composite line pattern light beam 111 reflected back by the scanned object to generate left and right composite line images and transmit them to the processor; the processor is used to receive the left and right composite line images and process the left and right composite line images to obtain depth information of the scanned object.

[0052] In some embodiments, Figure 1 As shown, the transmitting end 110 may include a light source 101 and a pattern modulation element 102. The light source 101 is used to emit a light beam to the pattern modulation element 102. The pattern modulation element 102 modulates the light beam emitted by the light source 101 to obtain a composite line pattern light beam 111 including a dense line pattern 140 and a sparse line pattern 130. Among them, the sparse multi-line pattern 130 can be regarded as a subset of the dense line pattern 140, that is, the coding lines in the projected sparse multi-line pattern 130 are aligned and distributed with at least part of the measurement lines in the dense line pattern 140, so that the uniqueness of the measurement lines can be encoded by arranging and numbering the coding lines.

[0053] Exemplarily, the light source 101 may be a light emitting diode (LED), an edge-emitting laser (EEL), a vertical-cavity surface-emitting laser (VCSEL), or the like, or may be a light source array composed of a plurality of light sources. The light emitted by the light source 101 may be any one or more combinations of visible light, infrared light, blue light, green light, or ultraviolet light.

[0054] Exemplarily, the pattern modulation element 102 may be a mask or a micro-electro-mechanical system (MEMS) galvanometer.

[0055] In some embodiments, the light source 101 may be a dual-band LED light source, and the pattern modulation element 102 may be a mask. The dual-band LED and the mask are used to realize the projection of dense line patterns and sparse line patterns.

[0056] Exemplarily, the dual-band LED light source can emit a first band light beam and a second band light beam, and the mask includes a substrate with a preset pattern, and the preset pattern on the substrate can be as follows: Figure 4 As shown, the preset pattern includes a first area (white shadow area) and a second area (black shadow area).

[0057] In one example, the first area only allows the first band light beam to pass through, and the second area allows the first band light beam and the second band light beam to pass through. That is, when the dual-band LED light source emits the first band light beam through the mask, the first band light beam passes through the first area and the second area on the mask to form a Figure 2 When the dual-band LED light source emits a second-band light beam through the mask, the second-band light beam can only pass through the second area on the mask to form a dense line pattern as shown in (b); Figure 2 The sparse line pattern shown in (a) can then realize the projection of multi-line patterns with different densities.

[0058] In another example, the first area only allows the first band beam to pass through, and the second area only allows the second band beam to pass through. That is to say, the dual-band LED light source can simultaneously project beams of different bands / colors (such as the first band beam and the second band beam), and form a composite line pattern after passing through different areas on the mask, wherein the first band beam corresponds to a dense line pattern, and the second band beam corresponds to a sparse line pattern. Furthermore, the receiving end 120 collects a frame of composite line image, and obtains a dense line image and a sparse line image by identifying different color line patterns in the composite line image, wherein the first band beam corresponds to a dense line image, and the second band beam corresponds to a sparse line image. Furthermore, each coded line in the sparse line image is first decoded, and then each measurement line in the dense line image is decoded based on the sparse line image to obtain the encoding of each coded line and each measurement line.

[0059] For example, Figure 4 The white shaded area of ​​the mask shown can be designed to allow only the blue light source to pass through, and the black shaded area of ​​the mask can be designed to allow only the green light source to pass through. The blue light source and the green light source simultaneously emit light beams to form a multi-line image including sparse green coding lines and dense blue measurement lines. A small number of green coding lines are first identified according to color, and the green coding lines are decoded using the technical solution of the present application. Since the blue measurement lines are located between the green coding lines, the blue measurement lines can be decoded based on the numbers of the green coding lines.

[0060] Exemplarily, the above dual-band LED light source can be replaced by a laser light source, which can include laser sub-light sources of different bands (e.g., a blue laser sub-light source and a green laser sub-light source), the blue laser sub-light source correspondingly forms a dense line pattern, and the green laser sub-light source forms a sparse line pattern. The above method can also realize multi-line pattern projection with different densities.

[0061] In some examples, the preset pattern on the mask can be achieved by coating, for example, a light-transmitting film that only allows the first band light beam to pass through is coated in the first area, and two films that allow the first band light beam and the second band light beam to pass through are coated in the second area; or, a light-transmitting film that only allows the first band light beam to pass through is coated in the first area, and a light-transmitting film that only allows the second band light beam to pass through is coated in the second area.

[0062] In some examples, a collimating element may be provided between the light source 101 and the pattern modulation element 102 so that the light beam emitted by the light source 101 forms a parallel light beam after passing through the collimating element. The collimating element may be, for example, a fly-eye lens or a collimating lens.

[0063] In some examples, a projection lens may be further provided on the light emitting surface of the pattern modulation element 102 , and the projection lens is used to amplify the multi-line pattern light beam formed by the pattern modulation element 102 (such as a mask), thereby expanding the projection field of view.

[0064] In other embodiments, the pattern modulation element 102 may be a MEMS galvanometer. If a MEMS galvanometer is used to realize multi-line pattern projection with different densities, when the transmitting end 110 is an LED light source, a collimating element may be provided between the LED light source and the MEMS galvanometer so that the light beam emitted by the LED light source is a parallel light beam. When the parallel light beam is a line parallel light beam, a two-dimensional dense line pattern and a two-dimensional sparse line pattern may be formed by controlling the frequency of the one-dimensional deflection of the MEMS galvanometer to project to the scanned object; when the parallel light beam is a point parallel light beam, a line beam may be formed by firstly using the same one-dimensional deflection frequency, and then using another dimension of different polarization frequencies to form a two-dimensional dense line pattern and a two-dimensional sparse line pattern to project to the scanned object.

[0065] It should be understood that the use of MEMS galvanometers is generally combined with high-precision control methods, for example, control methods with closed-loop feedback. This control method is a prior art and is not limited in this application.

[0066] In some embodiments, the receiving end 120 includes a left camera 121 and a right camera 122, and the left camera 121 and the right camera 122 both include a black and white image sensor and a filter, and the black and white image sensor is used to collect the composite line pattern light beam reflected back from the scanned object and generate a corresponding composite line image or to collect the dense line pattern in the composite line pattern light beam reflected back from the scanned object and generate a dense line image for transmission to the processor.

[0067] Exemplarily, the black and white image sensor can be composed of any one or more of a charge-coupled device (CCD), a complementary metal oxide semiconductor (CMOS), an avalanche diode (AD), a single photon avalanche diode (SPAD), etc.

[0068] The filter is arranged on the light incident side of the black and white image sensor, and can be, for example, a narrowband filter that matches the wavelength of the light source. The filter is used to suppress the background light noise of the remaining bands, so that only the composite line pattern beam or the dense line pattern in the composite line pattern beam passes through the filter and is collected by the black and white image sensor.

[0069] In some examples, the left camera 121 and the right camera 122 also include an imaging lens, which is used to receive the composite line pattern light beam reflected by the scanned object, so that the composite line pattern light beam propagates to the filter and passes through the filter to be imaged on the corresponding pixel of the image sensor. Exemplarily, the imaging lens includes a lens or a lens group composed of multiple lenses.

[0070] In some embodiments, the receiving end 120 may also include a third camera, which includes a color image sensor (or RGB image sensor, texture image sensor, etc.), and the color image sensor is used to collect texture images to achieve texture mapping of the three-dimensional model to obtain a textured three-dimensional model.

[0071] Exemplarily, the color image sensor includes an image sensor and a Bayer filter arranged on the light incident side of the image sensor, which is used to collect the texture of the scanned object and transmit the collected texture to the processor; the processor renders the point cloud map of the scanned object according to the texture of the scanned object to obtain a texture point cloud model, or performs texture mapping on the three-dimensional model of the scanned object to obtain a texture three-dimensional model.

[0072] When the black and white image sensor only captures the dense line pattern in the composite line pattern light beam to generate a dense line image for transmission to the processor, the light incident side of the color image sensor may also be provided with a narrow-band filter that matches the wavelength of the light source emitting the sparse line pattern, so that the color image sensor is only used to capture the sparse line pattern in the composite line pattern light beam to generate a sparse line image and transmit it to the processor, so that the processor can identify multiple measurement lines in the dense line image based on the sparse line image.

[0073] Compared with color image sensors, black and white image sensors have better sensitivity and a higher signal-to-noise ratio for acquired images. Using black and white sensors to capture composite line pattern light beams is beneficial for high-precision measurement of the scanned object. In this embodiment, the color image sensor is mainly used to capture sparse line patterns for encoding or the texture of the scanned object, and will not be directly used for the measurement of the scanned object, so it will not affect the measurement accuracy.

[0074] In some embodiments, the processor may be an independent dedicated circuit, such as a system on a chip (SOC) chip, a field programmable gate array (FPGA) chip, an application-specific integrated circuit (ASIC) chip, etc., which includes a memory, a central processing unit (CPU), a bus, etc., and may also include a general processing circuit. Exemplarily, when the three-dimensional scanning system 10 is integrated into an electronic device such as a mobile phone, a television, a computer, a scanner, etc., the processing circuit of the electronic device may serve as at least a part of the processor.

[0075] Based on the system design of the three-dimensional scanning system provided in the embodiment of the present application, Figure 5 A specific optical system architecture of a three-dimensional scanning system is exemplarily shown.

[0076] Figure 5 Schematic diagram of the optical system structure of a three-dimensional scanning system provided by this application. Figure 5 As shown, the optical system 20 may include a transmitting end 200 and a receiving end, the receiving end includes a first camera 300 and a second camera 400 (corresponding to the left camera and the right camera mentioned above), the transmitting end 200 is used to transmit a composite line pattern light beam including a sparse line pattern and a dense line pattern to the scanned object 500, and the first camera 300 and the second camera 400 are used to collect the composite line pattern light beam reflected back by the scanned object 500 and generate a composite line image.

[0077] The transmitting end 200 includes a first light source 211, a second light source 212 and a mask 270. The first light source 211 and the second light source 212 are used to respectively transmit light beams to the mask 270. The light beam emitted by the first light source 211 can form a dense line pattern after being modulated by the mask 270 and projected onto the scanned object 500. The light beam emitted by the second light source 212 can form a sparse line pattern after being modulated by the mask 270 and projected onto the scanned object 500. For example, the light beams emitted by the first light source 211 and the second light source 212 can form the following pattern after passing through the mask 270: Figure 2The dense line pattern shown in (b) and Figure 2 The sparse line pattern shown in (a).

[0078] Exemplarily, different light sources emit light beams with different wavelengths. The first light source 211 may be a blue laser light source for emitting blue light to the mask 270 ; the second light source 212 may be a green laser light source for emitting green light to the mask 270 , wherein the blue laser light source can form a dense line pattern after being modulated by the mask 270 , and the green laser light source can form a sparse line pattern after being modulated by the mask 270 .

[0079] In some embodiments, the transmitting end 200 further includes a collimating element, and the collimating element (or collimating lens) is located between the light source and the mask 270. The collimating element may include a first collimating element 221 disposed on the light emitting side of the first light source 211 and a second collimating element 222 disposed on the light emitting side of the second light source 212, the first collimating element 221 being located between the first light source 211 and the mask 270, and the second collimating element 222 being located between the second light source 212 and the mask 270. The first collimating element 221 is used to collimate the light beam emitted by the first light source 211 to form a parallel light beam, and the second collimating element 222 is used to collimate the light beam emitted by the second light source 212 to form a parallel light beam. Exemplarily, the first collimating element 221 and / or the second collimating element 222 may be a lens or a lens group consisting of a plurality of lenses.

[0080] In some embodiments, the transmitting end 200 further includes a beam combining element 230, which is used to combine the parallel light beams of the first light source 211 and the parallel light beams of the second light source 212 into one parallel light beam. The beam combining element 230 is located between the light source and the mask 270. Specifically, the first light source 211 and the second light source 212 (or the first collimating element 221 and the second collimating element 222) are respectively set up on different light-entering sides of the beam combining element 230, and the mask 270 is set on the light-exiting side of the beam combining element 230. The first light source 211 and the second light source 212 can respectively emit light beams of different wavelengths to the beam combining element 230, and the beam combining element 230 is used to combine the parallel light beams of two different wavelengths into one parallel light beam and transmit it to the mask 270.

[0081] Exemplarily, the beam combining element 230 may be a dichroic mirror, which is used to transmit the light beam emitted by the first light source 211 to the light homogenizing element 240 and reflect the light beam emitted by the second light source 212 to the light homogenizing element 240 .

[0082] In some embodiments, the transmitting end 200 further includes a light homogenizing element 240, which is located at the light emitting side of the beam combining element 230. Specifically, the light homogenizing element 240 is located between the beam combining element 230 and the mask 270, and is used to homogenize the parallel light beams combined by the beam combining element 230 to form parallel light beams with balanced light energy. Exemplarily, the light homogenizing element 240 can be a fly-eye lens or a diffuser.

[0083] In some embodiments, the transmitting end 200 further includes a first relay lens 250 and a second relay lens 260, wherein the first relay lens 250 is located between the light homogenizing element 240 and the second relay lens 260, and the second relay lens 260 is located between the first relay lens 250 and the mask 270. The first relay lens 250 is used to expand the parallel light beam after homogenization and then project it to the second relay lens 260, and the second relay lens 260 is used to reshape the expanded light beam into a parallel light beam and project it to the mask 270.

[0084] In some embodiments, the transmitting end 200 further includes a projection lens 280, which is used to amplify the light beam projected by the mask 270 to the scanned object 500. Exemplarily, the projection lens 280 may include a single lens or a combination of multiple lenses, which is not limited here.

[0085] It should be noted that the first light source 211, the second light source 212, the first collimating element 221, the second collimating element 222, the beam combining element 230 and the light homogenizing element 240 may constitute a light emitting assembly, and the optical axes of the light emitting assembly and the first relay lens 250 and the second relay lens 260 are not located on the same horizontal line. Figure 6 As shown in the right figure, this allows the light beam emitted by the transmitting end 200 to fall within the collection field of view of the receiving end (including the first camera 300 and the second camera 400 mentioned above), and no AA alignment is required during the module assembly process of the transmitting end 200, so the installation is more convenient.

[0086] At the transmitting end 200, a collimator is provided on the light emitting side of the first light source 211 and the second light source 212, which is used to collimate the light beams emitted by the first light source 211 and the second light source 212 respectively to form parallel light beams to be transmitted to the beam combining element 230. The beam combining element 230 is used to combine the parallel light beams formed by the first light source 211 and the second light source 212 into one parallel light beam; the parallel light beams combined by the beam combining element 230 are formed into a parallel light beam with balanced light energy after the homogenization effect of the homogenization element 240 and propagate to the first relay lens 250. The first relay lens 250 is used to expand the parallel light beam after homogenization and emit it to the second relay lens 260; the second relay lens 260 is used to reshape the expanded light beam into a parallel light beam again and project it to the mask 270. The light beams of the first light source 211 and the second light source 212 are formed as follows after passing through the mask 270 respectively. Figure 2 The dense line pattern shown in (b) and Figure 2 The sparse line pattern shown in (a) is projected onto the scanned object 500 through the magnifying effect of the projection lens 280.

[0087] In some examples, the first camera 300 and the second camera 400 may each include a black-and-white image sensor, a reflector, and an imaging lens. The light beams reflected by the scanned object 500 are focused by the imaging lens and then enter the black-and-white image sensors of each camera through the reflector to generate a sparse line image and a dense line image. Figure 5 As shown, the first camera 300 includes a first black-and-white image sensor 310, a first reflector 320, and a first imaging lens 330. The first imaging lens 330 is used to focus the light beam reflected back by the scanned object 500. The first reflector 320 is used to reflect the focused light beam so that the light beam enters the first black-and-white image sensor 310. The first black-and-white image sensor 310 is used to collect the light beam to form a first sparse line image and a first dense line image. The second camera 400 includes a second black-and-white image sensor 410, a second reflector 420, and a second imaging lens 430. The second imaging lens 430 is used to focus the light beam reflected back by the scanned object 500. The second reflector 420 is used to reflect the focused light beam so that the light beam enters the second black-and-white image sensor 410. The second black-and-white image sensor 410 is used to collect the light beam to form a second sparse line image and a second dense line image.

[0088] Exemplarily, the first imaging lens 330 and / or the second imaging lens 430 may include one or more lenses. When the first imaging lens 330 and / or the second imaging lens 430 are multiple lenses, they may be doublet lenses.

[0089] In some examples, such as Figure 6As shown in the left figure, viewed from the baseline direction of the first black-and-white image sensor 310 and the second black-and-white image sensor 410, the first black-and-white image sensor 310 and the second black-and-white image sensor 410 are not located on the same straight line as the optical axes of their respective imaging lenses, but are arranged relatively tilted. Such an arrangement can cause the acquisition field of view of the image sensor to be tilted, so that the acquisition field of view of the first black-and-white image sensor 310 and the second black-and-white image sensor 410 overlap without the need for an angle design themselves, and no AA alignment is required during the installation process, thereby greatly improving the installation efficiency.

[0090] In some embodiments, Figure 5 As shown, the receiving end also includes a third camera 600, which includes a color image sensor 610, a third reflector 620 and a third imaging lens 630. The third imaging lens 630 is used to focus the light beam reflected back from the scanned object 500, and the third reflector 620 is used to reflect the focused light beam so that the light beam enters the color image sensor 610. The color image sensor 610 is used to collect texture information of the scanned object 500.

[0091] The spatial arrangement of the first camera 300, the second camera 400, the third camera 600 and the transmitting end 200 in the receiving end can refer to Figure 7 The third camera 600 and the transmitting end 200 are located on the same plane (such as the first plane), the first camera 300 and the second camera 400 are symmetrically arranged about the first plane, and the first plane is the plane where the transmitting end 200 and the third camera 600 are located. The transmitting light path of the transmitting end 200 is relatively longer than the receiving light path of each receiving end. Such an arrangement can miniaturize the volume of the three-dimensional scanner and is more in line with the ergonomic design of the oral scanner.

[0092] Combination Figure 5 It can be seen that the receiving optical path of the third camera 600 includes: natural light is focused on the third reflector 620 by the focusing effect of the third imaging lens 630, and enters the color image sensor 610 through the total reflection effect of the third reflector 620 to form an image. Exemplarily, the optical axis of the third imaging lens 630 is not located on the same straight line as the optical axis of the color image sensor 610, so as to facilitate the overlap with the field of view of other devices. Among them, the third imaging lens 630 can adopt a double cemented lens.

[0093] In some examples, considering that the three-dimensional scanning system (three-dimensional scanner) provided in the present application can be applied to the field of oral scanning, and in view of the dark environment within the oral cavity, in order to improve the imaging quality of the color image sensor 610, the third camera 600 can also be provided with a fill light (such as a white LED) to achieve fill light for the scanned object 500.

[0094] In summary, the present application provides a three-dimensional scanning system that combines dense line patterns and sparse line patterns, which can obtain high-precision three-dimensional scanning information using only a few light sources. While ensuring accuracy, it not only reduces costs and power consumption, but also eliminates the need to attach external marker points to the scanned object, thereby reducing labor costs and increasing the scanning rate, thereby improving the rate of three-dimensional reconstruction.

[0095] Combination of the above Figures 1 to 7 The three-dimensional scanning system provided by this application is introduced. Figure 8 and Fig. 9 The present invention introduces a three-dimensional scanning method and an electronic device provided by the present invention.

[0096] Figure 8 800 is a flow chart of a three-dimensional scanning method provided in an embodiment of the present application. Figure 1 In the three-dimensional scanning system 10 shown, the method 800 may include S810 to S850.

[0097] S810, emitting a composite line pattern light beam toward the scanned object.

[0098] The composite line pattern light beam includes a dense line pattern and a sparse line pattern, and each encoding line in the sparse line pattern is aligned with at least part of the measurement lines in the dense line pattern to achieve unique encoding.

[0099] Exemplary, combined Figure 1 As shown, the transmitting end 110 in the three-dimensional scanning system 10 can emit a composite line pattern light beam 111 to the scanned object, and the composite line pattern light beam 111 includes a sparse line pattern 130 and a dense line pattern 140, wherein the measuring lines in the dense line pattern 140 are aligned and overlapped with at least one coding line in the sparse line pattern 130, that is, each measuring line in the dense line pattern 140 and the coding line aligned therewith are projected at the same position in space.

[0100] Combination Figure 2 As shown, the sparse line pattern 130 includes multiple coding lines (6 coding lines), and the dense line pattern 140 includes multiple measuring lines (26 measuring lines). The first coding line is aligned with the first measuring line, and the second coding line is aligned with the sixth measuring line. The coding of the multiple measuring lines between the first coding line and the second coding line can be determined according to the arrangement numbers of the first coding line and the second coding line, thereby achieving unique coding.

[0101] S820, collecting the composite line pattern light beam reflected by the scanned object and generating a first composite line image and a second composite line image, the first composite line image comprising a first dense line image and a first sparse line image, the second composite line image comprising a second dense line image and a second sparse line image.

[0102] Exemplary, combined Figure 1 As shown, the first camera and the second camera of the receiving end 120 can respectively collect the composite line pattern light beam reflected by the scanned object, and generate a first composite line image and a second composite line image accordingly, and the first composite line image and the second composite line image respectively include a dense line image and a sparse line image. In other words, the first composite line image can include a first dense line image and a first sparse line image, and the second composite line image can include a second dense line image and a second sparse line image.

[0103] S830: Decode the first dense line image according to the arrangement numbers of the encoding lines of the first sparse line image to identify a plurality of measurement lines of the first dense line image.

[0104] Exemplarily, the processor in the three-dimensional scanning system 10 can decode the first dense line image according to the arrangement number of each encoding line of the first sparse line image to identify multiple measurement lines of the first dense line image. The encoding and decoding principle can be referred to Figure 3 and Figure 4 The relevant introduction in will not be repeated here.

[0105] S840, decoding the second dense line image according to the arrangement numbers of the encoding lines of the second sparse line image to identify a plurality of measurement lines of the second dense line image.

[0106] Exemplarily, the processor in the three-dimensional scanning system 10 can also decode the second dense line image according to the arrangement number of each encoding line of the second sparse line image to identify multiple measurement lines of the second dense line image. The encoding and decoding principle can be referred to Figure 3 and Figure 4 The relevant introduction in will not be repeated here.

[0107] S850: Perform stereo matching using the identified multiple measurement lines of the first dense line image and the identified multiple measurement lines of the second dense line image to obtain depth information of the scanned object.

[0108] Exemplarily, the processor in the three-dimensional scanning system 10 may perform stereo matching using the identified multiple measurement lines of the first dense line image and the identified multiple measurement lines of the second dense line image to obtain depth information of the scanned object.

[0109] In some embodiments, the method 800 may further include: acquiring internal and external parameters of the transmitting end and the receiving end; and obtaining a point cloud image of the scanned object by combining the internal and external parameters with the depth information of the scanned object.

[0110] In some embodiments, the method 800 may further include: performing three-dimensional reconstruction on the point cloud image of the scanned object to obtain a three-dimensional model of the scanned object.

[0111] Through the three-dimensional scanning method provided in the present application, high-precision three-dimensional scanning information can be obtained, such as high-precision three-dimensional scanning information of teeth. On the basis of ensuring accuracy, it can also reduce costs and power consumption, increase the rate of three-dimensional scanning, and thus increase the rate of three-dimensional reconstruction.

[0112] Fig. 9 9 is a schematic block diagram of an electronic device provided in an embodiment of the present application. The electronic device 900 includes a memory 910, a processor 920, and a communication interface 930. The memory 910, the processor 920, and the communication interface 930 are connected through an internal connection path, the memory 910 is used to store instructions, and the processor 920 is used to execute the instructions stored in the memory 910 to control the communication interface 930 to obtain information, or to enable the electronic device 900 to implement the aforementioned three-dimensional scanning method. Optionally, the memory 910 can be coupled to the processor 920 through an interface, or can be integrated with the processor 920.

[0113] It should be noted that the communication interface 930 uses a transceiver device such as, but not limited to, a transceiver. The communication interface 930 may also include an input / output interface.

[0114] The memory 910 stores one or more computer programs, which include instructions. When the instructions are executed by the processor 920, the electronic device 900 executes the three-dimensional scanning method in the above-mentioned embodiments.

[0115] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 920 or an instruction in the form of software. The method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in the processor for execution. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 910, and the processor 920 reads the information in the memory 910 and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0116] In some embodiments, the electronic device 900 may be a mobile phone, a television, a computer, a scanner, or other device, which is not limited in the present application.

[0117] The embodiment of the present application also provides a computer-readable storage medium, which stores a program code. When the computer program code is executed on a computer, the computer executes the above Figure 8 The three-dimensional scanning method shown.

[0118] The present application also provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above Figure 8 The three-dimensional scanning method shown.

[0119] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0120] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0121] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0122] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0123] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0124] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0125] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A three-dimensional scanning system, characterized in that: include: A transmitting end, used for transmitting a composite line pattern light beam to a scanned object, wherein the composite line pattern light beam comprises a dense line pattern composed of M measuring lines and a sparse line pattern composed of N encoding lines, where M and N are positive integers and M is greater than N, and each encoding line in the sparse line pattern is aligned with at least part of the measuring lines in the dense line pattern to achieve unique encoding; A receiving end includes a first camera and a second camera, wherein the first camera is used to collect the composite line pattern light beam reflected by the scanned object and generate a first composite line image including a first dense line image and a first sparse line image; the second camera is used to collect the composite line pattern light beam reflected by the scanned object and generate a second composite line image including a second dense line image and a second sparse line image; A processor is used to decode the first dense line image according to the arrangement number of each encoding line of the first sparse line image to identify multiple measurement lines of the first dense line image, and decode the second dense line image according to the arrangement number of each encoding line of the second sparse line image to identify multiple measurement lines of the second dense line image; and perform stereo matching using the identified multiple measurement lines of the first dense line image and the multiple measurement lines of the second dense line image to obtain depth information of the scanned object.

2. The three-dimensional scanning system according to claim 1, characterized in that: The processor is further used to obtain internal and external parameters of the transmitting end and the receiving end, and obtain a point cloud or a three-dimensional model of the scanned object by combining the internal and external parameters with the depth information of the scanned object.

3. The three-dimensional scanning system according to claim 1 or 2, characterized in that: Decoding the first dense line image or the second dense line image according to the arrangement numbers of the encoded lines in the first sparse line image or the second sparse line image to identify a plurality of measurement lines of the first dense line image or the second dense line image, comprising: Acquire a ray formed by connecting any point on each coding line in the first sparse line image or the second sparse line image with the optical center of the first camera or the second camera, and solve a ray equation corresponding to the ray and a light plane equation corresponding to a plurality of coding lines projected by the transmitting end to obtain a plurality of intersection points; Back-projecting the plurality of intersection points to the second sparse line image or the first sparse line image based on a projection constraint to determine intersection points that just fall on corresponding coding lines in the second sparse line image or the first sparse line image, thereby obtaining a matching relationship between each coding line in the first sparse line image and the second sparse line image; The number of each coded line in the first sparse line image and the second sparse line image is obtained according to the matching relationship between each coded line in the first sparse line image and the second sparse line image, and the number of the measurement line of the first dense line image and the second dense line image is identified based on the alignment relationship between each coded line in the first sparse line image and the second sparse line image and each measurement line in the first dense line image and the second dense line image.

4. The three-dimensional scanning system according to claim 1 or 2, characterized in that: The transmitting end includes a light source and a pattern modulation element, the light source is used to emit a light beam to the pattern modulation element, and the pattern modulation element is used to modulate the light beam emitted by the light source and project the composite line pattern light beam including the dense line pattern and the sparse line pattern to the scanned object.

5. The three-dimensional scanning system according to claim 4, characterized in that: The light source includes at least two sub-light sources, and the at least two sub-light sources are used to respectively emit light beams to the pattern modulation element. After being modulated by the pattern modulation element, the light beams respectively project the dense line pattern and the sparse line pattern onto the scanned object.

6. The three-dimensional scanning system according to claim 5, characterized in that: The light source is used to emit a first-band light beam and a second-band light beam; the pattern modulation element is a mask plate, and the mask plate includes a first area and a second area, the first area allows the first-band light beam to pass through, and the second area allows the first-band light beam and the second-band light beam to pass through; or, the first area allows the first-band light beam to pass through, and the second area allows the second-band light beam to pass through.

7. The three-dimensional scanning system according to claim 4, characterized in that: The emitting end further comprises a collimating element, which is disposed between the light source and the pattern modulation element and is used for collimating the light beam emitted by the light source to form a parallel light beam.

8. The three-dimensional scanning system according to claim 7, characterized in that: The pattern modulation element is a MEMS galvanometer; When the parallel light beam is a line parallel light beam, a two-dimensional dense line pattern and a two-dimensional sparse line pattern are formed by controlling the frequency of the one-dimensional deflection of the MEMS galvanometer, and projected onto the scanned object; When the parallel light beam is a point parallel light beam, by controlling the frequency of the two-dimensional deflection of the MEMS galvanometer, a line beam is first formed through one-dimensional same deflection frequency, and then a two-dimensional dense line pattern and a two-dimensional sparse line pattern are formed through another dimensional different polarization frequency, and are projected onto the scanned object.

9. The three-dimensional scanning system according to claim 7, characterized in that: The transmitting end further comprises a beam combining element, which is arranged between the collimating element and the pattern modulation element and is used for combining parallel light beams of different wavelengths into one parallel light beam.

10. The three-dimensional scanning system according to claim 9, characterized in that: The transmitting end further includes a light homogenizing element, which is disposed between the beam combining element and the pattern modulation element and is used to homogenize the parallel light beams combined by the beam combining element.

11. The three-dimensional scanning system according to claim 10, characterized in that: The transmitting end further includes a first relay lens and a second relay lens, the first relay lens is located between the light homogenizing element and the second relay lens, and the second relay lens is located between the first relay lens and the pattern modulation element; The first relay lens is used to expand the parallel light beam after homogenization and then project it to the second relay lens, and the second relay lens is used to reshape the expanded light beam into a parallel light beam and project it to the pattern modulation element.

12. The three-dimensional scanning system according to claim 11, characterized in that: The optical axes of the first relay lens, the second relay lens, and the light-emitting assembly including the light source, the collimating element, the beam combining element, and the light homogenizing element are not located on the same horizontal line.

13. The three-dimensional scanning system according to any one of claims 5 to 12, characterized in that: The first camera and the second camera include black-and-white image sensors, and the black-and-white image sensors are used to collect the composite line pattern light beam reflected by the scanned object and generate a corresponding composite line image.

14. The three-dimensional scanning system according to claim 13, characterized in that: The receiving end further includes a third camera, and the third camera includes a color image sensor, and the color image sensor is used to collect the texture of the scanned object.

15. The three-dimensional scanning system according to claim 14, characterized in that: The first camera, the second camera and the third camera each include an imaging lens, wherein the image sensors in the first camera, the second camera and the third camera and the optical axes of the imaging lens are not located on the same straight line.

16. The three-dimensional scanning system according to claim 14, characterized in that: The third camera and the transmitting end are located on the same plane, the first camera and the second camera are symmetrically arranged about a first plane, and the first plane is the plane where the transmitting end and the third camera are located; the transmitting light path of the transmitting end is longer than the receiving light path of the receiving end.

17. A three-dimensional scanning method, applied to a three-dimensional scanning system including a transmitting end and a receiving end, characterized in that: include: Emittering a composite line pattern beam toward a scanned object, the composite line pattern beam comprising a dense line pattern consisting of M measuring lines and a sparse line pattern consisting of N encoding lines, M and N being positive integers and M being greater than N, each encoding line in the sparse line pattern being aligned with at least part of the measuring lines in the dense line pattern to achieve unique encoding; Collecting the composite line pattern light beam reflected by the scanned object and generating a first composite line image and a second composite line image, wherein the first composite line image includes a first dense line image and a first sparse line image, and the second composite line image includes a second dense line image and a second sparse line image; Decoding the first dense line image according to the arrangement numbers of the encoding lines of the first sparse line image to identify a plurality of measurement lines of the first dense line image; Decoding the second dense line image according to the arrangement numbers of the respective encoding lines of the second sparse line image to identify a plurality of measurement lines of the second dense line image; Stereo matching is performed using the identified multiple measurement lines of the first dense line image and the identified multiple measurement lines of the second dense line image to obtain depth information of the scanned object.

18. The three-dimensional scanning method according to claim 17, characterized in that: The method further comprises: Acquire internal and external parameters of the transmitting end and the receiving end; The point cloud or three-dimensional model of the scanned object is obtained by combining the internal and external parameters and the depth information of the scanned object.

19. The three-dimensional scanning method according to claim 17 or 18, characterized in that: Decoding the first dense line image or the second dense line image according to the arrangement numbers of the encoded lines in the first sparse line image or the second sparse line image to identify a plurality of measurement lines of the first dense line image or the second dense line image, comprising: Acquire a ray formed by connecting any point on each coding line in the first sparse line image or the second sparse line image with the optical center of the first camera or the second camera in the receiving end, and solve the ray equation corresponding to the ray and the light plane equation corresponding to the multiple coding lines projected by the transmitting end to obtain multiple intersection points; Back-projecting the plurality of intersection points to the second sparse line image or the first sparse line image based on a projection constraint to determine intersection points that just fall on corresponding coding lines in the second sparse line image or the first sparse line image, thereby obtaining a matching relationship between each coding line in the first sparse line image and the second sparse line image; The number of each coded line in the first sparse line image and the second sparse line image is obtained according to the matching relationship between each coded line in the first sparse line image and the second sparse line image, and the number of the measurement line of the first dense line image and the second dense line image is identified based on the alignment relationship between each coded line in the first sparse line image and the second sparse line image and each measurement line in the first dense line image and the second dense line image.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 17 to 19.

Images