Three-dimensional scanning device, method, apparatus, storage medium and processor
The three-dimensional scanning device and method address the inefficiency of multiple image sequences by projecting stripe-encoded images within a defined interval, ensuring encoding uniqueness and simplifying spatial encoding, thereby enhancing scanning efficiency.
Patent Information
- Application Number
- JP2023540479
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-12-31
- Filing Date
- 2021-12-31
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-12-31
AI Technical Summary
Structured optical encoding patterns in three-dimensional scanning require multiple image sequences for complex encoding, leading to inefficiencies.
A three-dimensional scanning device and method that projects a stripe-encoded image within a defined imaging interval, ensuring encoding uniqueness using fewer image sequences by controlling the effective depth of field and optical parameters, allowing for simpler spatial encoding and decoding.
This approach enhances scanning efficiency by reducing the need for multiple image sequences and complex encoding, improving the speed and accuracy of dynamic scanning.
Smart Images

Figure 0007581528000001 
Figure 0007581528000002 
Figure 0007581528000003
Abstract
Description
[Technical field]
[0001] <CROSS REFERENCE TO RELATED APPLICATIONS> This application claims priority to Chinese patent application No. 2020116406858, filed with the China Patent Office on December 31, 2020, for "Three-dimensional scanning method, apparatus, storage medium and processor," and to Chinese patent application No. 2020116421453, filed with the China Patent Office on December 31, 2020, for "Three-dimensional scanning apparatus and method," the entire contents of which are incorporated herein by reference.
[0002] The present invention relates to the field of three-dimensional scanning, and more particularly to three-dimensional scanning devices, methods, apparatus, storage media and processors. [Background technology]
[0003] At present, the method of obtaining dental model data in the field of dental diagnosis and treatment internationally is gradually shifting from impression three-dimensional scanning to intraoral three-dimensional scanning technology. The emergence of this technology can be said to be a new revolution in digital processing of teeth. This technology can abandon the method of obtaining dental model data by impression, molding, and three-dimensional scanning, and directly scan the oral cavity to obtain three-dimensional data of teeth. In terms of process time, the two steps of impression and molding are omitted; in terms of cost, the material costs, labor costs, and model delivery costs required for the above process are saved; and in terms of customer experience, the discomfort during impression can be avoided. From the above advantages, it is obvious that this technology will grow greatly. It will gain great benefits in the market.
[0004] 3D digital intraoral scanner, also known as intraoral scanner, is a device that uses a probe-type optical scan head to directly scan the inside of a patient's oral cavity to obtain three-dimensional shape and color texture information of the surfaces of soft and hard tissues such as teeth, gums, and mucosa. One method of this device is to use the principle of active structured light triangulation imaging to project an active light pattern using a digital projection system, and after the camera imaging system captures the pattern, it immediately performs 3D reconstruction and stitching through algorithm processing.
[0005] When designing structured light coding patterns, decoding of the whole image is usually considered, such as temporal phase unwrapping, spatial phase unwrapping, etc. In addition to obtaining the wrapped phase, the periodicity problem of the wrapped phase needs to be solved by obtaining the true absolute phase. To globally unwrap the phase, more image sequences or more complex spatial encoding and decoding processes are usually required.
[0006] No effective solution has yet been proposed to the problem that prior art structured light coding patterns require multiple image sequences for complex coding. Summary of the Invention [Problem to be solved by the invention]
[0007] Embodiments of the present invention provide a 3D scanning device, method, apparatus, storage medium and processor to overcome the problem that prior art structured light coding patterns require multiple image sequences for complex coding. [Means for solving the problem]
[0008] According to one aspect of an embodiment of the present invention, there is provided a three-dimensional scanning device comprising a projection device configured to project a stripe-encoded image including a first stripe group onto a scanning object, and a camera configured to image (collect) the scanning object to obtain a camera image, wherein the camera image is an image of the scanning object in an image plane of the camera, the image plane including a first imaging section, and when the scanning object is within an effective depth of field range of the three-dimensional scanning device, the image of the first stripe group in the image plane is within the first imaging section, and only the first stripe group is within the first imaging section.
[0009] Alternatively, a system angle α is formed between the projection optical axis of the projection device and the imaging optical axis of the camera, and the optical parameters of the camera include a lens magnification k1, where ΔL is the effective depth of field of the three-dimensional scanning device and d1 is the first imaging section, where d1 = ΔL × tgα ÷ k1.
[0010] Optionally, the projection device includes an image display element including a first display section in which the first stripe group is provided, and the optical parameters of the projection device include a lens magnification k2 of the projection device, where D1 is the first display section, and D1 = ΔL × tgα ÷ k2.
[0011] Optionally, the stripe-encoded image further includes a second stripe group adjacent to the first stripe group, the image plane includes a second imaging section adjacent to the first imaging section, and when the scanned object is within the effective depth of field of the three-dimensional scanning device, the second stripe group is imaged at the image plane within the second imaging section, and only the second stripe group is present within the second imaging section.
[0012] Optionally, the stripe-coded image includes a plurality of periodically arranged stripes, and the first stripes and the second stripes are each one cycle of stripes.
[0013] Optionally, a system angle α is formed between the projection optical axis of the projection device and the imaging optical axis of the camera, and the optical parameters of the camera include a lens magnification k1, where ΔL is the effective depth of field of the three-dimensional scanning device and d2 is the second imaging section, and d2=ΔL×tgα÷k1.
[0014] Optionally, the projection device includes an image display element including a second display section in which the second stripe group is provided, and the optical parameters of the projection device include a lens magnification k2 of the projection device, and when the second display section is D2, D2 = ΔL × tgα ÷ k2.
[0015] Optionally, the apparatus further comprises a processor configured to perform a three-dimensional reconstruction of the scanned object based on the camera images.
[0016] Optionally, the first imaging section coordinates are preset in the processor, and the processor determines pixel coordinates of centers of stripes in the camera image based on the camera image, determines serial numbers of each stripe in the camera image based on the pixel coordinates of the stripe and the coordinates of the first imaging section, and performs three-dimensional reconstruction based on the pixel coordinates of the stripe centers and the serial numbers to obtain a three-dimensional digital model of the scanned object.
[0017] Optionally, the processor is configured to preset a light plane and a corresponding serial number of each stripe in the stripe-encoded image, and the processor determines a light plane corresponding to a pixel coordinate of a center of the stripe based on the consistency between the serial number of each stripe in the camera image and the serial number corresponding to the light plane of each stripe, and performs triangulation based on the pixel coordinate of the center of the stripe and the corresponding light plane to reconstruct a three-dimensional digital model of the scanned object.
[0018] According to another aspect of an embodiment of the present invention, a three-dimensional scanning method is provided based on the above three-dimensional scanning device, comprising the steps of projecting a stripe-encoded image onto a scanning object by a projection device, capturing an image of the scanning object by a camera to obtain a camera image, the camera image being an image of the scanning object in an image plane of the camera, the image plane including a first imaging section, and when the scanning object is within the effective depth of field range of the three-dimensional scanning device, the image of the first stripe group in the image plane is within the first imaging section and only the first stripe group is present in the first imaging section, and performing a three-dimensional reconstruction of the scanning object by a processor based on the camera image.
[0019] Optionally, the method further includes the steps of determining pixel coordinates of centers of stripes in the camera image based on the camera image; coordinates of a first imaging section are preset in the processor, and determining a serial number of each stripe based on the pixel coordinates of the stripe and the coordinates of the first imaging section; and three-dimensionally reconstructing the pixel coordinates of the centers of the stripes based on the serial numbers to obtain a three-dimensional digital model of the scanned object.
[0020] According to another aspect of an embodiment of the present invention, there is provided a three-dimensional scanning method including the steps of projecting a stripe-coded image onto a scanned object, the stripe-coded image including a time-coded image or a color-coded image, the time-coded image including a plurality of time stripe patterns arranged in time, and the color-coded image including a color stripe pattern coded using a plurality of colors; capturing a three-dimensional reconstruction image of the scanned object, the stripe-coded image being on a surface of the scanned object in the three-dimensional reconstruction image; and reconstructing a three-dimensional model of the scanned object based on the three-dimensional reconstruction image. R . Effect of the Invention
[0021] In an embodiment of the present invention, a projection device is configured to project a stripe-coded image including a first group of stripes onto a scanning object, a camera is configured to capture the scanning object to obtain a camera image, the camera image is an image of the scanning object in an image plane of the camera, the image plane includes a first imaging section, when the scanning object is within an effective depth of field range of the three-dimensional scanning device, the image of the first group of stripes in the image plane is within the first imaging section, and only the first group of stripes is within the first imaging section, and the three-dimensional scanning device is configured to project the stripe-coded image according to the linear propagation characteristics of light not beyond the first imaging section limited by the hardware configuration of the three-dimensional scanning device. In order to achieve this, the first imaging section is made into one coding cycle, and the coding uniqueness of the stripe-coded image is ensured within the coding cycle, so that it is possible to ensure the coding uniqueness using less coding information of the stripe-coded image (i.e., less image sequences or less spatial coding), which can be used in combination with optical characteristics without relying on a high hardware level, and the speed of dynamic scanning can be improved using less image sequences or simple spatial coding and decoding methods, thereby achieving the technical effect of improving scanning efficiency, and solving the problem in the prior art that structured light coding patterns require multiple image sequences for complex coding. [Brief description of the drawings]
[0022] The accompanying drawings described in this specification are used to provide a further understanding of the present invention and constitute a part of this application, and the exemplary embodiments of the present invention and the description thereof are used to explain the present invention and are not to be construed as unduly limiting the present invention. The accompanying drawings are as follows: [Figure 1] 1 is a schematic diagram of a three-dimensional scanning apparatus according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a schematic diagram of optical parameters of a lens according to an embodiment of the present invention. [Figure 3a] FIG. 1 is a schematic diagram 1 of a first time stripe pattern according to an embodiment of the present invention. [Figure 3b] FIG. 2 is a schematic diagram 2 of a second time stripe pattern according to an embodiment of the present invention. [Figure 3c] FIG. 3 is a schematic diagram of a third time stripe pattern according to an embodiment of the present invention. [Figure 3d] FIG. 2 is a schematic representation of an encoding table for a temporally encoded image according to an embodiment of the present invention; [Figure 4a] FIG. 2 is a schematic diagram of a color-coded image according to an embodiment of the present invention; [Figure 4b] FIG. 2 is a schematic diagram of an encoding table for a color-encoded image according to an embodiment of the present invention; [Diagram 5] 1 is a flow chart 1 of a three-dimensional scanning method according to an embodiment of the present invention. [Figure 6] 2 is a flow chart 2 of a three-dimensional scanning method according to an embodiment of the present invention. [Figure 7] FIG. 2 is a schematic diagram of a blocked encoding according to an embodiment of the present invention; [Figure 8] FIG. 13 is a schematic diagram of a reconfiguration of stripe offsets according to an embodiment of the present invention. [Figure 9] 1 is a schematic diagram of a three-dimensional scanning apparatus according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0023] In the following, in order to allow those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention, but it is obvious that the described embodiments are only a part of the embodiments of the present invention, and are not all of them. Based on the embodiments of the present invention, all other embodiments that those skilled in the art can obtain without creative efforts shall be included in the protection scope of the present invention.
[0024] In addition, the terms "first", "second", and the like in the present specification and claims and the above-mentioned attached drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It is to be understood that the data used in this manner can be exchanged, where appropriate, so that the embodiments of the present invention described herein can be practiced in an order other than that shown or described herein. Furthermore, the terms "comprise" and "have", as well as any variations thereof, are intended to be non-exclusive inclusive, e.g., a process, method, system, product, or apparatus consisting of a series of steps or units need not be limited to those steps or units expressly described, but may include other steps or units not expressly described or inherent to the process, method, product, or apparatus.
[0025] According to an embodiment of the present invention, there is provided an embodiment of a projection beam offset method based on three-dimensional scanning. It is noted that the steps illustrated in the flowcharts in the accompanying drawings may be performed by a computer system, such as a computer capable of executing a set of instructions, and that although a logical sequence is shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than as shown.
[0026] Figure 1 is a schematic diagram of a three-dimensional scanning apparatus according to an embodiment of the present invention, as shown in Figure 1, the apparatus comprises a projection apparatus 10 configured to project a stripe-coded image comprising a first set of stripes onto a scanned object, and a camera 12 configured to image the scanned object to obtain a camera image, where the camera image is an image of the scanned object in an image plane of the camera, the image plane comprising a first imaging zone, and when the scanned object is within the effective depth of field range of the three-dimensional scanning apparatus, the image of the first set of stripes in the image plane is within the first imaging zone, and only the first set of stripes is within the first imaging zone.
[0027] In an embodiment of the invention, a projection device is configured to project a stripe-coded image comprising a first set of stripes onto a scanning object, and a camera is configured to image the scanning object to obtain a camera image, where the camera image is an image of the scanning object in an image plane of the camera, the image plane comprising a first imaging zone, and where when the scanning object is within an effective depth of field range of the 3D scanning device, the image of the first set of stripes in the image plane is within the first imaging zone, and only the first set of stripes is within the first imaging zone. In the above-mentioned three-dimensional scanning device, in order to prevent the stripe-coded image from exceeding the first imaging section limited by the hardware configuration of the three-dimensional scanning device according to the linear propagation characteristics of light, the first imaging section is made into one encoding cycle, and the encoding uniqueness of the stripe-coded image is ensured within the encoding cycle, thereby making it possible to ensure the uniqueness of the encoding using less encoding information of the stripe-coded image (i.e., less image sequences or less spatial encoding), which can be used in combination with optical characteristics without relying on a high hardware level, and the speed of dynamic scanning can be improved using less image sequences or simple spatial encoding and decoding methods, thereby achieving the technical effect of improving scanning efficiency, and solving the problem in the prior art that structured light coding patterns require multiple image sequences for complex encoding.
[0028] Alternatively, as shown in FIG. 1, the effective depth of field ΔL is the distance from Z0 to Z2, where Z0 is the near point position and Z2 is the far point position, and when the scanning object is between Z0 and Z2, the camera can capture a clear image of the scanning object.
[0029] That is, the effective depth of field ΔL=front depth of field ΔL1+rear depth of field ΔL2, and the value of ΔL1+ΔL2 is in the range of 10 mm to 20 mm.
[0030] Alternatively, the magnification of the optical system of the camera is usually about 3:1, and the imaging section of the fixed projection light on the camera image (e.g., the first imaging section or the second imaging section) has a range of d, i.e., one cycle.
[0031] The technical solution to be protected in this application is that, usually, in a scanning scene aimed at a small field of view within the effective depth of field, due to the angle of the binocular system and the magnification of the optical lens, the stripe pattern of the same code value of the structured light will inevitably move within the width of the camera or projection device, and this movement depends on three things: the effective depth of field, the angle of the optical system and the magnification of the lens.
[0032] Optionally, the moving range includes the display section (such as the first display section or the second display section) of the projection device and the imaging section (such as the first imaging section or the second imaging section) of the camera, and when the optical parameters of the projection device and the camera in the three-dimensional scanning device are determined, the moving range is determined, and the uniqueness of the stripe coding within this moving range can be designed to ensure the uniqueness of the code value globally across the width. Due to the linear propagation characteristics of light, the light within this display section will not jump out of the imaging section.
[0033] Alternatively, the imaging movement range is one encoding cycle, in which the uniqueness of the encoding is ensured, and the cycle can have a small range depending on the optical design, and the uniqueness of the encoding can be ensured with less encoding information (less image sequences or less spatial encoding).
[0034] Fig. 2 is a schematic diagram of optical parameters of a lens according to an embodiment of the present invention, as shown in Fig. 2, the optical parameters include the focal plane of the lens, the circle of confusion for obtaining the front focal depth and the rear focal depth before and after the focal plane, and the effective depth of field of the lens, where the effective depth of field of the lens includes the front depth of field based on the front focal depth, the rear depth of field based on the rear focal depth, and the shooting distance between the position of the object point (i.e. the scanning object) and the lens, and the shooting distance includes the object distance between the lens and the scanning object, the near point distance between the near point of the depth of field and the lens, and the far point distance between the far point of the depth of field and the lens.
[0035] Alternatively, the lens shown in FIG. 2 may be a camera lens or a projection device lens.
[0036] Alternatively, if the lens shown in FIG. 2 is a camera lens, the scanned object can be set within the effective depth of field of the camera, and the camera image capturing the scanned object can be set within the focal depth range, and an imaging section (e.g., a first imaging section or a second imaging section) can be calculated based on optical parameters determined by the camera lens.
[0037] Alternatively, if the lens shown in FIG. 2 is a lens of a projection device, the negative film (or negative phase) of the stripe-encoded image can be set within the focal depth range of the projection device, and the scanned object can be set within the effective depth of field range of the projection device, and a display interval (e.g., the first display interval or the second display interval) can be calculated based on the optical parameters determined by the lens of the projection device.
[0038] As an optional example, a system angle α is formed between the projection optical axis of the projection device and the imaging optical axis of the camera, the optical parameters of the camera include a lens magnification k1, the effective depth of field of the three-dimensional scanning device is ΔL, and the first imaging section is d1, where d1 = ΔL × tgα ÷ k1.
[0039] As an optional example, the projection device has an image display element including a first display section in which a first stripe group is provided, and the optical parameters of the projection device include a lens magnification k2 of the projection device, and when the first display section is D1, D1 = ΔL × tgα ÷ k2.
[0040] As an optional example, the stripe-encoded image further includes a second stripe group adjacent to the first stripe group, the image plane includes a second imaging section adjacent to the first imaging section, and when the scanned object is within the effective depth of field of the three-dimensional scanning device, the second stripe group is imaged at the image plane within the second imaging section, and only the second stripe group is present within the second imaging section.
[0041] Alternatively, the first group of stripes is projected at a near point in the depth of field of the projection device and the second group of stripes is projected at a far point in the depth of field of the projection device, or the first group of stripes is projected at a far point in the depth of field of the projection device and the second group of stripes is projected at a near point in the depth of field of the projection device.
[0042] As an alternative embodiment, the stripe-encoded image includes a plurality of periodically arranged stripes, the first stripes and the second stripes each being one cycle of stripes.
[0043] Optionally, the stripe-coded image comprises a time-coded image or a color-coded image, the time-coded image comprising a plurality of time stripe patterns arranged in time, and the color-coded image comprising a color stripe pattern encoded using a plurality of colors.
[0044] As an alternative embodiment, a binary coding is used for the stripe coded image: in a time striped image, pixels with stripes are coded 1 and pixels without stripes are coded 0. In a color coded pattern, pixels with red stripes (R) are coded 100, pixels with blue stripes (B) are coded 001, pixels with green stripes (G) are coded 010 and pixels without stripes are coded 000. Of course, if there are only two color stripes, then two codes are sufficient, e.g. pixels with red stripes are coded 10, pixels with blue stripes are coded 01 and pixels without stripes are coded 00.
[0045] Optionally, when the stripe-coded image is a time-coded image, the time-coded image includes a plurality of time-stripe patterns, which are projected sequentially in time order, where the plurality of time-stripe patterns correspond to one coding cycle.
[0046] Alternatively, the first set of stripes may be a time stripe pattern projected at a first time and the second set of stripes may be a time stripe pattern projected at a second time.
[0047] Optionally, the first set of stripes may be a color-coded pattern.
[0048] As shown in Figure 3a, Figure 3a is a schematic diagram 1 of a first time stripe pattern according to an embodiment of the present invention, as shown in Figure 3b, Figure 3b is a schematic diagram 2 of a second time stripe pattern according to an embodiment of the present invention, and as shown in Figure 3c, Figure 3c is a schematic diagram 3 of a third time stripe pattern according to an embodiment of the present invention, and the three time stripe patterns shown in Figures 3a to 3c correspond to one encoding cycle, and by decoding each stripe in the three time stripe patterns in the encoding cycle, one time image encoding table can be obtained, and the order of each stripe to be projected can be determined by the encoding table.
[0049] As shown in Figure 3d, Figure 3d is a schematic diagram of a coding table of a time-coded image according to an embodiment of the present invention, in the time stripe patterns shown in Figures 3a to 3c, the same pixel positions take values in sequence (in binary code 0 or 1), and when the three time stripe patterns are arranged in sequence according to the acquisition time, the binary stripe coding shown in Figure 3d is obtained.
[0050] Here, the single cycle stripe of the first time stripe pattern is 10101000, and 10101000 may be set periodically and repeatedly in the first time stripe pattern, the single cycle stripe of the second time stripe pattern is 10001010, and 10001010 may be set periodically and repeatedly in the second time stripe pattern, the single cycle stripe of the third time stripe pattern is 11111111, and 11111111 may be set periodically and repeatedly in the third time stripe pattern, of course, the repeat cycle numbers of 10101000, 10001010, and 11111111 are the same, and during projection, the three time stripe patterns are projected in chronological order, for example, the first time stripe pattern is projected at the first projection time, the second time stripe pattern is projected at the second projection time, and the third time stripe pattern is projected at the third projection time.
[0051] Optionally, when acquiring a stripe-coded image of the camera image before 3D reconstruction, the above codes can be used to identify the stripes despite various challenging environments such as object boundaries, obstructions, and reflections that destroy the stripes, thus avoiding the problem of code ambiguity.
[0052] In addition, the three time-stripe coded images shown in Figures 3a to 3c are designed as one reconstruction cycle, and the decoding and reconstruction operations can be completed based on the three time-stripe coded images, which greatly reduces the time required for continuous acquisition of time-stripe coded images during dynamic scanning, and avoids problems such as image offset, image blurring, and decoding errors caused by fast motion.
[0053] Fig. 4a is a schematic diagram of a color-coded image according to an embodiment of the present invention, as shown in Fig. 4a, each stripe in the coding cycle is coded with a color, the more the number of colors, the easier it is to design the uniqueness of the coding, but at the same time, the more difficult it is to identify the color coding, and the more the number of colors, the more difficult it is to distinguish the difference between colors. If the number of stripes is controlled to 8, etc., it can be distinguished by 3 colors, and the complexity of coding and decoding is greatly reduced.
[0054] FIG. 4b is a schematic diagram of a coding table of a color-coded image according to an embodiment of the present invention. As shown in FIG. 4b, the stripe code is a 3-bit binary number obtained based on the coding values of different color stripe-coded images (which use binary coding of 0 or 1 to represent three-channel color information).
[0055] For example, the stripe-coded image shown in Fig. 4a includes three colors, and each color stripe corresponds to one coding sequence, where the coding sequence corresponding to the red stripe (R) is 100, the coding sequence corresponding to the blue stripe (B) is 001, and the coding sequence corresponding to the green stripe (G) is 010. Of course, the stripe-coded image may be a color stripe sequence arranged according to the de Bruin sequence, or a plurality of stripe sequences repeatedly set as a single cycle of the color stripe sequence arranged according to the de Bruin sequence.
[0056] Optionally, when acquiring a stripe-coded image of the camera image before 3D reconstruction, the above codes can be used to identify the stripes despite various challenging environments such as object boundaries, obstructions, and reflections that destroy the stripes, thus avoiding the problem of code ambiguity.
[0057] In addition, by taking the different color stripe-coded images shown in Figure 4a as one cycle and realizing a color stripe-coded image based on one simple color coding, the operations of decoding and reconstruction can be completed, which greatly shortens the sustained acquisition time of the image sequence required for one frame of three-dimensional data during dynamic scanning, reduces the complexity and computational amount of encoding and decoding, and avoids problems such as complex and time-consuming algorithms, decoding errors, etc. caused by too many color types.
[0058] As an optional example, a system angle α is formed between the projection optical axis of the projection device and the imaging optical axis of the camera, the optical parameters of the camera include a lens magnification k1, the effective depth of field of the three-dimensional scanning device is ΔL, and the second imaging section is d2, where d2 = ΔL × tgα ÷ k1.
[0059] As an optional example, the projection device has an image display element including a first display section in which a first stripe group is provided, and the optical parameters of the projection device include the lens magnification k2 of the projection device, and when the second display section is D2, D2 = ΔL × tgα ÷ k2.
[0060] Optionally, the value range of the system angle α is between 6° and 10°.
[0061] As an optional example, the processor is configured to perform a three-dimensional reconstruction of the scanned object based on the camera images.
[0062] As an optional example, the first imaging section coordinates are preset in the processor, the processor determines pixel coordinates of the centers of the stripes in the camera image based on the camera image, determines serial numbers of each stripe in the camera image based on the pixel coordinates of the stripes and the coordinates of the first imaging section, and performs three-dimensional reconstruction based on the pixel coordinates of the centers of the stripes and the serial numbers to obtain a three-dimensional digital model of the scanned object.
[0063] As an optional example, the second imaging section coordinates are preset in the processor, and the processor determines pixel coordinates of the centers of the stripes in the camera image based on the camera image, determines serial numbers of each stripe in the camera image based on the pixel coordinates of the stripe, the coordinates of the first imaging section and the coordinates of the second imaging section, and performs three-dimensional reconstruction based on the pixel coordinates of the centers of the stripes and the serial numbers to obtain a three-dimensional digital model of the scanned object.
[0064] As an optional embodiment, the light plane and corresponding serial number of each stripe in the stripe-encoded image are preset in the processor, and the processor determines the light plane corresponding to the pixel coordinates of the center of the stripe based on the consistency between the serial number of each stripe in the camera image and the serial numbers corresponding to the light plane of each stripe, and performs triangulation based on the pixel coordinates of the center of the stripe and the corresponding light plane to reconstruct a three-dimensional digital model of the scanned object.
[0065] FIG. 5 is a flow chart 1 of a three-dimensional scanning method according to an embodiment of the present invention. As shown in FIG. 5, based on the above three-dimensional scanning device, A step S502 of projecting a stripe-encoded image onto a scanning object by a projection device; Step S504 of imaging the object to be scanned with a camera to obtain a camera image, the camera image being an image of the object to be scanned in an image plane of the camera, the image plane including a first imaging zone, and when the object to be scanned is within a useful depth of field range of the 3D scanning device, the image of the first set of stripes in the image plane is within the first imaging zone, and only the first set of stripes is present in the first imaging zone; A step S506 is performed in which a processor performs a three-dimensional reconstruction of the scanned object based on the camera images.
[0066] In an embodiment of the invention, the projection device is configured to project a stripe-encoded image comprising a first set of stripes onto a scanning object, and the camera is configured to image said scanning object to obtain a camera image. Here, the camera image is an image of the scanned object in an image plane of the camera, the image plane includes a first imaging section, and when the scanned object is within an effective depth of field range of the three-dimensional scanning device, the image of the first stripe group in the image plane is within the first imaging section, and only the first stripe group is within the first imaging section; in order to prevent the stripe-coded image from exceeding the first imaging section limited by the hardware configuration of the three-dimensional scanning device according to the linear propagation characteristics of light, the three-dimensional scanning device makes the first imaging section one coding cycle, and ensures the coding uniqueness of the stripe-coded image within the coding cycle, thereby making it possible to ensure the coding uniqueness using coding information of a small number of stripe-coded images (i.e., a small number of image sequences or a small number of spatial coding), which can be used in combination with optical characteristics without relying on a high hardware level, and can improve the speed of dynamic scanning using a small number of image sequences or a simple spatial coding and decoding method, thereby achieving the technical effect of improving scanning efficiency, and solving the problem in the prior art that the structured light coding pattern requires multiple image sequences for complex coding.
[0067] As an optional embodiment, the method further includes a step of determining pixel coordinates of the centers of the stripes in the camera image based on the camera image; the coordinates of the first imaging section are preset in the processor; a step of determining a serial number of each stripe based on the pixel coordinates of the stripe and the coordinates of the first imaging section; and a step of three-dimensionally reconstructing the pixel coordinates of the centers of the stripes based on the serial numbers to obtain a three-dimensional digital model of the scanned object.
[0068] In an alternative embodiment, the object is imaged by a camera to obtain a camera image, where the camera image is an image of the object in an image plane of the camera, the image plane including a first imaging section and a second imaging section, and when the object is within the effective depth of field of the three-dimensional scanning device, the image of the first stripe group in the image plane is within the first imaging section and only the first stripe group is within the first imaging section, and the image of the second stripe group in the image plane is within the second imaging section and only the second stripe group is within the second imaging section. Note that when the object moves within the effective depth of field, the first stripe group moves within the first imaging section but does not go beyond the first imaging section, and the second stripe group moves within the second imaging section but does not go beyond the second imaging section.
[0069] As an alternative embodiment, the projection device is configured to project a stripe-coded image including a first group of stripes and a second group of stripes onto the object to be scanned, and the camera is configured to image the object to obtain a camera image, where the camera image is an image of the object to be scanned in an image plane of the camera, the image plane including a first imaging section and a second imaging section, and when the object to be scanned is within an effective depth of field range of the three-dimensional scanning device, the image of the first group of stripes in the image plane is within the first imaging section and only the first group of stripes is within the first imaging section, the image of the second group of stripes in the image plane is within the second imaging section and only the second group of stripes is within the second imaging section, and the three-dimensional scanning device divides the first imaging section into one encoding section so that the stripe-coded image does not exceed the first imaging section and the second imaging section defined by the hardware of the three-dimensional scanning device according to the linear propagation characteristics of light. By imaging only one cycle in the first imaging section, imaging only another coding cycle in the second imaging section, and ensuring the coding uniqueness of the stripe coding image within each coding cycle, it is possible to ensure the coding uniqueness using the coding information of fewer stripe coding images (i.e., fewer image sequences or fewer spatial coding), which can be used in combination with optical characteristics without relying on a high hardware level, and the speed of dynamic scanning can be improved using fewer image sequences or simple spatial coding and decoding methods, thereby achieving the technical effect of improving scanning efficiency, and further solving the problem that the structured light coding pattern in the prior art requires multiple image sequences for complex coding. In the same projection pattern, a stripe sequence with unique coding can be repeatedly set, reducing the difficulty of coding.
[0070] As an optional embodiment, the method further includes a step of determining pixel coordinates of the centers of stripes in the camera image based on the camera image; the coordinates of the first imaging section and the coordinates of the second imaging section are preset in the processor; a step of determining a serial number of each stripe based on the pixel coordinates of the stripe, the coordinates of the first imaging section and the coordinates of the second imaging section; and a step of three-dimensionally reconstructing the pixel coordinates of the centers of the stripes based on the serial numbers to obtain a three-dimensional digital model of the scanned object.
[0071] FIG. 6 is a flow chart 2 of a three-dimensional scanning method according to an embodiment of the present invention. As shown in FIG. 6, the method includes: a step S602 of projecting a stripe-coded image onto a scanned object, the stripe-coded image comprising a time-coded image or a color-coded image, the time-coded image comprising a plurality of time stripe patterns arranged in time, and the color-coded image comprising a color stripe pattern encoded using a plurality of colors; S604, capturing a three-dimensional reconstructed image of the scanned object, the three-dimensional reconstructed image having a stripe-encoded image on a surface of the scanned object; and step S606 of reconstructing a three-dimensional model of the scanned object based on the three-dimensional reconstructed image.
[0072] Here, a stripe-coded image is projected onto a scanned object, the stripe-coded image is modulated and deformed by the scanned object, and the obtained three-dimensional reconstructed image of the scanned object is a surface image of the scanned object, and the image includes the deformed stripe-coded image.
[0073] In an embodiment of the present invention, a stripe-coded image is projected onto a scanning object, the stripe-coded image including a time-coded image or a color-coded image, the time-coded image including a plurality of time stripe patterns arranged in time, and the color-coded image including a color stripe pattern encoded by a plurality of colors. A three-dimensional reconstruction image of the scanning object is captured, and the stripe-coded image is on the surface of the scanning object in the three-dimensional reconstruction image. By reconstructing a three-dimensional model of the scanning object based on the three-dimensional reconstruction image, the time-coded image or the color-coded image can make the stripe-coded image have a unique stripe code, and the objective of ensuring the uniqueness of the stripe code of the stripe-coded image is achieved, the technical effect of improving the speed of dynamic scanning is achieved, and the problem of complex coding of the projected image required for the three-dimensional scanning process is solved.
[0074] Optionally, in the colour-coded image, the colour stripe pattern comprises at least a first set of stripes.
[0075] Optionally, in the time-encoded image, the time stripe pattern of the first time projection may be a first set of stripes and the time stripe pattern of the second time projection may be a second set of stripes.
[0076] Fig. 7 is a schematic diagram of the embodiment of the present invention when the coding is interrupted, as shown in Fig. 7, P1-P8 are coding stripes. Here, P1 and P2 are the camera's field of view is blocked by the object (i.e., the object to be scanned) itself, and the side stripe lines are interrupted and incomplete, which leads to the incompleteness of monolithic data, and the coding information of P1-P2 is very close to the coding information of P6-P7, which may lead to code ambiguity and noise and spurious data in three-dimensional reconstruction. However, according to the technical solution of the present application, the stripe image can be identified based on the image coding table (e.g., time image coding table, color image coding table), which improves the identification efficiency of stripe coding.
[0077] In the above step S604 of capturing a three-dimensional reconstruction image of the scanned object, one or more images may be obtained by projecting the stripe-encoded image onto the scanned object, where if the stripe-encoded image is a time-encoded image, multiple images with the stripe-encoded image on the surface may be collected, and the three-dimensional reconstruction image may be determined based on the multiple collected images, and if the stripe-encoded image is a color-encoded image, one image with the stripe-encoded image on the surface may be collected, and the three-dimensional reconstruction image may be determined based on the image.
[0078] The above step S606 of reconstructing a three-dimensional model of the scanned object based on the three-dimensional reconstructed image includes reconstructing the three-dimensional model using a monocular stereoscopic system or a binocular stereoscopic system.
[0079] For example, in a process of reconstructing a three-dimensional model based on a binocular stereoscopic vision system equipped with camera A and camera B, in the process of capturing a three-dimensional reconstructed image of a scanned object, the three-dimensional reconstructed image captured by camera A is a first three-dimensional reconstructed image, and the three-dimensional reconstructed image captured by camera B is a second three-dimensional reconstructed image, and a three-dimensional model of the scanned object is reconstructed based on a common stripe code of the first three-dimensional reconstructed image and the second three-dimensional reconstructed image.
[0080] As another example, in a process of reconstructing a three-dimensional model based on a monocular stereoscopic vision system, a camera captures a three-dimensional reconstructed image, and reconstructs a three-dimensional model of the scanned object based on the stripes in the three-dimensional reconstructed image and the corresponding light planes.
[0081] In the following, for the sake of explanation, the content projected on the surface of the scanning object is a stripe pattern, and the stripe pattern includes a time stripe pattern (e.g., a first time stripe pattern, a second time stripe pattern, a third time stripe pattern, a fourth time stripe pattern, and a fifth time stripe pattern) and a color stripe pattern (e.g., a first color stripe pattern and a second color stripe pattern). The imaged content having the scanning object is a stripe image, and the stripe image has the scanning object, and the surface of the scanning object in the stripe image has a stripe pattern, and the stripe image includes a time stripe image (e.g., a first time stripe image, a second time stripe image, a third time stripe image, a fourth time stripe image, and a fifth time stripe image) and a color stripe image (e.g., a first color stripe image and a second color stripe image).
[0082] For example, after projecting a first time stripe pattern onto the object to be scanned, there is the first time stripe pattern projected onto the surface of the object to be scanned, and at this time, an image of the object to be scanned is captured (i.e., a first time stripe image is captured), and the captured first time stripe image has the object to be scanned and the first time stripe pattern projected onto the surface of the object to be scanned.
[0083] The relationships between other stripe patterns and stripe images are similar, so a description thereof will be omitted here.
[0084] As an optional embodiment, when the stripe-encoded image is a time-encoded image, the three-dimensional scanning method further includes the steps of projecting a first time stripe pattern onto a surface of the scanned object at a first time, and acquiring a first time stripe image of the surface of the scanned object, projecting a second time stripe pattern onto the surface of the scanned object at a second time, and acquiring a second time stripe image of the surface of the scanned object, and determining a time image encoding table based on the first time stripe image and the second time stripe image.
[0085] Optionally, the first time is earlier than the second time.
[0086] In the above embodiment of the present invention, at a first time, a first time stripe pattern is projected onto the surface of the scanned object, and a first time stripe image of the surface of the scanned object is captured; at a second time, a second time stripe pattern is projected onto the surface of the scanned object, and a second time stripe image of the surface of the scanned object is captured; and further, a stripe image encoding table is commonly defined based on the time sequence of the first time stripe image and the second time stripe image.
[0087] The captured first time stripe image is a first three-dimensional reconstructed image, which includes a first time stripe pattern modulated by the scanned object, and the captured second time stripe image is a second three-dimensional reconstructed image, which includes a second time stripe pattern modulated by the scanned object.
[0088] As an optional embodiment, the step of determining the time image encoding table based on the first time stripe image and the second time stripe image includes the steps of determining a first encoding table based on the first time stripe image, determining a second encoding table based on the second time stripe image, and constructing the time image encoding table based on the first encoding table and the second encoding table.
[0089] As an optional example, the step of determining the first encoding table based on the first time stripe image includes a step of setting pixels with stripes in the first time stripe image as a first encoding value and pixels without stripes in the first time stripe image as a second encoding value, and constructing the first encoding table based on the first encoding value and the second encoding value of the pixel position distribution of the first time stripe image; the step of determining the second encoding table based on the second time stripe image includes a step of setting pixels with stripes in the second time stripe image as a first encoding value and pixels without stripes in the second time stripe image as a second encoding value, and constructing the second encoding table based on the first encoding value and the second encoding value of the pixel position distribution of the second time stripe image; and the step of constructing the time image encoding table based on the first encoding table and the second encoding table includes arranging the encoding values of the same pixel position in the first encoding table and the second encoding table in the order of acquisition of the first time stripe image and the second time stripe image to form an encoding sequence of corresponding pixels, and constructing the time image encoding table based on the encoding sequence.
[0090] As an alternative embodiment, the coding table is binary coded, with a coding value of 1 corresponding to a pixel with stripes in the temporally coded image and a coding value of 0 corresponding to a pixel without stripes in the temporally coded image.
[0091] In the above embodiment of the present invention, a plurality of image positions are set in the time stripe pattern (e.g., the first time stripe pattern and the second time stripe pattern), and each pixel can represent a binary code, for example, when pixels with stripes are distributed at the pixel positions, it represents a first coding value, for example, 1, and when pixels without stripes are distributed at the pixel positions, it represents a second coding value, for example, 0. Therefore, based on the first time stripe image, there is a corresponding first coding table, and based on the second time stripe image, there is a corresponding second coding table, and further, based on the first coding table and the second coding table, the corresponding coding sequence of the same pixel position can be determined based on the acquisition order of the stripes to form a time image coding table.
[0092] For example, if pixel position A in a first time stripe image is encoded as 1 and B as 0, and pixel position A in a second time stripe image is encoded as 0 and B as 1, then the first encoding table corresponding to the first time stripe image is (A:1, B:0) and the second encoding table corresponding to the second time stripe image is (A:0, B:1), and the time image encoding table determined based on the first encoding table and the second encoding table is (A:10, B:01).
[0093] Alternatively, two or more time stripe patterns may be projected, and multiple time stripe patterns may be arranged in a time sequence to generate a multi-bit code.
[0094] As an optional embodiment, after acquiring a second time stripe image of the surface of the scanned object, the method further includes a step of projecting a third time stripe pattern onto the surface of the scanned object at a third time, a step of acquiring a third time stripe image of the surface of the scanned object, and a step of determining a time image encoding table based on the first time stripe image, the second time stripe image and the third time stripe image.
[0095] For example, if pixel position A in a first time stripe image is encoded as 1 and B as 0, and pixel position A in a second time stripe image is encoded as 0 and B as 1, and based on this, pixel position A in a third time stripe image is encoded as 1 and B as 1, then the first encoding table corresponding to the first time stripe image is (A:1, B:0), the second encoding table corresponding to the second time stripe image is (A:0, B:1), and the third encoding table corresponding to the third time stripe image is (A:1, B:1), and therefore the time image encoding table determined based on the first encoding table, the second encoding table, and the third encoding table is (A:101, B:011).
[0096] As an optional embodiment, the step of determining the time image coding table based on the first time stripe image, the second time stripe image and the third time stripe image includes the steps of: setting pixels with stripes in the first time stripe image as a first coding value, setting pixels without stripes in the first time stripe image as a second coding value, and constructing a first coding table based on the first coding value and the second coding value of a pixel position distribution of the first time stripe image; and setting pixels with stripes in the second time stripe image as the first coding value, setting pixels without stripes in the second time stripe image as the second coding value, and constructing a first coding table based on the first coding value and the second coding value of a pixel position distribution of the second time stripe image. constructing a second coding table based on the first coding value and the second coding value of a pixel position distribution of the third time stripe image, setting pixels with stripes in the third time stripe image as a first coding value and pixels without stripes in the third time stripe image as a second coding value, and constructing a third coding table based on the first coding value and the second coding value of a pixel position distribution of the third time stripe image; arranging the coding values of the same pixel position in the first coding table, the second coding table, and the third coding table in the acquisition order of the first time stripe image, the second time stripe image, and the third time stripe image to form a coding sequence of corresponding pixels, and constructing a time image coding table based on the coding sequence.
[0097] As an optional example, after determining the time image encoding table based on the first time stripe image and the second time stripe image, the method further includes the steps of projecting a fourth time stripe pattern onto the scanned object, obtaining a fourth time stripe pattern on the surface of the scanned object, and determining a sequence of each stripe in the fourth time stripe image based on the time image encoding table, and projecting a fifth time stripe pattern onto the scanned object, obtaining a fifth time stripe pattern on the surface of the scanned object, and determining a sequence of each stripe in the fifth time stripe image based on the time image encoding table, wherein the fifth time stripe pattern is obtained based on an offset distance d in the same direction of each stripe in the fourth time stripe pattern.
[0098] 8 is a schematic diagram of the stripe offset reconfiguration according to an embodiment of the present invention, as shown in FIG. 8, when the stripe spacing is L, the reconfigured stripes can be designed as a dense stripe group with equidistant offset, so as to increase the density of the monolithic data. According to the requirements of stripe resolution, the stripe offset distance d can be designed to be 1 / 2, 1 / 3, 1 / 4, etc. of L, the smaller the stripe offset distance, the higher the resolution, and the larger the stripe offset distance, the smaller the number of images and the faster the scanning speed.
[0099] Optionally, based on the three-dimensional scanning equipment shown in FIG. 1, the stripe-encoded image is moved within the width of the camera based on the above equipment parameters, such as the effective depth of field of the projection system, the lens magnification of the camera, and the optical angle between the projection optical axis of the projection system and the imaging optical axis of the camera, which are determined by the physical characteristics of each hardware in the three-dimensional scanning equipment.
[0100] Furthermore, based on the equipment parameters of the above-mentioned three-dimensional scanning equipment, the stripe-encoded image can be prevented from exceeding the imaging range of the camera, making it easier to three-dimensionally reconstruct an image of the scanned object having the captured stripe code.
[0101] Note that, in general, in a scanning scene directed to a small field of view within the effective depth of field, due to the angle of the binocular system and the magnification of the optical lens, it is inevitable that the stripe pattern of the same code value of the structured light (i.e., stripe-coded image) will move within the width of the camera, and this movement depends on three things: the effective depth of field, the angle of the optical system, and the magnification of the lens. When the parameters of the optical system are fixed, this movement range (i.e., offset distance) is determined, and the uniqueness of the stripe coding within this movement range (i.e., offset distance) can be designed to ensure the uniqueness of the code value globally across the width. Due to the linear propagation characteristics of light, the projected light within this movement range (i.e., offset distance) will not jump out of the range. The movement range (i.e., offset distance) is taken as one coding cycle, and the coding cycle ensures the uniqueness of the coding, and the coding cycle can ensure a small range according to the optical design, so that the coding uniqueness can be ensured with less coding information (less image sequence or less spatial coding). Since the stripes of other coding cycles do not interfere with the coding cycle in the entire range, multiple coding cycles can be used across the width.
[0102] As an optional example, when the stripe-coded image is a color-coded image, the three-dimensional scanning method includes the steps of projecting a color-coded image including a first color stripe pattern and a second color stripe pattern onto a surface of a scanned object, obtaining a color stripe image of the surface of the scanned object including the first color stripe image and the second color stripe image, and determining a color image coding table based on the first color stripe image and the second color stripe image.
[0103] It should be noted that the first color stripe image and the second color stripe image are multiple color stripes in the same color stripe pattern acquired separately by corresponding color channels, for example, a color stripe pattern including a combined arrangement of red and green stripes, where the red channel of the camera acquires the red stripes to form a red stripe image, and the green channel of the camera acquires the green stripes to form a green stripe image.
[0104] As an optional example, the step of determining the color image encoding table based on the color stripe image includes the steps of determining a first color encoding table based on the first color stripe image, determining a second color encoding table based on the second color stripe image, and constructing the color image encoding table based on the first color encoding table and the second color encoding table.
[0105] As an optional embodiment, the step of determining the first color coding table based on the first color stripe image includes a step of setting pixels having the first color in the first color stripe image as a first coding sequence and pixels not having the first color in the first color stripe image as a fourth coding sequence, and constructing the first color coding table based on the first coding sequence and the fourth coding sequence of the pixel position distribution of the first color stripe image; the step of determining the second color coding table based on the second color stripe image includes a step of setting pixels having the second color in the second color stripe image as a second coding sequence and pixels not having the second color in the second color stripe image as a fourth coding sequence, and constructing the second color coding table based on the second coding sequence and the fourth coding sequence of the pixel position distribution of the second color stripe image; and the step of constructing the color image coding table based on the first color coding table and the second color coding table includes a step of superimposing the coding sequences of the same pixel positions in the first color coding table and the second color coding table as coding sequences of corresponding pixels, and the superimposed coding sequences corresponding to the distribution of each pixel constitute the color image coding table.
[0106] As an alternative embodiment, the coding table is binary coded, with a first coding sequence corresponding to pixels with a first color in the color-coded image being (0,0,1), a second coding sequence corresponding to pixels with a second color in the color-coded image being (0,1,0) and a fourth coding sequence corresponding to pixels with no color in the color-coded image being (0,0,0).
[0107] According to another aspect of an embodiment of the present invention, there is also provided a computer readable storage medium having a program stored thereon which, when executed, controls an apparatus in which the computer readable storage medium is disposed to perform the three-dimensional scanning method described above.
[0108] According to a further aspect of an embodiment of the present invention there is also provided a processor configured to execute a program which, when executed, performs the above-mentioned method of three-dimensional scanning.
[0109] According to an embodiment of the present invention, there is also provided an embodiment of a three-dimensional scanning device, which may be configured to perform the three-dimensional scanning method of the present invention, and the three-dimensional scanning method of the present invention may be performed on the three-dimensional scanning device.
[0110] FIG. 9 is a schematic diagram of a three-dimensional scanning device according to an embodiment of the present invention. As shown in FIG. 9, the device includes: The system includes a projection unit 92 configured to project a stripe-coded image onto a scanned object, the stripe-coded image including a time-coded image or a color-coded image, the time-coded image including a plurality of time stripe patterns arranged in time, and the color-coded image including a color stripe pattern coded using a plurality of colors; an imaging unit 94 configured to capture a three-dimensional reconstructed image of the scanned object, the imaging unit 94 having the stripe-coded image on a surface of the scanned object in the three-dimensional reconstructed image; and a reconstruction unit 96 that reconstructs a three-dimensional model of the scanned object based on the three-dimensional reconstructed image.
[0111] It should be noted that the projection unit 92 in this embodiment can be configured to perform step S602 in the embodiment of the present application, the imaging unit 94 in this embodiment can be configured to perform step S604 in the embodiment of the present application, and the reconstruction unit 96 in this embodiment can be configured to perform step S606 in the embodiment of the present application. The examples and application scenarios achieved by the above-mentioned devices and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiments.
[0112] In an embodiment of the present invention, a stripe-coded image is projected onto a scanning object, the stripe-coded image includes a time-coded image or a color-coded image, the time-coded image includes a plurality of time stripe patterns arranged in time, and the color-coded image includes a color stripe pattern encoded with a plurality of colors. A three-dimensional reconstruction image of the scanning object is captured, and the stripe-coded image is on the surface of the scanning object in the three-dimensional reconstruction image. By reconstructing a three-dimensional model of the scanning object based on the three-dimensional reconstruction image, the time-coded image or the color-coded image can make the stripe-coded image have a unique stripe code, and the objective of ensuring the uniqueness of the stripe code of the stripe-coded image is achieved, the technical effect of improving the speed of dynamic scanning is achieved, and the problem of complex coding of the projected image required for the three-dimensional scanning process is solved.
[0113] As an optional example, when the stripe-coded image is a time-coded image, the three-dimensional scanning apparatus further comprises a first projection unit configured to project a first time stripe pattern onto the surface of the scanned object at a first time, a first acquisition unit configured to acquire the first time stripe image of the surface of the scanned object, a second projection unit configured to project a second time stripe pattern onto the surface of the scanned object at a second time, a second acquisition unit configured to acquire the second time stripe image of the surface of the scanned object, and a first determination unit configured to determine a time image coding table based on the first time stripe image and the second time stripe image.
[0114] As an optional example, the first determination unit comprises a first determination module configured to determine a first encoding table based on a first time stripe image, a second determination module configured to determine a second encoding table based on a second time stripe image, and a first configuration module configured to configure a time image encoding table based on the first encoding table and the second encoding table.
[0115] As an optional example, the first determination module comprises a first determination sub-module configured to set pixels with stripes in the first time stripe image as a first coded value and pixels without stripes in the first time stripe image as a second coded value, and to construct a first coding table based on the first coded value and the second coded value of the pixel position distribution of the first time stripe image; the second determination module comprises a second determination sub-module configured to set pixels with stripes in the second time stripe image as a first coded value and pixels without stripes in the second time stripe image as a second coded value, and to construct a second coding table based on the first coded value and the second coded value of the pixel position distribution of the second time stripe image; and the first construction module comprises a first construction sub-module configured to arrange the coded values of the same pixel position in the first coding table and the second coding table in the order of acquisition of the first time stripe image and the second time stripe image to form a coding sequence of corresponding pixels, and to construct a time image coding table based on the coding sequence.
[0116] As an optional embodiment, the apparatus further includes a third projection unit configured to project a third time stripe pattern onto the surface of the scanned object at a third time after acquiring the second time stripe image of the surface of the scanned object, a third acquisition unit configured to acquire the third time stripe image of the surface of the scanned object, and a second determination unit configured to determine a time image encoding table based on the first time stripe image, the second time stripe image and the third time stripe image.
[0117] As an optional embodiment, the second determination module may include a first encoding module configured to set a pixel with stripes in the first time stripe image as a first encoding value, set a pixel without stripes in the first time stripe image as a second encoding value, and configure a first encoding table based on the first and second encoding values of the pixel position distribution of the first time stripe image; a second encoding module configured to set a pixel with stripes in the second time stripe image as a first encoding value, set a pixel without stripes in the second time stripe image as a second encoding value, and configure a second encoding table based on the first and second encoding values of the pixel position distribution of the second time stripe image; the third encoding module is configured to set pixels with stripes in the three time stripe image as a first encoding value and pixels without stripes in the third time stripe image as a second encoding value, and to construct a third encoding table based on the first encoding values and the second encoding values of the pixel position distribution of the third time stripe image; and a fourth encoding module is configured to arrange the encoding values of the same pixel position in the first encoding table, the second encoding table and the third encoding table in the order in which the first time stripe image, the second time stripe image and the third time stripe image were acquired to form an encoding sequence of corresponding pixels, and to construct a time image encoding table based on the encoding sequence.
[0118] As an alternative embodiment, the coding table is binary coded, with a coding value of 1 corresponding to pixels with stripes in the temporally coded image and a coding value of 0 corresponding to pixels without stripes in the temporally coded image.
[0119] As an optional embodiment, the apparatus further comprises a third determination unit configured to, after determining a time image encoding table based on the first time stripe image and the second time stripe image, project a fourth time stripe pattern onto the scanning object to obtain a fourth time stripe image on a surface of the scanning object and determine a sequence of each stripe in the fourth time stripe image based on the time image encoding table, and a fourth determination unit configured to project a fifth time stripe pattern onto the scanning object to obtain a fifth time stripe image on a surface of the scanning object and determine a sequence of each stripe in the fifth time stripe image based on the time image encoding table, wherein the fifth time stripe pattern is obtained based on an offset distance d in the same direction of each stripe in the fourth time stripe pattern.
[0120] As an optional example, when the stripe-coded image is a color-coded image, the three-dimensional scanning apparatus includes a fourth projection unit configured to project a color-coded image including a first color stripe pattern and a second color stripe pattern onto a surface of the object to be scanned, a fourth acquisition unit to acquire a color stripe image of the surface of the object to be scanned, including the first color stripe image and the second color stripe image, and a fifth determination unit to determine a color image coding table based on the first color stripe image and the second color stripe image.
[0121] As an optional example, the fifth determination unit includes a third determination module configured to determine a first color encoding table based on the first color stripe image, a fourth determination module configured to determine a second color encoding table based on the second color stripe image, and a second configuration module configured to configure a color image encoding table based on the first color encoding table and the second color encoding table.
[0122] As an optional embodiment, the third determination module comprises a third determination sub-module configured to set pixels with a first color in the first color stripe image as a first encoding sequence, pixels without the first color in the first color stripe image as a fourth encoding sequence, and construct a first color encoding table based on the first encoding sequence and the fourth encoding sequence of the pixel position distribution of the first color stripe image; the fourth determination module comprises a fourth determination sub-module configured to set pixels with a second color in the second color stripe image as a second encoding sequence, pixels without the second color in the second color stripe image as a fourth encoding sequence, and construct a second color encoding table based on the second encoding sequence and the fourth encoding sequence of the pixel position distribution of the second color stripe image; and the second construction module comprises a second construction sub-module configured to superimpose the encoding sequences of the same pixel positions in the first color encoding table and the second color encoding table as encoding sequences of corresponding pixels, and the superimposed encoding sequences corresponding to the distribution of each pixel constitute a color image coding table.
[0123] As an alternative embodiment, the coding table is binary coded, with a first coding sequence corresponding to pixels with a first color in the color-coded image being (0,0,1), a second coding sequence corresponding to pixels with a second color in the color-coded image being (0,1,0) and a fourth coding sequence corresponding to pixels with no color in the color-coded image being (0,0,0).
[0124] As an alternative embodiment, the first imaging section is equidistant from the second imaging section.
[0125] The numbers in the above embodiments of the present invention are for illustrative purposes only and do not represent advantages or disadvantages of the embodiments.
[0126] In the above embodiments of the present invention, each embodiment is described with particular attention to each other, and when a particular embodiment is not described in detail, reference may be made to the relevant descriptions of other embodiments.
[0127] It should be understood that in some embodiments of the present application, the disclosed technical contents can be implemented in other ways. Here, the above-mentioned device embodiments are merely examples, and for example, the division of units can be a logical division of functions, and can actually be implemented in other ways, for example, multiple units or components can be combined, or can be incorporated into another system, or some functions can be ignored or not performed. Meanwhile, the illustrated or discussed mutual couplings or direct couplings or communication connections can be via indirect couplings or communication connections of some interfaces, units or modules, and can be implemented in electrical or other forms.
[0128] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place or in multiple units. Some or all of these units may be selected according to practical needs to achieve the purpose of the solution of the embodiment.
[0129] In addition, each functional unit in each embodiment of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The integrated unit may be implemented in the form of hardware or a software functional unit.
[0130] The integrated unit may be implemented as a software functional unit and stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the essence of the technical solution of the present invention or a part that contributes to the prior art, or the whole or part of the technical solution, may be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes a plurality of instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The storage medium includes various media capable of storing program code, such as a USB memory, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, and an optical disk.
[0131] It should be noted that the above is only a preferred embodiment of the present invention, and many improvements and modifications are possible for those skilled in the art without departing from the principle of the present invention, and these improvements and modifications are also considered to be within the protection scope of the present invention. [Industrial Applicability]
[0132] The solution provided by the embodiment of the present invention can be applied to three-dimensional scanning process, which solves the problem that the structured light coding pattern in the prior art requires multiple image sequences for complex coding, and effectively improves the scanning efficiency.
Claims
1. 1. A three-dimensional scanning device, comprising: a projection device configured to project a stripe-encoded image including the first set of stripes onto a scanned object; a camera configured to capture an image of the scanning object to obtain a camera image; the camera image is an image of the object in an image plane of the camera, the image plane including a first imaging zone, and when the object is within a useful depth of field range of the three-dimensional scanning device, the image of the first set of stripes in the image plane is within the first imaging zone, and only the first set of stripes is within the first imaging zone; the stripe-encoded image includes a second group of stripes disposed adjacent to the first group of stripes, the image plane includes a second imaging section disposed adjacent to the first imaging section, and when the scanned object is within an effective depth of field of the three-dimensional scanning device, the second group of stripes is imaged at the image plane within the second imaging section, and only the second group of stripes is present within the second imaging section; Three-dimensional scanning equipment.
2. A system angle α is formed between the projection optical axis of the projector and the imaging optical axis of the camera, and the optical parameters of the camera include a lens magnification k 1 The effective depth of field of the three-dimensional scanning device is ΔL, and the first imaging section is d 1 In this case, d 1 = ΔL × tgα ÷ k 1 That is, 2. A three-dimensional scanning device according to claim 1.
3. the projection device includes an image display element including a first display section in which the first stripe group is provided, A system angle α is formed between the projection optical axis of the projection device and the imaging optical axis of the camera, and the optical parameters of the projection device include the lens magnification k of the projection device. 2 The effective depth of field of the three-dimensional scanning device is ΔL, and the first display section is D 1 In this case, D 1 = ΔL × tgα ÷ k 2 That is, 2. A three-dimensional scanning device according to claim 1.
4. the stripe-encoded image includes a plurality of periodically arranged stripe groups, the first stripe group and the second stripe group each being a cycle of stripe groups; 2. A three-dimensional scanning device according to claim 1.
5. A system angle α is formed between the projection optical axis of the projector and the imaging optical axis of the camera, and the optical parameters of the camera include a lens magnification k 1 The effective depth of field of the three-dimensional scanning device is ΔL, and the second imaging section is d 2 In this case, d 2 = ΔL × tgα ÷ k 1 That is, 2. A three-dimensional scanning device according to claim 1.
6. the projection device includes an image display element including a second display section in which the second stripe group is provided, A system angle α is formed between the projection optical axis of the projection device and the imaging optical axis of the camera, and the optical parameters of the projection device include the lens magnification k of the projection device. 2 The effective depth of field of the three-dimensional scanning device is ΔL, and the second display section is D 2 In this case, D 2 = ΔL × tgα ÷ k 2 That is, 2. A three-dimensional scanning device according to claim 1.
7. and a processor configured to perform a three-dimensional reconstruction of the scanned object based on the camera images.
2. A three-dimensional scanning device according to claim 1.
8. The processor is configured to preset coordinates of the first imaging section; The processor determines pixel coordinates of centers of stripes in the camera image based on the camera image; The processor determines a serial number for each stripe in the camera image based on pixel coordinates of the stripe and coordinates of the first imaging section; the processor performs a three-dimensional reconstruction based on pixel coordinates of the centers of the stripes and the serial number to obtain a three-dimensional digital model of the scanned object.
8. A three-dimensional scanning apparatus according to claim 7.
9. The processor is preset with a light plane and a corresponding serial number of each stripe in the stripe-encoded image; The processor determines a light plane corresponding to a pixel coordinate of a center of the stripe based on a match between a serial number of each stripe in the camera image and a serial number corresponding to a light plane of each stripe; the processor performs triangulation based on pixel coordinates of the centers of the stripes and the corresponding light planes to reconstruct a three-dimensional digital model of the scanned object; 8. A three-dimensional scanning apparatus according to claim 7.
10. 7. A method of three-dimensional scanning performed on a three-dimensional scanning device according to any one of claims 1 to 6, wherein the three-dimensional scanning device further comprises a processor configured to perform a three-dimensional reconstruction of the scanned object based on the camera images, projecting the stripe encoded image onto the scanned object by the projection device; capturing an image of the scanning object with the camera to obtain a camera image; and performing, by the processor, a three-dimensional reconstruction of the scanned object based on the camera images. Three-dimensional scanning method.
11. determining pixel coordinates of centers of stripes in the camera image based on the camera image; coordinates of a first imaging section are preset in the processor, and a serial number of each stripe is determined based on pixel coordinates of the stripe and the coordinates of the first imaging section; and reconstructing pixel coordinates of the centers of the stripes three-dimensionally based on the serial numbers to obtain a three-dimensional digital model of the scanned object.
11. The method of three-dimensional scanning according to claim 10.
12. 1. A three-dimensional scanning method comprising: projecting a stripe-coded image onto a scanned object, the stripe-coded image comprising a time-coded image or a color-coded image, the time-coded image comprising a plurality of time stripe patterns arranged in time, and the color-coded image comprising a color stripe pattern encoded with a plurality of colors; capturing a three-dimensional reconstructed image of the scanned object, the three-dimensional reconstructed image having the stripe encoded image on a surface of the scanned object; reconstructing a three-dimensional model of the scanned object based on the three-dimensional reconstructed image; The stripe-encoded image includes a first group of stripes and a second group of stripes adjacent to the first group of stripes, the object to be scanned is imaged by a camera to obtain a camera image, the camera image is an image of the object to be scanned in an image plane of the camera, the image plane includes a first imaging section and a second imaging section adjacent to the first imaging section, when the object to be scanned is within an effective depth of field of a three-dimensional scanning device, the image of the second group of stripes in the image plane is within the second imaging section, and only the second group of stripes is within the second imaging section. Three-dimensional scanning method.
Citation Information
Patent Citations
Three coding period gray scale trapezoid phase shift structured light three dimensional information obtaining method
CN102519390A
Object measuring system by coded grid projection
JP1977056556A
Shape-measuring device and shape-measuring method
JP2008180530A
Method for controlling the linear dimensions of a 3D object
JP2015534091A
Projection device, collection device, and three-dimensional scanning system including them
JP2022509094A
