Scanning chip, scanner and scanning system capable of realizing double scanning modes

Abstract

The utility model provides a scanning chip, a scanner and a scanning system for realizing double scanning modes. The scanning chip comprises an image input port, a multi-path input switch, a depth calculation engine and an output port so as to respond to a high-precision scanning mode or a rapid scanning mode to process left and right feature image data acquired by a three-dimensional scanner; in a high-precision scanning mode, left and right feature images are received through an image input port and enter a multi-path input switch so as to allow the left and right feature images to be output through an output port; in the fast scanning mode, left and right feature images are received through the image input port and enter the multi-path input switch, so that the left and right feature images are allowed to enter the depth calculation engine for stereo matching processing to obtain a disparity map or a depth map or light stripe extraction processing to obtain left and right interested images, and the left and right interested images are output through the output port. The three-dimensional scanner applying the scanning chip has different measurement precisions and can meet the scanning requirements of different application scenes.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional scanning technology, and in particular to a scanning chip, a scanner, and a scanning system for realizing a dual scanning mode. Background Art

[0002] A handheld scanner is a three-dimensional data acquisition device, which is usually constructed into a three-dimensional scanning system together with the modeling software applied to the host computer. Among them, the handheld scanner integrated with hardware such as a projection module and a camera is used to scan the scanning object to obtain the image data of the scanning object, while the modeling software applied to the host computer is responsible for processing the image data collected by the scanner to obtain a three-dimensional model of the scanned object.

[0003] In the existing technology, the scanning technology of handheld scanners mainly relies on the laser line scanning principle or the stripe scanning principle, wherein: the laser line scanning principle is to emit one or more laser lines to a part of the surface of the scanned object through the projection module within a single frame acquisition cycle, and the camera captures the laser lines reflected back by the part of the area to generate a laser line image and transmits it to the host computer. The host computer calculates the three-dimensional information of the scanned object through triangulation and then reconstructs the three-dimensional model of the scanned object; the stripe scanning principle is to project a series of stripe patterns onto the surface of the object to the scanned object through the projection module, and the camera captures the stripe pattern projected onto the surface of the scanned object to generate a stripe image and upload it to the host computer. The host computer calculates the phase change of the stripe pattern based on the captured stripe image to obtain the three-dimensional information of the scanned object and then reconstructs the three-dimensional model of the scanned object.

[0004] Specifically, existing handheld scanners generally do not support depth calculation and processing of the image data they collect to obtain three-dimensional information of the scanned object, and then transmit the three-dimensional information of the scanned object to the host computer for processing to obtain a three-dimensional model of the scanned object. They usually transmit the image directly to the host computer for processing by the modeling software in the host computer to obtain a three-dimensional model of the scanned object. Therefore, for handheld scanners with multi-eye imaging, compared with the former's processing method of performing depth calculation on the multi-eye collected images at the scanning end and then uploading them, the latter's processing method of directly transmitting the image will result in a large amount of image data needing to be transmitted, and has higher requirements on the output interface bandwidth.

[0005] Furthermore, although handheld scanners based on the laser line scanning principle or the stripe scanning principle can both provide high-precision scanning results, both are time-consuming and have low reconstruction efficiency. For example, the laser line projected based on the first scanning projection in a single frame acquisition cycle can only cover a small area of ​​the scanned object, that is, each frame of the image acquired includes less information about the scanned object and all the information of the image needs to be uploaded. If all the information of the scanned object is to be obtained, it is necessary to project the laser line to different areas of the scanned object multiple times in multiple acquisition cycles and acquire multiple frames of images to obtain a three-dimensional model of the scanned object, which leads to a large amount of data to be processed. In view of the phase periodicity characteristics, when using a handheld scanner based on the stripe scanning principle to obtain a stripe image to obtain a three-dimensional model of the scanned object, a phase wrapping phenomenon will be involved. Therefore, when calculating the stripe phase change, a phase unwrapping technology is required, which leads to an increase in the amount of calculation.

[0006] Furthermore, existing handheld scanners all achieve high-precision scans, but different scanning objects or scenarios require different 3D scanner performance. For example, industrial design requires a high-precision 3D scanner, while 3D printing requires a 3D scanner that can scan objects quickly. Existing 3D scanners, which rely solely on a single scanning principle, cannot meet the diverse scanning needs of various objects and scenarios and are difficult to adapt to all scanning objects or scenarios. Summary of the Invention

[0007] The present application provides a scanning chip, scanner and scanning system that realize dual scanning mode, which can meet the three-dimensional scanning needs of multiple scenarios, is conducive to the promotion and application of three-dimensional scanners, and reduces the cost of three-dimensional scanning.

[0008] In a first aspect, the present application provides a scanning chip for a three-dimensional scanner, which is used to process left and right feature images collected by the three-dimensional scanner in response to a fast scanning mode and / or a high-precision scanning mode. The scanning chip includes an image input port, a multi-channel input switch, a depth calculation engine and an output port. The left and right feature images include left and right speckle images and / or left and right multi-line images: the image input port includes a left camera input port and a right camera input port, which are used to respectively receive the left and right feature images for parallel input to the multi-channel input switch; the multi-channel input switch selectively allows the left and right feature images to enter the depth calculation engine in response to the fast scanning mode or the high-precision scanning mode; when the When responding to the fast scanning mode, the multi-channel input switch allows the left and right feature images to enter the depth calculation engine. When the left and right feature images include left and right speckle images, the depth calculation engine is used to perform stereo matching on the left and right speckle images to obtain a disparity map or a depth map and output it through the output port; or, when the left and right feature images include left and right multi-line images, the depth calculation engine is used to perform light stripe extraction on the left and right multi-line images to obtain left and right images of interest and output them through the output port; or, when responding to the high-precision scanning mode, the multi-channel input switch does not allow the left and right feature images to enter the depth calculation engine, and the multi-channel input switch allows the left and right feature images to be output through the output port.

[0009] In a second aspect, the present application provides a three-dimensional scanner for scanning a scanning object in response to a scanning mode to obtain multi-frame image data of the scanning object, the scanning mode including a high-precision scanning mode and / or a fast scanning mode; wherein the three-dimensional scanner includes a transmitting end, a receiving end and a scanning chip: the transmitting end includes a speckle projection unit and / or a multi-line projection unit for projecting a speckle pattern beam and / or a multi-line pattern beam onto the scanning object; the receiving end includes a left camera and a right camera, and the left camera and the right camera are used to collect the light beam reflected by the scanning object to form left and right speckle images and / or left and right multi-line images; the scanning chip includes an image input port, a multi-channel input switch, a depth calculation engine and an output port; the image input port includes a left camera input port and a right camera input port, and is used to receive left and right feature images respectively to input them in parallel into the multi-channel input switch; the multi-channel input switch, in response to The left and right feature images are selectively allowed to enter the depth calculation engine in response to a fast scanning mode or a high-precision scanning mode to achieve the fast scanning mode and / or the high-precision scanning mode; wherein: when responding to the fast scanning mode, the multi-channel input switch allows the left and right feature images to enter the depth calculation engine, and when the left and right feature images include left and right speckle images, the depth calculation engine is used to perform stereo matching on the left and right speckle images to obtain a disparity map or a depth map and output the map through an output port; or, when the left and right feature images include left and right multi-line images, the depth calculation engine is used to perform light stripe extraction on the left and right multi-line images to obtain left and right images of interest and output the images through the output port; or, when responding to the high-precision scanning mode, the multi-channel input switch does not allow the left and right feature images to enter the depth calculation engine, and the multi-channel input switch allows the left and right feature images to be output through the output port.

[0010] In a third aspect, the present application provides a three-dimensional scanning system, comprising the three-dimensional scanner of the second aspect and modeling software applied to a host computer, wherein the three-dimensional scanner is used to respond to different scanning modes to scan a scanning object to obtain left and right feature images of the scanning object or obtain left and right feature images and texture images of the scanning object, the different scanning modes include a high-precision scanning mode and / or a fast scanning mode, and the left and right feature images include left and right speckle images and / or left and right multi-line images; when the multi-channel input switch in the three-dimensional scanner allows the left and right feature images to enter the depth calculation engine to achieve a high-precision scanning mode or a fast scanning mode, the modeling software is based on the left and right feature images or the left and right features of interest obtained by the three-dimensional scanner. The image is three-dimensionally reconstructed to obtain a texture-free three-dimensional model of the scanned object; or, the left and right feature images or the left and right images of interest and the texture image obtained based on the three-dimensional scanner are received and the left and right feature images and the texture image are used to perform three-dimensional reconstruction to obtain a textured three-dimensional model of the scanned object; when the multi-channel input switch in the three-dimensional scanner does not allow the left and right feature images to enter the depth calculation engine to achieve a high-precision scanning mode, the modeling software performs three-dimensional reconstruction based on the disparity map or depth map or the left and right images of interest obtained by the three-dimensional scanner to obtain a texture-free three-dimensional model of the scanned object; or, the textured three-dimensional model of the scanned object is obtained based on the disparity map or depth map or the left and right images of interest and the texture image obtained by the three-dimensional scanner.

[0011] Compared with the prior art, the scanning chip provided by the present application includes left and right image data ports, a multi-channel input switch and a depth calculation engine, wherein the left and right image input ports in the scanning chip support the simultaneous input of left and right feature images, and the selection function of the multi-channel input switch selectively allows the depth calculation engine to realize fast scanning mode or high-precision scanning mode, by allowing the input left and right feature images to be directly output to the host computer, so that the host computer can directly use the original image captured by the scanner for high-precision three-dimensional reconstruction to realize high-precision scanning mode; or allowing the input left and right feature images to enter the depth calculation engine, so that the depth calculation engine can perform stereo matching processing on the left and right feature images to obtain a single-frame depth image or perform light strip extraction on the left and right feature images to obtain left and right images of interest that only include the area of ​​interest and output them to the host computer to realize fast scanning mode, thereby converting two separate frames of images into a single-frame image output or eliminating redundant information in the image, which not only reduces the amount of image data output by the scanner to the host computer, improves the image transmission efficiency, but also reduces the requirements for the output interface bandwidth.

[0012] Furthermore, within a single-frame acquisition cycle, a 3D scanner using the above-mentioned scanning chip can project a multi-line patterned light beam or a speckle patterned light beam, or both. When both are taken into account, the multi-channel input switch in the scanning chip also allows the 3D scanner to acquire corresponding left and right multi-line images or left and right speckle images, both of which enter the depth calculation engine. The depth calculation engine performs light stripe extraction processing on the left and right multi-line images to obtain left and right images of interest, which are output to the host computer to achieve a high-precision scanning mode. The left and right speckle images are stereo matched to obtain a disparity map or a depth map, which is output to the host computer to achieve a fast scanning mode. Compared to multi-line patterned beam projection, when the transmitting end in a 3D scanner projects a speckle patterned beam, it can cover most areas of the scanned object and be simultaneously captured by the left and right cameras at the receiving end to obtain left and right speckle images containing information about more areas of the scanned object. This allows for obtaining more three-dimensional information about the scanned object with a smaller number of image frames, quickly acquiring information about the entire area of ​​the scanned object, improving scanning efficiency, and achieving a fast scanning mode. Furthermore, when the transmitting end projects a multi-line patterned beam, it can capture finer surface features of the scanned object, achieving a high-precision scanning mode. Consequently, the 3D scanner provided by this application, including the above-mentioned scanning chip and beams with different projection patterns, can achieve high-precision scanning modes and fast scanning modes with different measurement accuracies to meet the accuracy requirements of different application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a schematic diagram of the system architecture of a scanning system that implements single / dual scanning modes provided by this application.

[0014] Figure 2 This is a schematic diagram of the architecture of a scanning head in the three-dimensional scanning system provided by this application.

[0015] Figure 3 This is a schematic diagram of the system architecture of the scanning chip in the three-dimensional scanning system provided in this application.

[0016] Figure 4 This is a schematic diagram of the architecture of another scanning head in the three-dimensional scanning system provided in this application.

[0017] Figure 5 This is a flow chart of the image coding and compression method in the three-dimensional scanning system provided by this application.

[0018] Figure 6 This is a schematic diagram of a replacement vector calculation in the image coding compression method in the three-dimensional scanning system provided by this application.

[0019] Figure 7 This is another replacement vector calculation diagram of the image coding compression method in the three-dimensional scanning system provided by this application.

[0020] Figure 8This is a schematic diagram of result vector calculation of the image coding compression method in the three-dimensional scanning system provided by this application.

[0021] Figure 9 It is a flow chart of the multi-mode three-dimensional reconstruction method in the three-dimensional scanning system provided in this application.

[0022] Figure 10 It is a flow chart of the three-dimensional scanning method provided by this application.

[0023] Figure 11 This is a flow chart of another three-dimensional scanning method provided by this application. DETAILED DESCRIPTION

[0024] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and should not be understood as limiting the present application.

[0025] The terms "first," "second," "third," "fourth," and so forth (if any) in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. The order of the sequence numbers of the processes below does not imply a specific order of execution. The order of execution of the processes should be determined by their functions and inherent logic, and should not constitute any limitation on the implementation of the embodiments of the present application.

[0026] References to "some embodiments" and the like in this specification mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in some embodiments" and the like that appear in different places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0027] The terms used in the embodiments of the present disclosure are for the purpose of describing specific embodiments only and are not intended to limit the embodiments of the present disclosure. The singular forms "a" and "the" used in the embodiments of the present disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0028] The following is an illustrative description of the technical solution in this application with reference to the accompanying drawings. Figures 1 to 11 The examples are only intended to help those skilled in the art understand the embodiments of the present application, and are not intended to limit the embodiments of the present application to Figures 1 to 11 Those skilled in the art can obviously make various equivalent modifications or changes based on the examples given, and such modifications and changes also fall within the scope of the embodiments of the present application.

[0029] Figure 1 This is an architectural diagram of a three-dimensional scanning system that implements a dual scanning mode according to the present application. The system includes a three-dimensional scanner, a host computer equipped with modeling software, and a display end. The three-dimensional scanner is used to scan the scanning object in response to different scanning modes to obtain multiple frames of image data of the scanning object. The scanning modes include high-precision scanning mode and fast scanning mode; the three-dimensional scanner is communicated with the modeling software in the host computer, and the modeling software in the host computer processes the multiple frames of image data according to a preset three-dimensional reconstruction method to obtain one or more combinations of two-dimensional images, depth images, point cloud data or three-dimensional models of the scanning object. The display end is used to display the processing results of the host computer, such as displaying the two-dimensional image, depth image, point cloud data or three-dimensional model of the scanning object.

[0030] In one embodiment, a three-dimensional scanner includes a scanning head and a scanning chip, wherein the scanning head includes a transmitting end and a receiving end, the transmitting end includes a speckle projection unit and / or a multi-line projection unit, and is used to project a characteristic pattern beam onto a scanned object, the characteristic pattern beam including a speckle pattern beam and / or a multi-line pattern beam; the receiving end includes a left camera, a right camera, and an RGB camera, the left camera and the right camera are used to collect light beams reflected by the scanned object to form left and right characteristic images, the left and right characteristic images including left and right speckle images and / or left and right multi-line images, and the RGB camera is used to collect the scanning object. The scanning chip is used to process the image data collected by the scanning head in response to different scanning modes for transmission to the host computer, such as performing stereo matching on the left and right speckle images to obtain a disparity map or a depth map and transmitting the disparity map or the depth map to the host computer, or performing light stripe extraction (or laser line center line extraction) on the left and right multi-line images to obtain the left and right images of interest and transmit them to the host computer, or only for transferring the left and right feature images to the host computer, or for transmitting the texture image to the host computer for processing by the modeling software in the host computer to obtain a three-dimensional model of the scanned object.

[0031] It should be noted that when a multi-line patterned light beam is used to scan a scanning object, it is generally necessary to paste marker points on the scanning object to assist in the subsequent splicing of multiple frames of left and right multi-line images based on the marker points to obtain a three-dimensional model of the scanned object. For this reason, in this application, when light strips are extracted from the left and right multi-line images to obtain left and right images of interest, the contours of the marker points are often also extracted. That is, in addition to the center lines of each laser line in the multi-line image, the left and right images of interest can also include the contours of the marker points to facilitate the subsequent acquisition of the three-dimensional model of the scanned object. This application does not impose any restrictions on this.

[0032] In one embodiment, a scanning chip includes an image input port, a multi-channel input switch, a depth calculation engine, and an output port. The image input port includes left and right camera input ports for respectively receiving left and right feature images for parallel input into the multi-channel input switch. The multi-channel input switch selectively allows the left and right feature images to enter the depth calculation engine in response to a fast scanning mode and / or a high-precision scanning mode. When responding to the fast scanning mode, the multi-channel input switch allows the left and right feature images to enter the depth calculation engine. When the left and right feature images include left and right speckle images, the depth calculation engine is configured to perform stereo matching on the left and right speckle images to obtain a disparity map or a depth map, and output the map via the output port. Alternatively, when the left and right feature images include left and right multi-line images, the depth calculation engine is configured to perform light stripe extraction on the left and right multi-line images to obtain left and right images of interest, and output the images via the output port. Alternatively, when responding to the high-precision scanning mode, the multi-channel input switch does not allow the left and right feature images to enter the depth calculation engine, and the multi-channel input switch allows the left and right feature images to be directly output via the output port.

[0033] Compared to the high-precision scanning mode, in which the present embodiment directly uploads the left and right feature images to a host computer for processing, the present application performs stereo matching on the left and right speckle images to merge the two separate speckle image frames into a single depth image frame for output, or performs light stripe extraction on the left and right multi-line images to eliminate redundant information in the image data and output only the image data including the region of interest. This not only increases the image upload rate but also reduces the bandwidth requirement for the output interface, thereby improving the efficiency of three-dimensional scanning.

[0034] In another embodiment, compared to a speckle pattern beam projected by a transmitting end, a multi-line pattern beam emitted by a transmitting end can capture finer surface features of a scanned object and generate left and right multi-line images with higher measurement accuracy. Therefore, when the left and right feature images include left and right speckle images and left and right multi-line images, a multi-input switch in the scanning chip allows the left and right feature images to enter the depth calculation engine in response to either a fast scanning mode or a high-precision scanning mode. When responding to the fast scanning mode, the left and right camera input ports are configured to respectively receive the left and right speckle images for parallel input to the multi-input switch. The multi-input switch allows the left and right speckle images to enter the depth calculation engine for stereo matching of the left and right speckle images to obtain a disparity map or a depth map, which is then output via an output port. When responding to the high-precision scanning mode, the left and right camera input ports are configured to respectively receive the left and right multi-line images for parallel input to the multi-input switch. The multi-input switch allows the left and right multi-line images to enter the depth calculation engine for light stripe extraction of the left and right multi-line images to obtain left and right images of interest, which are then output via the output port.

[0035] It should be understood that, compared with a multi-line patterned beam, a frame of speckle patterned beam is projected onto the scanned object within a single-frame acquisition cycle. This frame of patterned beam can cover most areas of the scanned object and be simultaneously captured by the left and right cameras at the receiving end to obtain left and right speckle images including information about more areas of the scanned object. This allows a smaller number of image frames to be acquired with a shorter acquisition cycle to obtain more three-dimensional information about the scanned object, quickly acquiring information about the entire area of ​​the scanned object, improving scanning efficiency, and thus achieving a fast scanning mode.

[0036] Furthermore, when the multi-input switch in the three-dimensional scanner allows the left and right feature images to enter the depth calculation engine to achieve a high-precision scanning mode or a fast scanning mode, the modeling software performs three-dimensional reconstruction based on the left and right feature images or the left and right images of interest obtained by the three-dimensional scanner to obtain a texture-free three-dimensional model of the scanned object; or, receives the left and right feature images or the left and right images of interest and the texture image obtained based on the three-dimensional scanner and uses the left and right feature images and the texture image to perform three-dimensional reconstruction to obtain a textured three-dimensional model of the scanned object; when the multi-input switch in the three-dimensional scanner does not allow the left and right feature images to enter the depth calculation engine to achieve a high-precision scanning mode, the modeling software performs three-dimensional reconstruction based on the disparity map or depth map or the left and right images of interest obtained by the three-dimensional scanner to obtain a texture-free three-dimensional model of the scanned object; or, performs three-dimensional reconstruction based on the disparity map or depth map or the left and right images of interest and the texture image obtained by the three-dimensional scanner to obtain a textured three-dimensional model of the scanned object.

[0037] In one embodiment, the scanning head also includes an IMU for detecting the position of the scanning head. When it is detected that the scanning head is facing upward, that is, the light outlet of the scanning head is facing upward, the transmitting end is closed to prevent the transmitting end from projecting a light beam to the user, thereby ensuring eye safety.

[0038] It should be noted that the host computer of the present application is a processing end with computing power greater than the scanning chip in the three-dimensional scanner, such as a computer, desktop computer, cloud or mobile phone terminal. Given that the host computer can perform complex and high-precision algorithms, the images captured by the left and right cameras are directly transmitted to the host computer without processing to obtain high-precision point cloud data or a three-dimensional model; and the images captured by the left and right cameras are processed by the scanning chip in the three-dimensional scanner to obtain a depth map and then transmitted to the host computer, which can reduce the processing time of the host computer and quickly generate point cloud data or a three-dimensional model of the scanned object.

[0039] Figure 2This is a system architecture diagram of a scanning head provided by the present application. The speckle projection unit in the transmitting end includes at least a first speckle projection unit 1 and a second speckle projection unit 2. The receiving end includes at least an RGB camera 7 and a first binocular camera 3 and a second binocular camera 4, each consisting of two left and right camera groups. The first binocular camera 3 is embedded between the left and right cameras of the second binocular camera 4, so that the two binocular camera groups have different baselines. The first speckle projection unit 1 and the second speckle projection unit 2 are arranged between the first binocular camera 3. The speckle density and / or field of view of the speckle pattern beam projected by the first speckle projection unit 1 are different from those of the speckle pattern beam projected by the second speckle projection unit 2. Thus, the first binocular camera 3 and the second binocular camera 4 form active binocular imaging systems with different measurement accuracies. The RGB camera is arranged between the left or right cameras of the first binocular camera 3 and the second binocular camera 4, so that the images captured by the RGB camera overlap with those captured by the binocular camera. This allows the acquisition of texture information of the scanned object, thereby better capturing detailed information about the scanned object and improving the accuracy of the scanned object's three-dimensional model.

[0040] Furthermore, the different baselines of the first binocular camera 3 and the second binocular camera 4 will also result in different measurement accuracies of the imaging systems respectively formed based on the first binocular camera 3 and the second binocular camera 4. That is, when the first speckle projection unit 1 and the second speckle projection unit 2 have different speckle densities and / or projection fields of view and / or wavelengths and / or working distances, they respectively form active binocular imaging systems with at least two measurement accuracies with the first binocular camera 3 and the second binocular camera 4, thereby enabling a scanner equipped with the imaging system to adapt to different scanning scene requirements. For example, the imaging system formed by the first speckle projection unit 1 and the first binocular camera 3 with a short baseline is suitable for short-range scanning, while the imaging system formed by the second speckle projection unit 2 and the second binocular camera 4 with a long baseline is suitable for long-range scanning. The short and long distances mentioned above can be divided according to a distance threshold.

[0041] In other words, in both high-precision scanning mode and fast scanning mode, the short-range imaging system formed by the first speckle projection unit 1 and the first binocular camera 3 and the long-range imaging system formed by the second speckle projection unit 2 and the second binocular camera 4 can be used. Users can select the corresponding imaging system and the appropriate scanning mode according to the size and distance of the scanned object to scan the scanned object, thereby meeting the scanning accuracy requirements of different scanned objects.

[0042] In one embodiment, a first speckle projection unit 1 includes an LED light source, a collimating lens, a mask, and a projection lens, and a second speckle projection unit 2 includes a VCSEL light source array, a collimating lens, and a diffractive optical element (DOE). The VCSEL light source array includes multiple VCSEL light sources that emit a spot beam. Due to limitations in existing manufacturing processes, the spacing between the multiple VCSEL light sources cannot be made smaller, that is, there is a certain gap between the multiple VCSEL light sources. The light source of the second speckle projection unit 2 is an LED light source, and its speckle density is determined by the pattern density on the mask and the focal length of the projection lens. As a result, the speckle pattern density projected by the second speckle projection unit 2 using the VCSEL light source array is lower than that of the first speckle projection unit 1, and the measurement accuracy of the imaging system corresponding to the second speckle projection unit 2 is also lower than that of the imaging system corresponding to the first speckle projection unit 1.

[0043] Specifically, the mask in the first speckle projection unit 1 is provided with a dot array pattern of a preset density. When a light beam passes through this circular array pattern, a high-density speckle pattern is formed. Specifically, in the first speckle projection unit 1, an LED light source disposed on the focal plane of a collimating lens emits a divergent light beam to the collimating lens. After being focused by the collimating lens, the focused light beam is projected onto the mask disposed on the focal plane of the projection lens. The focused light beam passes through the circular array on the mask to form a high-density speckle pattern beam. The magnification effect of the projection lens forms a high-density speckle pattern beam with a larger field of view than the original (i.e., without the projection field of view of the projection lens), making it suitable for scanning small objects and maximizing the restoration of details.

[0044] In another embodiment, a VCSEL light source array arranged on the focal plane of the collimating lens in the second speckle projection unit 2 emits a speckle pattern beam to the collimating lens. After being collimated by the collimating lens, a parallel speckle pattern beam is formed and propagates to the diffractive optical element. The diffractive optical element is used to replicate the parallel speckle pattern beam to form a speckle pattern beam with a projection field larger than that of the first speckle projection unit 1 but a lower speckle density than that of the first speckle projection unit 1.

[0045] In one embodiment, the projection lens is a lens with variable depth of field, which includes a fixed-focus lens and a micro-zoom mechanism. The micro-zoom mechanism includes a rotating handle or an angle limiter. By rotating the micro-zoom mechanism, the fixed-focus lens is slightly moved, so that the image distance of the fixed-focus lens changes, and the focal length of the fixed-focus lens is slightly changed, thereby making the working depth of field of the projection lens variable. The first speckle projection unit 1 or the second speckle projection unit 2 can project speckle patterns at different distances without changing at different distances, thereby avoiding insufficient depth of field caused by a single focal length and ensuring imaging quality at the same time.

[0046] The first speckle projection unit 1 in this embodiment uses a combination of LEDs, MASK masks, and projection lenses. It can project speckles of different wavelengths by replacing LEDs with different wavelengths. Furthermore, the speckle density can be freely adjusted by adjusting the pattern density of the MASK mask and the focal length of the projection lens, thereby ensuring the scanning accuracy of small objects at close range. The second speckle projection unit collects VCSEL laser light sources, collimating lenses, and diffractive optical elements. Its high energy efficiency ensures long-distance scanning of more than 1 meter and a large field of view, making it suitable for scanning medium and large objects.

[0047] In one embodiment, the optical axes of the left and right cameras in the first binocular camera 3 or the second binocular camera 4 are parallel or angled. When the optical axes of the left and right cameras in the first binocular camera 3 or the second binocular camera 4 are parallel, the overlapping area of ​​the acquisition fields of the left and right cameras in the above binocular cameras is small, and more information of the scanned object cannot be collected when the scanning head is used for a single scan, resulting in relatively low scanning efficiency; when the optical axes of the left and right cameras in the first binocular camera 3 or the second binocular camera 4 are angled, the overlapping area of ​​the acquisition fields of the left and right cameras in the binocular cameras is large, and more information of the scanned object can be collected when the three-dimensional scanner is used for a single scan, thereby improving the scanning efficiency of the scanning head.

[0048] In another embodiment, each of the first binocular camera 3 and the second binocular camera 4 includes an image sensor and an imaging lens. The imaging lens is used to modulate the light beam reflected from the scanned object so that it forms an image on the image sensor. Furthermore, the focal lengths of the imaging lens of the first binocular camera 3 and the imaging lens of the second binocular camera 4 can be the same or different. When the focal lengths of the imaging lenses are the same, i.e., their operating depths of field are the same, then, when used with speckle projection units having different projected speckle densities, scanning heads with different measurement accuracies can be formed.

[0049] When the focal lengths of the two imaging lenses are different, the corresponding binocular camera's acquisition field of view is also different. Generally speaking, when the imaging lens has a small focal length, the corresponding camera's acquisition field of view is large and the working depth of field is large. It can take into account both long and short distance scanning and is conducive to scanning large objects, but the disadvantage is that the spatial resolution is low and it cannot perform high-precision measurement of small objects. When the imaging lens has a large focal length, the corresponding camera's acquisition field of view is small and the spatial resolution is high, which is conducive to scanning small objects, but the working depth of field is small, it cannot take into account both long and short distance scanning and is not conducive to scanning large objects.

[0050] To this end, in one embodiment, the focal length of the imaging lens of the first binocular camera 3 is greater than the focal length of the imaging lens of the second binocular camera 4. Since the baseline of the first binocular camera 3 is smaller than that of the second binocular camera 4, the second binocular camera 4 with a smaller focal length can ensure higher scanning accuracy through a long baseline. As a result, the first binocular camera 3 with a short baseline and a large imaging lens focal length is more suitable for scanning small objects at close range, while the second binocular camera 4 with a long baseline and a small imaging lens focal length is more suitable for scanning long-range and medium-to-large objects.

[0051] Further, if Figure 2 As shown, the scanning head further includes a first switching circuit 51 and a second switching circuit 52. The scanning chip 6 in the three-dimensional scanner is electrically connected to the left camera 3a of the first binocular camera 3 and the left camera 4a of the second binocular camera 4 through the first switching circuit 51 to drive the left camera of the binocular camera and receive data; the scanning chip 6 in the three-dimensional scanner is electrically connected to the right camera 3b of the first binocular camera 3 and the right camera 4b of the second binocular camera 4 through the second switching circuit 52 to drive the right camera of the binocular camera and receive data. As a result, the two sets of binocular cameras can be connected to the scanning chip 6 using only four interfaces instead of eight interfaces, thereby reducing the number of interfaces and further reducing the size of the scanning chip 6.

[0052] In this application, the first speckle projection unit 1 and the second speckle projection unit 2, respectively, together with the first binocular camera 3 and the second binocular camera 4 of different baselines, form active binocular imaging systems with different measurement accuracies. Combined with an RGB camera, they form active binocular texture imaging systems with different measurement accuracies. The scanning chip 6 also includes a main control unit. Each component in each imaging system is controlled by a synchronization signal sent by the main control unit in the scanning chip 6 to ensure that the components in each imaging system operate synchronously. Specifically, the user adaptively selects the size and distance of the scan object and the desired scanning mode on the host computer. In response to the size and distance of the scan object and the corresponding scanning mode, the host computer generates a control instruction and sends it to the scanning chip 6. The main control unit in the scanning chip 6 parses the control instruction and generates a corresponding trigger signal to control the operation of the corresponding active binocular imaging system and control the data flow processing within the scanning chip 6.

[0053] The main control unit in the scanning chip 6 sends different trigger signals to the first speckle projection unit 1 or the second speckle projection unit 2, the first binocular camera 3 or the second binocular camera 4, and the RGB camera at the same time. Specifically, the scanning chip 6 sends a PWM signal to the first speckle projection unit 1 or the second speckle projection unit 2 to trigger the projection unit to project the speckle pattern beam, sends a first I2C signal to the first switching circuit 51 and the second switching circuit 52 to drive the first binocular camera 3 or the second binocular camera 4 to start to collect left and right speckle images, and sends a second I2C signal to the RGB camera to drive the RGB camera to start to collect texture images, thereby realizing simultaneous operation of each module in each active binocular imaging system.

[0054] After the imaging system captures an image, the main control unit in the scanning chip 6 controls the output and input of the data stream therein in response to the corresponding scanning mode. That is, when the scanning mode is the high-precision scanning mode, the main control unit triggers the image input port to receive the left and right speckle images and texture images, and controls the multi-channel input switch to allow the left and right speckle images and texture images to be directly transmitted to the multi-channel output switch for transmission to the host computer through the output port; when the scanning mode is the fast scanning mode, the main control unit triggers the image input port to receive the left and right speckle images and texture images, and controls the multi-channel input switch to allow the left and right speckle images and texture images to be transmitted to the depth calculation engine for processing to obtain a disparity map or a depth map, which is then transmitted to the host computer via the multi-channel output switch and the output port.

[0055] In one embodiment, the first switching circuit 51 includes a first signal switching switch 511 and a first data switching switch 512. The first signal switching switch 511 includes a signal input port and two signal output ports. The signal input port is electrically connected to the scanning chip 6, and the two signal output ports are electrically connected to the left camera 3a of the first binocular camera 3 and the left camera 4a of the second binocular camera 4, respectively. Under the control of the main control unit in the scanning chip 6, a first I2C signal including signals of different levels is input through the single signal input port of the first signal switching switch 511, and a corresponding signal output port is selected for output based on the level of the signal, thereby controlling the left camera 3a of the first binocular camera 3 or the left camera 4a of the second binocular camera 4 to operate independently.

[0056] Specifically, when the level signal output by the main control unit in the scanning chip 6 is at a high level, the first signal switching switch 511 receives the high level signal through the signal input port and sends a strobe signal to the left camera 3 a of the first binocular camera 3 through the signal output port to drive it to work; and when the level signal output by the main control unit is at a low level, the first signal switching switch 511 receives the low level signal through the signal input port and sends a strobe signal to the left camera 4 a of the second binocular camera 4 through the signal output port to drive it to work.

[0057] The first data switching switch 512 includes two data input ports and one data output port. The two data input ports are electrically connected to the left camera 3 a of the first binocular camera 3 and the left camera 4 a of the second binocular camera 4, respectively. The data signal output port is electrically connected to the scanning chip 6. Under the control of the main control unit in the scanning chip 6, the first data switching switch 512 receives image data captured by the left camera 3 a of the first binocular camera 3 or the left camera 4 a of the second binocular camera 4 through a single data output port, thereby reducing the number of interfaces of the scanning chip 6.

[0058] Specifically, the data input port corresponding to the first data switching switch 512 is controlled to be turned on according to different level signals. When the level signal output by the main control unit in the scanning chip 6 to the first data switching switch 512 is at a high level, the data input port in the first data switching switch 512 electrically connected to the left camera 3 a of the first binocular camera 3 is turned on, and the scanning chip 6 receives the image data captured by the left camera 3 a of the first binocular camera 3 through the data signal output port. When the level signal output by the main control unit is at a low level, the data input port in the first data switching switch 512 electrically connected to the right camera 3 b of the first binocular camera 3 is turned on, and the scanning chip 6 receives the image data captured by the right camera 3 b of the first binocular camera 3 through the data signal output port.

[0059] It should be noted that the second switching circuit 52 includes a second signal switching switch 521 and a second data switching switch 522. The second signal switching switch 521 and the second data switching switch 522 have the same structure and working mode as the above-mentioned first signal switching switch 511 and the first data switching switch 512, except that they are connected to the right camera 3b of the first binocular camera 3 and the right camera 4b of the second binocular camera 4, which will not be repeated here.

[0060] In one embodiment, Figure 2As shown, the scanning head also includes four groups of fill light arrays 8 surrounding the periphery of each camera of the first binocular camera 3 and the second binocular camera 4. Each group of fill light arrays 8 includes multiple fill light 81 for supplementary lighting, which is conducive to uniform brightness of the images captured by the first binocular camera 3 and the second binocular camera 4 to improve measurement accuracy. Furthermore, the fill light arrays surrounding the first binocular camera 3 and the second binocular camera 4 are electrically connected to the scanning chip 6, and the wavelengths of the light beams emitted by them are the same or different. Preferably, when the first binocular camera 3 and the first speckle projection unit 1 form a binocular imaging system, the wavelength of the light beam emitted by the fill light array surrounding the first binocular camera 3 is the same as the wavelength of the light beam emitted by the first speckle projection unit 1; when the second binocular camera 4 and the second speckle projection unit 2 form a binocular imaging system, the wavelength of the light beam emitted by the fill light array surrounding the second binocular camera 4 is the same as the wavelength of the light beam emitted by the second speckle projection unit 2. That is, when the wavelengths of the light beams emitted by the first speckle projection unit 1 and the second speckle projection unit 2 are the same, the wavelengths of the light beams emitted by the fill light arrays surrounding the first binocular camera 3 and the second binocular camera 4 are the same, and vice versa.

[0061] In order to achieve the above-mentioned high-precision scanning mode and fast scanning mode, the present application provides a detailed system architecture diagram of a scanning chip, such as Figure 3 Specifically, the scanning chip includes an image input port, a multi-channel input switch, a depth calculation engine, and an output interface. The image input port includes a left camera input port, a right camera input port, and an RGB camera input port. The left camera input port, the right camera input port, and the RGB camera input port are respectively connected in parallel to the multi-channel input switch. Preferably, the left camera input port, the right camera input port, and the RGB camera input port include a MIPI interface and / or a DVP interface, and the output port includes a MIPI interface and / or a USB interface. In response to different scanning modes, the multi-channel input switch selectively allows data from a desired input port to be input into the depth calculation engine to achieve a fast scanning mode, or selectively allows data from a desired input port to be directly output through the output port to achieve a high-precision scanning mode.

[0062] In one embodiment, the scanning chip further includes a left image processing unit and a right image processing unit disposed between the multi-channel input switch and the depth calculation engine, wherein: when the multi-channel input switch allows the left and right speckle images to enter the depth calculation engine, the left camera input port and the right camera input port respectively receive the left and right feature images captured by the left and right cameras, and the multi-channel input switch allows the left and right feature images to enter the left image processing unit and the right image processing unit respectively for image preprocessing, wherein the image preprocessing includes image enhancement processing such as image distortion correction and brightness correction; the depth calculation engine performs stereo matching processing on the preprocessed left and right feature images to obtain a disparity map or a depth map, and outputs the map through an output port; or the depth calculation engine performs light stripe extraction on the preprocessed left and right feature images to obtain left and right images of interest, and outputs the images through the output port; when the multi-channel input switch does not allow the left and right feature images to enter the depth calculation engine, the left and right camera input ports respectively receive the left and right feature images, and output the images through the output port of the multi-channel input switch.

[0063] It should be noted that the left and right feature images include left and right speckle images or left and right multi-line images, that is, when the multi-input switch allows the left and right speckle images or left and right multi-line images to enter the depth calculation engine, the left and right image processing units perform image enhancement preprocessing on the left and right speckle images or left and right multi-line images, and the depth calculation engine performs stereo matching processing on the preprocessed left and right speckle images to obtain disparity maps or depth maps and outputs them through the output port, or the depth calculation engine performs light stripe extraction on the preprocessed left and right multi-line images to obtain left and right images of interest and outputs them through the output port; when the multi-input switch does not allow the left and right speckle images or left and right multi-line images to enter the depth calculation engine, the left and right camera input ports respectively receive the left and right speckle images or left and right multi-line images and output them through the output port allowed by the multi-input switch.

[0064] In one embodiment, the scanning chip further includes a multi-channel output switch, the input end of which is directly connected to the output end of the multi-channel input switch and directly or indirectly connected to the output end of the depth calculation engine, respectively. The output end of the multi-channel output switch is connected to the output port, and the multi-channel input switch allows all or part of the image data in the left and right feature images, disparity maps or depth maps, left and right images of interest, and texture images to pass to the output port. Furthermore, in order to enable the host computer to obtain a more accurate textured three-dimensional model, the scanning chip further includes an image alignment unit connected to the depth calculation engine and the multi-channel output switch. The image alignment unit is used to rotate and transform the left and right images of interest or disparity maps or depth maps based on preset external parameters of the left and right cameras and the RGB camera, so that the left and right images of interest or disparity maps or depth images are aligned with the texture image to facilitate texture mapping and obtain a more accurate textured three-dimensional model.

[0065] It should be noted that when the RGB camera has an integrated texture image processing unit (i.e., Image Signal Processor), the texture image it collects is a color texture image (such as an RGB image or a YUV image); when the RGB camera does not have an integrated texture image processing unit, the texture image it collects is a grayscale texture image. Figure 3 As shown, the scanning chip also includes a texture image processing unit arranged between the multi-input switch and the multi-output switch, which is used to perform color mapping on the grayscale texture image to obtain a color texture image for transmission to the host computer, so as to better extract the texture information of the scanned object.

[0066] In addition, the scanning chip also includes a storage interface for reading data stored in an external memory, such as reading the preset external parameters of the left and right cameras and the RGB camera stored in the memory of the scanning head for image alignment; when the image input port is a MIPI interface and / or a DVP interface, and the output port is a USB interface, the scanning chip also includes a format conversion protocol unit arranged between the multi-channel output switch and the output interface, which is used to perform protocol conversion on the image data received based on the MIPI interface and / or DVP interface to obtain data that complies with the USB protocol and can be output through the USB interface, thereby making the three-dimensional scanner universal.

[0067] In one embodiment, the scanning chip further includes a bus and a main control unit electrically connected to the bus. The bus is electrically connected to the transmitter, the acquisition terminal, the multi-input switch, and the multi-output switch. The main control unit responds to different scanning modes and controls the 3D scanner, the multi-input switch, and the multi-output switch via the bus. It should be noted that the bus can be electrically connected to components such as the transmitter, the receiver, the fill light array, and the IMU in the 3D scanner, allowing the main control unit to control each component. This is not a limitation of the present application.

[0068] Figure 4This is a schematic structural diagram of another scanning head provided according to the present application. In the scanning head, the transmitting end includes a multi-line projection unit 110 and a speckle projection unit 120, and the left and right cameras in the receiving end include a left black and white camera 130, a right black and white camera 140, and an RGB camera 170. The multi-line projection unit 110 is used to project a multi-line patterned light beam onto a scanned object, and the speckle projection unit 120 is used to project a speckle patterned light beam onto the scanned object. The left black and white camera 130 and the right black and white camera 140 are used to collect the multi-line patterned light beam or the speckle patterned light beam reflected by the scanned object and generate corresponding multi-line images or speckle images to obtain point cloud data or a three-dimensional model of the scanned object. Preferably, the multi-line projection unit 110, the speckle projection unit 120, and the RGB camera 170 are arranged between the left black-and-white camera 130 and the right black-and-white camera 140. Therefore, when the multi-line projection unit 110 or the speckle projection unit 120 projects a patterned light beam, it can fill the acquisition field of view of the left black-and-white camera 130 and the right black-and-white camera 140, thereby better acquiring point cloud data or a three-dimensional model of the scanned object. Furthermore, the RGB camera 170 has an overlapping field of view with the left and right cameras, which allows it to better combine the texture information of the scanned object acquired by the RGB camera to acquire detailed information of the scanned object, thereby facilitating improved accuracy of the three-dimensional model of the scanned object.

[0069] In one embodiment, the multi-line projection unit 110 and the speckle projection unit 120 respectively form two binocular imaging systems with the left monochrome camera 130 and the right monochrome camera 140. Since the multi-line pattern projected by the multi-line projection unit 110 helps the left and right monochrome cameras obtain higher-density measurement data than the speckle pattern projected by the speckle projection unit 120, the measurement accuracy of the binocular imaging system formed by the multi-line projection unit 110 is greater than that of the binocular imaging system formed by the speckle projection unit 120. Specifically, in the high-precision scanning mode, the multi-line projection unit 110 projects a multi-line patterned beam onto the scanned object, or the speckle projection unit 120 projects a speckle patterned beam onto the scanned object. The left and right cameras respectively collect the beams reflected by the scanned object and generate left and right multi-line images or left and right speckle images. In this case, the scanning chip 6 does not process the left and right multi-line images or left and right speckle images, but receives the left and right multi-line images or left and right speckle images through the image input port. The images are then output to the host computer through the output port after the multi-way input switch allows the images to be processed. In the fast scanning mode, the speckle projection unit 120 projects a speckle patterned beam onto the scanned object, and the left and right cameras respectively collect the beams reflected by the scanned object and generate left and right speckle images. In this case, the scanning chip 6 receives the left and right speckle images through the image input port. The images are then input to the depth calculation engine after the multi-way input switch allows the images to be processed to obtain a disparity map or a depth map, which is then transmitted to the host computer through the output port for processing by the modeling software.

[0070] It should be noted that, compared to a speckle pattern beam, a multi-line pattern beam can provide high-density measurement data, and therefore has higher measurement accuracy. In this embodiment, by configuring the transmitting end to project beams of different patterns, the binocular imaging system composed of the multi-line projection unit 110 is more suitable for scanning objects with more complex textures or scanning scenes with high scanning accuracy requirements, and the binocular imaging system composed of the speckle projection unit 120 is more suitable for scanning scenes with high scanning speed requirements or low scanning accuracy requirements. Further combining the high-precision scanning mode and the fast scanning mode enables the three-dimensional scanner of the present application to have different performances, thereby adapting to different scanning scenes.

[0071] Furthermore, the multi-line projection unit 110 includes one or more multi-line projection units, and the speckle projection unit 120 includes one or more speckle projection units. When there are multiple multi-line projection units 110 or speckle projection units 120, each projection unit can be independently controlled to form a multi-line pattern beam or a speckle pattern beam of different densities. This allows the binocular imaging system composed of the multi-line projection unit 110 or the speckle projection unit 120 to have different measurement accuracies and be adaptable to a variety of different application scenarios, thereby greatly improving the integration and universality of the three-dimensional scanner.

[0072] In one embodiment, when multiple multi-line projection units are included, the multiple multi-line projection units project multi-line pattern beams of different shapes onto the scanned object, such as one or more combinations of parallel multi-line pattern beams, crossed multi-line pattern beams, and single-line pattern beams. Specifically, taking at least three multi-line projection units as an example, the three multi-line projection units are arranged in a triangle, wherein one multi-line projection unit is configured to project parallel multi-line pattern beams parallel or perpendicular to a baseline direction of the 3D scanner to perform a rough scan of the scanned object, where the baseline refers to the direction of the line connecting the binocular cameras in the 3D scanner; the other two multi-line projection units respectively project parallel multi-line pattern beams that are tilted relative to the baseline direction and in opposite directions, so that when the two multi-line projection units are simultaneously activated, a crossed multi-line pattern beam is formed to perform a fine scan of the scanned object.

[0073] It should be noted that when at least four multi-line projection units are included, three of the multi-line projection units, in addition to forming the above-mentioned parallel multi-line pattern beams and cross multi-line pattern beams, another multi-line projection unit is used to project a single line pattern beam to perform deep-hole scanning on a scanning object with holes. This application does not impose any restrictions here.

[0074] In one embodiment, a single multi-line projection unit 110 includes a laser light source, a collimating element, a modulating element, and a diffractive optical element sequentially disposed in an inner cavity of a housing. The laser light source emits a light beam to the collimating element for beam collimation. The collimated light beam enters the modulating element to form at least one line light beam. The at least one line light beam is replicated by the diffractive optical element to form a multi-line patterned light beam. A single speckle projection unit 120 includes a laser light source, a collimating element, and a diffractive optical element sequentially disposed in an inner cavity of a housing. The laser light source emits a spot light beam to the collimating element. The collimating element collimates the spot light beam to form a parallel spot light beam that propagates toward the diffractive optical element. The spot light beam is replicated by the diffractive optical element to form a speckle patterned light beam with a large field of view.

[0075] Preferably, the multi-line projection unit 110 and the speckle projection unit 120 are integrated into the emitting end of the scanning head through a separate modular assembly method. The emitting end of the scanning head includes an optoelectronic support, the multi-line projection unit 110, and the speckle projection unit 120. Before the multi-line projection unit 110 and the speckle projection unit 120 are assembled into the optoelectronic support, one or more components, such as a laser light source, a collimating element, a modulation element, and a diffractive optical element, are first integrated into a single module of one or more multi-line projection units 110 or speckle projection units 120. Positioning holes with fool-proofing are provided on the optoelectronic support to ensure that the integrated multi-line projection unit 110 and speckle projection unit 120 are assembled in a specific position to form the emitting end. For example, an asymmetric shape or feature is used to restrict the assembly direction or position of the emitting end, so that it can only be correctly assembled in a specific position; otherwise, it cannot be fixed. This eliminates the need for adjustment and eliminates the need for component alignment during each component assembly process. This reduces the cost and time loss caused by assembly errors, ensures product quality, and improves production efficiency, as well as the stability and reliability of the production process.

[0076] In one embodiment, the wavelengths of the light beams emitted by the multi-line projection unit 110 and the speckle projection unit 120 are the same or different. When the wavelengths of the light beams emitted by the multi-line projection unit 110 and the speckle projection unit 120 are the same, the left monochrome camera 130 or the right monochrome camera 140 includes a narrowband filter and an image sensor. The narrowband filter is used to filter out stray light in wavelengths other than the wavelengths of the light beams emitted by the multi-line projection unit 110 and the speckle projection unit 120, ensuring that only the light beam emitted by the projection unit enters the image sensor for imaging, thereby improving image quality. When the wavelengths of the light beams emitted by the multi-line projection unit 110 and the speckle projection unit 120 are different, for example, when the multi-line projection unit 110 projects a 450 nm wavelength light beam and the speckle projection unit 120 projects an 850 nm / 940 nm wavelength light beam, the left monochrome camera 130 or the right monochrome camera 140 includes a dual-pass filter and an image sensor. The dual-pass filter only allows the 450 nm / 850 nm / 940 nm wavelength light beams to pass through, while filtering out stray light in other wavelengths, ensuring that only the light beam emitted by the transmitting end enters the image sensor for imaging. The present application sets a double-pass filter so that light beams of different wavelengths can be collected using only two cameras instead of four cameras, thereby reducing the number of cameras, improving integration and reducing costs.

[0077] like Figure 4 As shown, the scanning head further includes a fill light array 150 surrounding the left monochrome camera 130 and the right monochrome camera 140. When the wavelengths of the multi-line projection unit 110 and the speckle projection unit 120 are different, the fill light array includes a plurality of LED lamps having the same wavelength as the multi-line projection unit 110 and the speckle projection unit 120. Each LED lamp has a common cathode, and the anode is connected to a power supply or other circuit via a current-limiting resistor. Then, by turning on the anode, the LED lamp with the corresponding wavelength is selected to emit light.

[0078] In one embodiment, given that Figure 2 and Figure 4 The transmitter or fill light of the scanning head shown in the figure requires a large current, but the scanning chip cannot meet the power consumption. Figure 2 and Figure 4 The illustrated scanning head also includes a light source driving circuit, which includes a signal input port and a current output port. The signal input port is connected to the scanning chip 6, and the current output port is connected to the transmitting end or the fill light. The light source selection signal sent by the scanning chip 6 enters the light source driving circuit through the signal input port. The light source driving circuit outputs current to the transmitting end or the fill light through the current output port based on the received light source selection signal to drive the transmitting end or the fill light to emit light. It should be noted that in this application, each projection unit and the fill light can share a light source driving circuit or each has a corresponding light source driving circuit, and it can select the corresponding device to emit light based on the received light source selection signal. However, sharing a light source driving circuit is conducive to miniaturization of the scanning head.

[0079] In one embodiment, the scanning head further comprises buttons, e.g. Figure 4 The buttons 170 and scanning chip 6 shown here facilitate user control of the scanning head's operating status during scanning. For example, the buttons communicate with the scanning chip to adjust the transmitter's drive current or start or pause the scanning head. The number of buttons can be configured based on functional requirements. Furthermore, the buttons on the scanning head can also be used to interact with a host computer to control the size of the scanning window used by the host computer to display the scanned object model.

[0080] In another embodiment, the scanning head further includes an indicator light which can indicate the difference between the current working distance and the optimal working distance by using different colors according to the actual distance between the 3D scanner and the target. Figure 4 The indicator light 180 shown can display five colors: blue, light blue, green, yellow, and orange, which are respectively used to indicate that the current working distance is too far, relatively far, optimal, relatively close, and too close. The user can move the scanning head according to the color of the indicator light 180 so that the scanning head is at the optimal working depth of field, which helps to improve the scanning accuracy.

[0081] After a 3D scanner is used to scan a scanning object to obtain a depth image or texture image, the image data can be uploaded to a host computer via a wired or wireless method for 3D reconstruction to obtain a 3D model of the scanned object. To this end, the 3D scanning system provided by the present application also includes a mobile handle, which is electrically connected to the 3D scanner and is connected to the host computer via a wired or wireless communication; the mobile handle includes a battery unit and a compression unit, the battery unit is used to power the 3D scanner; the compression unit is used to receive one or more of a disparity map or a depth map, a texture image, a left and right feature image, or a left and right image of interest obtained based on the 3D scanner, and encode and compress the image data using a preset image encoding and compression method for transmission to the host computer. Furthermore, when the mobile handle is wirelessly connected to the host computer, the mobile handle also includes a wireless transmission module, which transmits the image data collected by the 3D scanner to the host computer via WIFI technology or Bluetooth technology, and can also realize two-way interaction between the reconstruction software in the host computer and the 3D scanner, thereby allowing users to break free from the constraints of wired connections and improve the convenience of the 3D scanner.

[0082] This application configures an additional mobile handle for the three-dimensional scanner, and sets a battery unit and a compression unit in the mobile handle. This not only makes it easier for users to carry it for multi-directional scanning of the scanned object, but also reduces the transmission bandwidth requirements, making the transmission more stable, thereby improving the image transmission efficiency and achieving a higher frame rate. Moreover, a separate mobile handle can be adapted to different types of three-dimensional scanners, such as a three-dimensional scanner without image compression function or power supply, which can improve the portability and scanning efficiency of the three-dimensional scanner without changing the original structure of the three-dimensional scanner.

[0083] Furthermore, given that existing compression methods cannot achieve both compression time and compression rate, for example, lz4 and quicklz offer fast compression times but low compression rates; rvl and rle offer long compression times and low compression rates; and zstd and turbojpeg offer high compression rates but long compression times, to address these issues and ensure algorithmic effectiveness and user experience, this application provides a low-computational-complexity image compression method, pre-installed in the compression unit within the mobile controller, to address the aforementioned technical issues of low compression rate and long compression time.

[0084] Figure 5 The following is a flowchart of a low-computational-complexity image compression method provided by the present application, which includes:

[0085] S1: Receive input image data, select pixel values ​​corresponding to several pixels in the image data to form a pixel vector, and calculate a replacement vector for the pixel vector based on the pixel vector and an all-zero vector.

[0086] In one embodiment, calculating a replacement value vector of a pixel vector based on a pixel vector and an all-zero vector includes: taking two adjacent elements in the pixel vector as a group and selecting the maximum value of the elements in each group, and using the maximum value of the elements in each group and the all-zero vector to construct a first intermediate vector; taking adjacent elements in the first intermediate vector as a group and selecting the maximum value of the elements in each group, and using the maximum value of the elements in each group and the all-zero vector to construct a second intermediate vector; taking adjacent pixels in the second intermediate vector as a group and selecting the minimum value of the elements in each group, and using the minimum value of the elements in each group and the all-zero vector to construct a third intermediate vector; and selecting the value of the first element in the third intermediate vector to generate a replacement vector with the same replacement value and the same number of elements as the number of pixels.

[0087] Specifically, take the pixel values ​​corresponding to 8 pixels in the image data to form a pixel vector as an example, Figure 6 As shown, the pixel vector is [10, 11, 9, 8, 7, 15, 12, 13], and the number of elements in the all-zero vector is aligned with the number of elements in the pixel vector. That is, the all-zero vector is a vector composed of 8 zeros. Two adjacent elements in the pixel vector are grouped together, such as [10, 11], [9, 8], [7, 15], and [12, 13]. The maximum value in each group of elements, that is, 11, 9, 15, and 13, is selected to construct the first four bits of the first intermediate vector. The last four bits are padded with the last four bits of the all-zero vector so that the number of elements in the first intermediate vector is aligned with the pixel vector.

[0088] Furthermore, two adjacent elements in the first intermediate vector are grouped together, namely, [11,9], [15,13], [0,0], and [0,0], and the maximum value of each group is selected. Thus, 11, 15, 0, and 0 are selected to construct the first four bits of the second intermediate vector, and the last four bits are padded with the last four bits of the all-zero vector to align the number of elements with the pixel vector. Furthermore, adjacent elements in the second intermediate vector are grouped together, namely, [11,9], [0,0], [0,0], and [0,0], and the minimum value of each group is selected. Thus, the value of the first element in the third intermediate vector is selected to construct a replacement vector with the same replacement value and the same number of elements as the number of pixels. That is, element 11 is selected as the replacement value to construct a replacement vector with 8 elements and a value of 11.

[0089] In one embodiment, calculating a replacement value vector for a pixel vector based on a pixel vector and an all-zero vector includes: circularly shifting the pixel vector left by N elements to obtain a motion vector, comparing the numerical values ​​of each pixel in the pixel vector with the numerical values ​​of the elements at the corresponding position of the motion vector and taking the smaller value element between the two to construct a fourth intermediate vector, where N is a positive integer; grouping adjacent elements in the fourth intermediate vector into a group and selecting the minimum value in each group of elements, and using the minimum value in each group of elements and the all-zero vector to construct a fifth intermediate vector; grouping adjacent elements in the fifth intermediate vector into a group and selecting the maximum value in each group of elements, and using the maximum value in each group of elements and the all-zero vector to construct an intermediate vector and iterating at least A sixth intermediate vector is obtained at one time; the numerical values ​​of each pixel of the pixel vector and the corresponding position elements of the motion vector are compared and the larger value element of the two is taken to construct a seventh intermediate vector, the adjacent elements in the seventh intermediate vector are grouped together and the maximum value of each group of elements is selected, and the maximum value of each group of elements and the all-zero vector are used to construct an eighth intermediate vector; the adjacent elements in the eighth intermediate vector are grouped together and the minimum value of each group of elements is selected, and the minimum value of each group of elements and the all-zero vector are used to construct an intermediate vector and iterate at least once to obtain a ninth intermediate vector; the smaller value is selected as the replacement value in the first element of the sixth intermediate vector and the first element of the ninth intermediate vector to construct a replacement vector.

[0090] like Figure 7As shown in the left frame, the pixel vector is circularly shifted left by one element to obtain the moving vector. Specifically, the first element 10 of the pixel vector [10,11,9,8,7,15,12,13] is moved to the end of the queue, and the remaining elements are shifted left to obtain the moving vector [11,9,8,7,15,12,13,10]. Each element of the pixel vector is compared with the element at the corresponding position of the moving vector, and the smaller value of the two is taken to obtain the fourth intermediate vector [10,9,8,7,7,12 ,12,10], the adjacent elements in the fourth intermediate vector are grouped as a group, that is, [10,9], [8,7], [7,12], and [12,10] are each a group, and the minimum value in each group of elements, that is, 9, 7, 7, and 10, are selected to construct the fifth intermediate vector [9,7,7,10,0,0,0,0] with the all-zero vector; the adjacent elements in the fifth intermediate vector [9,7,7,10,0,0,0,0] are grouped as a group, [9,7], [7,10], [0,0], and [0, 0] are each a group, and the maximum value of each group is taken to construct the intermediate vector [9,7,0,0,0,0,0,0] and iterate once, that is, based on the vector [9,7,0,0,0,0,0,0], the steps of “grouping adjacent elements and taking the maximum value of each group to construct the intermediate vector” are executed to obtain the sixth intermediate vector.

[0091] It should be noted that Figure 7 The right-hand diagram illustrates a specific implementation method for obtaining the ninth intermediate vector. This method is similar to that used to obtain the sixth intermediate vector, except that the element selection method is opposite. This application uses the example of shifting the pixel vector left by one position only. N can be any other value, and this application will not elaborate further here. Furthermore, the smaller of the leading elements of the sixth and ninth intermediate vectors is selected as the element at each position in the replacement vector to ensure that the number of elements in the resulting replacement vector matches that in the pixel vector.

[0092] The above specific embodiment only takes 8 pixels to construct a pixel vector as an example. In fact, other numbers of pixels can be selected to construct a pixel vector. If the number of selected pixels is an odd number and cannot be evenly grouped, it can be solved by padding zeros at the end of the queue. This application does not impose any restrictions on this.

[0093] S2: Calculate the absolute difference between the elements of the pixel vector and the replacement value vector to obtain an absolute difference vector, compare the numerical value of each element of the absolute difference vector with the element at the corresponding position of the preset threshold vector, and obtain a first result vector based on the comparison result.

[0094] In one embodiment, comparing the numerical value of each element of the absolute difference vector with the element at the corresponding position of the preset threshold vector and obtaining a first result vector based on the comparison result includes: numerically comparing each element in the absolute difference vector with the element at the corresponding position of the preset threshold vector; if an element in the absolute difference vector is less than or equal to the element at the corresponding position of the budget threshold vector, then the element at the same position as the element in the first result vector is recorded as 255; if an element in the absolute difference vector is greater than the element at the corresponding position of the budget threshold vector, then the element at the same position as the element in the first result vector is recorded as 0; and traversing each element in the absolute difference vector to obtain its corresponding element in the first result vector, so as to obtain a complete first result vector.

[0095] Specifically, if Figure 8 As shown, taking the preset threshold vector elements are all 2 as an example, based on Figure 6 In the embodiment shown, the pixel vector is [10, 11, 9, 8, 7, 15, 12, 13], and the replacement vector is [11, 11, 11, 11, 11, 11, 11, 11]. The difference between each element in the pixel vector and the element at the corresponding position on the replacement vector is calculated, and the absolute value is taken to obtain the absolute difference vector. For example, the first element 10 of the pixel vector is compared with the first element 11 of the replacement vector to obtain the first element 1 of the absolute difference vector. The elements at other positions are deduced in the same way to obtain the complete absolute difference vector [1, 0, 2, 3, 4, 4, 1, 2] corresponding to the pixel vector.

[0096] Then, each element in the absolute difference vector is compared with the pixel at the corresponding position in the preset threshold vector. For example, the first element 1 in the absolute difference vector is compared with the first element 2 in the preset threshold vector. It can be seen that 1 is less than 2, that is, the first element in the absolute difference vector is less than or equal to the element at the corresponding position of the preset threshold vector, and the corresponding first element in the first result vector is recorded as 255; for another example, the fourth element 3 in the absolute difference vector is compared with the fourth element 2 in the budget threshold vector. It can be seen that 3 is greater than 2, that is, the fourth element in the absolute difference vector is greater than the element at the corresponding position of the preset threshold vector, and the corresponding fourth element of the first result vector is recorded as 0. The elements in the remaining positions of the first result vector are deduced in the same way to obtain the complete first result vector corresponding to the pixel vector [255, 255, 255, 0, 0, 0, 255, 255].

[0097] S3: Traverse and count the elements in the first result vector, take the non-zero elements as the number of pixels that meet the threshold condition, traverse each element in turn, and when the traversed element is not zero, continue traversing until the traversed element is zero or when all the traversed elements are not zero, count the number of pixels that meet the threshold condition in the traversed pixels, and use the number of pixels that meet the threshold condition and the replacement value in the replacement vector to encode and compress the traversed pixels; when the traversed element is zero, start with the pixel corresponding to the traversed element being zero, and continue traversing until the number of pixels is the same as the pixel vector selected in step S1. The number of pixels having the same value as the threshold is determined and steps S1-S2 are executed to obtain a second result vector to traverse the second result vector. If the element is still zero based on the second result vector, the element in the result vector is encoded and compressed by using the number of pixels that meet the threshold condition of zero and the original value of the pixel. If the element is not zero based on the second result vector, the traversal is continued until the element is zero or if all elements based on the second result vector are not zero, the number of pixels that meet the threshold condition in the traversed pixels is counted, and the traversed pixels are encoded and compressed by using the number of pixels that meet the threshold condition and the replacement value in the replacement vector.

[0098] by Figure 9 For example, specifically, Figure 9 The elements of the result vector [255,255,255,0,0,0,255,255] shown in the figure are traversed and counted from left to right. When the fourth element is traversed, it is zero. Then the number of non-zero elements in the first three elements is counted, that is, the number of pixels that meet the threshold condition is 3. The traversed pixels are encoded and compressed using the number of pixels that meet the threshold condition and the replacement value in the replacement vector. The corresponding encoding method is [3]

[11] . When the fourth element is traversed, starting from the pixel corresponding to the fourth element, the same number of pixels as in step S1 are selected to form a new pixel vector. Steps S1-S2 are executed again to obtain the second result vector and the second result vector is traversed. If the first element in the second result vector obtained based on the new pixel vector is still 0 (i.e., the fourth element in the above), the element in the result vector is encoded and compressed using the number of pixels that meet the threshold condition of zero and the original value of the pixel (the original value of the pixel corresponding to the fourth element is 8), that is, the encoding method of the pixel is [0][8]; if the first element in the second result vector obtained based on the new pixel vector is not 0, the elements in the second result vector are traversed from left to right until the traversed element is zero, then the number of non-zero elements in the second result vector is counted, and the traversed pixels are encoded and compressed using the number of pixels that meet the threshold condition and the replacement value in the replacement vector.

[0099] For the next zero element, starting with this element, the same number of pixels as in step S1 are selected to form a new round of pixel vectors, and steps S1-S3 are executed. Based on the same steps as the fourth pixel mentioned above, the encoding method corresponding to the pixel is obtained. The same process is repeated for the remaining pixels until the encoding of all pixels is completed, thereby achieving compression of the corresponding image data. This application will not go into details here.

[0100] This application customizes the threshold conditions, and then in the image coding and compression process, adopts the encoding method of "the number of pixels that meet the threshold conditions is zero and the original value of the pixels" for areas where the difference between adjacent pixels in the image is large, and adopts the encoding method of "the number of pixels that meet the threshold conditions and the replacement value in the replacement vector" for areas where the difference between adjacent pixels in the image is small. This makes the image coding and compression process flexible, and the coding method can be implemented only through simple addition, subtraction and numerical comparison, with a small amount of calculation, which is conducive to improving the compression rate and reducing time consumption.

[0101] After the image data is encoded and compressed and uploaded to the host computer, the host computer decodes the received image data to obtain the image to be processed. The decoding method is opposite to the encoding method, that is, [3]

[11] , [0][8,7,15] and [2]

[11] are restored to [11,11,11,8,7,5,11,11] for back-end application algorithm processing. The modeling software in the host computer processes the multi-frame disparity map or depth map or left and right feature images or left and right images of interest according to the multi-mode three-dimensional reconstruction method provided by one or more embodiments of the present application to obtain a three-dimensional model of the scanned object, wherein the multi-mode three-dimensional reconstruction method includes different target reconstruction modes, which are divided according to the type of the scanned object. When the scanned object is an object, it includes an object feature mode and an object texture mode; when the scanned object is a face, it includes a face texture mode. The user can select an appropriate target reconstruction mode in the modeling software of the host computer according to the type of the scanned object to achieve accurate three-dimensional reconstruction of the scanned object.

[0102] Furthermore, before using the target reconstruction mode to achieve three-dimensional reconstruction of the scanned object, it is necessary to perform point cloud conversion processing based on the image uploaded to the host computer by the three-dimensional scanner to obtain corresponding point cloud data in order to reconstruct the three-dimensional model of the scanned object based on the point cloud data. Specifically, if the multi-channel input switch in the three-dimensional scanner does not allow the left and right feature images to enter the depth calculation engine, then when the images uploaded to the host computer modeling software are left and right feature images, when the left and right feature images are left and right speckle images, the modeling software performs stereo matching processing on the left and right speckle images to obtain a disparity map or depth map; or, when the left and right feature images are left and right multi-line images, the modeling software performs light strip extraction on the left and right multi-line images to perform stereo matching to obtain a disparity map or depth map; the disparity map or depth map is converted into a point cloud and the point cloud is processed according to the target reconstruction mode to obtain a texture-free three-dimensional model of the scanned object, or the point cloud and texture image are processed according to the target reconstruction mode to obtain a textured three-dimensional model of the scanned object;

[0103] If the multi-input switch in the 3D scanner allows the left and right feature images to enter the depth calculation engine, then when the image data uploaded to the host computer modeling software is a disparity map or a depth map or left and right images of interest, when the 3D scanner outputs the disparity map or the depth map, the modeling software converts the disparity map or the depth map into a point cloud; and / or, when the 3D scanner outputs the left and right images of interest, the modeling software performs stereo matching on the left and right images of interest to obtain the disparity map or the depth map to convert into a point cloud; and then processes the point cloud according to the target reconstruction mode to obtain a textureless 3D model of the scanned object, or processes the point cloud and the texture image according to the target reconstruction mode to obtain a textured 3D model of the scanned object; wherein the target reconstruction mode includes one or more combinations of object feature mode, object texture mode and face texture mode.

[0104] Figure 9 This is a flow chart of a multi-modal three-dimensional reconstruction method applied to modeling software in a host computer according to the present application, including object feature patterns, object texture patterns and face texture patterns, so that users can adaptively select the corresponding mode of three-dimensional reconstruction method according to different types of scanning objects to obtain a more accurate three-dimensional model of the scanned object.

[0105] In one embodiment, the modeling software processes the point cloud according to the object feature pattern to obtain a textureless three-dimensional model of the scanned object. The object feature pattern includes an online scanning method and an offline reconstruction method. The online scanning method includes: obtaining a multi-frame disparity map or depth map corresponding to the current moment and converting it into a current frame point cloud, detecting whether the current frame point cloud meets the preset registration conditions for subsequent real-time point cloud registration, the preset registration conditions including whether the current frame point cloud includes more than a preset threshold number of geometric features, such as identifiable corners, edges or unique shapes, etc., to help establish a reliable correspondence between different frame point clouds; if the current frame point cloud does not meet the preset registration conditions, the next frame point cloud is obtained for processing; if the current frame point cloud meets the registration conditions, the current frame point cloud is compared with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set through the Iterative Closest Point Algorithm. Point, ICP) is matched and the matching is constrained by a preset first cost function; when the constraint value obtained according to the preset first cost function is within the preset range, the current frame point cloud is saved, otherwise it is judged whether the current frame point cloud and the rest of the frame point cloud data in the historical frame point cloud set meet the relocation requirements, if so, the current frame point cloud is saved, if not, the current frame point cloud is discarded.

[0106] Specifically, judging whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set meet the relocation requirements includes: matching the current frame point cloud with the remaining frame point cloud data in the historical frame point cloud set and obtaining a constraint value based on a preset first cost function. If the constraint value is within a preset range, it indicates that the current frame point cloud meets the relocation requirements, otherwise it does not meet the requirements.

[0107] After the target object is scanned, the offline reconstruction mode is entered. The offline reconstruction mode includes: obtaining each frame point cloud obtained and saved based on scanning at different times, screening multi-frame key frame point clouds from each frame point cloud according to a preset time threshold or scanning movement distance, and using the multi-frame key frame point cloud and the historical frame point cloud set to perform global registration to obtain an optimized matching relationship to reduce matching drift, and then splicing and meshing all frame point clouds based on the optimized matching relationship to obtain a textureless three-dimensional network model.

[0108] In another embodiment, the modeling software processes the point cloud and the texture image according to the object texture mode to obtain a textured three-dimensional model of the scanned object. The object texture mode includes an online drawing method and an offline reconstruction method. The online drawing method includes: obtaining the current frame disparity map or the current frame depth map and the current frame texture image corresponding to the current moment, converting the current frame disparity map or the current frame depth map and judging whether the current frame texture image and the current frame point cloud corresponding to the current moment meet the preset registration conditions. The preset registration conditions include whether the current frame point cloud and the current frame texture image include more than a preset threshold number of geometric features, such as identifiable corners, edges or unique shapes, etc., to help establish a reliable matching relationship between different frame texture images and different frame point clouds.

[0109] If the current frame texture image or the current frame point cloud does not meet the preset registration conditions, the next frame texture image and point cloud are obtained for processing; if the current frame texture image and the current frame point cloud meet the preset registration conditions, the current frame point cloud is matched with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set through the iterative closest point (ICP) method and different preset cost functions are used to calculate different constraint values ​​for the first matching relationship between point cloud frames and the second matching relationship between texture image frames.

[0110] Specifically, a preset first cost function is used to constrain the first matching relationship between the current frame point cloud and the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set to obtain a first constraint value, and a preset second cost function is used to constrain the second matching relationship between the current frame texture image and the previous frame texture image corresponding to the previous moment in the historical frame texture image set to obtain a second constraint value.

[0111] In one embodiment, the first cost function is expressed by the following formula:

[0112]

[0113] in, Represents the first constraint value, [ , ] respectively represent the rotation matrix and translation matrix in the first matching relationship, Represents the i-th point in the current frame point cloud, Represents the i-th point in the previous frame point cloud in the historical frame point cloud set; when the first constraint value of the above-mentioned first cost function is the smallest, it means that the first matching relationship between the current frame point cloud and the previous frame point cloud reaches the best, and the corresponding [ , ] is the optimal value.

[0114] In another embodiment, the second cost function is expressed by the following formula:

[0115]

[0116] in, represents the second constraint value, Represents the color value of the i-th pixel in the current frame texture image, represents the color value of the j-th pixel in the previous frame texture image corresponding to the previous moment in the historical frame texture image set; when the second constraint value of the above second cost function is minimum, it means that the second matching relationship between the current frame texture image and the previous frame texture image reaches the best.

[0117] When both the first constraint value and the second constraint value are within the preset range or the sum of the first constraint value and the second constraint value is within the preset range, the current frame point cloud or the current frame texture image is saved; when the first constraint value or the second constraint value is not within the preset range or the sum of the first constraint value and the second constraint value is not within the preset range, the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set are judged for relocation or the current frame texture image and the remaining frame texture images in the historical frame texture image set are judged whether they meet the relocation requirements. If they meet the relocation requirements, the current frame point cloud or the current frame texture image is saved; if they do not meet the relocation requirements, the current frame point cloud or the current frame texture image is discarded.

[0118] In one embodiment, determining whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set are to be relocated or determining whether the current frame texture image and the remaining frame texture images in the historical frame texture image set meet the relocation includes: calculating a first constraint value for the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set according to a preset first cost function, and calculating a second constraint value for the current frame texture image and the remaining frame texture images in the historical frame texture image set according to a preset second cost function. If the first constraint value and the second constraint value are within a preset range or the first constraint value and the second constraint value are within a preset range, it indicates that the current frame point cloud and the current frame texture image meet the relocation requirements, otherwise they do not meet the requirements.

[0119] After the scanning of the target object is completed, the offline reconstruction mode is entered. The offline reconstruction mode includes: obtaining each frame point cloud and each frame texture image obtained based on scanning at different times, optimizing the first matching relationship and the second matching relationship based on the global optimization of all frame point clouds and all frame texture images to obtain the third matching relationship and the fourth matching relationship, splicing and meshing all frame point clouds based on the third matching relationship to obtain a texture-free mesh model, and texture mapping the texture-free mesh model through the fourth matching relationship and multiple frames of texture images to obtain a textured mesh model including texture information.

[0120] Furthermore, after obtaining the third matching relationship and the fourth matching relationship, multiple frames of key frame point clouds can be screened from the frame point clouds obtained and saved based on the preset time threshold or scanning movement distance, and the key frame point clouds can be used to further optimize the third matching relationship to obtain the fifth matching relationship. Based on the fifth matching relationship, all frame point clouds are spliced ​​and meshed to obtain a textureless mesh model, and the textureless mesh model is texture mapped through the fourth matching relationship and multiple frames of texture images to obtain a textured mesh model including texture information.

[0121] In another embodiment, the modeling software processes the point cloud and texture image according to the face texture mode to obtain a textured three-dimensional model of the scanned object. The face texture mode includes an online scanning mode and an offline reconstruction mode. Before entering the face texture mode, it is necessary to detect whether there is a frontal face in the current frame depth image and the current frame texture image. If so, the online scanning mode in the face texture mode is started. After the scan is completed, the offline reconstruction mode in the face texture mode is started to obtain a three-dimensional model of the frontal face.

[0122] Specifically, the online scanning mode includes: obtaining the current frame depth image and the current frame texture image corresponding to the current moment, wherein the current frame depth image and the current frame texture image both include a frontal face; converting the current frame depth image into a current frame point cloud and splicing it with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set through the ICP algorithm; constraining the splicing result using a preset first cost function; when the constraint value obtained according to the preset first cost function is within a preset range, the current frame point cloud is saved; otherwise, it is determined that the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set meet the relocation condition; if the relocation condition is met, that is, the constraint value obtained according to the preset first cost function is within a preset range, the current frame point cloud is saved; otherwise, it is discarded. It should be noted that during the scanning process, the corresponding point cloud splicing result can be displayed on the host computer while scanning. When the point cloud splicing result displayed by the host computer basically includes a complete frontal face, it indicates that the scan is complete; otherwise, the scan continues.

[0123] In one embodiment, the offline reconstruction method includes: obtaining each frame of point cloud obtained and saved based on scanning at different times, screening multiple frames of key frame point cloud from each frame of point cloud data according to a preset time threshold or scanning movement distance, using the key frame point cloud to globally optimize the stitching result to obtain a first point cloud set, and performing denoising, resampling and other processing on the optimized point cloud to improve the point cloud quality to obtain a second point cloud set; then meshing and meshing post-processing (such as smoothing, simplification, hole filling, etc.) the second point cloud set to obtain a texture-free mesh model, and then using a texture image including a frontal face to perform texture mapping on the texture-free mesh model to obtain a textured three-dimensional mesh model.

[0124] Furthermore, in the above-mentioned different reconstruction modes, the target object and background area can be divided according to the distance information in the depth image, and only the target object is reconstructed, while the background interference is eliminated, which is conducive to improving the reconstruction efficiency. In addition, each reconstruction mode in the three-dimensional reconstruction method of the present application is divided into online scanning and offline reconstruction. During the online scanning process, the user can adjust the scanning angle in real time according to the scanning situation to dynamically update the collected data, while offline reconstruction allows the use of more complex algorithms and more sufficient computing resources to process the collected data, which can achieve higher quality three-dimensional models. The present application realizes three-dimensional model reconstruction by combining online scanning and offline reconstruction, which not only ensures real-time interaction, but also improves the accuracy and details of the model to meet diverse application needs.

[0125] Based on the above three-dimensional scanning system, the present application also provides a three-dimensional scanning method, which is applied to a three-dimensional scanner including a depth calculation engine and a modeling software applied to a host computer, such as Figure 10 As shown, the method includes:

[0126] Projecting a characteristic pattern beam including a speckle pattern beam and / or a multi-line pattern beam onto a scanned object, and collecting the beam reflected by the scanned object to form left and right characteristic images, the left and right characteristic images including left and right speckle images and / or left and right multi-line images; selectively allowing the left and right characteristic images to enter a depth calculation engine to process the left and right characteristic images in response to a scanning mode, the scanning mode including a high-precision scanning mode and / or a fast scanning mode, wherein:

[0127] When the left and right feature images are not allowed to enter the depth calculation engine, the scanning mode is a high-precision scanning mode, which directly outputs the left and right feature images to the modeling software to use the left and right feature images for three-dimensional reconstruction to obtain a three-dimensional model of the scanned object;

[0128] When the left and right feature images are allowed to enter the depth calculation engine, the scanning mode is the fast scanning mode. When the left and right feature images are left and right speckle images, the depth calculation engine performs stereo matching on the left and right speckle images to obtain a disparity map or a depth map, and outputs the stereo matching to the modeling software to use the disparity map or the depth map for three-dimensional reconstruction to obtain a three-dimensional model of the scanned object. Alternatively, when the left and right feature images are left and right multi-line images, the depth calculation engine performs light stripe extraction on the left and right multi-line images to obtain left and right images of interest, and outputs the stereo matching to the modeling software to use the left and right images of interest for three-dimensional reconstruction to obtain a three-dimensional model of the scanned object.

[0129] In another embodiment, Figure 11As shown in FIG, when the left and right feature images include left and right speckle images and left and right multi-line images and both the left and right feature images are allowed to enter the depth calculation engine to realize the fast scanning mode or the high-precision scanning mode; in the fast scanning mode, stereo matching is performed on the left and right speckle images to obtain a disparity map or a depth map and output it; in the high-precision scanning mode, light stripe extraction is performed on the left and right multi-line images to obtain left and right images of interest and output it, and the modeling software in the host computer uses the disparity map or the depth map or the left and right images of interest to perform three-dimensional reconstruction to obtain a three-dimensional model of the scanned object.

[0130] In order to improve the uploading efficiency of image data collected by the 3D scanner, before the image data in different scanning modes are transmitted to the modeling software of the host computer, the image data obtained by the 3D scanner is encoded and compressed by a preset image compression method, such as encoding and compressing one or a combination of left and right feature images, depth maps or disparity maps, left and right images of interest, and texture images. The preset image compression method includes: selecting pixel values ​​corresponding to several pixels in the image data to form a pixel vector and calculating a replacement vector of the pixel vector based on the pixel vector and the all-zero vector; calculating the absolute difference of the elements of the pixel vector and the replacement value vector to obtain the absolute difference vector The method comprises the following steps: comparing the numerical value of each element of the absolute difference vector with the element at the corresponding position of the preset threshold vector and obtaining a result vector according to the comparison result; performing traversal and statistics on the elements in the result vector, taking the non-zero elements as the number of pixels that meet the threshold condition, traversing each element in turn, and when the traversed element is zero, counting the number of pixels that meet the threshold condition in the traversed pixels, and encoding and compressing the traversed pixels using the number of pixels that meet the threshold condition and the replacement value in the replacement vector; and when the traversed element is not zero, encoding and compressing the part of the zero element in the result vector using the number of pixels that meet the threshold condition and the original value of the pixel.

[0131] In some embodiments, the 3D scanning method further includes acquiring a texture image of the scanned object and transmitting it to modeling software on a host computer, thereby combining the image data uploaded to the host computer by the 3D scanner with the target reconstruction mode to obtain a 3D model of the scanned object. Specifically, if the left and right feature images are not allowed to enter the depth calculation engine, then when the image data uploaded to the host computer are left and right feature images, or when the left and right feature images are left and right speckle images, the modeling software performs stereo matching on the left and right feature images to obtain a disparity map or depth map; and / or, when the left and right feature images are left and right multi-line images, light stripe extraction and stereo matching are performed on the left and right multi-line images to obtain a disparity map or depth map; the disparity map or depth map is converted into a point cloud, and the point cloud is processed according to the target reconstruction mode to obtain a texture-free 3D model of the scanned object, or the point cloud and texture image are processed according to the target reconstruction mode to obtain a textured 3D model of the scanned object.

[0132] If the left and right feature images are allowed to enter the depth calculation engine, when the image data uploaded to the upper computer modeling software is a depth map or disparity map or left and right images of interest, when the modeling software receives the disparity map or depth map, the disparity map or depth map is converted into a point cloud; or, when the modeling software receives the left and right images of interest, the left and right images of interest are stereo matched to obtain a disparity map or depth map, which is then converted into a point cloud; the point cloud is processed according to the target reconstruction mode to obtain a textureless three-dimensional model of the scanned object, or the point cloud and texture image are processed according to the target reconstruction mode to obtain a textured three-dimensional model of the scanned object; wherein the target reconstruction mode includes one or more combinations of object feature mode, object texture mode and face texture mode.

[0133] Furthermore, in order to improve the accuracy of the textured three-dimensional model of the scanned object in the fast scanning mode, when the left and right feature images are allowed to enter the depth calculation engine, before the disparity map or depth map or the left and right images of interest and the texture image are transmitted to the modeling software, it also includes using the preset external parameters of the three-dimensional scanner to rotate the disparity map or depth map or the left and right images of interest so that the disparity map or depth map or the left and right images of interest are aligned with the texture image, and then the texture image is better used to accurately map the texture-free three-dimensional model to ensure the accuracy of the textured three-dimensional model of the scanned object. It should be noted that in the high-precision scanning mode or the fast scanning mode, if the texture image needs to be uploaded to the host computer for processing, the texture images under different scanning modes can also be encoded and compressed using the above-mentioned preset image compression method to improve the upload efficiency of the texture image.

[0134] In one embodiment, when the target reconstruction mode is the object feature mode, the point cloud is processed according to the object feature mode to obtain a textureless three-dimensional model of the scanned object, and the object feature mode includes online scanning and offline reconstruction, wherein the online scanning includes: obtaining the current frame point cloud corresponding to the current moment, detecting whether the current frame point cloud meets the preset registration conditions, the preset registration conditions including whether the current frame point cloud includes geometric features exceeding a preset threshold number; if the current frame point cloud does not meet the preset registration conditions, obtaining the next frame point cloud for processing; if the current frame point cloud meets the registration conditions, matching the current frame point cloud with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set and constraining the matching using a preset first cost function; when the constraint value obtained according to the preset first cost function is within a preset range, saving the current frame point cloud, otherwise determining whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set meet relocation, if so, saving the current frame point cloud, if not, discarding the current frame point cloud.

[0135] Offline reconstruction includes: obtaining each frame point cloud obtained and saved based on scanning at different times, screening multi-frame keyframe point clouds from each frame point cloud according to a preset time threshold or scanning movement distance, using the multi-frame keyframe point cloud and the historical frame point cloud set to perform global registration to obtain an optimized matching relationship, and based on the optimized matching relationship, splicing and meshing all frame point clouds to obtain a textureless three-dimensional network model.

[0136] When the target reconstruction mode is the object texture mode, the point cloud and the texture image are processed according to the object texture mode to obtain a textured three-dimensional model of the scanned object. The object texture mode includes online scanning and offline reconstruction, wherein the online scanning includes: obtaining the current frame point cloud and the current frame texture image corresponding to the current moment, and judging whether the geometric features of the current frame texture image and the current frame point cloud meet the preset registration conditions. The preset registration conditions include whether the current frame point cloud and the current frame texture image include geometric features exceeding a preset threshold number.

[0137] If the current frame texture image or the current frame point cloud does not meet the preset registration conditions, the next frame texture image and point cloud are obtained for processing; if the current frame texture image and the current frame point cloud meet the preset registration conditions, the current frame point cloud is matched with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set, and the current frame texture image is matched with the previous frame texture image corresponding to the previous moment in the historical frame texture image set, and different preset cost functions are used to perform different constraint value calculations on the first matching relationship between point cloud frames and the second matching relationship between texture image frames; when the different constraint values ​​are all within the preset range or the sum of the different constraint values ​​is within the preset range, the current frame point cloud or the current frame texture image is saved; otherwise, it is determined whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set or the current frame texture image and the remaining frame texture images in the historical frame texture image set meet repositioning. If repositioning is met, the current frame point cloud or the current frame texture image is saved; if repositioning is not met, the current frame point cloud or the current frame texture image is discarded.

[0138] Offline reconstruction includes: obtaining each frame of point cloud and each frame of texture image obtained and saved based on scanning at different times, optimizing the first matching relationship and the second matching relationship based on the global optimization of each frame of point cloud and each frame of texture image to obtain a third matching relationship and a fourth matching relationship, splicing and meshing each frame of point cloud based on the third matching relationship to obtain a texture-free mesh model, and texture mapping the texture-free mesh model through the fourth matching relationship and each frame of texture image to obtain a textured mesh model including texture information.

[0139] When the target reconstruction mode is the face texture mode, the point cloud and the texture image are processed according to the face texture mode to obtain a texture three-dimensional model of the scanned object. The face texture mode includes online scanning and offline reconstruction, wherein the online scanning includes: obtaining the current frame depth image and the current frame texture image corresponding to the current moment, and the current frame depth image and the current frame texture image both include a frontal face; converting the current frame depth image into a current frame point cloud and splicing it with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set, and using a preset first cost function to constrain the splicing result. When the constraint value obtained according to the preset first cost function is within a preset range, the current frame point cloud is saved. Otherwise, it is determined whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set meet the relocation requirements. If the relocation requirements are met, the current frame point cloud is saved. If the relocation requirements are not met, the current frame point cloud is discarded.

[0140] Offline reconstruction includes: obtaining each frame of point cloud obtained and saved based on scanning at different times, screening multiple frames of key frame point clouds from each frame of point cloud according to a preset time threshold or scanning movement distance, and using the multiple frames of key frame point clouds to globally optimize the stitching results to obtain a point cloud set; meshing the point cloud set to obtain a textureless mesh model, and using a texture image including a frontal face to texture map the textureless mesh model to obtain a textured three-dimensional mesh model of the frontal face.

[0141] The present application also provides a computer-readable storage medium for storing a computer program. When the computer program is executed, the present application is realized. Figure 10 or Figure 11 The three-dimensional scanning method in the embodiment.

[0142] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0143] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working process of the method described above can refer to the corresponding process in the aforementioned system, device, and unit embodiments, and will not be repeated here.

[0144] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0145] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0146] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0147] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A scanning chip used in a three-dimensional scanner, characterized in that: Used to process the left and right feature images collected by the 3D scanner in response to the fast scanning mode and / or the high-precision scanning mode, the scanning chip includes an image input port, a multi-channel input switch, a depth calculation engine and an output port, and the left and right feature images include left and right speckle images and / or left and right multi-line images: The image input port includes a left camera input port and a right camera input port, which are used to receive the left and right feature images respectively and input them into the multi-channel input switch in parallel; The multi-input switch selectively allows the left and right feature images to enter the depth calculation engine in response to a fast scanning mode or a high-precision scanning mode; When responding to the fast scanning mode, the multi-input switch allows the left and right feature images to enter the depth calculation engine. When the left and right feature images include left and right speckle images, the depth calculation engine is used to perform stereo matching on the left and right speckle images to obtain a disparity map or a depth map and output the map through the output port. Or, when the left and right feature images include left and right multi-line images, the depth calculation engine is used to perform light stripe extraction on the left and right multi-line images to obtain left and right images of interest and output the images through the output port. Or, When responding to the high-precision scanning mode, the multi-way input switch does not allow the left and right feature images to enter the depth calculation engine, and the multi-way input switch allows the left and right feature images to be output through the output port.

2. The scanning chip according to claim 1, wherein: include: When the left and right characteristic images include left and right speckle images and left and right multi-line images, the multi-input switch allows the left and right characteristic images to enter the depth calculation engine in response to a fast scanning mode or a high-precision scanning mode; When responding to the fast scanning mode, the left camera input port and the right camera input port are used to respectively receive the left and right speckle images for parallel input into the multi-channel input switch, and the multi-channel input switch allows the left and right speckle images to enter the depth calculation engine to perform stereo matching on the left and right speckle images to obtain a disparity map or a depth map and output the disparity map through the output port; When responding to the high-precision scanning mode, the left camera input port and the right camera input port are used to receive the left and right multi-line images for parallel input into the multi-channel input switch. The multi-channel input switch allows the left and right multi-line images to enter the depth calculation engine to perform light strip extraction on the left and right multi-line images to obtain left and right images of interest and output them through the output port.

3. The scanning chip according to claim 1 or 2, wherein: When the three-dimensional scanner includes a left camera, a right camera and an RGB camera for collecting the left and right feature images and texture images, the image input port also includes an RGB camera input port, and the left camera input port, the right camera input port and the RGB camera input port are respectively connected to the multi-channel input switch in parallel. The multi-channel input switch selectively allows the image data of the corresponding input port to be input into the multi-channel output switch or the depth calculation engine in response to different scanning modes.

4. The scanning chip according to claim 3, wherein: The scanning chip further includes a left image processing unit and a right image processing unit disposed between the multi-input switch and the depth calculation engine, wherein: When the multi-channel input switch allows the left and right feature images to enter the depth calculation engine, the left camera input port and the right camera input port respectively receive the left and right feature images captured by the left and right cameras, and the multi-channel input switch allows the left and right feature images to enter the left image processing unit and the right image processing unit respectively for image preprocessing; the depth calculation engine performs stereo matching processing on the preprocessed left and right feature images to obtain a disparity map or a depth map, or the depth calculation engine performs light strip extraction on the preprocessed left and right feature images to obtain left and right images of interest and output them through the output port; When the multi-input switch does not allow the left and right feature images to enter the depth calculation engine, the left and right camera input ports respectively receive the left and right feature images and output them through the output port after being allowed by the multi-input switch.

5. The scanning chip according to claim 4, wherein: The scanning chip also includes a multi-way output switch, the input ends of the multi-way output switch are directly connected to the output ends of the multi-way input switch and directly or indirectly connected to the output ends of the depth calculation engine, the output ends of the multi-way output switch are connected to the output port, and the multi-way input switch allows all or part of the image data in the left and right feature images, disparity maps or depth maps, left and right images of interest, and texture images to pass through to the output port.

6. The scanning chip according to claim 5, wherein: The scanning chip also includes an image alignment unit connected between the depth calculation engine and the multi-channel output switch, and the image alignment unit is used to rotate the left and right images of interest or disparity maps or depth maps based on preset external parameters of the left and right cameras and the RGB camera, so that the left and right images of interest or disparity maps or the depth images are aligned with the texture image.

7. The scanning chip according to any one of claims 1 to 6, characterized in that: The scanning chip also includes a bus and a main control unit electrically connected to the bus. The bus is electrically connected to the three-dimensional scanner, the multi-input switch and the multi-output switch. The main control unit responds to different scanning modes and controls the three-dimensional scanner, the multi-input switch and the multi-output switch through the bus.

8. A three-dimensional scanner, characterized in that: Used to scan a scanning object in response to a scanning mode to obtain multiple frames of image data of the scanning object, wherein the scanning mode includes a high-precision scanning mode and / or a fast scanning mode; wherein the three-dimensional scanner includes a transmitting end, a receiving end and a scanning chip: The transmitting end includes a speckle projection unit and / or a multi-line projection unit for projecting a speckle pattern beam and / or a multi-line pattern beam onto the scanning object; The receiving end includes a left camera and a right camera, and the left camera and the right camera are used to collect the light beam reflected by the scanned object to form left and right characteristic images, and the left and right characteristic images include left and right speckle images and / or left and right multi-line images; The scanning chip includes an image input port, a multi-channel input switch, a depth calculation engine, and an output port; the image input port includes a left camera input port and a right camera input port, respectively receiving the left and right feature images for parallel input to the multi-channel input switch; the multi-channel input switch selectively allows the left and right feature images to enter the depth calculation engine in response to a fast scanning mode or a high-precision scanning mode; wherein: When responding to the fast scanning mode, the multi-input switch allows the left and right feature images to enter the depth calculation engine. When the left and right feature images include left and right speckle images, the depth calculation engine is used to perform stereo matching on the left and right speckle images to obtain a disparity map or a depth map and output the map through the output port. Or, when the left and right feature images include left and right multi-line images, the depth calculation engine is used to perform light stripe extraction on the left and right multi-line images to obtain left and right images of interest and output the images through the output port. Or, When responding to the high-precision scanning mode, the multi-way input switch does not allow the left and right feature images to enter the depth calculation engine, and the multi-way input switch allows the left and right feature images to be output through the output port.

9. The three-dimensional scanner according to claim 8, wherein: include: When the left and right feature images include left and right speckle images and left and right multi-line images, the multi-input switch allows the left and right feature images to enter the depth calculation engine in response to either the fast scanning mode or the high-precision scanning mode. When responding to the fast scanning mode, the left camera input port and the right camera input port are used to respectively receive the left and right speckle images for parallel input to the multi-input switch, and the multi-input switch allows the left and right speckle images to enter the depth calculation engine so as to perform stereo matching on the left and right speckle images to obtain a disparity map or a depth map and output the disparity map through the output port. When responding to the high-precision scanning mode, the left camera input port and the right camera input port are used to receive the left and right multi-line images for parallel input into the multi-channel input switch. The multi-channel input switch allows the left and right multi-line images to enter the depth calculation engine to perform light strip extraction on the left and right multi-line images to obtain left and right images of interest and output them through the output port.

10. The three-dimensional scanner according to claim 9, wherein: The receiving end also includes an RGB camera, which is used to capture a texture image of the scanned object. When responding to the high-precision scanning mode and / or the fast scanning mode, the scanning chip receives the texture image through the image input port and outputs the texture image through the multi-input switch and the output port.

11. The three-dimensional scanner according to claim 10, wherein: The image input port also includes an RGB camera input port, wherein the left camera input port, the right camera input port, and the RGB camera input port are respectively connected in parallel to the multi-channel input switch, and the multi-channel input switch selectively allows the image data of the corresponding input port to be input into the multi-channel output switch or the depth calculation engine in response to different scanning modes.

12. The three-dimensional scanner according to claim 11, wherein: The scanning chip further includes a left image processing unit and a right image processing unit disposed between the multi-input switch and the depth calculation engine, wherein: When the multi-channel input switch allows the left and right feature images to enter the depth calculation engine, the left camera input port and the right camera input port respectively receive the left and right feature images captured by the left and right cameras, and the multi-channel input switch allows the left and right feature images to enter the left image processing unit and the right image processing unit respectively for image preprocessing; the depth calculation engine performs stereo matching processing on the preprocessed left and right feature images to obtain a disparity map or a depth map, or the depth calculation engine performs light strip extraction on the preprocessed left and right feature images to obtain left and right images of interest and output them through the output port; When the multi-input switch does not allow the left and right feature images to enter the depth calculation engine, the left and right camera input ports respectively receive the left and right feature images and output them through the output port after being allowed by the multi-input switch.

13. The three-dimensional scanner according to claim 11, wherein: The scanning chip also includes a multi-channel output switch, the input end of the multi-channel output switch is directly connected to the output end of the multi-channel input switch and is directly or indirectly connected to the output end of the depth calculation engine, the output end of the multi-channel output switch is connected to the output port, and the multi-channel input switch allows all or part of the image data in the left and right speckle images, left and right multi-line images, disparity maps or depth maps, left and right images of interest, and texture images to pass to the output port.

14. The three-dimensional scanner according to claim 13, wherein: The scanning chip also includes an image alignment unit connected to the depth calculation engine and the multi-channel output switch, and the image alignment unit is used to rotate the disparity map or the depth map based on preset external parameters of the left and right cameras and the RGB camera to align the disparity map or the depth image with the texture image.

15. The three-dimensional scanner according to any one of claims 9 to 14, wherein: The scanning chip also includes a bus and a main control unit electrically connected to the bus. The bus is electrically connected to the transmitting end, the collecting end, the multi-way input switch, and the multi-way output switch. The main control unit responds to different scanning modes and controls the transmitting end, the collecting end, the multi-way input switch, and the multi-way output switch through the bus.

16. The three-dimensional scanner according to any one of claims 9 to 14, wherein: When the speckle projection unit in the transmitting end is used to project a speckle pattern beam, the speckle projection unit includes at least a first speckle projection unit and a second speckle projection unit, and the receiving end includes at least a first binocular camera and a second binocular camera respectively composed of two groups of left and right cameras, wherein: The first binocular camera is embedded between the left and right cameras of the second binocular camera so that the two groups of binocular cameras have different baselines. The first speckle projection unit and the second speckle projection unit are arranged between the first binocular camera. The speckle density of the speckle pattern beam projected by the first speckle projection unit is greater than that of the speckle pattern beam projected by the second speckle projection unit. The projection field of view of the speckle pattern beam projected by the first speckle projection unit is smaller than that of the projection field of view of the speckle pattern beam projected by the second speckle projection unit. The first speckle projection unit and the first binocular camera constitute a short-range imaging system suitable for scanning small-sized objects; the second speckle projection unit and the second binocular camera constitute a long-range imaging system suitable for scanning medium-to-large-sized objects, so that the corresponding imaging system can be selected for scanning according to the size and distance of the scanning object in different scanning modes.

17. The three-dimensional scanner according to claim 16, wherein: The first speckle projection unit includes an LED light source, a collimating lens, a mask, and a projection lens. The LED light source, which is arranged on the focal plane of the collimating lens, emits a divergent light beam to the collimating lens. After being focused by the collimating lens, the divergent light beam is projected onto the mask, which is arranged on the focal plane of the projection lens. The focused light beam passes through the circular array on the mask to form a high-density speckle pattern beam, and the magnification effect of the projection lens forms a speckle pattern beam with high density and a large field of view.

18. The three-dimensional scanner according to claim 17, wherein: The depth of field of the projection lens is variable. The projection lens includes a fixed-focus lens and a micro-zoom mechanism. The fixed-focus lens is slightly moved by rotating the micro-zoom mechanism, thereby changing the image distance of the fixed-focus lens to change the working depth of field of the projection lens. In addition, the first speckle projection unit or the second speckle projection unit can project speckle patterns at different distances, and the speckle pattern does not change at different distances.

19. The three-dimensional scanner according to claim 16, wherein: The three-dimensional scanner further includes a first switching circuit and a second switching circuit. The scanning chip is electrically connected to the left camera of the first binocular camera and the left camera of the second binocular camera through the first switching circuit, and is electrically connected to the right camera of the first binocular camera and the right camera of the second binocular camera through the second switching circuit. The first switching circuit or the second switching circuit includes a signal switching switch and a data switching switch. The scanning chip includes a main control unit, wherein: The signal switching switch includes a signal input port and two signal output ports, the signal input port is electrically connected to the scanning chip, and the two signal output ports are electrically connected to the left camera or the right camera of the first binocular camera and the second binocular camera, respectively, so that under the control of the main control unit in the scanning chip, the left camera or the right camera of the first binocular camera and the second binocular camera can be controlled to operate independently through the single signal input port of the signal switching switch; The data switching switch includes two data input ports and one data output port. The two data input ports are electrically connected to the left camera or the right camera of the first binocular camera or the second binocular camera, respectively. The data output port is electrically connected to the left camera input port and the right camera input port of the scanning chip. Under the control of the main control unit in the scanning chip, the data switching switch receives image data captured by the left camera or the right camera of the first binocular camera or the second binocular camera through the single data output port.

20. The three-dimensional scanner according to any one of claims 11 to 17, wherein: When the projection unit includes a speckle projection unit and a multi-line projection unit, the transmitting end projects a speckle pattern beam and a multi-line pattern beam, the left and right cameras in the receiving end include left and right black and white cameras, and the multi-line projection unit, the speckle projection unit, and the RGB camera are arranged between the left and right black and white cameras. When the multi-line projection unit or the speckle projection unit projects the patterned beam, the acquisition field of view of the left and right black and white cameras can be filled.

21. The three-dimensional scanner according to claim 20, wherein: The multi-line projection unit and the speckle projection unit are integrated into the projection module by assembling separate modules. The projection module includes an optoelectronic support, a multi-line projection unit, and a speckle projection unit. One or more multi-line projection units or speckle projection units are integrated into a single module using one or more of a laser light source, a collimating element, a modulation element, and a diffractive optical element. Positioning holes with fool-proofing principles are provided on the optoelectronic support so that the integrated multi-line projection unit and the speckle projection unit can be assembled in a specific position to obtain the projection module.

22. The three-dimensional scanner according to claim 20, wherein: When the multi-line projection unit includes multiple units, the multi-line projection unit is used to project a line pattern beam of at least one or more combinations of parallel multi-line pattern beams parallel or perpendicular to the baseline of the three-dimensional scanner, crossed multi-line pattern beams inclined relative to the baseline and in opposite directions, or a single line pattern beam to the scanned object.

23. A three-dimensional scanning system, characterized in that: The invention comprises a three-dimensional scanner according to any one of claims 7 to 20 and modeling software applied to a host computer, wherein the three-dimensional scanner is in communication with the modeling software, wherein: The three-dimensional scanner is configured to scan a scanned object in response to different scanning modes to obtain left and right characteristic images of the scanned object or obtain left and right characteristic images and texture images of the scanned object, wherein the different scanning modes include a high-precision scanning mode and / or a fast scanning mode, and the left and right characteristic images include left and right speckle images and / or left and right multi-line images; When the multi-input switch in the three-dimensional scanner allows the left and right feature images to enter the depth calculation engine to achieve a high-precision scanning mode or a fast scanning mode, the modeling software performs three-dimensional reconstruction based on the left and right feature images or the left and right images of interest obtained by the three-dimensional scanner to obtain a texture-free three-dimensional model of the scanned object; or receives the left and right feature images or the left and right images of interest and the texture image obtained based on the three-dimensional scanner and performs three-dimensional reconstruction using the left and right feature images and the texture image to obtain a textured three-dimensional model of the scanned object; When the multi-input switch in the three-dimensional scanner does not allow the left and right feature images to enter the depth calculation engine to achieve a high-precision scanning mode, the modeling software performs three-dimensional reconstruction based on the disparity map or depth map or left and right images of interest obtained by the three-dimensional scanner to obtain a texture-free three-dimensional model of the scanned object; or, performs three-dimensional reconstruction based on the disparity map or depth map or left and right images of interest and texture images obtained by the three-dimensional scanner to obtain a textured three-dimensional model of the scanned object.

24. The three-dimensional scanning system according to claim 23, wherein: The three-dimensional scanning system further includes a mobile handle, which is electrically connected to the three-dimensional scanner and connected to the host computer via wired or wireless communication; wherein the mobile handle includes a battery unit and a compression unit: The battery unit is used to supply power to the three-dimensional scanner; The compression unit is used to receive the disparity map or depth map or left and right images of interest and / or the texture image obtained based on the three-dimensional scanner in the fast scanning mode, and use a preset image coding and compression method to encode and compress the image data for transmission to the host computer; receive the left and right feature images and / or the texture image obtained based on the three-dimensional scanner in the high-precision scanning mode, and use a preset image coding and compression method to encode and compress the image data for processing by the modeling software.

Images