Scanning chip, scanner, scanning system and scanning method for realizing double scanning modes

Abstract

The invention provides a scanning chip, a scanner, a scanning system and a scanning method for realizing double scanning modes. The scanning chip comprises an image input port, a multi-path input switch, a depth calculation engine and an output port so as to respond to a high-precision scanning mode or a rapid scanning mode to process left and right feature image data acquired by a three-dimensional scanner; in a high-precision scanning mode, left and right feature images are received through an image input port and enter a multi-path input switch so as to allow the left and right feature images to be output through an output port; in the fast scanning mode, left and right feature images are received through the image input port and enter the multi-path input switch, so that the left and right feature images are allowed to enter the depth calculation engine for stereo matching processing to obtain a disparity map or a depth map or light stripe extraction processing to obtain left and right interested images, and the left and right interested images are output through the output port. The three-dimensional scanner applying the scanning chip has different measurement precisions and can meet the scanning requirements of different application scenes.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional scanning technology, and in particular to a scanning chip, a scanner, a scanning system and a scanning method for realizing a dual scanning mode. Background Art

[0002] A handheld scanner is a three-dimensional data acquisition device, which is usually combined with a modeling software applied to a host computer to form a three-dimensional scanning system. The handheld scanner, which integrates hardware such as a projection module and a camera, is used to scan the scanned object to obtain image data of the scanned object, while the modeling software applied to the host computer is responsible for processing the image data collected by the scanner to obtain a three-dimensional model of the scanned object.

[0003] In the prior art, the scanning technology of handheld scanners mainly relies on the laser line scanning principle or the stripe scanning principle, wherein: the laser line scanning principle is to emit one or more laser lines to a partial area of ​​the surface of the scanned object through a projection module within a single frame acquisition cycle, and the camera captures the laser lines reflected back by the partial area to generate a laser line image and transmits it to the host computer, and the host computer calculates the three-dimensional information of the scanned object through triangulation and then reconstructs the three-dimensional model of the scanned object; the stripe scanning principle is to project a series of stripe patterns onto the surface of the object to the scanned object through the projection module, and the camera captures the stripe pattern projected onto the surface of the scanned object to generate a stripe image and upload it to the host computer, and the host computer calculates the phase change of the stripe pattern based on the captured stripe image to obtain the three-dimensional information of the scanned object and then reconstructs the three-dimensional model of the scanned object.

[0004] Specifically, existing handheld scanners generally do not support deep calculation and processing of the image data they collect to obtain three-dimensional information of the scanned object, and then transmit the three-dimensional information of the scanned object to a host computer for processing to obtain a three-dimensional model of the scanned object. They usually transmit the image directly to the host computer for processing by the modeling software in the host computer to obtain the three-dimensional model of the scanned object. Therefore, for handheld scanners with multi-eye imaging, compared with the former's processing method of performing depth calculation on the multi-eye collected images at the scanning end and then uploading them, the latter's processing method of directly transmitting the image will result in a large amount of image data needing to be transmitted, and will have higher requirements on the output interface bandwidth.

[0005] Furthermore, although handheld scanners based on the laser line scanning principle or the stripe scanning principle can provide high-precision scanning results, both are time-consuming and have low reconstruction efficiency. For example, the laser line projected based on the first scanning projection in a single frame acquisition cycle can only cover a small area of ​​the scanned object, that is, each frame of the image acquired includes less information about the scanned object and all the information of the image needs to be uploaded. If all the information of the scanned object needs to be obtained, it is necessary to project the laser line to different areas of the scanned object multiple times in multiple acquisition cycles and collect multiple frames of images to obtain a three-dimensional model of the scanned object, which leads to a large amount of data processing; and in view of the phase periodicity characteristics, when using a handheld scanner based on the stripe scanning principle to obtain a stripe image to obtain a three-dimensional model of the scanned object, a phase wrapping phenomenon is involved. Therefore, when calculating the stripe phase change, a phase unwrapping technology is required, which leads to an increase in the amount of calculation.

[0006] Furthermore, existing handheld scanners can only achieve high-precision scanning results, but different scanning objects or scenes have different performance requirements for 3D scanners. For example, industrial design scenes require high-precision 3D scanners, while 3D printing scenes require 3D scanners that can quickly scan objects. For existing 3D scanners based only on a single scanning principle, they cannot meet the diversity of scanning objects and scenes, and are difficult to adapt to all scanning objects or scenes. Summary of the invention

[0007] The present application provides a scanning chip, scanner, scanning system and scanning method for realizing dual scanning modes, which can meet the three-dimensional scanning requirements of multiple scenes, and is conducive to the promotion and application of three-dimensional scanners and the reduction of three-dimensional scanning costs.

[0008] In a first aspect, the present application provides a scanning chip applied to a three-dimensional scanner, which is used to process left and right feature images collected by the three-dimensional scanner to realize a fast scanning mode and / or a high-precision scanning mode. The scanning chip includes an image input port, a multi-channel input switch, a depth calculation engine and an output port. The left and right feature images include left and right speckle images and / or left and right multi-line images: the image input port is used to receive the left and right feature images into the multi-channel input switch; the multi-channel input switch selectively allows the left and right feature images to enter the depth calculation engine to realize the fast scanning mode and / or the high-precision scanning mode; if the multi-channel input switch allows the left When the left and right feature images enter the depth calculation engine to realize the fast scanning mode, when the left and right feature images include the left and right speckle images, the depth calculation engine is used to perform stereo matching on the left and right speckle images to obtain the disparity map or the depth map and output it through the output port; or, when the left and right feature images include the left and right multi-line images, the depth calculation engine is used to perform light strip extraction on the left and right multi-line images to obtain the left and right images of interest and output them through the output port; or, if the multi-input switch does not allow the left and right feature images to enter the depth calculation engine to realize the high-precision scanning mode, the multi-input switch allows the left and right feature images to be output through the output port.

[0009] In a second aspect, the present application provides a three-dimensional scanner for scanning a scanning object in response to a scanning mode to obtain multi-frame image data of the scanning object, the scanning mode including a high-precision scanning mode and / or a fast scanning mode; wherein the three-dimensional scanner includes a transmitting end, a receiving end and a scanning chip: the transmitting end includes a projection unit for projecting a speckle pattern beam and / or a multi-line pattern beam onto the scanning object; the receiving end includes a left camera and a right camera, and the left camera and the right camera are used to collect the light beam reflected back by the scanning object to form left and right speckle images and / or left and right multi-line images; the scanning chip includes an image input port, a multi-channel input switch, a depth calculation engine and an output port, the image input port is used to receive left and right feature images into the multi-channel input switch; the multi-channel input switch selectively allows left and right feature images to be displayed on the scanning object. The feature image enters the depth calculation engine to realize the fast scanning mode and / or the high-precision scanning mode; wherein: if the multi-input switch allows the left and right feature images to enter the depth calculation engine to realize the fast scanning mode, when the left and right feature images include the left and right speckle images, the depth calculation engine is used to perform stereo matching on the left and right speckle images to obtain the disparity map or the depth map and output it through the output port; or, when the left and right feature images include the left and right multi-line images, the depth calculation engine is used to perform light strip extraction on the left and right multi-line images to obtain the left and right images of interest and output them through the output port; or, if the multi-input switch does not allow the left and right feature images to enter the depth calculation engine to realize the high-precision scanning mode, the multi-input switch allows the left and right feature images to be output through the output port.

[0010] In a third aspect, the present application provides a three-dimensional scanning system, including the three-dimensional scanner of the second aspect and modeling software applied to a host computer, wherein the three-dimensional scanner is used to respond to different scanning modes to scan a scanning object to obtain left and right feature images of the scanning object or obtain left and right feature images and texture images of the scanning object, the different scanning modes include a high-precision scanning mode and / or a fast scanning mode, and the left and right feature images include left and right speckle images and / or left and right multi-line images; when the multi-channel input switch in the three-dimensional scanner allows the left and right feature images to enter the depth calculation engine to achieve a high-precision scanning mode or a fast scanning mode, the modeling software obtains the left and right feature images or the left and right objects of interest according to the three-dimensional scanner. The image is three-dimensionally reconstructed to obtain a texture-free three-dimensional model of the scanned object; or, the left and right feature images or the left and right images of interest and the texture image obtained based on the three-dimensional scanner are received and the left and right feature images and the texture image are used to perform three-dimensional reconstruction to obtain a textured three-dimensional model of the scanned object; when the multi-channel input switch in the three-dimensional scanner does not allow the left and right feature images to enter the depth calculation engine to achieve a high-precision scanning mode, the modeling software performs three-dimensional reconstruction based on the disparity map or depth map or the left and right images of interest obtained by the three-dimensional scanner to obtain a texture-free three-dimensional model of the scanned object; or, the disparity map or depth map or the left and right images of interest and the texture image obtained by the three-dimensional scanner are three-dimensionally reconstructed to obtain a textured three-dimensional model of the scanned object.

[0011] In a fourth aspect, the present application provides a three-dimensional scanning method, which is applied to a three-dimensional scanner including a depth calculation engine and a modeling software applied to a host computer, the method comprising: projecting a characteristic pattern beam including a speckle pattern beam and / or a multi-line pattern beam onto a scanned object, and collecting the beam reflected back by the scanned object to form left and right characteristic images, the left and right characteristic images including left and right speckle images and / or left and right multi-line images; selectively allowing the left and right characteristic images to enter the depth calculation engine to process the left and right characteristic images in response to a scanning mode, the scanning mode including a high-precision scanning mode and / or a fast scanning mode, wherein: when the left and right characteristic images are not allowed to enter the depth calculation engine, the scanning mode is a high-precision scanning mode, and the left and right characteristic images are directly output. the left and right feature images are sent to the modeling software to perform three-dimensional reconstruction using the left and right feature images to obtain a three-dimensional model of the scanned object; when the left and right speckle images are allowed to enter the depth calculation engine, the scanning mode is a fast scanning mode, and when the left and right feature images are left and right speckle images, the depth calculation engine is used to perform stereo matching on the left and right speckle images to obtain a disparity map or a depth map and output it to the modeling software to perform three-dimensional reconstruction using the disparity map or the depth map to obtain a three-dimensional model of the scanned object; and / or, when the left and right feature images are left and right multi-line images, the depth calculation engine is used to perform light strip extraction on the left and right multi-line images to obtain left and right images of interest and output them to the modeling software to perform three-dimensional reconstruction using the left and right images of interest to obtain a three-dimensional model of the scanned object.

[0012] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed, the three-dimensional scanning method of the fourth aspect is implemented.

[0013] Compared with the prior art, the scanning chip provided by the present application includes left and right image data ports, a multi-channel input switch and a depth calculation engine, wherein the left and right image input ports in the scanning chip support the simultaneous input of left and right feature images, and the selection function of the multi-channel input switch selectively allows the depth calculation engine to realize a fast scanning mode or a high-precision scanning mode, by allowing the input left and right feature images to be directly output to the host computer, so that the host computer can directly use the original images collected by the scanner to perform high-precision three-dimensional reconstruction to realize a high-precision scanning mode; or allowing the input left and right feature images to enter the depth calculation engine, so that the depth calculation engine can perform stereo matching processing on the left and right feature images to obtain a single-frame depth image or perform light strip extraction on the left and right feature images to obtain left and right images of interest that only include the area of ​​interest and output them to the host computer to realize a fast scanning mode, thereby converting two separate frames of images into a single-frame image output or eliminating redundant information in the image, which not only reduces the amount of image data output by the scanner to the host computer, improves the image transmission efficiency, but also reduces the requirements for the output interface bandwidth.

[0014] Furthermore, within a single frame acquisition cycle, the three-dimensional scanner using the above scanning chip can project a multi-line pattern light beam or a speckle pattern light beam, or both. When both are taken into account, the multi-channel input switch in the scanning chip also allows the three-dimensional scanner to acquire corresponding left and right multi-line images or left and right speckle images, which enter the depth calculation engine. The depth calculation engine performs light strip extraction processing on the left and right multi-line images to obtain left and right images of interest and outputs them to the host computer to achieve a high-precision scanning mode. The left and right speckle images are stereo matched to obtain a disparity map or a depth map and output them to the host computer to achieve a fast scanning mode. Compared with multi-line pattern beam projection, when the transmitting end in the three-dimensional scanner projects a speckle pattern beam, it can cover most areas of the scanned object and be simultaneously captured by the left and right cameras of the receiving end to obtain left and right speckle images including more regional information of the scanned object, thereby achieving more three-dimensional information of the scanned object with a smaller number of image frames, quickly acquiring information of the entire area of ​​the scanned object, improving scanning efficiency to achieve a fast scanning mode; and when the transmitting end projects a multi-line pattern beam, it can capture finer surface features of the scanned object to achieve a high-precision scanning mode, thereby enabling the three-dimensional scanner provided by the present application including the above-mentioned scanning chip and different projection pattern beams to achieve a high-precision scanning mode and a fast scanning mode with different measurement accuracies to meet the accuracy requirements of different application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the system architecture of a scanning system for realizing single / dual scanning modes provided by the present application.

[0016] Figure 2 It is a schematic diagram of the architecture of a scanning head in the three-dimensional scanning system provided by this application.

[0017] Figure 3 It is a schematic diagram of the system architecture of the scanning chip in the three-dimensional scanning system provided in this application.

[0018] Figure 4 This is a schematic diagram of the architecture of another scanning head in the three-dimensional scanning system provided in this application.

[0019] Figure 5 It is a flow chart of the image coding and compression method in the three-dimensional scanning system provided by the present application.

[0020] Figure 6 It is a schematic diagram of replacement vector calculation in the image coding compression method in the three-dimensional scanning system provided by the present application.

[0021] Figure 7 This is another replacement vector calculation schematic diagram of the image coding compression method in the three-dimensional scanning system provided by the present application.

[0022] Figure 8 It is a schematic diagram of result vector calculation of the image coding compression method in the three-dimensional scanning system provided by the present application.

[0023] Fig. 9 It is a flow chart of a multi-mode three-dimensional reconstruction method in a three-dimensional scanning system provided in this application.

[0024] Fig.10 It is a flow chart of the three-dimensional scanning method provided by this application.

[0025] Fig.11 It is a flow chart of another three-dimensional scanning method provided by the present application. DETAILED DESCRIPTION

[0026] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In the drawings, the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present application, and should not be construed as limitations on the present application.

[0027] The terms "first", "second", "third", "fourth", etc. (if any) in this application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The size of the sequence number of each process below does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0028] References to "some embodiments" and the like described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in some embodiments" and the like that appear at different places in this specification do not necessarily all refer to the same embodiment, but rather mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0029] The terms used in the disclosed embodiments are only for the purpose of describing specific embodiments and are not intended to limit the disclosed embodiments. The singular forms of "a", "the" used in the disclosed embodiments and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0030] The following will be combined with the accompanying drawings to exemplify the technical solution in this application. It should be understood that Figures 1 to 11 The examples are only intended to help those skilled in the art understand the embodiments of the present application, and are not intended to limit the embodiments of the present application to Figures 1 to 11 Those skilled in the art can obviously make various equivalent modifications or changes based on the examples given, and such modifications and changes also fall within the scope of the embodiments of the present application.

[0031] Figure 1This is an architectural schematic diagram of a three-dimensional scanning system that implements a dual scanning mode according to the present application. The system includes a three-dimensional scanner, a host computer equipped with modeling software, and a display end. The three-dimensional scanner is used to scan a scanning object in response to different scanning modes to obtain multiple frames of image data of the scanning object. The scanning modes include a high-precision scanning mode and a fast scanning mode. The modeling software in the host computer processes the multiple frames of image data according to a preset three-dimensional reconstruction method to obtain one or more combinations of a two-dimensional image, a depth image, point cloud data, or a three-dimensional model of the scanning object. The display end is used to display the processing results of the host computer, such as displaying a two-dimensional image, a depth image, point cloud data, or a three-dimensional model of the scanning object.

[0032] In one embodiment, a three-dimensional scanner includes a scanning head and a scanning chip, wherein the scanning head includes a transmitting end and a receiving end, the transmitting end includes a projection unit, and is used to project a characteristic pattern beam onto a scanned object, the characteristic pattern beam including a speckle pattern beam and / or a multi-line pattern beam; the receiving end includes a left camera, a right camera and an RGB camera, the left camera and the right camera are used to collect the beam reflected by the scanned object to form a left and right characteristic image, the left and right characteristic images including a left and right speckle image and / or a left and right multi-line image, and the RGB camera is used to collect a texture image of the scanned object; the scanning chip is used to perform stereo matching on the left and right speckle images to obtain a disparity map or a depth map and transmit the disparity map or the depth map to a host computer, or to perform light strip extraction (or laser line center line extraction) on the left and right multi-line images to obtain left and right images of interest and transmit them to the host computer, or to only transfer the left and right characteristic images to the host computer, or to transmit the texture image to the host computer for processing by modeling software in the host computer to obtain a three-dimensional model of the scanned object.

[0033] It should be noted that when a multi-line patterned light beam is used to scan a scanning object, it is generally necessary to paste marker points on the scanning object to assist in the subsequent stitching of multiple frames of left and right multi-line images based on the marker points to obtain a three-dimensional model of the scanned object. For this reason, in the present application, when light strips are extracted from the left and right multi-line images to obtain the left and right images of interest, the contours of the marker points are often also extracted. That is, in addition to the center lines of each laser line in the multi-line image, the left and right images of interest may also include the contours of the marker points to facilitate the subsequent acquisition of the three-dimensional model of the scanned object. The present application does not impose any restrictions on this.

[0034] In one embodiment, the scanning chip includes an image input port, a multi-input switch, a depth calculation engine and an output port, wherein the image input port is used to receive left and right feature images and enter the multi-input switch; the multi-input switch selectively allows the left and right feature images to enter the depth calculation engine to achieve a fast scanning mode and / or a high-precision scanning mode; wherein, if the multi-input switch allows the left and right feature images to enter the depth calculation engine to achieve a fast scanning mode, when the left and right feature images include left and right speckle images, the depth calculation engine is used to perform stereo matching on the left and right speckle images to obtain a disparity map or a depth map and output it through the output port; or, when the left and right feature images include left and right multi-line images, the depth calculation engine is used to perform light strip extraction on the left and right multi-line images to obtain left and right images of interest and output them through the output port; or, if the multi-input switch does not allow the left and right feature images to enter the depth calculation engine to achieve a high-precision scanning mode, the multi-input switch allows the left and right feature images to be directly output through the output port.

[0035] Compared with the high-precision scanning mode, in this embodiment, the left and right feature images are directly uploaded to the host computer for processing. In this application, two separate speckle image frames are combined into a single depth image frame for output by performing stereo matching processing on the left and right speckle images, or light strip extraction is performed on the left and right multi-line images to eliminate redundant information in the image data, and only the image data including the area of ​​interest is output, which not only improves the image upload rate, but also reduces the bandwidth requirement for the output interface, which is conducive to improving the efficiency of three-dimensional scanning.

[0036] In another embodiment, compared with the speckle pattern beam projected by the transmitting end, the multi-line pattern beam emitted by the transmitting end can collect finer surface features of the scanning object to generate left and right multi-line images with higher measurement accuracy. Therefore, when the left and right feature images include the left and right speckle images and the left and right multi-line images, the multi-way input switch in the scanning chip allows the left and right feature images to enter the depth calculation engine to achieve a fast scanning mode or a high-precision scanning mode; in the fast scanning mode, the image input port is used to receive the left and right speckle images to enter the multi-way input switch, and the multi-way input switch allows the left and right speckle images to enter the depth calculation engine to perform stereo matching on the left and right speckle images to obtain a disparity map or a depth map and output it through the output port; in the high-precision scanning mode, the image input port is used to receive the left and right multi-line images to enter the multi-way input switch, and the multi-way input switch allows the left and right multi-line images to enter the depth calculation engine to perform light strip extraction on the left and right multi-line images to obtain left and right images of interest and output them through the output port.

[0037] It should be understood that, compared with a multi-line patterned light beam, a frame of speckle patterned light beam is projected onto the scanned object within a single-frame acquisition period, and the frame of patterned light beam can cover most areas of the scanned object and be simultaneously acquired by the left and right cameras of the receiving end to obtain left and right speckle images including more regional information of the scanned object, thereby achieving more three-dimensional information of the scanned object by acquiring a small number of image frames corresponding to a relatively small acquisition period, quickly acquiring information of all areas of the scanned object, improving scanning efficiency, and thus achieving a fast scanning mode.

[0038] Furthermore, when the multi-input switch in the three-dimensional scanner allows the left and right feature images to enter the depth calculation engine to achieve a high-precision scanning mode or a fast scanning mode, the modeling software performs three-dimensional reconstruction based on the left and right feature images or the left and right images of interest obtained by the three-dimensional scanner to obtain a texture-free three-dimensional model of the scanned object; or, receives the left and right feature images or the left and right images of interest and the texture image obtained based on the three-dimensional scanner and uses the left and right feature images and the texture image to perform three-dimensional reconstruction to obtain a textured three-dimensional model of the scanned object; when the multi-input switch in the three-dimensional scanner does not allow the left and right feature images to enter the depth calculation engine to achieve a high-precision scanning mode, the modeling software performs three-dimensional reconstruction based on the disparity map or depth map or the left and right images of interest obtained by the three-dimensional scanner to obtain a texture-free three-dimensional model of the scanned object; or, performs three-dimensional reconstruction based on the disparity map or depth map or the left and right images of interest and the texture image obtained by the three-dimensional scanner to obtain a textured three-dimensional model of the scanned object.

[0039] In one embodiment, the scanning head also includes an IMU for detecting the position of the scanning head. When it is detected that the scanning head is facing upward, that is, the light outlet of the scanning head is facing upward, the transmitting end is closed to prevent the transmitting end from projecting a light beam to the user, thereby ensuring eye safety.

[0040] It should be noted that the host computer of the present application is a processing end with computing power greater than the scanning chip in the three-dimensional scanner, such as a computer, desktop computer, cloud or mobile phone terminal. Given that the host computer can perform complex and high-precision algorithms, the images captured by the left and right cameras can be directly transmitted to the host computer without processing to obtain high-precision point cloud data or a three-dimensional model; and the images captured by the left and right cameras can be processed by the scanning chip in the three-dimensional scanner to obtain a depth map and then transmitted to the host computer, which can reduce the processing time of the host computer and quickly generate point cloud data or a three-dimensional model of the scanned object.

[0041] Figure 2The system architecture diagram of a scanning head provided by the present application, the speckle projection unit in the transmitting end includes at least a first speckle projection unit 1 and a second speckle projection unit 2, and the receiving end includes at least an RGB camera 7 and a first binocular camera 3 and a second binocular camera 4 respectively composed of two groups of left and right cameras, wherein the first binocular camera 3 is embedded between the left and right cameras of the second binocular camera 4 so that the two groups of binocular cameras have different baselines, the first speckle projection unit 1 and the second speckle projection unit 2 are arranged between the first binocular camera 3, and the speckle density and / or field of view of the speckle pattern beam projected by the first speckle projection unit 1 are different from the speckle pattern beam projected by the second speckle projection unit 2, thereby respectively forming an active binocular imaging system with different measurement accuracy with the first binocular camera 3 and the second binocular camera 4. The RGB camera is arranged between the left camera or the right camera of the first binocular camera 3 and the second binocular camera 4, so that the image collected by the RGB camera has an overlapping area with the image collected by the binocular camera, thereby obtaining the texture information of the scanned object to better obtain the detail information of the scanned object, which is conducive to improving the accuracy of the three-dimensional model of the scanned object.

[0042] Furthermore, the different baselines of the first binocular camera 3 and the second binocular camera 4 will also result in different measurement accuracies of the imaging systems respectively formed based on the first binocular camera 3 and the second binocular camera 4. That is, when the first speckle projection unit 1 and the second speckle projection unit 2 have different speckle densities and / or projection fields of view and / or wavelengths and / or working distances, they respectively form active binocular imaging systems with at least two measurement accuracies with the first binocular camera 3 and the second binocular camera 4, so that the scanner equipped with the imaging system can adapt to different scanning scene requirements. For example, the imaging system formed by the first speckle projection unit 1 and the first binocular camera 3 with a short baseline is suitable for short-distance scanning, and the imaging system formed by the second speckle projection unit 2 and the second binocular camera 4 with a long baseline is suitable for long-distance scanning. The short and long distances can be divided according to the distance threshold.

[0043] In other words, in the two different scanning modes of high-precision scanning mode and fast scanning mode, both the short-range imaging system formed by the first speckle projection unit 1 and the first binocular camera 3 and the long-range imaging system formed by the second speckle projection unit 2 and the second binocular camera 4 can be used. The user can select the corresponding imaging system and the adapted scanning mode to scan the scanning object according to the size and distance of the scanning object, thereby meeting the scanning accuracy requirements of different scanning objects.

[0044] In one embodiment, the first speckle projection unit 1 includes an LED light source, a collimating lens, a MASK mask and a projection lens, and the second speckle projection unit 2 includes a VCSEL light source array, a collimating lens and a diffractive optical element (DOE). The VCSEL light source array includes a plurality of VCSEL light sources, which emit a spot light beam. Since the VCSEL light source array is limited by the existing process, the filling interval between the plurality of VCSEL light sources cannot be smaller, that is, there is a certain gap between the plurality of VCSEL light sources; and the light source of the second speckle projection unit 2 is an LED light source, and its speckle density is determined by the pattern density on the MASK mask and the focal length of the projection lens, so that the speckle pattern density projected by the second speckle projection unit 2 using the VCSEL light source array is smaller than that of the first speckle projection unit 1, and the measurement accuracy of the imaging system corresponding to the second speckle projection unit 2 is also smaller than that of the imaging system corresponding to the first speckle projection unit 1.

[0045] Specifically, a dot array pattern of a preset density is provided on the MASK mask in the first speckle projection unit 1, and a high-density speckle pattern can be formed when the light beam passes through the circular array pattern. Specifically, in the first speckle projection unit 1, an LED light source arranged on the focal plane of the collimating lens emits a divergent light beam to the collimating lens, and after being focused by the collimating lens, a focused light beam is projected to the MASK mask arranged on the focal plane of the projection lens, and the focused light beam passes through the circular array on the MASK mask to form a high-density speckle pattern beam, and through the magnification effect of the projection lens, a high-density speckle pattern beam with a larger field of view than the original (i.e., without the projection field of view of the projection lens) is formed, so that it is suitable for scanning small objects and restoring details to the greatest extent.

[0046] In another embodiment, a VCSEL light source array arranged on the focal plane of the collimating lens in the second speckle projection unit 2 emits a speckle pattern light beam to the collimating lens, which forms a parallel speckle pattern light beam after being collimated by the collimating lens and propagates to the diffractive optical element, and the diffractive optical element is used to replicate the parallel speckle pattern light beam to form a speckle pattern light beam with a projection field larger than that of the first speckle projection unit 1, but a lower speckle density than that of the first speckle projection unit 1.

[0047] In one embodiment, the projection lens is a lens with variable depth of field, which includes a fixed-focus lens and a micro-zoom mechanism. The micro-zoom mechanism includes a rotating handle or an angle limiter. The fixed-focus lens is slightly moved by rotating the micro-zoom mechanism, so that the image distance of the fixed-focus lens is changed, and the focal length of the fixed-focus lens is slightly changed, so that the working depth of field of the projection lens is variable. The first speckle projection unit 1 or the second speckle projection unit 2 can project speckle patterns at different distances and the speckle patterns will not change at different distances, thereby avoiding insufficient depth of field due to a single focal length and ensuring imaging quality at the same time.

[0048] The first speckle projection unit 1 in this embodiment uses a combination of LED, MASK mask and projection lens, which can project speckles of different wavelengths by replacing LEDs of different wavelengths, and the speckle density can be freely adjusted by adjusting the pattern density of the MASK mask and the focal length of the projection lens, thereby ensuring the scanning accuracy of small objects at close range; the second speckle projection unit collects VCSEL laser light source, collimating lens and diffractive optical element, which has high energy efficiency, thereby ensuring long-distance scanning of more than 1m, and a large field of view, which is suitable for scanning medium and large objects.

[0049] In one embodiment, the optical axes of the left and right cameras in the first binocular camera 3 or the second binocular camera 4 are parallel or angled. When the optical axes of the left and right cameras in the first binocular camera 3 or the second binocular camera 4 are parallel, the overlapping area of ​​the acquisition fields of the left and right cameras in the above binocular cameras is small, and more information of the scanned object cannot be collected when the scanning head is used for a single scan, resulting in relatively low scanning efficiency; and when the optical axes of the left and right cameras in the first binocular camera 3 or the second binocular camera 4 are angled, the overlapping area of ​​the acquisition fields of the left and right cameras in the binocular cameras is large, and more information of the scanned object can be collected when the three-dimensional scanner is used for a single scan, thereby improving the scanning efficiency of the scanning head.

[0050] In another embodiment, each camera in the first binocular camera 3 or the second binocular camera 4 includes an image sensor and an imaging lens, and the imaging lens is used to modulate the light beam reflected by the scanned object to image it on the image sensor. Further, the focal lengths of the imaging lens of the first binocular camera 3 and the imaging lens of the second binocular camera 4 are the same or different. When the focal lengths of the imaging lenses of the two are the same, that is, their working depths of field are the same, at this time, a scanning head with different measurement accuracy can be formed by using a speckle projection unit with different projection speckle densities.

[0051] When the focal lengths of the imaging lenses of the two are different, the corresponding binocular cameras have different acquisition fields of view. Generally speaking, when the focal length of the imaging lens is small, the corresponding camera has a large acquisition field of view and a large working depth of field. It can take into account both long and short distance scanning and is beneficial for scanning large objects, but its disadvantage is that the spatial resolution is low and it is impossible to perform high-precision measurements on small objects. When the focal length of the imaging lens is large, the corresponding camera has a small acquisition field of view and a high spatial resolution, which is beneficial for scanning small objects, but its working depth of field is small and it cannot take into account both long and short distance scanning and is not beneficial for scanning large objects.

[0052] To this end, in one embodiment, the focal length of the imaging lens of the first binocular camera 3 is greater than the focal length of the imaging lens of the second binocular camera 4. Given that the baseline of the first binocular camera 3 is smaller than that of the second binocular camera 4, the second binocular camera 4 with a small focal length can ensure higher scanning accuracy through a long baseline, thereby making the first binocular camera 3 with a short baseline and a large imaging lens focal length more suitable for scanning small-sized objects at close range, while the second binocular camera 4 with a long baseline and a small imaging lens focal length is more suitable for scanning long-distance and medium-to-large objects.

[0053] Furthermore, if Figure 2 As shown, the scanning head also includes a first switching circuit 51 and a second switching circuit 52. The scanning chip 6 in the three-dimensional scanner is electrically connected to the left camera 3a in the first binocular camera 3 and the left camera 4a in the second binocular camera 4 through the first switching circuit 51 to drive the left camera in the binocular camera and receive data; the scanning chip 6 in the three-dimensional scanner is electrically connected to the right camera 3b in the first binocular camera 3 and the right camera 4b in the second binocular camera 4 through the second switching circuit 52 to drive the right camera in the binocular camera and receive data, so that the two groups of binocular cameras can be connected to the scanning chip 6 using only 4 interfaces instead of 8 interfaces, thereby reducing the number of interfaces and thus reducing the size of the scanning chip 6.

[0054] In the present application, the first speckle projection unit 1 and the second speckle projection unit 2 respectively form an active binocular imaging system with different measurement accuracy with the first binocular camera 3 and the second binocular camera 4 of different baselines, and form an active binocular texture imaging system with different measurement accuracy in combination with the RGB camera. The scanning chip 6 also includes a main control unit. Each component in each imaging system is controlled by a synchronization signal sent by the main control unit in the scanning chip 6 to ensure that the components in each imaging system work synchronously. Specifically, the user adaptively selects the size distance of the scanning object and the required scanning mode in the host computer. The host computer generates a control instruction in response to the size distance of the scanning object and the corresponding scanning mode and sends it to the scanning chip 6. The main control unit in the scanning chip 6 parses the control instruction and generates a corresponding trigger signal to control the corresponding active binocular imaging system to work and control the data flow processing inside the scanning chip 6.

[0055] The main control unit in the scanning chip 6 sends different trigger signals to the first speckle projection unit 1 or the second speckle projection unit 2, the first binocular camera 3 or the second binocular camera 4 and the RGB camera at the same time. Specifically, the scanning chip 6 sends a PWM signal to the first speckle projection unit 1 or the second speckle projection unit 2 to trigger the projection unit to project the speckle pattern beam, sends a first I2C signal to the first switching circuit 51 and the second switching circuit 52 to drive the first binocular camera 3 or the second binocular camera 4 to start to collect the left and right speckle images, and sends a second I2C signal to the RGB camera to drive the RGB camera to start to collect the texture image, thereby realizing that each module in each active binocular imaging system works simultaneously.

[0056] After the imaging system acquires an image, the main control unit in the scanning chip 6 controls the output and input of the data stream therein in response to the corresponding scanning mode, that is, when the scanning mode is the high-precision scanning mode, the main control unit triggers the image input port to receive the left and right speckle images and the texture image, and controls the multi-channel input switch to allow the left and right speckle images and the texture image to be directly transmitted to the multi-channel output switch to be transmitted to the host computer through the output port; when the scanning mode is the fast scanning mode, the main control unit triggers the image input port to receive the left and right speckle images and the texture image, and controls the multi-channel input switch to allow the left and right speckle images and the texture image to be transmitted to the depth calculation engine for processing to obtain a disparity map or a depth map, and then transmits it to the host computer via the multi-channel output switch and the output port.

[0057] In one embodiment, the first switching circuit 51 includes a first signal switching switch 511 and a first data switching switch 512; wherein the first signal switching switch 511 includes a signal input port and two signal output ports, the signal input port is electrically connected to the scanning chip 6, and the two signal output ports are electrically connected to the left camera 3a of the first binocular camera 3 and the left camera 4a of the second binocular camera 4, respectively, so that under the control of the main control unit in the scanning chip 6, a first I2C signal including signals of different levels is input through a single signal input port of the first signal switching switch 511, and the corresponding signal output port is selected for output according to the level of the signal to control the left camera 3a of the first binocular camera 3 or the left camera 4a of the second binocular camera 4 to work separately.

[0058] Specifically, when the level signal output by the main control unit in the scanning chip 6 is at a high level, the first signal switching switch 511 receives the high level signal through the signal input port and sends a strobe signal to the left camera 3a of the first binocular camera 3 through the signal output port to drive it to work; and when the level signal output by the main control unit is at a low level, the first signal switching switch 511 receives the low level signal through the signal input port and sends a strobe signal to the left camera 4a of the second binocular camera 4 through the signal output port to drive it to work.

[0059] The first data switching switch 512 includes two data input ports and one data output port. The two data input ports are electrically connected to the left camera 3a of the first binocular camera 3 and the left camera 4a of the second binocular camera 4, respectively. The data signal output port is electrically connected to the scanning chip 6 to receive the image data collected by the left camera 3a of the first binocular camera 3 or the left camera 4a of the second binocular camera 4 through a single data output port under the control of the main control unit in the scanning chip 6, thereby reducing the number of interfaces of the scanning chip 6.

[0060] Specifically, the data input port corresponding to the first data switching switch 512 is controlled to be turned on according to different level signals. When the level signal output by the main control unit in the scanning chip 6 to the first data switching switch 512 is at a high level, the data input port electrically connected to the left camera 3a of the first binocular camera 3 in the first data switching switch 512 is turned on, and the scanning chip 6 receives the image data collected from the left camera 3a of the first binocular camera 3 through the data signal output port; when the level signal output by the main control unit is at a low level, the data input port electrically connected to the right camera 3b of the first binocular camera 3 in the first data switching switch 512 is turned on, and the scanning chip 6 receives the image data collected from the right camera 3b of the first binocular camera 3 through the data signal output port.

[0061] It should be noted that the second switching circuit 52 includes a second signal switching switch 521 and a second data switching switch 522. The second signal switching switch 521 and the second data switching switch 522 have the same structure and working method as the above-mentioned first signal switching switch 511 and the first data switching switch 512, except that they are connected to the right camera 3b of the first binocular camera 3 and the right camera 4b of the second binocular camera 4, which will not be repeated here.

[0062] In one embodiment, Figure 2As shown, the scanning head also includes four groups of fill light arrays 8 surrounding the periphery of each camera of the first binocular camera 3 and the second binocular camera 4. Each group of fill light arrays 8 includes a plurality of fill light arrays 81 for supplementary lighting, which is beneficial to uniformly brighten the images captured by the first binocular camera 3 and the second binocular camera 4 to improve the measurement accuracy. Further, the fill light arrays surrounding the first binocular camera 3 and the second binocular camera 4 are electrically connected to the scanning chip 6, and the wavelengths of the light beams emitted by them are the same or different. Preferably, when the first binocular camera 3 and the first speckle projection unit 1 form a binocular imaging system, the wavelength of the light beam emitted by the fill light array surrounding the first binocular camera 3 is the same as the wavelength of the light beam emitted by the first speckle projection unit 1; when the second binocular camera 4 and the second speckle projection unit 2 form a binocular imaging system, the wavelength of the light beam emitted by the fill light array surrounding the second binocular camera 4 is the same as the wavelength of the light beam emitted by the second speckle projection unit 2, that is, when the wavelengths of the light beams emitted by the first speckle projection unit 1 and the second speckle projection unit 2 are the same, the wavelengths of the light beams emitted by the fill light arrays surrounding the first binocular camera 3 and the second binocular camera 4 are the same, otherwise they are different.

[0063] In order to realize the above-mentioned high-precision scanning mode and fast scanning mode, the present application provides a detailed system architecture diagram of a scanning chip, such as Figure 3 As shown. Specifically, the scanning chip includes an image input port, a multi-channel input switch, a depth calculation engine and an output interface, wherein the image input port includes a left camera input port, a right camera input port and an RGB camera input port, and the left camera input port, the right camera input port and the RGB camera input port are respectively connected in parallel to the multi-channel input switch, preferably, the left camera input port, the right camera input port and the RGB camera input port include a MIPI interface and / or a DVP interface, and the output port includes a MIPI interface and / or a USB interface. In response to different scanning modes, the multi-channel input switch selectively allows the data of the required input port to be input into the depth calculation engine to achieve a fast scanning mode or selectively allows the data of the required input port to be directly output through the output port to achieve a high-precision scanning mode.

[0064] In one embodiment, the scanning chip further includes a left image processing unit and a right image processing unit arranged between the multi-input switch and the depth calculation engine, wherein: when the multi-input switch allows the left and right speckle images to enter the depth calculation engine, the left camera input port and the right camera input port respectively receive the left and right feature images collected by the left and right cameras, and the multi-input switch allows the left and right feature images to enter the left image processing unit and the right image processing unit respectively for image preprocessing, and the image preprocessing includes image enhancement processing such as image distortion correction and brightness correction; the depth calculation engine performs stereo matching processing on the preprocessed left and right feature images to obtain a disparity map or a depth map and outputs them through an output port, or the depth calculation engine performs light strip extraction on the preprocessed left and right feature images to obtain left and right images of interest and outputs them through the output port; when the multi-input switch does not allow the left and right feature images to enter the depth calculation engine, the left and right camera input ports respectively receive the left and right feature images and output them through the output port allowed by the multi-input switch.

[0065] It should be noted that the left and right feature images include left and right speckle images or left and right multi-line images, that is, when the multi-input switch allows the left and right speckle images or the left and right multi-line images to enter the depth calculation engine, the left and right image processing units perform image enhancement preprocessing on the left and right speckle images or the left and right multi-line images, the depth calculation engine performs stereo matching processing on the preprocessed left and right speckle images to obtain a disparity map or a depth map and outputs it through an output port, or the depth calculation engine performs light strip extraction on the preprocessed left and right multi-line images to obtain left and right images of interest and outputs them through an output port; when the multi-input switch does not allow the left and right speckle images or the left and right multi-line images to enter the depth calculation engine, the left and right camera input ports respectively receive the left and right speckle images or the left and right multi-line images and output them through the output port allowed by the multi-input switch.

[0066] In one embodiment, the scanning chip also includes a multi-channel output switch, the input end of the multi-channel output switch is directly connected to the output end of the multi-channel input switch and is directly or indirectly connected to the output end of the depth calculation engine, the output end of the multi-channel output switch is connected to the output port, and the multi-channel input switch allows all or part of the image data in the left and right feature images, disparity maps or depth maps, left and right images of interest, and texture images to pass to the output port. Further, in order to enable the host computer to obtain a more accurate textured three-dimensional model, the scanning chip also includes an image alignment unit connected to the depth calculation engine and the multi-channel output switch, and the image alignment unit is used to rotate and transform the left and right images of interest or disparity maps or depth maps based on the preset external parameters of the left and right cameras and the RGB camera, so that the left and right images of interest or disparity maps or depth images are aligned with the texture image, so as to facilitate texture mapping and obtain a more accurate textured three-dimensional model.

[0067] It should be noted that when the RGB camera has an integrated texture image processing unit (i.e., ISP, Image Signal Processor), the texture image it collects is a color texture image (such as an RGB image or a YUV image); when the RGB camera does not have an integrated texture image processing unit, the texture image it collects is a grayscale texture image. Figure 3 As shown, the scanning chip also includes a texture image processing unit arranged between the multi-input switch and the multi-output switch, which is used to perform color mapping on the grayscale texture image to obtain a color texture image for transmission to the host computer, so as to better extract the texture information of the scanned object.

[0068] In addition, the scanning chip also includes a storage interface for reading data stored in an external memory, such as reading the preset external parameters of the left and right cameras and the RGB camera stored in the memory of the scanning head for image alignment; when the image input port is a MIPI interface and / or a DVP interface, and the output port is a USB interface, the scanning chip also includes a format conversion protocol unit arranged between the multi-channel output switch and the output interface, which is used to perform protocol conversion on the image data received based on the MIPI interface and / or the DVP interface to obtain data that complies with the USB protocol and can be output through the USB interface, thereby making the three-dimensional scanner universal.

[0069] In one embodiment, the scanning chip further includes a bus and a main control unit electrically connected to the bus, the bus is electrically connected to the transmitting end, the collecting end, the multi-way input switch and the multi-way output switch, and the main control unit responds to different scanning modes and controls the three-dimensional scanner, the multi-way input switch and the multi-way output switch through the bus. It should be noted that the bus can be electrically connected to the transmitting end, the receiving end, the fill light array, the IMU and other devices in the three-dimensional scanner so that the main control unit can control each device, and this application does not limit this.

[0070] Figure 4This is a schematic diagram of the structure of another scanning head provided by the present application, in which a transmitting end of the scanning head includes a multi-line projection unit 110 and a speckle projection unit 120, and a left and a right camera of the receiving end include a left black-and-white camera 130, a right black-and-white camera 140 and an RGB camera 170, wherein the multi-line projection unit 110 is used to project a multi-line patterned light beam to a scanning object, the speckle projection unit 120 is used to project a speckle patterned light beam to the scanning object, and the left black-and-white camera 130 and the right black-and-white camera 140 are used to collect a multi-line patterned light beam or a speckle patterned light beam reflected by the scanning object and generate a corresponding multi-line image or a speckle image to obtain point cloud data or a three-dimensional model of the scanning object. Preferably, the multi-line projection unit 110, the speckle projection unit 120 and the RGB camera 170 are arranged between the left black-and-white camera 130 and the right black-and-white camera 140, so that when the multi-line projection unit 110 or the speckle projection unit 120 projects a patterned light beam, the acquisition field of view of the left black-and-white camera 130 and the right black-and-white camera 140 can be filled, so as to better acquire the point cloud data or the three-dimensional model of the scanned object; and the RGB camera 170 has an overlapping field of view with the left and right cameras, so as to better acquire the detail information of the scanned object in combination with the texture information of the scanned object acquired by the RGB camera, which is beneficial to improving the accuracy of the three-dimensional model of the scanned object.

[0071] In one embodiment, the multi-line projection unit 110 and the speckle projection unit 120 respectively form two sets of binocular imaging systems with the left black-and-white camera 130 and the right black-and-white camera 140. Since the multi-line pattern projected by the multi-line projection unit 110 is more helpful for the left and right black-and-white cameras to obtain higher density measurement data than the speckle pattern projected by the speckle projection unit 120, the measurement accuracy of the binocular imaging system formed by the multi-line projection unit 110 is greater than that of the binocular imaging system formed by the speckle projection unit 120. Specifically, in the high-precision scanning mode, the multi-line projection unit 110 projects a multi-line patterned light beam onto the scanning object or the speckle projection unit 120 projects a speckle patterned light beam onto the scanning object, and the left and right cameras respectively collect the light beams reflected by the scanning object and generate left and right multi-line images or generate left and right speckle images. At this time, the scanning chip 6 does not process the left and right multi-line images or left and right speckle images, but receives the left and right multi-line images or left and right speckle images through the image input port, and outputs them to the host computer through the output port after the multi-way input switch allows. In the fast scanning mode, the speckle projection unit 120 projects a speckle patterned light beam onto the scanning object, and the left and right cameras respectively collect the light beams reflected by the scanning object and generate left and right speckle images. At this time, the scanning chip 6 receives the left and right speckle images through the image input port, and enters the depth calculation engine through the multi-way input switch to process them to obtain a disparity map or a depth map, so as to be transmitted to the host computer through the output port for processing by the modeling software.

[0072] It should be noted that, compared with the speckle pattern beam, the multi-line pattern beam can provide high-density measurement data, so its measurement accuracy is higher. In this embodiment, the transmitting end is set to project light beams of different patterns, so that the binocular imaging system composed of the multi-line projection unit 110 is more suitable for scanning objects with complex textures or scanning scenes with high scanning accuracy requirements, and the binocular imaging system composed of the speckle projection unit 120 is more suitable for scanning scenes with high scanning speed requirements or low scanning accuracy requirements. Further combined with the high-precision scanning mode and the fast scanning mode, the three-dimensional scanner of the present application has different performances, thereby adapting to different scanning scenes.

[0073] Furthermore, the multi-line projection unit 110 includes one or more multi-line projection units, and the speckle projection unit 120 includes one or more speckle projection units. When there are multiple multi-line projection units 110 or speckle projection units 120, each projection unit can be independently controlled to form a multi-line pattern beam or a speckle pattern beam with different densities, so that the binocular imaging system based on the multi-line projection unit 110 or the speckle projection unit 120 has different measurement accuracy and can be adapted to a variety of different application scenarios, thereby greatly improving the integration and universality of the three-dimensional scanner.

[0074] In one embodiment, when there are multiple multi-line projection units, the multiple multi-line projection units project multi-line pattern beams of different shapes onto the scanned object, such as one or more combinations of parallel multi-line pattern beams, cross multi-line pattern beams, and single line pattern beams. Specifically, taking at least three multi-line projection units as an example, the three multi-line projection units are arranged in a triangle, wherein one multi-line projection unit is used to project parallel multi-line pattern beams parallel or perpendicular to the baseline direction of the three-dimensional scanner to complete a rough scan of the scanned object, the baseline refers to the direction of the connection between the binocular cameras in the three-dimensional scanner; the other two multi-line projection units respectively project parallel multi-line pattern beams that are inclined relative to the baseline direction and in opposite directions, so as to form a cross multi-line pattern beam when the two multi-line projection units are turned on at the same time to complete a fine scan of the scanned object.

[0075] It should be noted that when at least four multi-line projection units are included, three of the multi-line projection units, in addition to forming the above-mentioned parallel multi-line pattern beams and crossed multi-line pattern beams, another multi-line projection unit is used to project a single line pattern beam to perform deep hole scanning on a scanning object with holes, and the present application does not impose any restrictions thereon.

[0076] In one embodiment, a single multi-line projection unit 110 includes a laser light source, a collimating element, a modulating element, and a diffractive optical element sequentially disposed in an inner cavity of a shell. The laser light source emits a light beam to the collimating element for light beam collimation. The collimated light beam enters the modulating element to form at least one line light beam. The at least one line light beam is replicated by the diffractive optical element to form a multi-line pattern light beam. A single speckle projection unit 120 includes a laser light source, a collimating element, and a diffractive optical element sequentially disposed in an inner cavity of a shell. The laser light source emits a spot light beam to the collimating element. The collimating element collimates the spot light beam to form a parallel spot light beam that propagates toward the diffractive optical element. The spot light beam is replicated by the diffractive optical element to form a speckle pattern light beam with a large field of view.

[0077] Preferably, the multi-line projection unit 110 and the speckle projection unit 120 are integrated into the transmitting end of the scanning head by assembling the separated modules. The transmitting end of the scanning head includes an optoelectronic support, a multi-line projection unit 110 and a speckle projection unit 120. Before assembling the multi-line projection unit 110 and the speckle projection unit 120 into the optoelectronic support, one or more multi-line projection units 110 or speckle projection units 120 are integrated into a single module form by using one or more devices such as a laser light source, a collimating element, a modulation element and a diffractive optical element; by providing a positioning hole with a foolproof principle on the optoelectronic support, the integrated multi-line projection unit 110 and the speckle projection unit 120 are assembled in a specific position to obtain the transmitting end, for example, the asymmetric shape or feature is used to limit the assembly direction or position of the transmitting end, so that it can only be correctly assembled in a specific position, otherwise it cannot be fixed, and no adjustment is required, and no device alignment is required in each device assembly process, thereby reducing the cost and time loss caused by assembly errors, ensuring product quality, and improving production efficiency and the stability and reliability of the production process.

[0078] In one embodiment, the wavelengths of the light beams emitted by the multi-line projection unit 110 and the speckle projection unit 120 are the same or different. When the wavelengths of the light beams emitted by the two are the same, the left black-and-white camera 130 or the right black-and-white camera 140 includes a narrow-band filter and an image sensor. The narrow-band filter is used to filter the stray light of the remaining wavelength bands except the wavelengths of the light beams emitted by the multi-line projection unit 110 and the speckle projection unit 120, so that the light beam entering the image sensor for imaging is only the light beam emitted by the projection unit, thereby improving the imaging quality. When the wavelengths of the light beams emitted by the two are different, such as when the multi-line projection unit 110 projects a 450 nm wavelength light beam and the speckle projection unit 120 projects an 850 nm / 940 nm wavelength light beam, the left black-and-white camera 130 or the right black-and-white camera 140 includes a double-pass filter and an image sensor. The double-pass filter only allows the 450 nm / 850 nm / 940 nm wavelength light beam to pass through, and filters out the stray light of the remaining wavelength bands, so that the light beam entering the image sensor for imaging is only the light beam emitted by the transmitting end. The present application uses a dual-pass filter to collect light beams of different wavelengths using only two cameras instead of four cameras, thereby reducing the number of cameras, improving integration and reducing costs.

[0079] like Figure 4 As shown, the scanning head further includes a fill light array 150 surrounding the left black and white camera 130 and the right black and white camera 140. When the wavelengths of the multi-line projection unit 110 and the speckle projection unit 120 are different, the fill light array includes a plurality of LED lamps having the same wavelength as the multi-line projection unit 110 and the speckle projection unit 120, respectively. Each LED lamp has a common cathode, and an anode is connected to a power supply or other circuits through a current limiting resistor, and then the LED lamp with the corresponding wavelength is selected to emit light by turning on the anode.

[0080] In one embodiment, given that Figure 2 and Figure 4 The transmitter or fill light of the scanning head shown in the figure requires a large current, but the scanning chip cannot meet the power consumption. Figure 2 and Figure 4 The scanning head shown also includes a light source driving circuit, which includes a signal input port and a current output port. The signal input port is connected to the scanning chip 6, and the current output port is connected to the transmitting end or the fill light. The light source selection signal sent by the scanning chip 6 enters the light source driving circuit through the signal input port. The light source driving circuit outputs current to the transmitting end or the fill light through the current output port according to the received light source selection signal to drive it to emit light. It should be noted that in the present application, each projection unit and the fill light can share a light source driving circuit or each corresponds to a light source driving circuit, which can select the corresponding device to emit light according to the received light source selection signal, but sharing a light source driving circuit is conducive to miniaturization of the scanning head.

[0081] In one embodiment, the scanning head further comprises a button, such as Figure 4 The buttons 170 and the scanning chip 6 are shown to facilitate the user to control the working state of the scanning head through the buttons during the scanning process, such as communicating with the scanning chip through the buttons to adjust the driving current of the transmitting end or control the start or pause of the scanning head. The number of buttons can be set according to functional requirements. Furthermore, the buttons on the scanning head can also be used to interact with the host computer to control the size of the scanning window used by the host computer to display the scanned object model.

[0082] In another embodiment, the scanning head further includes an indicator light, which can indicate the difference between the current working distance and the optimal working distance through different colors according to the actual distance between the three-dimensional scanner and the target. Figure 4 The indicator light 180 shown can display five colors: blue, light blue, green, yellow, and orange, which are respectively used to indicate that the current working distance is too far, far, optimal, close, and too close. The user can move the scanning head according to the color of the indicator light 180 to make the scanning head at the optimal working depth of field, which helps to improve the scanning accuracy.

[0083] After using a three-dimensional scanner to scan the scanned object to obtain a depth image or a texture image, the image data can be uploaded to the host computer by wired or wireless means to perform three-dimensional reconstruction to obtain a three-dimensional model of the scanned object. To this end, the three-dimensional scanning system provided by the present application also includes a mobile handle, which is electrically connected to the three-dimensional scanner and connected to the host computer by wired or wireless communication; the mobile handle includes a battery unit and a compression unit, and the battery unit is used to power the three-dimensional scanner; the compression unit is used to receive one or more of the disparity map or depth map, texture image, left and right feature image or left and right image of interest obtained based on the three-dimensional scanner, and encodes and compresses the image data using a preset image encoding compression method for transmission to the host computer. Further, when the mobile handle is wirelessly connected to the host computer, the mobile handle also includes a wireless transmission module, which transmits the image data collected by the three-dimensional scanner to the host computer by wireless means through WIFI technology or Bluetooth technology, and can also realize the two-way interaction between the reconstruction software in the host computer and the three-dimensional scanner, thereby enabling the user to get rid of the constraints of wired connection and improve the convenience of the three-dimensional scanner.

[0084] The present application configures an additional mobile handle for the three-dimensional scanner, and sets a battery unit and a compression unit in the mobile handle, which not only makes it convenient for users to carry it for multi-directional scanning of the scanning object, but also reduces the transmission bandwidth requirement, making the transmission more stable, thereby improving the image transmission efficiency and achieving a higher frame rate. Moreover, a separate mobile handle can be adapted to different types of three-dimensional scanners, such as a three-dimensional scanner without image compression function or power supply, which can improve the portability and scanning efficiency of the three-dimensional scanner without changing the original structure of the three-dimensional scanner.

[0085] Furthermore, in view of the fact that the existing compression methods cannot achieve both compression time and compression rate, for example, lz4 and quicklz have short compression time but low compression rate; rvl and rle have long compression time and low compression rate; zstd and turbojpeg have high compression rate but long compression time. In order to solve the above problems and ensure the algorithm effect and user experience, the present application provides an image compression method with low computational complexity, which is preset in the compression unit in the mobile handle to solve the above technical problems of low compression rate and long compression time.

[0086] Figure 5 The following is a flowchart of a method for image compression with low computational complexity provided by the present application, wherein the method comprises: S1: receiving input image data, selecting pixel values ​​corresponding to a number of pixels in the image data to form a pixel vector, and calculating a replacement vector of the pixel vector based on the pixel vector and the all-zero vector.

[0087] In one embodiment, calculating a replacement value vector of a pixel vector based on a pixel vector and an all-zero vector includes: taking two adjacent elements in the pixel vector as a group and selecting the maximum value of the elements in each group, and using the maximum value of the elements in each group and the all-zero vector to construct a first intermediate vector; taking adjacent elements in the first intermediate vector as a group and selecting the maximum value of the elements in each group, and using the maximum value of the elements in each group and the all-zero vector to construct a second intermediate vector; taking adjacent pixels in the second intermediate vector as a group and selecting the minimum value of the elements in each group, and using the minimum value of the elements in each group and the all-zero vector to construct a third intermediate vector; selecting the value of the first element in the third intermediate vector to generate a replacement vector with the same replacement value and the same number of elements as the number of pixels.

[0088] Specifically, take the pixel values ​​corresponding to 8 pixels in the image data to form a pixel vector as an example, Figure 6 As shown, the pixel vector is [10,11,9,8,7,15,12,13], and the number of elements of the all-zero vector is aligned with the number of elements of the pixel vector, that is, the all-zero vector is a vector composed of 8 zeros, and two adjacent elements in the pixel vector are grouped as a group, such as [10,11], [9,8], [7,15], and [12,13] are each a group. The maximum value of each group of elements, that is, 11, 9, 15, and 13 are selected to construct the first four bits of the first intermediate vector, and the last four bits are padded with the last four bits of the all-zero vector to keep the number of elements of the first intermediate vector aligned with the pixel vector.

[0089] Further, two adjacent elements in the first intermediate vector are grouped as one, that is, [11,9], [15,13], [0,0], [0,0] are each grouped as one, and the maximum value in each group of elements is selected, then 11, 15, 0 and 0 are selected to construct the first four bits of the second intermediate vector, and the last four bits are padded by the last four bits of the all-zero vector so that the number of elements is aligned with the pixel vector; the adjacent elements in the second intermediate vector are grouped as one, that is, [11,9], [0,0], [0,0], [0,0] are each grouped as one, and the minimum value in each group of elements is selected, then 11, 0, 0 and 0 are selected to construct the first four bits of the third intermediate vector, and the last four bits are padded by the last four bits of the all-zero vector so that the number of elements is aligned with the pixel vector. Thus, the value of the first element in the third intermediate vector is selected to construct a replacement vector with the same replacement value and the same number of elements as the number of pixels, that is, element 11 is selected as the replacement value to construct a replacement vector with 8 elements and a value of 11.

[0090] In one embodiment, calculating a replacement value vector of a pixel vector based on a pixel vector and an all-zero vector includes: circularly shifting the pixel vector left by N elements to obtain a motion vector, comparing the numerical values ​​of each pixel of the pixel vector and the corresponding position element of the motion vector and taking the smaller value element between the two to construct a fourth intermediate vector, where N is a positive integer; taking adjacent elements in the fourth intermediate vector as a group and selecting the minimum value in each group of elements, and using the minimum value in each group of elements and the all-zero vector to construct a fifth intermediate vector; taking adjacent elements in the fifth intermediate vector as a group and selecting the maximum value in each group of elements, and using the maximum value in each group of elements and the all-zero vector to construct an intermediate vector and iterating at least The sixth intermediate vector is obtained at one time; the numerical values ​​of each pixel of the pixel vector and the corresponding position elements of the moving vector are compared and the larger value element of the two is taken to construct the seventh intermediate vector, the adjacent elements in the seventh intermediate vector are grouped as a group and the maximum value in each group of elements is selected, and the eighth intermediate vector is constructed using the maximum value in each group of elements and the all-zero vector; the adjacent elements in the eighth intermediate vector are grouped as a type and the minimum value in each group of elements is selected, and the intermediate vector is constructed using the minimum value in each group of elements and the all-zero vector and iterate at least once to obtain the ninth intermediate vector; the smaller value is selected as the replacement value in the first element of the sixth intermediate vector and the ninth intermediate vector to construct the replacement vector.

[0091] like Figure 7As shown in the left frame, taking the example of circularly shifting the pixel vector left by one element to obtain the moving vector, specifically, the first element 10 of the pixel vector [10,11,9,8,7,15,12,13] is moved to the end of the queue, and the remaining elements are shifted left to obtain the moving vector [11,9,8,7,15,12,13,10], and each element of the pixel vector is compared with the element at the corresponding position of the moving vector to obtain the smaller value of the two to obtain the fourth intermediate vector [10,9,8,7,7,12 ,12,10], the adjacent elements in the fourth intermediate vector are grouped as a group, that is, [10,9], [8,7], [7,12], and [12,10] are each a group, and the minimum value in each group of elements, that is, 9, 7, 7, 10, are selected to construct the fifth intermediate vector [9,7,7,10,0,0,0,0] with the all-zero vector; the adjacent elements in the fifth intermediate vector [9,7,7,10,0,0,0,0] are grouped as a group, [9,7], [7,10], [0,0], and [0, 0] are each a group, and the maximum value of each group is taken to construct the intermediate vector [9,7,0,0,0,0,0,0] and iterate once, that is, based on the vector [9,7,0,0,0,0,0,0], the step of “grouping adjacent elements and taking the maximum value of each group to construct the intermediate vector” is executed to obtain the sixth intermediate vector.

[0092] It should be noted that Figure 7 The right frame diagram is a specific implementation method for obtaining the ninth intermediate vector, which is similar to the method for obtaining the sixth intermediate vector, except that the method for selecting elements is opposite to that for the sixth intermediate vector. This application only takes the example of moving the pixel vector to the left by one position as an example, and N can be other values, which will not be described in detail in this application. In addition, a smaller value is selected from the first elements of the sixth intermediate vector and the ninth intermediate vector as the element of each position of the replacement vector, so that the number of elements of the obtained replacement vector and the pixel vector is consistent.

[0093] The above specific embodiment only takes 8 pixels to construct a pixel vector as an example. In fact, other numbers of pixels can be selected to construct a pixel vector. If the number of selected pixels is an odd number and cannot be evenly grouped, it can be solved by padding zeros at the end of the queue. This application does not impose any restrictions on this.

[0094] S2: Calculate the absolute difference between the elements of the pixel vector and the replacement value vector to obtain an absolute difference vector, compare the numerical value of each element of the absolute difference vector with the element at the corresponding position of the preset threshold vector, and obtain a first result vector according to the comparison result.

[0095] In one embodiment, comparing the numerical value of each element of the absolute difference vector with the element at the corresponding position of the preset threshold vector and obtaining the first result vector according to the comparison result includes: numerically comparing each element in the absolute difference vector with the element at the corresponding position of the preset threshold vector, if an element in the absolute difference vector is less than or equal to the element at the corresponding position of the budget threshold vector, then the element at the same position as the element in the first result vector is recorded as 255; if an element in the absolute difference vector is greater than the element at the corresponding position of the budget threshold vector, then the element at the same position as the element in the first result vector is recorded as 0; traversing each element in the absolute difference vector to obtain its corresponding element in the first result vector, so as to obtain a complete first result vector.

[0096] Specifically, Figure 8 As shown, taking the preset threshold vector elements are all 2 as an example, based on Figure 6 In the illustrated embodiment, the pixel vector is [10, 11, 9, 8, 7, 15, 12, 13], and the replacement vector is [11, 11, 11, 11, 11, 11, 11, 11]. The difference between each element in the pixel vector and the element at the corresponding position on the replacement vector is calculated, and the absolute value is taken to obtain the absolute difference vector. For example, the first element 10 of the pixel vector is compared with the first element 11 of the replacement vector to obtain the first element 1 of the absolute difference vector, and the elements at other positions are deduced in the same way to obtain the complete absolute difference vector [1, 0, 2, 3, 4, 4, 1, 2] corresponding to the pixel vector.

[0097] Then, each element in the absolute difference vector is compared with the pixel at the corresponding position in the preset threshold vector. For example, the first element 1 in the absolute difference vector is compared with the first element 2 in the preset threshold vector. It can be seen that 1<2, that is, the first element in the absolute difference vector is less than or equal to the element at the corresponding position of the preset threshold vector, and the corresponding first element in the first result vector is recorded as 255. For another example, the fourth element 3 in the absolute difference vector is compared with the fourth element 2 in the budget threshold vector. It can be seen that 3>2, that is, the fourth element in the absolute difference vector is greater than the element at the corresponding position of the preset threshold vector. The fourth element corresponding to the first result vector is recorded as 0, and the elements at other positions in the first result vector are deduced in the same way to obtain the complete first result vector [255,255,255,0,0,0,255,255] corresponding to the pixel vector.

[0098] S3: Traverse and count the elements in the first result vector, take the non-zero elements as the number of pixels that meet the threshold condition, traverse each element in turn, and when the traversed element is not zero, continue to traverse until the traversed element is zero or when all the traversed elements are not zero, count the number of pixels that meet the threshold condition in the traversed pixels, and use the number of pixels that meet the threshold condition and the replacement value in the replacement vector to encode and compress the traversed pixels; when the traversed element is zero, start with the pixel corresponding to the traversed element being zero, and continue to traverse until the number of pixels is equal to the pixel vector selected in step S1. The number of pixels having the same value is obtained by executing steps S1-S2 to obtain a second result vector to traverse the second result vector. If the element is still zero based on the second result vector, the element in the result vector is encoded and compressed by using the number of pixels satisfying the threshold condition of zero and the original value of the pixel; if the element is not zero based on the second result vector, the traversal is continued until the element is zero or if all elements based on the second result vector are not zero, the number of pixels satisfying the threshold condition in the traversed pixels is counted, and the traversed pixels are encoded and compressed by using the number of pixels satisfying the threshold condition and the replacement value in the replacement vector.

[0099] by Fig. 9 For example, specifically, Fig. 9 The elements of the result vector [255,255,255,0,0,0,255,255] shown in FIG. 1 are traversed and counted from left to right. When the fourth element is traversed, it is zero. Then, the number of elements whose first three elements are not zero is counted, that is, the number of pixels that meet the threshold condition is 3. The traversed pixels are encoded and compressed using the number of pixels that meet the threshold condition and the replacement value in the replacement vector. That is, the corresponding encoding method is [3]

[11] . When the fourth element is traversed, starting from the pixel corresponding to the fourth element, the same number of pixels as in step S1 are selected to form a new pixel vector. Steps S1-S2 are executed again to obtain a second result vector and the second result vector is traversed. If the first element of the second result vector obtained based on the new pixel vector is still 0 (i.e., the fourth element in the above), the element in the result vector is encoded and compressed by using the number of pixels that meet the threshold condition of zero and the original value of the pixel (the original value of the pixel corresponding to the fourth element is 8), that is, the encoding method of the pixel is [0][8]; if the first element of the second result vector obtained based on the new pixel vector is not 0, traverse the elements in the second result vector from left to right until the traversed element is zero, then count the number of non-zero elements in the second result vector, and use the number of pixels that meet the threshold condition and the replacement value in the replacement vector to encode and compress the traversed pixels.

[0100] For the next element that is zero, starting with this element, the same number of pixels as in step S1 are selected to form a new round of pixel vectors, and steps S1-S3 are executed. Based on the same steps as the fourth pixel mentioned above, the encoding method corresponding to the pixel is obtained. The same process is repeated for the remaining pixels until the encoding of all pixels is completed, thereby achieving compression of the corresponding image data. This application will not go into details here.

[0101] The present application customizes the threshold conditions, and then in the image coding and compression process, adopts the encoding method of "the number of pixels that meet the threshold conditions is zero and the original value of the pixels" for the areas in the image where the difference between adjacent pixels is large, and adopts the encoding method of "the number of pixels that meet the threshold conditions and the replacement value in the replacement vector" for the areas in the image where the difference between adjacent pixels is small. This makes the image coding and compression process flexible, and the coding method can be implemented only through simple addition, subtraction and numerical comparison, with a small amount of calculation, which is conducive to improving the compression rate and reducing time consumption.

[0102] After the image data is encoded and compressed and uploaded to the host computer, the host computer decodes the received image data to obtain the image to be processed. The decoding method is opposite to the encoding method, that is, [3]

[11] , [0][8,7,15] and [2]

[11] are restored to [11,11,11,8,7,5,11,11] for back-end application algorithm processing. The modeling software in the host computer processes the multi-frame disparity map or depth map or left and right feature images or left and right images of interest according to the multi-mode three-dimensional reconstruction method provided by one or more embodiments of the present application to obtain a three-dimensional model of the scanned object, wherein the multi-mode three-dimensional reconstruction method includes different target reconstruction modes, which are divided according to the type of the scanned object. When the scanned object is an object, it includes an object feature mode and an object texture mode; when the scanned object is a face, it includes a face texture mode. The user can select an appropriate target reconstruction mode in the modeling software of the host computer according to the type of the scanned object to achieve accurate three-dimensional reconstruction of the scanned object.

[0103] Furthermore, before realizing the three-dimensional reconstruction of the scanned object by using the target reconstruction mode, it is necessary to perform point cloud conversion processing on the image uploaded to the host computer by the three-dimensional scanner to obtain corresponding point cloud data so as to reconstruct the three-dimensional model of the scanned object based on the point cloud data. Specifically, if the multi-channel input switch in the three-dimensional scanner does not allow the left and right feature images to enter the depth calculation engine, then when the image uploaded to the host computer modeling software is the left and right feature image, when the left and right feature image is the left and right speckle image, the modeling software performs stereo matching processing on the left and right speckle image to obtain a disparity map or a depth map; or, when the left and right feature image is the left and right multi-line image, the modeling software performs light strip extraction on the left and right multi-line image to perform stereo matching to obtain a disparity map or a depth map; convert the disparity map or the depth map into a point cloud and process the point cloud according to the target reconstruction mode to obtain a texture-free three-dimensional model of the scanned object, or process the point cloud and the texture image according to the target reconstruction mode to obtain a textured three-dimensional model of the scanned object; If the multi-input switch in the three-dimensional scanner allows the left and right feature images to enter the depth calculation engine, then when the image data uploaded to the host computer modeling software is a disparity map or a depth map or left and right images of interest, when the three-dimensional scanner outputs a disparity map or a depth map, the modeling software converts the disparity map or the depth map into a point cloud; and / or, when the three-dimensional scanner outputs left and right images of interest, the modeling software stereo matches the left and right images of interest to obtain a disparity map or a depth map to convert into a point cloud; and then processes the point cloud according to the target reconstruction mode to obtain a textureless three-dimensional model of the scanned object, or processes the point cloud and the texture image according to the target reconstruction mode to obtain a textured three-dimensional model of the scanned object; wherein the target reconstruction mode includes one or more combinations of object feature mode, object texture mode and face texture mode.

[0104] Fig. 9 This is a flow chart of a multi-modal 3D reconstruction method applied to modeling software in a host computer according to the present application, including object feature mode, object texture mode and face texture mode, so that users can adaptively select the corresponding mode of 3D reconstruction method according to different types of scanning objects to obtain a more accurate 3D model of the scanned object.

[0105] In one embodiment, the modeling software processes the point cloud according to the object feature pattern to obtain a textureless three-dimensional model of the scanned object. The object feature pattern includes an online scanning method and an offline reconstruction method. The online scanning method includes: obtaining a multi-frame disparity map or depth map corresponding to the current moment and converting it into a current frame point cloud, detecting whether the current frame point cloud meets a preset registration condition for subsequent real-time point cloud registration, and the preset registration condition includes whether the current frame point cloud includes a number of geometric features exceeding a preset threshold, such as identifiable corners, edges or unique shapes, etc., to help establish a reliable correspondence between different frame point clouds; if the current frame point cloud does not meet the preset registration condition, the next frame point cloud is obtained for processing; if the current frame point cloud meets the registration condition, the current frame point cloud is compared with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set through the nearest point iteration method (Iterative Closest Point Algorithm). Point, ICP) is matched and the match is constrained by a preset first cost function; when the constraint value obtained according to the preset first cost function is within the preset range, the current frame point cloud is saved, otherwise it is determined whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set meet the relocation requirements, if so, the current frame point cloud is saved, if not, the current frame point cloud is discarded.

[0106] Specifically, judging whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set satisfy the repositioning includes: matching the current frame point cloud with the remaining frame point cloud data in the historical frame point cloud set and obtaining a constraint value according to a preset first cost function. If the constraint value is within a preset range, it means that the current frame point cloud satisfies the repositioning, otherwise it does not satisfy it.

[0107] After the scanning of the target object is completed, the offline reconstruction mode is entered. The offline reconstruction method includes: obtaining each frame point cloud obtained and saved based on the scanning at different times, screening multiple frame key frame point clouds from each frame point cloud according to a preset time threshold or scanning movement distance, and using the multi-frame key frame point cloud and the historical frame point cloud set to perform global registration to obtain an optimized matching relationship to reduce matching drift, and then splicing and meshing all frame point clouds based on the optimized matching relationship to obtain a texture-free three-dimensional network model.

[0108] In another embodiment, the modeling software processes the point cloud and the texture image according to the object texture mode to obtain a textured three-dimensional model of the scanned object. The object texture mode includes an online drawing method and an offline reconstruction method. The online drawing method includes: obtaining a current frame disparity map or a current frame depth map and a current frame texture image corresponding to the current moment, converting the current frame disparity map or the current frame depth map and determining whether the current frame texture image and the current frame point cloud corresponding to the current moment meet preset alignment conditions. The preset alignment conditions include whether the current frame point cloud and the current frame texture image include a number of geometric features exceeding a preset threshold, such as identifiable corners, edges or unique shapes, etc., to help establish a reliable matching relationship between different frame texture images and different frame point clouds.

[0109] If the current frame texture image or the current frame point cloud does not meet the preset registration conditions, the next frame texture image and point cloud are obtained for processing; if the current frame texture image and the current frame point cloud meet the preset registration conditions, the current frame point cloud is matched with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set through the iterative closest point (ICP) method and different preset cost functions are used to calculate different constraint values ​​for the first matching relationship between point cloud frames and the second matching relationship between texture image frames.

[0110] Specifically, a preset first cost function is used to constrain the first matching relationship between the current frame point cloud and the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set to obtain a first constraint value, and a preset second cost function is used to constrain the second matching relationship between the current frame texture image and the previous frame texture image corresponding to the previous moment in the historical frame texture image set to obtain a second constraint value.

[0111] In one embodiment, the first cost function is expressed by the following formula:

[0112] in, represents the first constraint value, [ , ] respectively represent the rotation matrix and translation matrix in the first matching relationship, represents the i-th point in the point cloud of the current frame, represents the i-th point in the previous frame point cloud in the historical frame point cloud set; when the first constraint value of the above first cost function is the smallest, it means that the first matching relationship between the current frame point cloud and the previous frame point cloud is optimal, and the corresponding [ , ] is the optimal value.

[0113] In another embodiment, the second cost function is expressed by the following formula:

[0114] in, represents the second constraint value, Represents the color value of the i-th pixel in the current frame texture image, represents the color value of the j-th pixel in the previous frame texture image corresponding to the previous moment in the historical frame texture image set; when the second constraint value of the above second cost function is the smallest, it means that the second matching relationship between the current frame texture image and the previous frame texture image is optimal.

[0115] When both the first constraint value and the second constraint value are within the preset range or the sum of the first constraint value and the second constraint value is within the preset range, the current frame point cloud or the current frame texture image is saved; when the first constraint value or the second constraint value is not within the preset range or the sum of the first constraint value and the second constraint value is not within the preset range, the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set are judged for repositioning or the current frame texture image and the remaining frame texture images in the historical frame texture image set are judged whether they meet the repositioning requirements. If the repositioning requirements are met, the current frame point cloud or the current frame texture image is saved; if the repositioning requirements are not met, the current frame point cloud or the current frame texture image is discarded.

[0116] In one embodiment, determining whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set are to be relocated or determining whether the current frame texture image and the remaining frame texture images in the historical frame texture image set satisfy relocation includes: calculating a first constraint value for the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set according to a preset first cost function, and calculating a second constraint value for the current frame texture image and the remaining frame texture images in the historical frame texture image set according to a preset second cost function. If the first constraint value and the second constraint value are within a preset range or the first constraint value and the second constraint value are within a preset range, it indicates that the current frame point cloud and the current frame texture image satisfy relocation, otherwise they do not satisfy relocation.

[0117] After the scanning of the target object is completed, the offline reconstruction mode is entered, and the offline reconstruction mode includes: obtaining each frame point cloud and each frame texture image obtained based on scanning at different times, optimizing the first matching relationship and the second matching relationship based on the global optimization of all frame point clouds and all frame texture images to obtain the third matching relationship and the fourth matching relationship, splicing and meshing all frame point clouds based on the third matching relationship to obtain a texture-free mesh model, and texture mapping the texture-free mesh model through the fourth matching relationship and multiple frames of texture images to obtain a textured mesh model including texture information.

[0118] Furthermore, after obtaining the third matching relationship and the fourth matching relationship, multiple frames of key frame point clouds can be screened from each frame point cloud obtained and saved based on scanning at different times according to a preset time threshold or scanning movement distance, and the third matching relationship can be further optimized using the key frame point cloud to obtain a fifth matching relationship. Based on the fifth matching relationship, all frame point clouds are spliced ​​and meshed to obtain a texture-free mesh model, and the texture-free mesh model is texture mapped through the fourth matching relationship and multiple frames of texture images to obtain a textured mesh model including texture information.

[0119] In another embodiment, the modeling software processes the point cloud and texture image according to the face texture mode to obtain a textured three-dimensional model of the scanned object. The face texture mode includes an online scanning method and an offline reconstruction method. Before entering the face texture mode, it is necessary to detect whether there is a frontal face in the current frame depth image and the current frame texture image. If so, the online scanning method in the face texture mode is started. After the scan is completed, the offline reconstruction method in the face texture mode is started to obtain a three-dimensional model of the frontal face.

[0120] Specifically, the online scanning mode includes: obtaining the current frame depth image and the current frame texture image corresponding to the current moment, and the current frame depth image and the current frame texture image both include a frontal face; converting the current frame depth image into the current frame point cloud and splicing it with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set through the ICP algorithm, and constraining the splicing result using the preset first cost function. When the constraint value obtained according to the preset first cost function is within the preset range, the current frame point cloud is saved, otherwise it is judged that the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set meet the relocation. If the relocation condition is met, that is, the constraint value obtained according to the preset first cost function is within the preset range, the current frame point cloud is saved, otherwise it is eliminated. It should be noted that during the scanning process, the corresponding point cloud splicing result can be displayed on the host computer while scanning. When the point cloud splicing result displayed by the host computer basically includes a complete frontal face, it indicates that the scanning is over, otherwise it continues to scan.

[0121] In one embodiment, the offline reconstruction method includes: obtaining each frame point cloud obtained and saved based on scanning at different times, selecting multiple frames of key frame point clouds from each frame point cloud data according to a preset time threshold or scanning movement distance, using the key frame point cloud to globally optimize the stitching result to obtain a first point cloud set, and performing denoising, resampling and other processing on the optimized point cloud to improve the point cloud quality to obtain a second point cloud set; then meshing and meshing post-processing (such as smoothing, simplification, hole filling, etc.) the second point cloud set to obtain a texture-free mesh model, and then using a texture image including a frontal face to texture map the texture-free mesh model to obtain a textured three-dimensional mesh model.

[0122] Furthermore, in the above-mentioned different reconstruction modes, the target object and the background area can be divided according to the distance information in the depth image, and only the target object is reconstructed, while the background interference is eliminated, which is conducive to improving the reconstruction efficiency. In addition, each reconstruction mode in the three-dimensional reconstruction method of the present application is divided into online scanning and offline reconstruction. During the online scanning process, the user can adjust the scanning angle in real time according to the scanning situation to dynamically update the collected data, and offline reconstruction allows the use of more complex algorithms and more sufficient computing resources to process the collected data, which can achieve higher quality three-dimensional models. The present application realizes three-dimensional model reconstruction by combining online scanning and offline reconstruction, which not only ensures real-time interaction, but also improves the accuracy and details of the model to meet diverse application needs.

[0123] Based on the above three-dimensional scanning system, the present application also provides a three-dimensional scanning method, which is applied to a three-dimensional scanner including a depth calculation engine and a modeling software applied to a host computer, such as Fig.10 As shown, the method includes: Projecting a characteristic pattern beam including a speckle pattern beam and / or a multi-line pattern beam onto a scanned object, and collecting the beam reflected by the scanned object to form left and right characteristic images, the left and right characteristic images including left and right speckle images and / or left and right multi-line images; selectively allowing the left and right characteristic images to enter a depth calculation engine to process the left and right characteristic images in response to a scanning mode, the scanning mode including a high-precision scanning mode and / or a fast scanning mode, wherein: When the left and right feature images are not allowed to enter the depth calculation engine, the scanning mode is a high-precision scanning mode, and the left and right feature images are directly output to the modeling software to use the left and right feature images for three-dimensional reconstruction to obtain a three-dimensional model of the scanned object; When the left and right feature images are allowed to enter the depth calculation engine, the scanning mode is the fast scanning mode, and when the left and right feature images are left and right speckle images, the depth calculation engine performs stereo matching on the left and right speckle images to obtain a disparity map or a depth map and outputs it to the modeling software so as to perform three-dimensional reconstruction using the disparity map or the depth map to obtain a three-dimensional model of the scanned object; or, when the left and right feature images are left and right multi-line images, the depth calculation engine performs light strip extraction on the left and right multi-line images to obtain left and right images of interest and outputs it to the modeling software so as to perform three-dimensional reconstruction using the left and right images of interest to obtain a three-dimensional model of the scanned object.

[0124] In another embodiment, Fig.11As shown, when the left and right feature images include the left and right speckle images and the left and right multi-line images and both allow the left and right feature images to enter the depth calculation engine to realize the fast scanning mode or the high-precision scanning mode; in the fast scanning mode, the left and right speckle images are stereo matched to obtain the disparity map or the depth map and output; in the high-precision scanning mode, the left and right multi-line images are light strip extracted to obtain the left and right images of interest and output, and the modeling software in the host computer uses the disparity map or the depth map or the left and right images of interest to perform three-dimensional reconstruction to obtain the three-dimensional model of the scanned object.

[0125] In order to improve the uploading efficiency of image data collected by the 3D scanner, before the image data in different scanning modes are transmitted to the modeling software of the upper computer, the image data obtained by the 3D scanner is encoded and compressed by a preset image compression method, such as encoding and compressing one or a combination of left and right feature images, depth maps or disparity maps, left and right images of interest, and texture images. The preset image compression method includes: selecting pixel values ​​corresponding to several pixels in the image data to form a pixel vector and calculating a replacement vector of the pixel vector based on the pixel vector and the all-zero vector; calculating the element absolute difference between the pixel vector and the replacement value vector to obtain an absolute difference vector The method comprises the following steps: comparing the numerical values ​​of each element of the absolute difference vector with the elements at the corresponding positions of the preset threshold vector and obtaining the result vector according to the comparison result; performing traversal and statistics on the elements in the result vector, taking the non-zero elements as the number of pixels satisfying the threshold condition, traversing each element in turn, and when the traversed element is zero, counting the number of pixels satisfying the threshold condition in the traversed pixels, and encoding and compressing the traversed pixels by using the number of pixels satisfying the threshold condition and the replacement value in the replacement vector; and when the traversed element is not zero, encoding and compressing the zero part of the element in the result vector by using the number of pixels satisfying the threshold condition and the original value of the pixel.

[0126] In some embodiments, the three-dimensional scanning method further includes collecting the texture image of the scanned object and transmitting it to the modeling software of the host computer, so as to obtain the three-dimensional model of the scanned object in combination with the image data uploaded to the host computer by the three-dimensional scanner and the target reconstruction mode. Specifically, if the left and right feature images are not allowed to enter the depth calculation engine, when the image data uploaded to the host computer are left and right feature images, when the left and right feature images are left and right speckle images, the modeling software performs stereo matching processing on the left and right feature images to obtain a disparity map or a depth map; and / or, when the left and right feature images are left and right multi-line images, light strip extraction and stereo matching are performed on the left and right multi-line images to obtain a disparity map or a depth map; the disparity map or the depth map is converted into a point cloud and the point cloud is processed according to the target reconstruction mode to obtain a texture-free three-dimensional model of the scanned object, or the point cloud and the texture image are processed according to the target reconstruction mode to obtain a textured three-dimensional model of the scanned object.

[0127] If the left and right feature images are allowed to enter the depth calculation engine, when the image data uploaded to the upper computer modeling software is a depth map or a disparity map or a left and right image of interest, when the modeling software receives the disparity map or the depth map, the disparity map or the depth map is converted into a point cloud; or, when the modeling software receives the left and right images of interest, the left and right images of interest are stereo matched to obtain a disparity map or a depth map, and then converted into a point cloud; the point cloud is processed according to the target reconstruction mode to obtain a textureless three-dimensional model of the scanned object, or the point cloud and the texture image are processed according to the target reconstruction mode to obtain a textured three-dimensional model of the scanned object; wherein the target reconstruction mode includes one or more combinations of object feature mode, object texture mode and face texture mode.

[0128] Furthermore, in order to improve the accuracy of the textured three-dimensional model of the scanned object in the fast scanning mode, when the left and right feature images are allowed to enter the depth calculation engine, before the disparity map or depth map or the left and right images of interest and the texture image are transmitted to the modeling software, it also includes using the preset external parameters of the three-dimensional scanner to rotate the disparity map or depth map or the left and right images of interest so that the disparity map or depth map or the left and right images of interest are aligned with the texture image, and then the texture image is better used to accurately texture the textureless three-dimensional model to ensure the accuracy of the textured three-dimensional model of the scanned object. It should be noted that in the high-precision scanning mode or the fast scanning mode, if the texture image needs to be uploaded to the host computer for processing, the texture images in different scanning modes can also be encoded and compressed by the above-mentioned preset image compression method to improve the upload efficiency of the texture image.

[0129] In one embodiment, when the target reconstruction mode is the object feature mode, the point cloud is processed according to the object feature mode to obtain a textureless three-dimensional model of the scanned object, and the object feature mode includes online scanning and offline reconstruction, wherein the online scanning includes: obtaining the current frame point cloud corresponding to the current moment, detecting whether the current frame point cloud satisfies a preset registration condition, the preset registration condition includes whether the current frame point cloud includes a number of geometric features exceeding a preset threshold; if the current frame point cloud does not satisfy the preset registration condition, obtaining the next frame point cloud for processing; if the current frame point cloud satisfies the registration condition, matching the current frame point cloud with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set and constraining the matching using a preset first cost function; when the constraint value obtained according to the preset first cost function is within a preset range, the current frame point cloud is saved, otherwise it is determined whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set meet relocation, if the relocation is met, the current frame point cloud is saved, if the relocation is not met, the current frame point cloud is discarded.

[0130] Offline reconstruction includes: obtaining each frame point cloud obtained and saved based on scanning at different times, screening multi-frame key frame point clouds from each frame point cloud according to a preset time threshold or scanning movement distance, using the multi-frame key frame point cloud and the historical frame point cloud set to perform global registration to obtain an optimized matching relationship, and based on the optimized matching relationship, splicing and meshing all frame point clouds to obtain a texture-free three-dimensional network model.

[0131] When the target reconstruction mode is the object texture mode, the point cloud and the texture image are processed according to the object texture mode to obtain a textured three-dimensional model of the scanned object. The object texture mode includes online scanning and offline reconstruction, wherein the online scanning includes: obtaining the current frame point cloud and the current frame texture image corresponding to the current moment, and determining whether the geometric features of the current frame texture image and the current frame point cloud meet the preset registration conditions, and the preset registration conditions include whether the current frame point cloud and the current frame texture image include geometric features exceeding a preset threshold number.

[0132] If the current frame texture image or the current frame point cloud does not meet the preset registration conditions, the next frame texture image and point cloud are obtained for processing; if the current frame texture image and the current frame point cloud meet the preset registration conditions, the current frame point cloud is matched with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set and the current frame texture image is matched with the previous frame texture image corresponding to the previous moment in the historical frame texture image set, and different preset cost functions are used to perform different constraint value calculations on the first matching relationship between point cloud frames and the second matching relationship between texture image frames; when the different constraint values ​​are all within the preset range or the sum of the different constraint values ​​is within the preset range, the current frame point cloud or the current frame texture image is saved; otherwise, it is determined whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set or the current frame texture image and the remaining frame texture images in the historical frame texture image set meet repositioning, if repositioning is met, the current frame point cloud or the current frame texture image is saved, if repositioning is not met, the current frame point cloud or the current frame texture image is discarded.

[0133] Offline reconstruction includes: obtaining each frame of point cloud and each frame of texture image obtained and saved based on scanning at different times, optimizing the first matching relationship and the second matching relationship based on the global optimization of each frame of point cloud and each frame of texture image to obtain a third matching relationship and a fourth matching relationship, splicing and meshing each frame of point cloud based on the third matching relationship to obtain a texture-free mesh model, and texture mapping the texture-free mesh model through the fourth matching relationship and each frame of texture image to obtain a textured mesh model including texture information.

[0134] When the target reconstruction mode is the face texture mode, the point cloud and the texture image are processed according to the face texture mode to obtain a texture three-dimensional model of the scanned object. The face texture mode includes online scanning and offline reconstruction, wherein the online scanning includes: obtaining the current frame depth image and the current frame texture image corresponding to the current moment, the current frame depth image and the current frame texture image both include a frontal face; converting the current frame depth image into a current frame point cloud and splicing it with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set, and constraining the splicing result using a preset first cost function. When the constraint value obtained according to the preset first cost function is within a preset range, the current frame point cloud is saved, otherwise it is determined whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set meet the repositioning requirements. If the repositioning requirements are met, the current frame point cloud is saved, and if the repositioning requirements are not met, the current frame point cloud is discarded.

[0135] Offline reconstruction includes: obtaining each frame point cloud obtained and saved based on scanning at different times, screening multiple frames of key frame point clouds from each frame point cloud according to a preset time threshold or scanning movement distance, and using the multiple frames of key frame point clouds to globally optimize the stitching results to obtain a point cloud set; meshing the point cloud set to obtain a texture-free mesh model, and using a texture image including a frontal face to texture map the texture-free mesh model to obtain a textured three-dimensional mesh model of the frontal face.

[0136] The present application also provides a computer-readable storage medium for storing a computer program. When the computer program is executed, the present application is realized. Fig.10 or Fig.11 The three-dimensional scanning method in the embodiment.

[0137] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0138] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the method described above can refer to the corresponding process in the aforementioned system, device, and unit embodiments, and will not be repeated here.

[0139] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0140] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0142] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A scanning chip used in a three-dimensional scanner, characterized in that: Used to process the left and right characteristic images collected by the three-dimensional scanner to achieve a fast scanning mode and / or a high-precision scanning mode, the scanning chip includes an image input port, a multi-channel input switch, a depth calculation engine and an output port, and the left and right characteristic images include left and right speckle images and / or left and right multi-line images: The image input port is used to receive the left and right feature images into the multi-channel input switch; The multi-input switch selectively allows the left and right feature images to enter the depth calculation engine to achieve a fast scanning mode and / or a high-precision scanning mode; If the multi-input switch allows the left and right feature images to enter the depth calculation engine to implement the fast scanning mode, when the left and right feature images include left and right speckle images, the depth calculation engine is used to perform stereo matching on the left and right speckle images to obtain a disparity map or a depth map and output it through the output port; or, when the left and right feature images include left and right multi-line images, the depth calculation engine is used to perform light strip extraction on the left and right multi-line images to obtain left and right images of interest and output them through the output port; or, If the multi-input switch does not allow the left and right feature images to enter the depth calculation engine to achieve a high-precision scanning mode, the multi-input switch allows the left and right feature images to be output through the output port.

2. The scanning chip according to claim 1, characterized in that: include: When the left and right characteristic images include left and right speckle images and left and right multi-line images, the multi-input switch allows both the left and right characteristic images to enter the depth calculation engine to achieve a fast scanning mode or a high-precision scanning mode; In the fast scanning mode, the image input port is used to receive the left and right speckle images into the multi-way input switch, and the multi-way input switch allows the left and right speckle images to enter the depth calculation engine to perform stereo matching on the left and right speckle images to obtain a disparity map or a depth map and output it through the output port; In the high-precision scanning mode, the image input port is used to receive the left and right multi-line images into the multi-way input switch, and the multi-way input switch allows the left and right multi-line images to enter the depth calculation engine to perform light strip extraction on the left and right multi-line images to obtain left and right images of interest and output them through the output port.

3. The scanning chip according to claim 1 or 2, characterized in that: When the three-dimensional scanner includes a left camera, a right camera and an RGB camera to collect the left and right feature images and the texture image, the image input port includes a left feature input port, a right camera input port and an RGB camera input port, wherein the left camera input port, the right camera input port and the RGB camera input port are respectively connected to the multi-way input switch in parallel, and the multi-way input switch selectively allows the image data of the corresponding input port to be input into the multi-way output switch or the depth calculation engine in response to different scanning modes.

4. The scanning chip according to claim 3, characterized in that: The scanning chip further includes a left image processing unit and a right image processing unit disposed between the multi-input switch and the depth calculation engine, wherein: When the multi-input switch allows the left and right feature images to enter the depth calculation engine, the left camera input port and the right camera input port receive the left and right feature images captured by the left and right cameras respectively, and the multi-input switch allows the left and right feature images to enter the left image processing unit and the right image processing unit respectively for image preprocessing; the depth calculation engine performs stereo matching processing on the preprocessed left and right feature images to obtain a disparity map or a depth map, or the depth calculation engine performs light strip extraction on the preprocessed left and right feature images to obtain left and right images of interest and output them through the output port; When the multi-input switch does not allow the left and right feature images to enter the depth calculation engine, the left and right camera input ports receive the left and right feature images respectively and are allowed to be output through the output port by the multi-input switch.

5. The scanning chip according to claim 4, characterized in that: The scanning chip also includes a multi-way output switch, the input ends of the multi-way output switch are directly connected to the output ends of the multi-way input switch and are directly or indirectly connected to the output ends of the depth calculation engine, the output ends of the multi-way output switch are connected to the output port, and the multi-way input switch allows all or part of the image data in the left and right feature images, disparity maps or depth maps, left and right images of interest, and texture images to pass through to the output port.

6. The scanning chip according to claim 5, characterized in that: The scanning chip also includes an image alignment unit connected between the depth calculation engine and the multi-channel output switch, and the image alignment unit is used to rotate the left and right images of interest or the disparity map or the depth map based on preset external parameters of the left and right cameras and the RGB camera to align the left and right images of interest or the disparity map or the depth image with the texture image.

7. The scanning chip according to any one of claims 1 to 6, characterized in that: The scanning chip also includes a bus and a main control unit electrically connected to the bus, the bus is electrically connected to the three-dimensional scanner, the multi-input switch and the multi-output switch, and the main control unit responds to different scanning modes and controls the three-dimensional scanner, the multi-input switch and the multi-output switch through the bus.

8. A three-dimensional scanner, characterized in that: Used to scan the scanned object in response to the scanning mode to obtain multiple frames of image data of the scanned object, the scanning mode includes a high-precision scanning mode and / or a fast scanning mode; wherein the three-dimensional scanner includes a transmitting end, a receiving end and a scanning chip: The transmitting end comprises a projection unit for projecting a speckle pattern beam and / or a multi-line pattern beam onto the scanning object; The receiving end comprises a left camera and a right camera, wherein the left camera and the right camera are used to collect light beams reflected by the scanned object to form left and right characteristic images, wherein the left and right characteristic images comprise left and right speckle images and / or left and right multi-line images; The scanning chip includes an image input port, a multi-way input switch, a depth calculation engine and an output port, wherein the image input port is used to receive the left and right feature images and enter the multi-way input switch; the multi-way input switch selectively allows the left and right feature images to enter the depth calculation engine to achieve a fast scanning mode and / or a high-precision scanning mode; wherein: If the multi-input switch allows the left and right feature images to enter the depth calculation engine to implement the fast scanning mode, when the left and right feature images include left and right speckle images, the depth calculation engine is used to perform stereo matching on the left and right speckle images to obtain a disparity map or a depth map and output it through the output port; or, when the left and right feature images include left and right multi-line images, the depth calculation engine is used to perform light strip extraction on the left and right multi-line images to obtain left and right images of interest and output them through the output port; or, If the multi-input switch does not allow the left and right feature images to enter the depth calculation engine to achieve a high-precision scanning mode, the multi-input switch allows the left and right feature images to be output through the output port.

9. The three-dimensional scanner according to claim 8, characterized in that: include: When the left and right feature images include left and right speckle images and left and right multi-line images, the multi-input switch allows the left and right feature images to enter the depth calculation engine to achieve a fast scanning mode or a high-precision scanning mode; in the fast scanning mode, the image input port is used to receive the left and right speckle images to enter the multi-input switch, and the multi-input switch allows the left and right speckle images to enter the depth calculation engine to perform stereo matching on the left and right speckle images to obtain a disparity map or a depth map and output it through the output port; In the high-precision scanning mode, the image input port is used to receive the left and right multi-line images into the multi-way input switch, and the multi-way input switch allows the left and right multi-line images to enter the depth calculation engine to perform light strip extraction on the left and right multi-line images to obtain left and right images of interest and output them through the output port.

10. The three-dimensional scanner according to claim 9, characterized in that: The receiving end also includes an RGB camera, which is used to collect a texture image of the scanned object. In the high-precision scanning mode and / or the fast scanning mode, the scanning chip receives the texture image through the image input port and outputs the texture image through the multi-input switch and the output port.

11. The three-dimensional scanner according to claim 10, wherein: The image input port includes a left camera input port, a right camera input port, and an RGB camera input port, wherein the left camera input port, the right camera input port, and the RGB camera input port are respectively connected in parallel to the multi-way input switch, and the multi-way input switch selectively allows the image data of the corresponding input port to be input into the multi-way output switch or the depth calculation engine in response to different scanning modes.

12. The three-dimensional scanner according to claim 11, wherein: The scanning chip further includes a left image processing unit and a right image processing unit disposed between the multi-input switch and the depth calculation engine, wherein: When the multi-input switch allows the left and right feature images to enter the depth calculation engine, the left camera input port and the right camera input port receive the left and right feature images captured by the left and right cameras respectively, and the multi-input switch allows the left and right feature images to enter the left image processing unit and the right image processing unit respectively for image preprocessing; the depth calculation engine performs stereo matching processing on the preprocessed left and right feature images to obtain a disparity map or a depth map, or the depth calculation engine performs light strip extraction on the preprocessed left and right feature images to obtain left and right images of interest and output them through the output port; When the multi-input switch does not allow the left and right feature images to enter the depth calculation engine, the left and right camera input ports receive the left and right feature images respectively and are allowed to be output through the output port by the multi-input switch.

13. The three-dimensional scanner according to claim 11, wherein: The scanning chip also includes a multi-way output switch, the input ends of the multi-way output switch are directly connected to the output ends of the multi-way input switch and are directly or indirectly connected to the output ends of the depth calculation engine, the output ends of the multi-way output switch are connected to the output port, and the multi-way input switch allows all or part of the image data in the left and right speckle images, left and right multi-line images, disparity maps or depth maps, left and right images of interest, and texture images to pass to the output port.

14. The three-dimensional scanner according to claim 13, wherein: The scanning chip also includes an image alignment unit connected to the depth calculation engine and the multi-channel output switch, and the image alignment unit is used to rotate the disparity map or the depth map based on preset external parameters of the left and right cameras and the RGB camera to align the disparity map or the depth image with the texture image.

15. The three-dimensional scanner according to any one of claims 9 to 14, characterized in that: The scanning chip also includes a bus and a main control unit electrically connected to the bus, the bus is electrically connected to the transmitting end, the collecting end, the multi-way input switch and the multi-way output switch, and the main control unit responds to different scanning modes and controls the transmitting end, the collecting end, the multi-way input switch and the multi-way output switch through the bus.

16. The three-dimensional scanner according to any one of claims 9 to 14, characterized in that: When the projection unit in the transmitting end is used to project a speckle pattern beam, the projection unit at least includes a first speckle projection unit and a second speckle projection unit, and the receiving end at least includes a first binocular camera and a second binocular camera respectively composed of two groups of left and right cameras, wherein: The first binocular camera is embedded between the left camera and the right camera of the second binocular camera so that the two groups of binocular cameras have different baselines, the first speckle projection unit and the second speckle projection unit are arranged between the first binocular camera, and the speckle density of the speckle pattern beam projected by the first speckle projection unit is greater than that of the speckle pattern beam projected by the second speckle projection unit, and the projection field of view of the speckle pattern beam projected by the first speckle projection unit is smaller than that of the projection field of view of the speckle pattern beam projected by the second speckle projection unit; The first speckle projection unit and the first binocular camera constitute a short-range imaging system suitable for small-sized scanning objects; the second speckle projection unit and the second binocular camera constitute a long-range imaging system suitable for medium-to-large-sized scanning objects, so that the corresponding imaging system is selected for scanning according to the size and distance of the scanning object in different scanning modes.

17. The three-dimensional scanner according to claim 16, wherein: The first speckle projection unit comprises an LED light source, a collimating lens, a MASK mask and a projection lens. The LED light source arranged on the focal plane of the collimating lens emits a divergent light beam to the collimating lens, and after being focused by the collimating lens, a focused light beam is projected to the MASK mask arranged on the focal plane of the projection lens. The focused light beam passes through the circular array on the MASK mask to form a high-density speckle pattern beam, and forms a speckle pattern beam with high density and a large field of view through the magnification effect of the projection lens.

18. The three-dimensional scanner according to claim 17, wherein: The depth of field of the projection lens is variable. The projection lens comprises a fixed-focus lens and a micro-zoom mechanism. The fixed-focus lens is slightly moved by rotating the micro-zoom mechanism, thereby changing the image distance of the fixed-focus lens to change the working depth of field of the projection lens. The first speckle projection unit or the second speckle projection unit can project speckle patterns at different distances and the speckle patterns do not change at different distances.

19. The three-dimensional scanner according to claim 16, wherein: The three-dimensional scanner further includes a first switching circuit and a second switching circuit, the scanning chip is electrically connected to the left camera of the first binocular camera and the left camera of the second binocular camera through the first switching circuit, and is electrically connected to the right camera of the first binocular camera and the right camera of the second binocular camera through the second switching circuit; the first switching circuit or the second switching circuit includes a signal switching switch and a data switching switch, and the scanning chip includes a main control unit, wherein: The signal switching switch includes a signal input port and two signal output ports, the signal input port is electrically connected to the scanning chip, and the two signal output ports are electrically connected to the left camera or the right camera of the first binocular camera and the second binocular camera, respectively, so as to control the left camera or the right camera of the first binocular camera and the second binocular camera to work separately through the single signal input port of the signal switching switch under the control of the main control unit in the scanning chip; The data switching switch includes two data input ports and one data output port, the two data input ports are electrically connected to the left camera or the right camera of the first binocular camera and the second binocular camera, respectively, and the data output port is electrically connected to the left camera input port and the right camera input port in the scanning chip, so as to receive the image data collected by the left camera or the right camera of the first binocular camera and the second binocular camera through a single data output port under the control of the main control unit in the scanning chip.

20. The three-dimensional scanner according to any one of claims 11 to 17, characterized in that: When the projection unit in the transmitting end is used to project a speckle pattern beam and a multi-line pattern beam, the projection unit includes a speckle projection unit and a multi-line projection unit, the left and right cameras in the receiving end include a left and right black-and-white camera, the multi-line projection unit, the speckle projection unit and the RGB camera are arranged between the left and right black-and-white cameras, and when the multi-line projection unit or the speckle projection unit projects the patterned beam, the collection field of view of the left and right black-and-white cameras can be filled.

21. The three-dimensional scanner according to claim 20, wherein: The multi-line projection unit and the speckle projection unit are integrated into the projection module by assembling separate modules. The projection module comprises an optoelectronic support, a multi-line projection unit and a speckle projection unit. One or more multi-line projection units or speckle projection units are respectively integrated into a single module form by using one or more of a laser light source, a collimating element, a modulation element and a diffractive optical element. Positioning holes with a foolproof principle are provided on the optoelectronic support so that the integrated multi-line projection unit and the speckle projection unit are assembled in a specific position to obtain the projection module.

22. The three-dimensional scanner according to claim 20, wherein: When the multi-line projection unit includes multiple units, the multi-line projection unit is used to project a line pattern beam of at least one or more combinations of parallel multi-line pattern beams parallel or perpendicular to the baseline of the three-dimensional scanner, crossed multi-line pattern beams inclined relative to the baseline and in opposite directions, or a single line pattern beam to the scanned object.

23. A three-dimensional scanning system, characterized in that: The invention comprises a three-dimensional scanner as claimed in any one of claims 7 to 20 and modeling software applied to a host computer, wherein: The three-dimensional scanner is used to scan the scanned object in response to different scanning modes to obtain left and right characteristic images of the scanned object or obtain left and right characteristic images and texture images of the scanned object, wherein the different scanning modes include a high-precision scanning mode and / or a fast scanning mode, and the left and right characteristic images include left and right speckle images and / or left and right multi-line images; When the multi-input switch in the three-dimensional scanner allows the left and right feature images to enter the depth calculation engine to achieve a high-precision scanning mode or a fast scanning mode, the modeling software performs three-dimensional reconstruction based on the left and right feature images or the left and right images of interest obtained by the three-dimensional scanner to obtain a texture-free three-dimensional model of the scanned object; or receives the left and right feature images or the left and right images of interest and the texture images obtained based on the three-dimensional scanner and performs three-dimensional reconstruction using the left and right feature images and the texture images to obtain a textured three-dimensional model of the scanned object; When the multi-input switch in the three-dimensional scanner does not allow the left and right feature images to enter the depth calculation engine to achieve a high-precision scanning mode, the modeling software performs three-dimensional reconstruction based on the disparity map or depth map or left and right images of interest obtained by the three-dimensional scanner to obtain a texture-free three-dimensional model of the scanned object; or, performs three-dimensional reconstruction based on the disparity map or depth map or left and right images of interest and texture images obtained by the three-dimensional scanner to obtain a textured three-dimensional model of the scanned object.

24. The three-dimensional scanning system according to claim 23, characterized in that: The three-dimensional scanning system further includes a mobile handle, which is electrically connected to the three-dimensional scanner and connected to the host computer by wired or wireless communication; wherein the mobile handle includes a battery unit and a compression unit: The battery unit is used to supply power to the three-dimensional scanner; The compression unit is used to receive the disparity map or depth map or left and right images of interest and / or the texture image obtained based on the three-dimensional scanner in the fast scanning mode, and encode and compress the image data using a preset image encoding and compression method for transmission to the host computer; receive the left and right feature images and / or the texture image obtained based on the three-dimensional scanner in the high-precision scanning mode, and encode and compress the image data using a preset image encoding and compression method for processing by the modeling software.

25. The three-dimensional scanning system according to claim 24, characterized in that: The preset image compression method comprises: receiving input image data, selecting pixel values ​​corresponding to a number of pixels in the image data to form a pixel vector, and calculating a replacement vector of the pixel vector based on the pixel vector and an all-zero vector; Calculate the absolute difference between the elements of the pixel vector and the replacement value vector to obtain an absolute difference vector, compare the numerical value of each element of the absolute difference vector with the element at the corresponding position of the preset threshold vector, and obtain a first result vector according to the comparison result; Performing traversal and statistics on the elements in the first result vector, taking the non-zero elements as the number of pixels satisfying the threshold condition, traversing each element in turn, and when the traversed element is not zero, continuing to traverse until the traversed element is zero or when all the traversed elements are not zero, counting the number of pixels satisfying the threshold condition in the traversed pixels, and encoding and compressing the traversed pixels using the number of pixels satisfying the threshold condition and the replacement value in the replacement vector; When the traversed element is zero, starting from the pixel corresponding to the traversed element being zero, continue to traverse until the number of pixels is the same as the number of pixels in the pixel vector to obtain a second result vector and traverse the second result vector. If the element is still zero based on the second result vector, the element in the result vector is encoded and compressed using the number of pixels that meet the threshold condition of zero and the original value of the pixel; if the element is not zero based on the second result vector, continue to traverse until the element is zero or if all elements based on the second result vector are not zero, count the number of pixels that meet the threshold condition in the traversed pixels, and use the number of pixels that meet the threshold condition and the replacement value in the replacement vector to encode and compress the traversed pixels.

26. The three-dimensional scanning system according to claim 23, characterized in that: If the multi-input switch in the three-dimensional scanner does not allow the left and right feature images to enter the depth calculation engine, when the left and right feature images are the left and right speckle images, the modeling software performs stereo matching processing on the left and right speckle images to obtain a disparity map or a depth map; and / or, when the left and right feature images are the left and right multi-line images, the modeling software performs light strip extraction on the left and right multi-line images to perform stereo matching to obtain a disparity map or a depth map; converts the disparity map or the depth map into a point cloud and processes the point cloud according to a target reconstruction mode to obtain a texture-free three-dimensional model of the scanned object, or processes the point cloud and the texture image according to a target reconstruction mode to obtain a textured three-dimensional model of the scanned object; If the multi-input switch in the three-dimensional scanner allows the left and right feature images to enter the depth calculation engine, when the three-dimensional scanner outputs the disparity map or depth map, the modeling software converts the disparity map or depth map into a point cloud; and / or, when the three-dimensional scanner outputs the left and right images of interest, the modeling software performs stereo matching on the left and right images of interest to obtain a disparity map or depth map to convert into a point cloud; and then processes the point cloud according to a target reconstruction mode to obtain a textureless three-dimensional model of the scanned object, or processes the point cloud and the texture image according to a target reconstruction mode to obtain a textured three-dimensional model of the scanned object; wherein the target reconstruction mode includes one or more combinations of an object feature mode, an object texture mode, and a face texture mode.

27. The three-dimensional scanning system according to claim 26, characterized in that: The modeling software processes the point cloud according to the object feature pattern to obtain a texture-free three-dimensional model of the scanned object, wherein the object feature pattern includes online scanning and offline reconstruction, wherein: The online scanning includes: obtaining a current frame point cloud corresponding to the current moment, and detecting whether the current frame point cloud satisfies a preset registration condition, wherein the preset registration condition includes whether the current frame point cloud includes a number of geometric features exceeding a preset threshold; If the current frame point cloud does not meet the preset registration condition, the next frame point cloud is obtained for processing; if the current frame point cloud meets the registration condition, the current frame point cloud is matched with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set and the matching is constrained by using the preset first cost function; When the constraint value obtained according to the preset first cost function is within the preset range, the current frame point cloud is saved. Otherwise, it is determined whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set meet the requirements of relocation. If the requirements are met, the current frame point cloud is saved. If the requirements are not met, the current frame point cloud is discarded. The offline reconstruction includes: obtaining each frame point cloud obtained and saved based on scanning at different times, screening multiple frame key frame point clouds from the each frame point cloud according to a preset time threshold or scanning movement distance, using the multiple frame key frame point clouds and a historical frame point cloud set to perform global registration to obtain an optimized matching relationship, and based on the optimized matching relationship, splicing and meshing all frame point clouds to obtain a texture-free three-dimensional network model.

28. The three-dimensional scanning system according to claim 26, characterized in that: The modeling software processes the point cloud and the texture image according to the object texture mode to obtain a textured three-dimensional model of the scanned object, wherein the object texture mode includes online scanning and offline reconstruction, wherein: The online scanning includes: obtaining a current frame point cloud and a current frame texture image corresponding to the current moment, and determining whether the geometric features of the current frame texture image and the current frame point cloud meet a preset registration condition, wherein the preset registration condition includes whether the current frame point cloud and the current frame texture image include a number of geometric features exceeding a preset threshold; If the current frame texture image or the current frame point cloud does not meet the preset registration condition, the next frame texture image and point cloud are obtained for processing; if the current frame texture image and the current frame point cloud meet the preset registration condition, the current frame point cloud is matched with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set and the current frame texture image is matched with the previous frame texture image corresponding to the previous moment in the historical frame texture image set, and different preset cost functions are used to calculate different constraint values ​​for the first matching relationship between point cloud frames and the second matching relationship between texture image frames; When the different constraint values ​​are all within the preset range or the sum of the different constraint values ​​is within the preset range, the current frame point cloud or the current frame texture image is saved; otherwise, it is determined whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set or the current frame texture image and the remaining frame texture images in the historical frame texture image set meet the requirements of repositioning. If the requirements of repositioning are met, the current frame point cloud or the current frame texture image is saved; if the requirements of repositioning are not met, the current frame point cloud or the current frame texture image is discarded; The offline reconstruction includes: obtaining each frame point cloud and each frame texture image obtained and saved based on scanning at different times, optimizing the first matching relationship and the second matching relationship based on global optimization of the each frame point cloud and the each frame texture image to obtain a third matching relationship and a fourth matching relationship, splicing and meshing the each frame point cloud based on the third matching relationship to obtain a texture-free mesh model, and texture mapping the texture-free mesh model through the fourth matching relationship and the each frame texture image to obtain a textured mesh model including texture information.

29. The three-dimensional scanning system according to claim 26, characterized in that: The modeling software processes the point cloud and the texture image according to the face texture mode to obtain a texture three-dimensional model of the scanned object, wherein the face texture mode includes online scanning and offline reconstruction, wherein: The online scanning includes: obtaining a current frame depth image and a current frame texture image corresponding to the current moment, wherein the current frame depth image and the current frame texture image both include a frontal face; converting the current frame depth image into a current frame point cloud and splicing it with a previous frame point cloud corresponding to a previous moment in a historical frame point cloud set, and constraining the splicing result using a preset first cost function. When the constraint value obtained according to the preset first cost function is within a preset range, the current frame point cloud is saved. Otherwise, it is determined whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set meet repositioning. If so, the current frame point cloud is saved. If not, the current frame point cloud is discarded. The offline reconstruction includes: obtaining each frame point cloud obtained and saved based on scanning at different times, screening multiple frames of key frame point clouds from each frame point cloud according to a preset time threshold or scanning movement distance, and using the multiple frames of key frame point clouds to globally optimize the stitching results to obtain a point cloud set; meshing the point cloud set to obtain a texture-free mesh model, and using a texture image including a frontal face to texture map the texture-free mesh model to obtain a textured three-dimensional mesh model of the frontal face.

30. A three-dimensional scanning method, characterized in that: Applied to a three-dimensional scanner including a depth calculation engine and to modeling software applied to a host computer, the method comprises: Projecting a characteristic pattern beam including a speckle pattern beam and / or a multi-line pattern beam onto a scanned object, and collecting the beam reflected back by the scanned object to form left and right characteristic images, wherein the left and right characteristic images include left and right speckle images and / or left and right multi-line images; selectively allowing the left and right feature images to enter the depth calculation engine to process the left and right feature images in response to a scanning mode, wherein the scanning mode includes a high-precision scanning mode and / or a fast scanning mode, wherein: When the left and right feature images are not allowed to enter the depth calculation engine, the scanning mode is the high-precision scanning mode, and the left and right feature images are directly output to the modeling software to perform three-dimensional reconstruction using the left and right feature images to obtain a three-dimensional model of the scanned object; When the left and right feature images are allowed to enter the depth calculation engine, the scanning mode is the fast scanning mode, and when the left and right feature images are the left and right speckle images, the depth calculation engine performs stereo matching on the left and right speckle images to obtain a disparity map or a depth map, and outputs the map to the modeling software so as to perform three-dimensional reconstruction using the disparity map or the depth map to obtain a three-dimensional model of the scanned object; or, when the left and right feature images are the left and right multi-line images, the depth calculation engine performs light strip extraction on the left and right multi-line images to obtain left and right images of interest, and outputs the images to the modeling software so as to perform three-dimensional reconstruction using the left and right images of interest to obtain a three-dimensional model of the scanned object.

31. The three-dimensional scanning method according to claim 30, characterized in that: When the left and right feature images include left and right speckle images and left and right multi-line images, and both allow the left and right feature images to enter the depth calculation engine to realize a fast scanning mode or a high-precision scanning mode; in the fast scanning mode, stereo matching is performed on the left and right speckle images to obtain a disparity map or a depth map and output it; in the high-precision scanning mode, light stripes are extracted from the left and right multi-line images to obtain left and right images of interest and output them.

32. The three-dimensional scanning method according to claim 30, characterized in that: The three-dimensional scanning method further includes: collecting the texture image of the scanned object and outputting it to the modeling software; if the left and right feature images are not allowed to enter the depth calculation engine, when the left and right feature images are left and right speckle images, the modeling software performs stereo matching processing on the left and right feature images to obtain a disparity map or a depth map; and / or, when the left and right feature images are left and right multi-line images, extracting light stripes and performing stereo matching on the left and right multi-line images to obtain a disparity map or a depth map; converting the disparity map or the depth map into a point cloud and processing the point cloud according to a target reconstruction mode to obtain a texture-free three-dimensional model of the scanned object, or processing the point cloud and the texture image according to a target reconstruction mode to obtain a textured three-dimensional model of the scanned object; and / or, If the left and right feature images are allowed to enter the depth calculation engine, when the modeling software receives the disparity map or depth map, the disparity map or depth map is converted into a point cloud; or, when the modeling software receives the left and right images of interest, the left and right images of interest are stereo matched to obtain a disparity map or depth map, and then converted into a point cloud; the point cloud is processed according to a target reconstruction mode to obtain a textureless three-dimensional model of the scanned object, or the point cloud and the texture image are processed according to a target reconstruction mode to obtain a textured three-dimensional model of the scanned object; wherein the target reconstruction mode includes one or more combinations of an object feature mode, an object texture mode and a face texture mode.

33. The three-dimensional scanning method according to claim 32, characterized in that: The point cloud is processed according to the object feature pattern to obtain a texture-free three-dimensional model of the scanned object, wherein the object feature pattern includes online scanning and offline reconstruction, wherein: The online scanning includes: obtaining a current frame point cloud corresponding to the current moment, and detecting whether the current frame point cloud satisfies a preset registration condition, wherein the preset registration condition includes whether the current frame point cloud includes a number of geometric features exceeding a preset threshold; If the current frame point cloud does not meet the preset registration condition, the next frame point cloud is obtained for processing; if the current frame point cloud meets the registration condition, the current frame point cloud is matched with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set and the matching is constrained by using the preset first cost function; When the constraint value obtained according to the preset first cost function is within the preset range, the current frame point cloud is saved. Otherwise, it is determined whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set meet the requirements of relocation. If the requirements are met, the current frame point cloud is saved. If the requirements are not met, the current frame point cloud is discarded. The offline reconstruction includes: obtaining each frame point cloud obtained and saved based on scanning at different times, screening multiple frame key frame point clouds from the each frame point cloud according to a preset time threshold or scanning movement distance, using the multiple frame key frame point clouds and a historical frame point cloud set to perform global registration to obtain an optimized matching relationship, and based on the optimized matching relationship, splicing and meshing all frame point clouds to obtain a texture-free three-dimensional network model.

34. The three-dimensional scanning method according to claim 32, characterized in that: The point cloud and the texture image are processed according to the object texture mode to obtain a texture three-dimensional model of the scanned object, wherein the object texture mode includes online scanning and offline reconstruction, wherein: The online scanning includes: obtaining a current frame point cloud and a current frame texture image corresponding to the current moment, and determining whether the geometric features of the current frame texture image and the current frame point cloud meet a preset registration condition, wherein the preset registration condition includes whether the current frame point cloud and the current frame texture image include a number of geometric features exceeding a preset threshold; If the current frame texture image or the current frame point cloud does not meet the preset registration condition, the next frame texture image and point cloud are obtained for processing; if the current frame texture image and the current frame point cloud meet the preset registration condition, the current frame point cloud is matched with the previous frame point cloud corresponding to the previous moment in the historical frame point cloud set and the current frame texture image is matched with the previous frame texture image corresponding to the previous moment in the historical frame texture image set, and different preset cost functions are used to calculate different constraint values ​​for the first matching relationship between point cloud frames and the second matching relationship between texture image frames; When the different constraint values ​​are all within the preset range or the sum of the different constraint values ​​is within the preset range, the current frame point cloud or the current frame texture image is saved; otherwise, it is determined whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set or the current frame texture image and the remaining frame texture images in the historical frame texture image set meet the requirements of repositioning. If the requirements of repositioning are met, the current frame point cloud or the current frame texture image is saved; if the requirements of repositioning are not met, the current frame point cloud or the current frame texture image is discarded; The offline reconstruction includes: obtaining each frame point cloud and each frame texture image obtained and saved based on scanning at different times, optimizing the first matching relationship and the second matching relationship based on global optimization of the each frame point cloud and the each frame texture image to obtain a third matching relationship and a fourth matching relationship, splicing and meshing the each frame point cloud based on the third matching relationship to obtain a texture-free mesh model, and texture mapping the texture-free mesh model through the fourth matching relationship and the each frame texture image to obtain a textured mesh model including texture information.

35. The three-dimensional scanning method according to claim 32, characterized in that: The point cloud and the texture image are processed according to the face texture pattern to obtain a texture three-dimensional model of the scanned object, wherein the face texture pattern includes online scanning and offline reconstruction, wherein: The online scanning includes: obtaining a current frame depth image and a current frame texture image corresponding to the current moment, wherein the current frame depth image and the current frame texture image both include a frontal face; converting the current frame depth image into a current frame point cloud and splicing it with a previous frame point cloud corresponding to a previous moment in a historical frame point cloud set, and constraining the splicing result using a preset first cost function. When the constraint value obtained according to the preset first cost function is within a preset range, the current frame point cloud is saved. Otherwise, it is determined whether the current frame point cloud and the remaining frame point cloud data in the historical frame point cloud set meet the requirements of relocation. If the requirements are met, the current frame point cloud is saved. If the requirements are not met, the current frame point cloud is discarded. The offline reconstruction includes: obtaining each frame point cloud obtained and saved based on scanning at different times, screening multiple frames of key frame point clouds from each frame point cloud according to a preset time threshold or scanning movement distance, and using the multiple frames of key frame point clouds to globally optimize the stitching results to obtain a point cloud set; meshing the point cloud set to obtain a texture-free mesh model, and using a texture image including a frontal face to texture map the texture-free mesh model to obtain a textured three-dimensional mesh model of the frontal face.

36. The three-dimensional scanning method according to any one of claims 32 to 35, characterized in that: When the left and right feature images are allowed to enter the depth calculation engine, before the disparity map or depth map or the left and right images of interest and the texture image are output to the modeling software, the disparity map or depth map or the left and right images of interest are rotated and transformed using the preset external parameters of the three-dimensional scanner to align the disparity map or depth map or the left and right images of interest with the texture image.

37. The three-dimensional scanning method according to any one of claims 31 to 35, characterized in that: Before transmitting the image data in different scanning modes to the modeling software of the host computer, the image data obtained by the three-dimensional scanner is encoded and compressed by a preset image compression method, and the preset image compression method includes: Selecting pixel values ​​corresponding to a number of pixels in the image data to form a pixel vector and calculating a replacement vector of the pixel vector based on the pixel vector and the all-zero vector; Calculate the absolute difference between the elements of the pixel vector and the replacement value vector to obtain an absolute difference vector, compare the numerical value of each element of the absolute difference vector with the element at the corresponding position of the preset threshold vector, and obtain a result vector according to the comparison result; The elements in the result vector are traversed and counted, and the non-zero elements are regarded as the number of pixels that meet the threshold condition. Each element is traversed in turn. When the traversed element is zero, the number of pixels that meet the threshold condition in the traversed pixels is counted, and the traversed pixels are encoded and compressed using the number of pixels that meet the threshold condition and the replacement value in the replacement vector; when the traversed element is not zero, the part of the zero element in the result vector is encoded and compressed using the number of pixels that meet the threshold condition and the original value of the pixel.

38. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the three-dimensional scanning method based on speckle projection as described in any one of claims 31 to 37 is implemented.

Images