Three-dimensional scanning method and apparatus, computer device, and storage medium

Abstract

A three-dimensional scanning method and apparatus, a computer device, and a storage medium. The method comprises: on the basis of a first scanning mode, scanning an object to be scanned, to obtain first point cloud data; obtaining a switching instruction; scanning said object on the basis of the switching instruction and a second scanning mode, to obtain second point cloud data and third point cloud data; and performing three-dimensional reconstruction on the basis of the first point cloud data, the second point cloud data and the third point cloud data to obtain a three-dimensional model.

Description

Three-dimensional scanning method, device, computer equipment and storage medium

[0001] Related applications

[0002] This application claims priority to Chinese patent application number 202410060599.1, filed on January 16, 2024, entitled “Three-dimensional scanning method, device, computer equipment and storage medium,” the entire text of which is incorporated herein by reference. Technical Field

[0003] The present application relates to the field of three-dimensional scanning technology, and in particular to a three-dimensional scanning method, apparatus, computer equipment, and storage medium. Background Art

[0004] 3D scanners are primarily used to detect and analyze the shape (geometry) and appearance (such as color, surface albedo, and other properties) of objects or environments in the real world. 3D scanners scan objects to generate point cloud data corresponding to the object's surface. These point cloud data are then reconstructed 3D to create a digital model of the actual object in the virtual world. The denser the point cloud data collected during scanning, the more accurate the reconstructed 3D data model. The resulting 3D data models are widely used in various fields.

[0005] In current 3D scanner technology, when using laser scanning, markers must first be placed on the object to be scanned. After the markers are placed, the object is laser scanned by the 3D scanner. After obtaining each frame of point cloud data, the data must be spliced ​​based on the marker information and then reconstructed to create a 3D model. However, placing markers on the object requires manual labor, which consumes significant labor and material costs. Summary of the Invention

[0006] According to various embodiments of the present application, a three-dimensional scanning method, apparatus, computer device, and storage medium are provided.

[0007] In a first aspect, the present application provides a three-dimensional scanning method, the method comprising: scanning an object to be scanned based on a first scanning mode to obtain first point cloud data; the first point cloud data includes point cloud data reconstructed based on speckle; obtaining a switching instruction; according to the switching instruction, scanning the object to be scanned based on a second scanning mode to obtain second point cloud data and third point cloud data; the second point cloud data includes point cloud data reconstructed based on speckle, and the third point cloud data includes point cloud data reconstructed based on linear stripe features; performing three-dimensional reconstruction based on the first point cloud data, the second point cloud data, and the third point cloud data to obtain a three-dimensional model.

[0008] In one embodiment, according to the switching instruction, the object to be scanned is scanned based on the second scanning mode to obtain second point cloud data and third point cloud data, and then includes: obtaining a switching instruction; according to the switching instruction, the object to be scanned is scanned based on the first scanning mode to obtain first point cloud data.

[0009] In one embodiment, obtaining the switching instruction includes: obtaining the switching instruction input by a user.

[0010] In one embodiment, obtaining the switching instruction includes: obtaining distance information between the scanner and the object to be scanned; generating the switching instruction based on the distance information; if the distance information is greater than or equal to a preset threshold, scanning the object to be scanned based on a first scanning mode; if the distance information is less than the preset threshold, scanning the object to be scanned based on a second scanning mode.

[0011] In one embodiment, scanning the object to be scanned based on the first scanning mode to obtain first point cloud data includes: acquiring two frames of grayscale images of the object to be scanned in real time; reconstructing based on the speckle in the grayscale images according to the two frames of the grayscale images to obtain first point cloud data; calculating a first transformation relationship between the first point cloud data and the world coordinate system based on a point cloud registration algorithm; adding the first point cloud data reconstructed from each frame to a first scattered speckle cloud set, and adding the first transformation relationship corresponding to each frame of the first point cloud data to a transformation relationship set.

[0012] In one embodiment, scanning the object to be scanned based on the second scanning mode to obtain second point cloud data and third point cloud data includes: acquiring two frames of grayscale images of the object to be scanned in real time; reconstructing based on the speckle in the grayscale images according to the two frames of grayscale images to obtain second point cloud data; reconstructing based on the linear stripe features in the grayscale images to obtain third point cloud data; calculating the second transformation relationship between the second point cloud data and the third point cloud data and the world coordinate system based on a point cloud registration algorithm; adding the second point cloud data reconstructed from each frame to the second scattered speckle point cloud set, adding the third point cloud data reconstructed from each frame to the first linear stripe feature point cloud set, and adding the second transformation relationship corresponding to the second point cloud data and the third point cloud data from each frame to the transformation relationship set.

[0013] In one embodiment, before acquiring two frames of grayscale images of the object to be scanned in real time, the method includes: projecting speckles onto the surface of the object to be scanned by a speckle projector; projecting linear stripe features onto the surface of the object to be scanned by a linear stripe feature projector; and both the speckle projector and the linear stripe feature projector are infrared light projectors.

[0014] In one embodiment, the three-dimensional reconstruction based on the first point cloud data, the second point cloud data and the third point cloud data to obtain a three-dimensional model includes: point cloud stitching based on the first scattered spot cloud set, the second scattered spot cloud set and the transformation relationship set, and the corresponding first point cloud data, the second point cloud data, the first transformation relationship and the second transformation relationship; three-dimensional reconstruction based on the first scattered spot cloud set, the second scattered spot cloud set, the first linear stripe feature point cloud set and the transformation relationship set, and the corresponding first point cloud data, the second point cloud data, the third point cloud data, the first transformation relationship and the second transformation relationship to obtain a three-dimensional model.

[0015] In one embodiment, before point cloud stitching is performed based on the first scattered speckle cloud set, the second scattered speckle cloud set and the transformation relationship set, the corresponding first point cloud data, the second point cloud data, the first transformation relationship and the second transformation relationship, it also includes: performing global alignment based on the first scattered speckle cloud set and the second scattered speckle cloud set corresponding to the first scattered speckle cloud set and the second scattered speckle cloud set, and updating the first transformation relationship and the second transformation relationship corresponding to the transformation relationship set.

[0016] In one embodiment, the three-dimensional reconstruction is performed on the first point cloud data, the second point cloud data, the third point cloud data, the first transformation relationship and the second transformation relationship corresponding to the first scattered spot cloud set, the second scattered spot cloud set, the first linear stripe feature point cloud set and the transformation relationship set to obtain a three-dimensional model, including: obtaining the scattered spot cloud weight and the linear stripe feature point cloud weight; the scattered spot cloud weight is the weight of the second point cloud data in the second scattered spot cloud set; the linear stripe feature point cloud weight is the weight of the third point cloud data in the first linear stripe feature point cloud set; three-dimensional reconstruction is performed on the first scattered spot cloud set, the second scattered spot cloud set, the first linear stripe feature point cloud set, the transformation relationship set, the scattered spot cloud weight and the linear stripe feature point cloud weight to obtain a three-dimensional model.

[0017] In one embodiment, the scanning of the object to be scanned based on the first scanning mode to obtain the first point cloud data also includes: reconstructing based on the linear stripe features in the two frames of the grayscale images to obtain the fourth point cloud data; calculating the first transformation relationship between the fourth point cloud data and the world coordinate system based on the point cloud registration algorithm; adding the fourth point cloud data reconstructed from each frame to the second linear stripe feature point cloud set; the three-dimensional reconstruction of the first point cloud data, the second point cloud data, the third point cloud data, the first transformation relationship and the second transformation relationship corresponding to the first scattered spot cloud set, the second scattered spot cloud set, the first linear stripe feature point cloud set and the transformation relationship set to obtain the three-dimensional model also includes: obtaining the first scattered spot cloud weight, the second scattered spot cloud weight, the first The linear stripe feature point cloud weight and the second linear stripe feature point cloud weight; the first scattered spot cloud weight is the weight of the first point cloud data in the first scattered spot cloud set, the second scattered spot cloud weight is the weight of the second point cloud data in the second scattered spot cloud set, the first linear stripe feature point cloud weight is the weight of the fourth point cloud data in the second linear stripe feature point cloud set, and the second linear stripe feature point cloud weight is the weight of the third point cloud data in the first linear stripe feature point cloud set; three-dimensional reconstruction is performed based on the first scattered spot cloud set, the second scattered spot cloud set, the first linear stripe feature point cloud set, the second linear stripe feature point cloud set, the transformation relationship set, the first scattered spot cloud weight, the second scattered spot cloud weight, the first linear stripe feature point cloud weight and the second linear stripe feature point cloud weight to obtain a three-dimensional model.

[0018] In a second aspect, the present application also provides a three-dimensional scanning device, comprising: a scanning module for scanning an object to be scanned based on a first scanning mode to obtain first point cloud data; the first point cloud data includes point cloud data reconstructed based on speckle; an instruction acquisition module for obtaining a switching instruction; the scanning module is also used to scan the object to be scanned based on a second scanning mode according to the switching instruction to obtain second point cloud data and third point cloud data; the second point cloud data includes point cloud data reconstructed based on speckle, and the third point cloud data includes point cloud data reconstructed based on linear stripe features; a reconstruction module is used to perform three-dimensional reconstruction based on the first point cloud data, the second point cloud data, and the third point cloud data to obtain a three-dimensional model.

[0019] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements any of the three-dimensional scanning methods described in the first aspect when executing the computer program.

[0020] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the three-dimensional scanning methods described in the first aspect.

[0021] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the disclosed drawings without any creative work.

[0023] FIG1 is a schematic flow chart of a three-dimensional scanning method in one embodiment.

[0024] FIG2 is a flow chart of a method for generating a switching instruction in one embodiment.

[0025] FIG3 is a flow chart of a method for generating a switching instruction in one embodiment.

[0026] FIG4 is a flow chart of a method for generating a switching instruction in one embodiment.

[0027] FIG5 is a schematic diagram of a flow chart of a first scanning mode in one embodiment.

[0028] FIG6 is a schematic diagram of a flow chart of a second scanning mode in one embodiment.

[0029] FIG7 is a structural block diagram of a three-dimensional scanning device in one embodiment.

[0030] FIG8 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0032] The present application provides a 3D scanner, which can be a handheld 3D scanner or a drone 3D scanner. This embodiment is not specifically limited to this embodiment; it only needs to be able to perform 3D scanning of the object to be scanned. Taking a handheld 3D scanner as an example, the 3D scanner includes at least a binocular camera and a color camera. The color camera can be integrated into the 3D scanner or a standalone camera, but this application does not specifically limit this. When the 3D scanner scans the object to be scanned, the binocular camera captures a grayscale image of the object to be scanned and reconstructs a 3D model based on the grayscale image. While the binocular camera is capturing images, the color camera simultaneously captures a color image of the object to be scanned. Based on the registration relationship between the color image and the grayscale image, texture mapping is performed on the reconstructed 3D model to obtain the final 3D model. When scanning the object to be scanned, the 3D scanner can project light onto the surface of the object to be scanned using a speckle projector and a linear stripe feature projector. Light of either the same frequency or different frequencies can be projected, but this application does not specifically limit this. The speckle projector can be a vertical-cavity surface-emitting laser (VCSEL), which projects an irregular speckle pattern onto the surface of the object to be scanned. The linear stripe projector projects linear stripe features onto the surface of the object to be scanned, such as a laser line projected by a laser, a linear stripe pattern projected by a projector or a projection lamp, or linear stripe features projected by other devices, which are not specifically limited in this application. In this case, the collected grayscale image will contain speckle or linear stripe features. When performing three-dimensional reconstruction, three-dimensional reconstruction can be performed based on the speckle or linear stripe features to obtain a three-dimensional model.

[0033] It should be noted that the process of reconstructing a three-dimensional point cloud using linear stripe features is the same, so the subsequent description will mainly use the laser line projected by the laser as an example.

[0034] In current 3D scanners, if a laser projector projects a laser line onto the surface of an object, markers are required to be placed on the surface. This laser-based 3D reconstruction yields a highly detailed 3D model. However, if a speckle projector projects a speckle pattern onto the surface, markers are not required, but the reconstructed 3D model may suffer from poor detail.

[0035] In one embodiment, as shown in FIG1 , a three-dimensional scanning method is provided, comprising the following steps:

[0036] Step 102 : Scan the object to be scanned based on a first scanning mode to obtain first point cloud data.

[0037] When using a 3D scanner to scan an object, the object can be scanned using a first scanning mode. After receiving a switching instruction, the scanning mode can be switched and the object can be scanned using a second scanning mode. Alternatively, the object can be scanned using a second scanning mode. After receiving a switching instruction, the scanning mode can be switched and the object can be scanned using the first scanning mode. This embodiment of the present application does not specifically limit this. The following description uses the example of starting a scan using the first scanning mode.

[0038] The object to be scanned is scanned using a first scanning mode to obtain first point cloud data. The object to be scanned can be any structure for which a three-dimensional model is to be constructed. The first point cloud data includes point cloud data reconstructed based on speckle patterns. That is, when scanning using the first scanning mode, reconstruction is performed based solely on the acquired speckle patterns to obtain first point cloud data reconstructed based on speckle patterns. When using the first scanning mode, only a speckle projector can be used to project a speckle image onto the surface of the object to be scanned, without using a linear fringe feature projector. In this case, the data collected by the 3D scanner consists solely of speckle patterns, and the first point cloud data is reconstructed based on the speckle patterns. Alternatively, neither the speckle projector nor the linear fringe feature projector is used, and a speckle image is formed on the surface of the object to be scanned using ambient light. In this case, the data collected by the 3D scanner consists solely of speckle patterns, and the first point cloud data is reconstructed based on the speckle patterns. Alternatively, both the speckle projector and the linear fringe feature projector are used. In this case, the surface of the object to be scanned has both a speckle image and linear fringe features. After the 3D scanner collects the data, reconstruction is performed based solely on the speckle patterns to obtain the first point cloud data. The first scanning mode is used to scan a large area of ​​the object to be scanned. Among them, the speckle projector and the linear stripe feature projector are both infrared light projectors. Infrared light is invisible light. The projection based on infrared light is invisible to the naked eye, thereby avoiding damage to the eyes.

[0039] Step 104: Obtain a switching instruction.

[0040] During the scanning process of the 3D scanner, a switching instruction is obtained in real time. The switching instruction is used to switch the operating mode of the 3D scanner. Specifically, when the 3D scanner is operating in the first scanning mode, upon receiving the switching instruction, the 3D scanner switches to the second scanning mode. When the 3D scanner is operating in the second scanning mode, upon receiving the switching instruction, the 3D scanner switches to the first scanning mode.

[0041] The switching instructions can be input by the user or generated based on data collected in real time by the 3D scanner during the scanning process. For example, during the scanning process, the 3D scanner can collect real-time statistics on the scanning time and generate the switching instructions based on the time information; can obtain real-time information about the 3D scanner's angle and generate the switching instructions based on the 3D scanner's tilt angle; and can obtain real-time information about the distance between the 3D scanner and the object to be scanned and generate the switching instructions based on the distance information.

[0042] Step 106 : Scan the object to be scanned based on the second scanning mode according to the switching instruction to obtain second point cloud data and third point cloud data.

[0043] After switching the scanning mode according to the switching instruction, the object to be scanned is scanned using the second scanning mode. The second point cloud data includes point cloud data reconstructed based on speckle patterns, and the third point cloud data includes point cloud data reconstructed based on linear fringe features. That is, when scanning using the second scanning mode, it is necessary to simultaneously reconstruct the second point cloud data based on speckle patterns and the third point cloud data based on linear fringe features. When scanning using the second scanning mode, a speckle projector and a linear fringe feature projector can be used simultaneously to project a speckle image and linear fringe features (i.e., laser-generated linear fringe features) onto the surface of the object to be scanned. In this case, the data collected by the 3D scanner contains both speckle patterns and linear fringe features. The second point cloud data is reconstructed based on the speckle patterns, and the third point cloud data is reconstructed based on the linear fringe features. The linear fringe feature projector can be used to project linear fringe features onto the surface of the object to be scanned. In this case, natural light forms a speckle image on the surface of the object to be scanned. The data collected by the 3D scanner contains both speckle and linear fringe features. The second point cloud data is reconstructed based on the speckle pattern, and the third point cloud data is reconstructed based on the linear fringe features. The second scanning mode performs a fine scan of the local details of the object to be scanned.

[0044] Step 108 : Perform three-dimensional reconstruction based on the first point cloud data, the second point cloud data, and the third point cloud data to obtain a three-dimensional model.

[0045] After the 3D scan is completed, 3D reconstruction is performed based on the first, second, and third point cloud data to obtain a 3D model. Both the first and second point cloud data are point cloud data reconstructed based on speckle patterns, and the first and second point cloud data include scanning information for all areas of the object to be scanned. Therefore, the 3D model is obtained by splicing the first and second point cloud data, and then by 3D reconstruction based on the first, second, and third point cloud data. It is understood that the first point cloud data is point cloud data reconstructed based on speckle patterns when in the first scanning mode; the second point cloud data is point cloud data reconstructed based on speckle patterns when in the second scanning mode; and the third point cloud data is point cloud data reconstructed based on linear fringe features when in the second scanning mode. When performing three-dimensional reconstruction based on the first point cloud data, the second point cloud data, and the third point cloud data, splicing is performed based on the speckle features in the first point cloud data and the second point cloud data. Since the second point cloud data and the third point cloud data are point cloud data acquired synchronously, the third point cloud data and the first point cloud data are also spliced ​​together. Finally, three-dimensional reconstruction is performed based on the first point cloud data and the third point cloud data to obtain a three-dimensional model.

[0046] In one embodiment, the first scanning mode and the second scanning mode can be the same scanning mode, that is, both the first scanning mode and the second scanning mode utilize both a speckle projector and a linear fringe feature projector. In this case, the first scanning mode and the second scanning mode differ only in their data processing methods. In the first scanning mode, at least the first point cloud data must be reconstructed based on speckle. In the second scanning mode, both the second point cloud data and the third point cloud data must be reconstructed based on speckle. The first scanning mode and the second scanning mode can also be different scanning modes. That is, in the first scanning mode, only the speckle projector or ambient light is used to form a speckle image on the surface of the scanned object. In the second scanning mode, both the speckle projector and the linear fringe feature projector are used. In this case, the data processing methods of the first scanning mode and the second scanning mode are also different. In the first scanning mode, only the first point cloud data must be reconstructed based on speckle. In the second scanning mode, both the second point cloud data and the third point cloud data must be reconstructed based on speckle.

[0047] In an embodiment of the present application, the object to be scanned is first scanned based on a first scanning mode to obtain first point cloud data. During the scanning process, a switching instruction is obtained in real time, and according to the switching instruction, the object to be scanned is scanned based on a second scanning mode to obtain second point cloud data and third point cloud data. The first point cloud data includes point cloud data reconstructed based on speckle; the second point cloud data includes point cloud data reconstructed based on speckle; and the third point cloud data includes point cloud data reconstructed based on linear stripe features. Finally, a three-dimensional reconstruction is performed based on the first point cloud data, the second point cloud data, and the third point cloud data to obtain a three-dimensional model. By generating point cloud data reconstructed based on speckle in both scanning modes and performing three-dimensional stitching using speckle feature stitching, the need to attach marker points to the object to be scanned when using laser scanning is avoided. This saves manpower and material costs and further improves the efficiency and accuracy of three-dimensional scanning.

[0048] In one embodiment, a switching instruction is obtained; and according to the switching instruction, the object to be scanned is scanned based on the first scanning mode to obtain first point cloud data. If the first scanning mode is used at the beginning of the scan, upon receiving the switching instruction, the 3D scanner switches to the second scanning mode, and the switching instruction is obtained in real time during the scanning process. Upon receiving the switching instruction again, the 3D scanner switches back to the first scanning mode. If the second scanning mode is used at the beginning of the scan, upon receiving the switching instruction, the 3D scanner switches to the first scanning mode, and the switching instruction is obtained during the scanning process. Upon receiving the switching instruction again, the 3D scanner switches back to the second scanning mode. That is, during the entire 3D scanning process, the first scanning mode and the second scanning mode can be switched at any time, as long as the 3D scanner receives the switching instruction.

[0049] In an embodiment of the present application, a switching instruction is received in real time while the 3D scanner is scanning an object. Upon receiving the switching instruction, the 3D scanner switches between a first scanning mode and a second scanning mode. In other words, during the scanning process, the scanning mode of the 3D scanner is switched in real time based on the structural state of the object to be scanned, enabling a more detailed scan of the object to be scanned.

[0050] In one embodiment, a switching instruction input by the user can be obtained based on the user's input. A mode switching button can be set on the three-dimensional scanner. During the scanning process of the object to be scanned, the user triggers the button to generate a switching instruction. A touch screen can also be set on the three-dimensional scanner. During the scanning process of the object to be scanned, the user generates a switching instruction by clicking a virtual button for switching the scanning mode on the touch screen. The three-dimensional scanner can also be configured with an external device. The external device is connected to the three-dimensional scanner via a wired or wireless method. The external device can be: a touchpad, a remote control, a keyboard, a touch screen, etc. The user uses the external device to input the switching instruction through the external device during the scanning process.

[0051] In the embodiment of the present application, by inputting a switching instruction by the user, it is possible to combine the user's scanning experience to specifically determine that different scanning modes are used for different structures of the object to be scanned, thereby making the three-dimensional scanning more accurate.

[0052] In one embodiment, as shown in FIG2 , a method for generating a switching instruction is provided, comprising the following steps:

[0053] Step 202: Acquire the distance information between the scanner and the object to be scanned.

[0054] A distance sensor can be provided on the 3D scanner. During the process of scanning the object to be scanned by the 3D scanner, the distance information between the 3D scanner and the object to be scanned can be detected in real time by the distance sensor. Alternatively, during the process of scanning the object to be scanned by the 3D scanner, a grayscale image captured by the camera can be obtained, and the distance information between the 3D scanner and the object to be scanned can be calculated based on the grayscale image. The distance information can be the distance information obtained at a single moment, or the distance information can be obtained at each moment in a preset time period. The average value of the distance information can then be calculated based on the distance information at each moment, and the average value can be used as the final distance information. That is, if the scanner has a binocular camera, the distance information corresponding to each frame can be calculated based on multiple frames of grayscale images captured by the binocular camera. The distance information of multiple frames can be continuously calculated, and the average value based on the distance information of the multiple frames can be calculated as the final distance information.

[0055] Step 204: Generate a handover instruction based on the distance information.

[0056] If the distance information is greater than or equal to a preset threshold, the object to be scanned is scanned using the first scanning mode; if the distance information is less than the preset threshold, the object to be scanned is scanned using the second scanning mode. When generating a switching instruction based on the distance information between the 3D scanner and the object to be scanned, the first scanning mode is used to scan a large area of ​​the object to be scanned, so a larger distance is required to perform such a large-scale scan. The second scanning mode is used to scan fine details of the object to be scanned, so a smaller distance is required to scan the details of the object to be scanned. Therefore, the switching instruction can be generated by setting a preset threshold. The preset threshold is a distance threshold, which can be set by the user based on the actual structure of the object to be scanned and scanning requirements, and is not specifically limited in this embodiment. After the preset threshold is set, the distance information is compared with the preset threshold to generate a switching instruction. If the distance information is greater than or equal to the preset threshold, the generated switching instruction controls the 3D scanner to switch to the first scanning mode; if the distance information is less than the preset threshold, the generated switching instruction controls the 3D scanner to switch to the second scanning mode.

[0057] It is understandable that when a preset threshold is used as the basis for mode switching, frequent switching of scanning modes is likely to occur when the 3D scanner fluctuates around the preset threshold. Therefore, a variable value can be added to the preset threshold. When the distance information is greater than the sum of the preset threshold and the variable value, a switching instruction is generated to control the 3D scanner to switch to the first scanning mode; when the distance information is less than the difference between the preset threshold and the variable value, a switching instruction is generated to control the 3D scanner to switch to the second scanning mode. By setting the variable value, it can buffer the switching of scanning modes and avoid frequent switching of scanning modes.

[0058] In one embodiment, as shown in FIG3 , a switching instruction generation method is provided. When the three-dimensional scanner is currently in the first scanning mode, the distance D between the scanner and the object to be scanned is calculated, and D is compared with YD3, where YD3 is a preset threshold. If D < YD3, it is determined whether to use the second scanning mode. If so, the second scanning mode is switched to; if not, the current scanning mode is maintained. If D ≥ YD3, the current scanning mode is maintained. When the three-dimensional scanner is currently in the second scanning mode, the distance D between the scanner and the object to be scanned is calculated, and D is compared with YD3. If D ≥ YD3, it is determined whether to use the first scanning mode. If so, the first scanning mode is switched to; if not, the current scanning mode is maintained. If D < YD3, the current scanning mode is maintained.

[0059] In one embodiment, as shown in FIG4 , a switching instruction generation method is provided. When the three-dimensional scanner is currently in the first scanning mode, the distance D between the scanner and the object to be scanned is calculated, and D is compared with YD3-B, where YD3 is a preset threshold value and B is a variable value. If D < YD3-B, it is determined whether to use the second scanning mode. If so, the second scanning mode is switched to; if not, the current scanning mode is maintained. If D ≥ YD3-B, the current scanning mode is maintained. When the three-dimensional scanner is currently in the second scanning mode, the distance D between the scanner and the object to be scanned is calculated, and D is compared with YD3+B. If D > YD3+B, it is determined whether to use the first scanning mode. If so, the first scanning mode is switched to; if not, the current scanning mode is maintained. If D ≤ YD3, the current scanning mode is maintained.

[0060] In one embodiment, as shown in FIG5 , a method step of a first scanning mode is provided:

[0061] Step 502: Acquire two frames of grayscale images of the object to be scanned in real time.

[0062] When the 3D scanner operates in the first scanning mode, the binocular camera acquires two grayscale images of the scanned object in real time. That is, the binocular camera captures two grayscale images simultaneously. These two grayscale images can contain only speckle or both speckle and linear streak features, i.e., linear streak features formed by the laser.

[0063] Step 504 : Reconstruct the two grayscale images based on the speckles in the grayscale images to obtain first point cloud data.

[0064] After obtaining two frames of grayscale images, speckle reconstruction is performed based on the speckles in the two frames of grayscale images to obtain first point cloud data.

[0065] Step 506 : Calculate a first transformation relationship between the first point cloud data and the world coordinate system based on a point cloud registration algorithm.

[0066] The first point cloud data is registered with the world coordinate system through a point cloud registration algorithm to obtain a first conversion relationship between the first point cloud data and the world coordinate system.

[0067] In step 508 , the first point cloud data obtained by reconstructing each frame is added to the first scattered speckle cloud set, and the first transformation relationship corresponding to the first point cloud data of each frame is added to the transformation relationship set.

[0068] Two grayscale images are acquired at each moment during the 3D scanner's operation in the first scanning mode. Speckle reconstruction is performed on the two grayscale images at each moment to obtain first point cloud data for each frame. Point cloud registration is then performed on each frame of the first point cloud data to obtain a first transformation relationship between each frame of the first point cloud data and the world coordinate system. Finally, the reconstructed first point cloud data for each frame is added to a first speckle cloud set, and the first transformation relationship corresponding to each frame of the first point cloud data is added to a relationship transformation set.

[0069] In one embodiment, as shown in FIG6 , a method step of a second scanning mode is provided:

[0070] Step 602: Acquire two frames of grayscale images of the object to be scanned in real time.

[0071] When the 3D scanner operates in the second scanning mode, the binocular camera captures two grayscale images of the object in real time. This means that the binocular camera captures two grayscale images simultaneously. These images contain both speckle and linear streak features, specifically those created by the laser.

[0072] Step 604 : Reconstruct the two grayscale images based on the speckle in the grayscale images to obtain second point cloud data; and reconstruct the two grayscale images based on the linear stripe features to obtain third point cloud data.

[0073] After obtaining two frames of grayscale images, speckle reconstruction is performed based on the speckles in the two frames of grayscale images to obtain second point cloud data. Multi-line laser reconstruction is performed based on the linear stripe features in the two frames of grayscale images to obtain third point cloud data.

[0074] Step 606 : Calculate a second transformation relationship between the second point cloud data and the third point cloud data and the world coordinate system based on a point cloud registration algorithm.

[0075] The second point cloud data is registered with the world coordinate system using a point cloud registration algorithm to obtain a second transformation relationship between the second point cloud data and the world coordinate system. Since both the second and third point cloud data are reconstructed based on two grayscale image frames, the transformation relationship between the third point cloud data and the world coordinate system is the same as the transformation relationship between the second point cloud data and the world coordinate system, namely, a second transformation relationship. It is understood that the third point cloud data can also be registered with the world coordinate system using a point cloud registration algorithm to obtain a second transformation relationship between the third point cloud data and the world coordinate system. This second transformation relationship is also the transformation relationship between the second point cloud data and the world coordinate system. Of course, when the speckle projected by the speckle projector and the laser line projected by the linear fringe feature projector are in different wavelength bands, a transformation relationship exists between the second and third point cloud data. In this case, the second transformation relationship between the second and third point cloud data and the world coordinate system is not the same transformation relationship and needs to be solved in conjunction with the existing transformation relationship between the second and third point cloud data.

[0076] In step 608, the second point cloud data reconstructed from each frame is added to the second scattered spot cloud set, the third point cloud data reconstructed from each frame is added to the first linear stripe feature point cloud set, and the second transformation relationship corresponding to the second point cloud data and the third point cloud data of each frame is added to the transformation relationship set.

[0077] In one embodiment, before acquiring two frames of grayscale images of the object to be scanned in real time, the method includes: projecting speckles onto the surface of the object to be scanned using a speckle projector; and projecting linear stripe features onto the surface of the object to be scanned using a linear stripe feature projector. Both the speckle projector and the linear stripe feature projector are infrared light projectors. Infrared light is invisible to the naked eye, thereby preventing damage to the eyes.

[0078] Acquire two grayscale images at each moment while the 3D scanner is operating in the second scanning mode. Perform speckle reconstruction on the two grayscale images at each moment to obtain second point cloud data for each frame. Perform multi-line laser reconstruction on the two grayscale images at each moment to obtain third point cloud data for each frame. Point cloud registration is then performed on each frame of the second point cloud data to obtain a second transformation relationship between the second point cloud data and the world coordinate system, as well as a second transformation relationship between the third point cloud data and the world coordinate system. Finally, the reconstructed second point cloud data for each frame is added to the second speckle cloud set, the reconstructed third point cloud data for each frame is added to the first linear stripe feature point cloud set, and the second transformation relationship corresponding to the second and third point cloud data for each frame is added to the relationship transformation set.

[0079] In the embodiment of the present application, speckle reconstruction is performed on the grayscale image during scanning in both the first scanning mode and the second scanning mode, so that speckle features can be used for splicing during 3D reconstruction, thus avoiding the need to attach marker points to the scanned object.

[0080] In one embodiment, performing three-dimensional reconstruction based on the first point cloud data, the second point cloud data, and the third point cloud data to obtain a three-dimensional model includes:

[0081] Point clouds are stitched together based on the first scattered speckle cloud set, the second scattered speckle cloud set, and the transformation relationship set, and the corresponding first point cloud data, second point cloud data, first transformation relationship, and second transformation relationship. That is, during three-dimensional reconstruction, the scanned data first needs to be stitched together. Therefore, the first point cloud data in the first scattered speckle cloud set, the second point cloud data in the second scattered speckle cloud set, and the first transformation relationship and second transformation relationship in the transformation relationship set are obtained. Since the first point cloud data and the second point cloud data are both point cloud data reconstructed based on speckle, they can be stitched together using the first point cloud data, the second point cloud data, the first transformation relationship, and the second transformation relationship. After the stitching is completed, three-dimensional reconstruction is performed based on the first scattered speckle cloud set, the second scattered speckle cloud set, the first linear stripe feature point cloud set, and the transformation relationship set, and the corresponding first point cloud data, second point cloud data, third point cloud data, first transformation relationship, and second transformation relationship to obtain a three-dimensional model. It can be understood that the first point cloud data is point cloud data reconstructed based on speckle in the first scanning mode; the second point cloud data is point cloud data reconstructed based on speckle in the second scanning mode; and the third point cloud data is point cloud data reconstructed based on linear stripe features in the second scanning mode. During 3D reconstruction, point clouds are first spliced ​​based on the first and second point cloud data, the first and second transformation relationships. Since the third and second point cloud data are synchronously acquired, the transformation relationships between the second and third point cloud data and the world coordinate system are both the second transformation relationships. Therefore, the third point cloud data is also spliced ​​with the first point cloud data. Finally, 3D reconstruction is performed based on the first and third point cloud data, the first and second transformation relationships, and a 3D model is obtained.

[0082] In this embodiment of the present application, 3D reconstruction is completed by stitching together point cloud data obtained through speckle reconstruction. Using speckle as a feature for stitching and reconstruction avoids the need to place markers on the surface of the scanned object, saving manpower and material costs, and further improving the efficiency and accuracy of 3D scanning.

[0083] In one embodiment, before performing point cloud stitching, it is also necessary to perform global registration using the first point cloud data and the second point cloud data, and further adjust the first conversion relationship and the second conversion relationship. For example, global registration is performed based on the first point cloud data and the second point cloud data corresponding to the first scattered speckle cloud set and the second scattered speckle cloud set, and the first conversion relationship and the second conversion relationship corresponding to the conversion relationship set are updated. Since the first conversion relationship and the second conversion relationship before the update are both obtained by registration based on the local point cloud data collected at a certain moment, the registration accuracy of the local point cloud data is relatively low for the object to be scanned as a whole. Therefore, after the three-dimensional scan is completed, it is necessary to perform global registration based on the first point cloud data and the second point cloud data obtained for each frame, and update the conversion relationship obtained by the local registration, so as to improve the accuracy of the conversion relationship.

[0084] In one embodiment, since the point cloud data reconstructed by the second scanning mode includes both the second point cloud data obtained by speckle reconstruction and the third point cloud data obtained by multi-line laser reconstruction, when performing three-dimensional reconstruction, the second point cloud data and the third point cloud data can be respectively assigned the weights occupied during reconstruction, thereby adjusting the accuracy of the reconstructed three-dimensional model. For example, first, the scattered speckle cloud weight and the linear stripe feature point cloud weight are obtained; the scattered speckle cloud weight is the weight of the second point cloud data in the second scattered speckle cloud set; the linear stripe feature point cloud weight is the weight of the third point cloud data in the first linear stripe feature point cloud set. Among them, the scattered speckle cloud weight and the linear stripe feature point cloud weight can be set according to the actual use requirements, and this embodiment does not make specific limitations. For example, if the three-dimensional model needs to be reconstructed with higher accuracy, the linear stripe feature point cloud weight can be set higher. After obtaining the scattered spot cloud weights and the linear stripe feature point cloud weights, three-dimensional reconstruction is performed based on the first scattered spot cloud set, the second scattered spot cloud set, the first linear stripe feature point cloud set, the transformation relationship set, the scattered spot cloud weights and the linear stripe feature point cloud weights to obtain a three-dimensional model.

[0085] In the embodiment of the present application, by setting the scattered spot cloud weight and the linear stripe feature point cloud weight, the accuracy of the three-dimensional model can be adjusted in real time according to the user's needs to better meet the user's usage needs.

[0086] In one embodiment, there's no clear distinction between the first and second scanning modes; the two modes are merged into one, simultaneously projecting both a speckle feature pattern and a laser feature pattern. The only difference in subsequent data processing is the data processing. For example, the first scanning mode uses both a speckle projector and a linear fringe feature projector to project a speckle image and linear fringe features onto the surface of the object being scanned. In this case, the data collected by the 3D scanner includes both speckle and linear fringe features. In addition to the above embodiment, the first scanning mode also requires reconstructing two grayscale images based on the linear fringe features in the grayscale images to obtain fourth point cloud data. A point cloud registration algorithm is used to calculate a first transformation between the fourth point cloud data and the world coordinate system. Finally, the reconstructed fourth point cloud data for each frame is added to the second linear fringe feature point cloud set. In this case, weights must be assigned to the first, second, third, and fourth point cloud data during 3D reconstruction. For example, a first scattered speckle cloud weight, a second scattered speckle cloud weight, a first linear stripe feature point cloud weight, and a second linear stripe feature point cloud weight are obtained; the first scattered speckle cloud weight is the weight of the first point cloud data in the first scattered speckle cloud set, the second scattered speckle cloud weight is the weight of the second point cloud data in the second scattered speckle cloud set, the first linear stripe feature point cloud weight is the weight of the fourth point cloud data in the second linear stripe feature point cloud set, and the second linear stripe feature point cloud weight is the weight of the third point cloud data in the first linear stripe feature point cloud set. The first point cloud data and the fourth point cloud data are point cloud data reconstructed under the first scanning mode. Therefore, the first scattered speckle cloud weight corresponding to the first point cloud data and the first linear stripe feature point cloud weight corresponding to the fourth point cloud data constitute a set of weights, i.e., the sum of the first scattered speckle cloud weight and the first linear stripe feature point cloud weight is 1. The second point cloud data and the third point cloud data are point cloud data reconstructed under the second scanning mode. Therefore, the second scattered speckle cloud weight corresponding to the second point cloud data and the second linear stripe feature point cloud weight corresponding to the third point cloud data constitute a set of weights, i.e., the sum of the second scattered speckle cloud weight and the second linear stripe feature point cloud weight is 1. The first scattered speckle cloud weight, the second scattered speckle cloud weight, the first linear stripe feature point cloud weight, and the second linear stripe feature point cloud weight can be set based on actual usage requirements, and this embodiment does not impose specific limitations on their values. After obtaining the weights of all point cloud data, three-dimensional reconstruction is performed based on the first scattered speckle cloud set, the second scattered speckle cloud set, the first linear stripe feature point cloud set, the second linear stripe feature point cloud set, the transformation relationship set, the first scattered speckle cloud weight, the second scattered speckle cloud weight, the first linear stripe feature point cloud weight, and the second linear stripe feature point cloud weight to obtain a three-dimensional model.It can be understood that when the first scanning mode only requires the first point cloud data for three-dimensional reconstruction, the first scattered spot cloud weight corresponding to the first point cloud data can be set to 1, and the first linear stripe feature point cloud weight corresponding to the fourth point cloud data can be set to 0.

[0087] Of course, when the speckle projected by the speckle projector and the laser line projected by the linear stripe feature projector are in different bands, there is a conversion relationship between the second point cloud data and the third point cloud data. When scanning using the first scanning mode, the conversion relationship is a unit matrix.

[0088] This embodiment can further adjust the accuracy of the three-dimensional model by configuring weights for the first point cloud data, the second point cloud data, the third point cloud data, and the fourth point cloud data respectively, so as to better meet the user's usage needs.

[0089] In one embodiment, a color camera may be used to acquire a color image of the object to be scanned during the three-dimensional scanning, and texture mapping may be performed based on the color image.

[0090] The first scanning mode also includes: acquiring a color image of the object to be scanned, calibrating the first point cloud data with the color image to obtain a first point cloud image transformation relationship; and obtaining a third transformation relationship between the color image and a world coordinate system based on the first transformation relationship and the first point cloud image transformation relationship. Each frame of the color image is added to an image set, and the third transformation relationship between each frame of the image and the world coordinate system is added to the image transformation set. The third transformation relationship between the color image and the world coordinate system is obtained based on the first transformation relationship and the first point cloud image transformation relationship, and the third transformation relationship can be obtained by multiplying the first transformation relationship by the first point cloud image transformation relationship.

[0091] The second scanning mode also includes: acquiring a color image of the object to be scanned, calibrating the second point cloud data with the color image to obtain a second point cloud image transformation relationship; and obtaining a fourth transformation relationship between the color image and the world coordinate system based on the second transformation relationship and the second point cloud image transformation relationship. Each frame of the color image is added to the image set, and the fourth transformation relationship between each frame of the image and the world coordinate system is added to the image transformation set. The fourth transformation relationship between the color image and the world coordinate system is obtained based on the second transformation relationship and the second point cloud image transformation relationship, and the fourth transformation relationship can be obtained by multiplying the second transformation relationship by the second point cloud image transformation relationship.

[0092] After the three-dimensional scanning is completed and the three-dimensional model is obtained, the image set and the image transformation set can be used to perform texture mapping on the three-dimensional model, so that the obtained three-dimensional model can better reflect the characteristics of the scanned object.

[0093] The present application provides a three-dimensional scanning method, which first adopts a speckle scanning mode when performing three-dimensional scanning and scanning large-scale data. The speckle scanning is specifically:

[0094] Step 1: The projector projects a speckle pattern onto the object to be scanned.

[0095] Step 2: The binocular black and white camera and the color camera are triggered simultaneously to obtain two frames of grayscale images G0 and G1 and one frame of color image M.

[0096] Step 3: The two grayscale images G0 and G1 are reconstructed by speckle to obtain the point cloud C. Then, the coordinate transformation relationship between the point cloud C and the color image M is obtained as RT0 through calibration.

[0097] Step 4: Using the point cloud registration algorithm, the transformation relationship RT_C between the point cloud C and the world coordinate system is calculated. Further, the transformation relationship RT_M = RT_C × RT0 between the color image M and the world coordinate system is obtained.

[0098] Step 5: Add the point cloud C to the global point cloud set CA, add the color image M to the global image set MA, add the transformation relationship RT_C to the global point cloud pose transformation set RT_CA, and add the transformation relationship RT_M to the global image pose transformation set RT_MA.

[0099] After scanning a large area of ​​the object to be scanned, the scanning mode is switched to the speckle + laser scanning mode to perform fine scanning on local details. The speckle + laser scanning mode is specifically as follows:

[0100] Step 1: The projector projects speckle patterns and multi-line lasers onto the object to be scanned.

[0101] Step 2: The binocular black and white camera and the color camera are triggered simultaneously to obtain two frames of grayscale images G0' and G1' and one frame of color image M'.

[0102] Step 3: Perform speckle reconstruction on the two grayscale images G0' and G1' to obtain point cloud C'. Perform multi-line laser reconstruction on the two grayscale images G0' and G1' to obtain point cloud L. Point cloud C' and point cloud L are in the same coordinate system. Then, through calibration, the coordinate transformation relationship between point cloud C' and image M' is obtained as RT0'.

[0103] Step 4: Using the point cloud registration algorithm, calculate the transformation relationship RT_C' between point cloud C' and the world coordinate system. Since both point clouds L and C' are reconstructed based on the two grayscale images G0' and G1', the transformation relationship between point cloud L and the world coordinate system is RT_C'. Furthermore, the transformation relationship between color image M' and the world coordinate system is RT_M' = RT_C' × RT0'.

[0104] Step 5: Add point cloud C' to the global point cloud set CA, add point cloud L' to the global laser point cloud set LA, add color image M' to the global image set MA, add the transformation relationship RT_C' to the global point cloud pose transformation set RT_CA, and add the transformation relationship RT_M' to the global image pose transformation set RT_MA.

[0105] After scanning is complete, point clouds C and C' in the global point cloud set CA are transformed to the world coordinate system using the transformation relationships RT_C and RT_C' in the global point cloud pose transformation set RT_CA. Point cloud L in the global laser point cloud set LA is also transformed to the world coordinate system using the transformation relationship RT_C' in the global point cloud pose transformation set RT_CA. A TSDF (Truncated Signed Distance Function) is then used for real-time fusion to generate the point cloud model PM, which is then displayed.

[0106] Before reconstructing the point cloud model, global registration is performed through the global point cloud set CA, and all transformation relationships in the global point cloud pose transformation set RT_CA are updated.

[0107] When performing 3D reconstruction, if the point cloud data of a certain frame only comes from the global point cloud set CA, only the point cloud data in the corresponding global point cloud set CA will be used for TSDF fusion. When the point cloud data of a certain frame includes both the global point cloud set CA and the global laser point cloud set LA, different weights will be assigned to the two point cloud data during TSDF fusion. For example, the weight from the global point cloud set CA during TSDF fusion is WC, and the weight from the global laser point cloud set LA during TSDF fusion is WL. When WC=0, WL=1, it means that only LA is used for fusion. When WC is not equal to 0, and WL is not equal to 0, it means that both point clouds are involved in the fusion.

[0108] In this embodiment of the present application, even without attaching markers to the scanned object, the system can freely switch between the first and second scanning modes and reconstruct a three-dimensional model. Furthermore, in areas where details of the object need to be scanned, laser reconstruction can improve the accuracy of local details. Furthermore, during fusion, the point cloud data obtained from speckle reconstruction and the point cloud data obtained from laser reconstruction are weighted separately, effectively utilizing both speckle and laser data.

[0109] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0110] Based on the same inventive concept, embodiments of the present application also provide a three-dimensional scanning device for implementing the three-dimensional scanning method described above. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations of one or more of the three-dimensional scanning device embodiments provided below can be found in the above-mentioned limitations of the three-dimensional scanning method and will not be further elaborated here.

[0111] In one embodiment, as shown in FIG7 , a three-dimensional scanning device is provided, comprising: a scanning module 100 , an instruction acquisition module 200 , and a reconstruction module 300 ; wherein:

[0112] The scanning module 100 is configured to scan the object to be scanned based on a first scanning mode to obtain first point cloud data; the first point cloud data includes point cloud data obtained by reconstructing the point cloud based on speckle;

[0113] The instruction acquisition module 200 is used to acquire a switching instruction;

[0114] The scanning module 100 is further configured to scan the object to be scanned based on the second scanning mode according to the switching instruction to obtain second point cloud data and third point cloud data; the second point cloud data includes point cloud data reconstructed based on speckle patterns, and the third point cloud data includes point cloud data reconstructed based on linear fringe features;

[0115] The reconstruction module 300 is used to perform three-dimensional reconstruction based on the first point cloud data, the second point cloud data and the third point cloud data to obtain a three-dimensional model.

[0116] The instruction acquisition module 200 is further configured to acquire a switching instruction.

[0117] The scanning module 100 is further configured to scan the object to be scanned based on the first scanning mode according to the switching instruction to obtain first point cloud data.

[0118] The instruction acquisition module 200 is further configured to acquire the switching instruction input by the user.

[0119] The instruction acquisition module 200 is also used to obtain the distance information between the scanner and the object to be scanned; generate the switching instruction based on the distance information; if the distance information is greater than or equal to a preset threshold, the object to be scanned is scanned based on the first scanning mode; if the distance information is less than the preset threshold, the object to be scanned is scanned based on the second scanning mode.

[0120] The scanning module 100 is further configured to acquire two frames of grayscale images of the object to be scanned in real time; reconstruct the two frames of grayscale images based on speckle in the grayscale images to obtain first point cloud data; calculate a first transformation relationship between the first point cloud data and a world coordinate system based on a point cloud registration algorithm; add the first point cloud data obtained by reconstructing each frame to a first speckle cloud set, and add the first transformation relationship corresponding to each frame of the first point cloud data to a transformation relationship set.

[0121] The scanning module 100 is also used to acquire two frames of grayscale images of the object to be scanned in real time; reconstruct the two frames of grayscale images based on the speckle in the grayscale images to obtain second point cloud data; reconstruct the linear stripe features in the grayscale images to obtain third point cloud data; calculate the second transformation relationship between the second point cloud data and the third point cloud data and the world coordinate system based on the point cloud registration algorithm; add the second point cloud data reconstructed from each frame to the second scattered speckle point cloud set, add the third point cloud data reconstructed from each frame to the first linear stripe feature point cloud set, and add the second transformation relationship corresponding to the second point cloud data and the third point cloud data from each frame to the transformation relationship set.

[0122] The reconstruction module 300 is also used to perform point cloud stitching based on the first scattered speckle cloud set, the second scattered speckle cloud set and the transformation relationship set, and the corresponding first point cloud data, second point cloud data, first transformation relationship and second transformation relationship; and to perform three-dimensional reconstruction based on the first scattered speckle cloud set, the second scattered speckle cloud set, the first linear stripe feature point cloud set and the transformation relationship set, and the corresponding first point cloud data, second point cloud data, third point cloud data, first transformation relationship and second transformation relationship to obtain a three-dimensional model.

[0123] The reconstruction module 300 is further configured to perform global registration based on the first point cloud data and the second point cloud data corresponding to the first scattered speckle cloud set and the second scattered speckle cloud set, and update the first transformation relationship and the second transformation relationship corresponding to the transformation relationship set.

[0124] The reconstruction module 300 is also used to obtain the scattered spot cloud weight and the linear stripe feature point cloud weight; the scattered spot cloud weight is the weight of the second point cloud data in the second scattered spot cloud set; the linear stripe feature point cloud weight is the weight of the third point cloud data in the first linear stripe feature point cloud set; three-dimensional reconstruction is performed based on the first scattered spot cloud set, the second scattered spot cloud set, the first linear stripe feature point cloud set, the transformation relationship set, the scattered spot cloud weight and the linear stripe feature point cloud weight to obtain a three-dimensional model.

[0125] The scanning module 100 is also used to reconstruct the two frames of grayscale images based on the linear stripe features in the grayscale images to obtain fourth point cloud data; calculate the first transformation relationship between the fourth point cloud data and the world coordinate system based on the point cloud registration algorithm; and add the fourth point cloud data reconstructed from each frame to the second linear stripe feature point cloud set.

[0126] The reconstruction module 300 is also used to obtain a first scattered spot cloud weight, a second scattered spot cloud weight, a first linear stripe feature point cloud weight and a second linear stripe feature point cloud weight; the first scattered spot cloud weight is the weight of the first point cloud data in the first scattered spot cloud set, the second scattered spot cloud weight is the weight of the second point cloud data in the second scattered spot cloud set, the first linear stripe feature point cloud weight is the weight of the fourth point cloud data in the second linear stripe feature point cloud set, and the second linear stripe feature point cloud weight is the weight of the third point cloud data in the first linear stripe feature point cloud set; three-dimensional reconstruction is performed according to the first scattered spot cloud set, the second scattered spot cloud set, the first linear stripe feature point cloud set, the second linear stripe feature point cloud set, the transformation relationship set, the first scattered spot cloud weight, the second scattered spot cloud weight, the first linear stripe feature point cloud weight and the second linear stripe feature point cloud weight to obtain a three-dimensional model.

[0127] Each module in the above-mentioned 3D scanning device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0128] In one embodiment, a computer device is provided, the internal structure of which can be shown in Figure 8. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a three-dimensional scanning method is implemented.

[0129] Those skilled in the art will understand that the structure shown in FIG8 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0130] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements any one of the three-dimensional scanning methods in the above embodiments when executing the computer program.

[0131] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any one of the three-dimensional scanning methods in the above embodiments is implemented.

[0132] In one embodiment, a computer program product is provided, comprising a computer program, which implements any one of the three-dimensional scanning methods in the above embodiments when executed by a processor.

[0133] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0134] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0135] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0136] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A three-dimensional scanning method, characterized in that, The method includes: Scanning the object to be scanned based on a first scanning mode to obtain first point cloud data; the first point cloud data includes point cloud data reconstructed based on speckles; Obtaining a switching instruction; According to the switching instruction, scanning the object to be scanned based on a second scanning mode to obtain second point cloud data and third point cloud data; the second point cloud data includes point cloud data reconstructed based on speckles, and the third point cloud data includes point cloud data reconstructed based on linear stripe features; Performing three-dimensional reconstruction based on the first point cloud data, the second point cloud data, and the third point cloud data to obtain a three-dimensional model.

2. The method according to claim 1, wherein After scanning the object to be scanned based on the second scanning mode according to the switching instruction to obtain second point cloud data and third point cloud data, it includes: Obtaining a switching instruction; According to the switching instruction, scanning the object to be scanned based on the first scanning mode to obtain first point cloud data.

3. The method according to claim 2, wherein The obtaining of the switching instruction includes: Obtaining the switching instruction input by the user.

4. The method according to claim 1, wherein The obtaining of the switching instruction includes: Obtaining distance information between the scanner and the object to be scanned; Generating the switching instruction according to the distance information; wherein, If the distance information is greater than or equal to a preset threshold, scanning the object to be scanned based on the first scanning mode; If the distance information is less than the preset threshold, scanning the object to be scanned based on the second scanning mode.

5. The method according to claim 2, wherein Scanning the object to be scanned based on the first scanning mode to obtain first point cloud data includes: Real-time obtaining two grayscale images of the object to be scanned; According to the two grayscale images, reconstructing based on the speckles in the grayscale images to obtain first point cloud data; Calculating a first transformation relationship between the first point cloud data and the world coordinate system based on a point cloud registration algorithm; Adding the first point cloud data reconstructed from each frame to a first speckle point cloud set, and adding the first transformation relationship corresponding to each frame of the first point cloud data to a transformation relationship set.

6. The method according to claim 5, wherein, Scanning the object to be scanned based on the second scanning mode to obtain second point cloud data and third point cloud data includes: Real-time obtaining two grayscale images of the object to be scanned; According to the two grayscale images, reconstructing based on the speckles in the grayscale images to obtain second point cloud data; reconstructing based on the linear stripe features in the grayscale images to obtain third point cloud data; Calculating a second transformation relationship between the second point cloud data and the third point cloud data and the world coordinate system based on a point cloud registration algorithm; Adding the second point cloud data reconstructed from each frame to a second speckle point cloud set, adding the third point cloud data reconstructed from each frame to a first linear stripe feature point cloud set, and adding the second transformation relationship corresponding to each frame of the second point cloud data and the third point cloud data to a transformation relationship set.

7. The method according to claim 6, wherein Before the real-time obtaining of two grayscale images of the object to be scanned, it includes: Projecting speckles onto the surface of the object to be scanned through a speckle projector; Projecting linear stripe features onto the surface of the object to be scanned through a linear stripe feature projector; Both the speckle projector and the linear stripe feature projector are infrared light projectors.

8. The method according to claim 6, wherein The three-dimensional reconstruction based on the first point cloud data, the second point cloud data, and the third point cloud data to obtain a three-dimensional model includes: Performing point cloud stitching according to the first speckle point cloud set, the second speckle point cloud set, and the conversion relationship set, and the corresponding first point cloud data, second point cloud data, first conversion relationship, and second conversion relationship; Performing three-dimensional reconstruction according to the first speckle point cloud set, the second speckle point cloud set, the first linear stripe feature point cloud set, and the conversion relationship set, and the corresponding first point cloud data, second point cloud data, third point cloud data, first conversion relationship, and second conversion relationship to obtain a three-dimensional model.

9. The method according to claim 8, wherein Before performing point cloud stitching according to the first speckle point cloud set, the second speckle point cloud set, and the conversion relationship set, and the corresponding first point cloud data, second point cloud data, first conversion relationship, and second conversion relationship, it further includes: Performing global registration according to the first point cloud data and the second point cloud data corresponding to the first speckle point cloud set and the second speckle point cloud set, and updating the first conversion relationship and the second conversion relationship corresponding to the conversion relationship set.

10. The method according to claim 8, wherein The three-dimensional reconstruction according to the first speckle point cloud set, the second speckle point cloud set, the first linear stripe feature point cloud set, and the conversion relationship set, and the corresponding first point cloud data, second point cloud data, third point cloud data, first conversion relationship, and second conversion relationship to obtain a three-dimensional model includes: Obtaining the speckle point cloud weight and the linear stripe feature point cloud weight; the speckle point cloud weight is the weight of the second point cloud data in the second speckle point cloud set; the linear stripe feature point cloud weight is the weight of the third point cloud data in the first linear stripe feature point cloud set; Performing three-dimensional reconstruction according to the first speckle point cloud set, the second speckle point cloud set, the first linear stripe feature point cloud set, the conversion relationship set, the speckle point cloud weight, and the linear stripe feature point cloud weight to obtain a three-dimensional model.

11. The method according to claim 8, wherein, The scanning of the object to be scanned based on the first scanning mode to obtain the first point cloud data further includes: Reconstructing based on the linear stripe features in the grayscale images according to two frames of the grayscale images to obtain the fourth point cloud data; Calculating the first conversion relationship between the fourth point cloud data and the world coordinate system based on the point cloud registration algorithm; Adding the fourth point cloud data reconstructed from each frame to the second linear stripe feature point cloud set; The three-dimensional reconstruction according to the first speckle point cloud set, the second speckle point cloud set, the first linear stripe feature point cloud set, and the conversion relationship set, and the corresponding first point cloud data, second point cloud data, third point cloud data, first conversion relationship, and second conversion relationship to obtain a three-dimensional model further includes: Obtain the first speckle point cloud weight, the second speckle point cloud weight, the first linear stripe feature point cloud weight, and the second linear stripe feature point cloud weight; the first speckle point cloud weight is the weight of the first point cloud data in the first speckle point cloud set, the second speckle point cloud weight is the weight of the second point cloud data in the second speckle point cloud set, the first linear stripe feature point cloud weight is the weight of the fourth point cloud data in the second linear stripe feature point cloud set, and the second linear stripe feature point cloud weight is the weight of the third point cloud data in the first linear stripe feature point cloud set; Perform three-dimensional reconstruction based on the first speckle point cloud set, the second speckle point cloud set, the first linear stripe feature point cloud set, the second linear stripe feature point cloud set, the conversion relationship set, the first speckle point cloud weight, the second speckle point cloud weight, the first linear stripe feature point cloud weight, and the second linear stripe feature point cloud weight to obtain a three-dimensional model.

12. A three-dimensional scanning device, characterized in that, The device includes: A scanning module, configured to scan an object to be scanned based on a first scanning mode to obtain first point cloud data; the first point cloud data includes point cloud data reconstructed based on speckles; An instruction acquisition module, configured to acquire a switching instruction; The scanning module is further configured to scan the object to be scanned based on a second scanning mode according to the switching instruction to obtain second point cloud data and third point cloud data; the second point cloud data includes point cloud data reconstructed based on speckles, and the third point cloud data includes point cloud data reconstructed based on linear stripe features; A reconstruction module, configured to perform three-dimensional reconstruction based on the first point cloud data, the second point cloud data, and the third point cloud data to obtain a three-dimensional model.

13. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method according to any one of claims 1 to 11.

Images