Handheld Scanner and Its Scanning Method

Abstract

The present invention discloses a handheld scanner and a scanning method thereof. The handheld scanner includes a texture camera, a first black-and-white camera and a second black-and-white camera, and the first black-and-white camera and the second black-and-white camera are arranged at intervals; the handheld scanner further includes a laser projector, and the texture camera and the first black-and-white camera are respectively arranged on both sides of the laser projector. When scanning a highly reflective or dark object, while performing laser scanning, the texture is acquired through the texture camera. After point cloud fusion is completed, the texture images are screened and fused according to the shooting angle and the highlight degree of the image, so as to obtain the texture image of the overall point cloud. In this way, the problem of using laser scanning and then using a digital camera for subsequent color mapping in the prior art can be avoided, and the real-time performance of scanning is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional model reconstruction, and more particularly, to a handheld scanner and a scanning method thereof. Background Art

[0002] Currently, for the scanning of objects that do not allow powder spraying, such as dark or reflective objects, like the scanning of cultural relics, there are mainly two techniques. One is to use a laser scanner to obtain three-dimensional data and then use a digital camera for subsequent color mapping; the second is to perform color scanning with a laser on an articulated arm. These two scanning methods have great limitations in terms of scanning real-time performance and the breadth during scanning. Summary of the Invention

[0003] Embodiments of the present invention provide a handheld scanner and a scanning method thereof to at least solve the technical problem of poor real-time performance of the existing scanning methods.

[0004] According to one aspect of the embodiments of the present invention, there is provided a handheld scanner, including: a texture camera; a first black-and-white camera and a second black-and-white camera, the first black-and-white camera and the second black-and-white camera being spaced apart; a laser projector, the texture camera and the first black-and-white camera being respectively disposed on both sides of the laser projector.

[0005] Further, the handheld scanner further includes: a speckle projector, the speckle projector being spaced apart from the laser projector.

[0006] Further, the handheld scanner further includes: a first scanning unit, the laser projector and the first black-and-white camera being disposed on the first scanning unit, the laser projector and the first black-and-white camera being adjacent to each other.

[0007] Further, the handheld scanner further includes: a second scanning unit, the speckle projector and the second black-and-white camera being disposed on the second scanning unit.

[0008] Further, the handheld scanner further includes: a handheld part, the first scanning unit and the second scanning unit being respectively disposed at both ends of the handheld part.

[0009] Further, there is an exit angle A between the exit light paths of the first black-and-white camera and the second black-and-white camera, wherein the exit angle A is between 5° and 20°.

[0010] Further, the handheld scanner further includes: a first fill light, there being a plurality of first fill lights, the plurality of first fill lights surrounding the first black-and-white camera.

[0011] Further, the handheld scanner further includes: a second fill light, there being a plurality of second fill lights, the plurality of second fill lights surrounding the second black-and-white camera.

[0012] Further, the handheld scanner further includes: a third fill light, where there are multiple third fill lights, and the multiple third fill lights are arranged around the texture camera.

[0013] Further, the handheld scanner further includes: a third black and white camera, and the third black and white camera is spaced apart from both the first black and white camera and the second black and white camera.

[0014] According to one aspect of the embodiments of the present invention, there is provided a scanning method for a handheld scanner. This scanning method is applied to any of the above-mentioned handheld scanners. Among them, the scanning method includes: using a laser projector to project laser light onto the surface of the object to be measured, and using the first black and white camera and the second black and white camera to synchronously obtain a laser line map modulated by the surface of the object to be measured. Among them, the laser light map is used for three-dimensional reconstruction of the object to be measured; stopping the laser projector from projecting laser light onto the surface of the object to be measured, and using the texture camera to obtain a texture map of the surface of the object to be measured; where the texture map is used for texture reconstruction of the object to be measured; based on the internal and external parameters of the first black and white camera and the second black and white camera, performing point cloud reconstruction on the laser line maps obtained by the first black and white camera and the second black and white camera; and using the texture map to perform texture mapping on the point cloud to obtain a final three-dimensional scanned image.

[0015] Further, using the texture map to perform texture mapping on the point cloud to obtain a final three-dimensional scanned image includes: based on the internal and external parameters of the first black and white camera, the second black and white camera, and the texture camera, determining the correspondence between each pixel in the texture map and each point in the point cloud; according to the correspondence between each pixel in the texture map and each point in the point cloud, performing a texture mapping operation on the point cloud to determine the final three-dimensional scanned image.

[0016] Further, based on the internal and external parameters of the first black and white camera, the second black and white camera, and the texture camera, determining the correspondence between each pixel in the texture map and each point in the point cloud includes: based on the internal and external parameters of the first black and white camera, the second black and white camera, and the texture camera, determining the correspondence between each pixel in the laser line maps obtained by the first black and white camera and the second black and white camera and each pixel in the texture map obtained by the texture camera; according to the correspondence between each pixel in the laser line map and the texture map and the correspondence between each pixel in the laser line map and each point in the point cloud, determining the correspondence between each pixel in the texture map and each point in the point cloud.

[0017] Further, according to the correspondence between each pixel in the texture map and each point in the point cloud, perform a texturing operation on the point cloud to determine the final 3D scanned image, including: splicing the current frame point cloud with the previous frame point cloud; and determining a first region of the current frame point cloud and the texturing image of the first region based on at least the texture information of the texture map corresponding to the current frame point cloud and the texture information of the texture map corresponding to the previous frame point cloud; process the point clouds of all frames using the above steps to obtain the final 3D scanned image.

[0018] Further, determine a first region of the current frame point cloud and the texturing image of the first region based on at least the texture information of the texture map corresponding to the current frame point cloud and the texture information of the texture map corresponding to the previous frame point cloud, including: determining the first region of the current frame point cloud based on at least the gray-scale change between the texture map corresponding to the current frame point cloud and the texture map corresponding to the previous frame point cloud, where the gray-scale change of the first region satisfies a preset condition; compare multiple texture maps corresponding to the first region of the current frame point cloud to determine the target texture map for frontal shooting at the corresponding position of the first region on the object to be measured; perform a texturing process on the first region using the target texture map.

[0019] Further, through the circuit design in the handheld scanner, make the time interval between obtaining the laser line map modulated by the surface of the object to be measured using the first black-and-white camera and the second black-and-white camera and obtaining the texture map of the surface of the object to be measured using the texture camera lower than a preset threshold.

[0020] According to one aspect of the embodiments of the present invention, a scanning method for a handheld scanner is provided. The scanning method is applied to any of the above handheld scanners, where the scanning method includes: Step S1: Project a speckle pattern onto the surface of the object to be measured using a speckle projector, and synchronously obtain a target speckle pattern modulated by the surface of the object to be measured using the first black-and-white camera and the second black-and-white camera, where the target speckle pattern is used for 3D reconstruction of the object to be measured; Step S2: Stop the speckle projector from projecting the speckle pattern onto the surface of the object to be measured, and use a texture camera to obtain the texture map of the surface of the object to be measured; where the texture map is used for texture reconstruction of the object to be measured; Step S3: Based on the internal and external parameters of the first black-and-white camera and the second black-and-white camera, perform point cloud reconstruction on the target speckle pattern obtained by the first black-and-white camera and the second black-and-white camera; and perform a texturing process on the point cloud using the texture map to obtain the final 3D scanned image.

[0021] Further, steps S1 and S2 in the scanning method are alternately executed; or, step S1 in the scanning method is executed multiple times, and then step S2 is executed.

[0022] According to one aspect of the embodiments of the present invention, a scanning method for a handheld scanner is provided. The scanning method is applied to any of the above handheld scanners. The scanning method includes: projecting a speckle pattern onto the surface of the object to be measured by a speckle projector, and synchronously acquiring a target speckle pattern modulated by the surface of the object to be measured by a first black-and-white camera and a second black-and-white camera, where the target speckle pattern is used for three-dimensional reconstruction of the object to be measured; stopping the speckle projector from projecting the speckle pattern onto the surface of the object to be measured, and projecting a laser beam onto the surface of the object to be measured by a laser projector, and synchronously acquiring a laser line pattern modulated by the surface of the object to be measured by the first black-and-white camera and the second black-and-white camera, where the laser beam pattern is used for three-dimensional reconstruction of the object to be measured; stopping the laser projector from projecting the laser beam onto the surface of the object to be measured, and acquiring a texture map of the surface of the object to be measured by a texture camera; where the texture map is used for texture reconstruction of the object to be measured; based on the internal and external parameters of the first black-and-white camera and the second black-and-white camera, performing point cloud reconstruction on the target speckle patterns acquired by the first black-and-white camera and the second black-and-white camera; and performing point cloud reconstruction on the laser line patterns acquired by the first black-and-white camera and the second black-and-white camera; performing screening processing on the first point cloud obtained by point cloud reconstruction of the target speckle pattern and the second point cloud obtained by point cloud reconstruction of the laser line pattern to obtain a target point cloud; and performing texture mapping on the target point cloud using the texture map to obtain a final three-dimensional scanned image.

[0023] In the embodiments of the present invention, the handheld scanner includes a texture camera, a first black-and-white camera, and a second black-and-white camera, and the first black-and-white camera and the second black-and-white camera are arranged at intervals; the handheld scanner further includes a laser projector, and the texture camera and the first black-and-white camera are respectively arranged on both sides of the laser projector. When scanning a highly reflective or dark object, while performing laser scanning, the texture is acquired by the texture camera. After the point cloud fusion is completed through the three-dimensional point cloud fusion technology, the texture image is screened and fused according to the shooting angle and the highlight degree of the image, so as to obtain the texture image of the overall point cloud. In this way, the problem of using laser scanning and then using a digital camera for subsequent color mapping in the prior art can be avoided, and the real-time performance of scanning is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0025] Figure 1 is a schematic diagram of a handheld scanner provided according to an embodiment of the present application;

[0026] Figure 2It is the flowchart of the scanning method of the handheld scanner provided by the embodiment of the present application Figure 1 ;

[0027] Figure 3 It is the flowchart of the scanning method of the handheld scanner provided by the embodiment of the present application Figure 2 ;

[0028] Figure 4 It is the flowchart of the scanning method of the handheld scanner provided by the embodiment of the present application Figure 3 .

[0029] Among them, the above-mentioned drawings include the following reference numerals:

[0030] 10, texture camera; 21, first black and white camera; 22, second black and white camera; 23, third black and white camera; 30, laser projector; 40, speckle projector; 51, first fill light; 52, second fill light; 53, third fill light; 60, first scanning unit; 70, second scanning unit; 80, handheld part. Detailed implementation manners

[0031] In order to enable those skilled in the art of the present technology to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order different from those illustrated or described here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a system, product or device including a series of units does not necessarily have to be limited to those units clearly listed, but may include other units not clearly listed or inherent to these products or devices.

[0033] According to the first embodiment of the present invention, a handheld scanner is provided. As Figure 1 shown, the handheld scanner according to the embodiment of the present invention includes a texture camera 10, a first black and white camera 21 and a second black and white camera 22, and the first black and white camera 21 and the second black and white camera 22 are arranged at intervals; a laser projector 30, and the texture camera 10 and the first black and white camera 21 are respectively arranged on both sides of the laser projector 30.

[0034] In this embodiment, when scanning a highly reflective or dark object, while the handheld scanner performs laser scanning, it obtains texture through the texture camera. After the point cloud fusion is completed by the 3D point cloud fusion technology, the texture images are screened and fused according to the shooting angle and the highlight degree of the image, so as to obtain the texture image of the overall point cloud. In this way, the problem of using laser scanning and then using a digital camera for subsequent color mapping in the prior art can be avoided, and the real-time performance of scanning is improved. This scanning mode is the first scanning mode.

[0035] Specifically, the shooting angle is obtained according to the camera RT obtained after scanning and stitching and the relationship between the texture camera and the black-and-white camera during calibration, and the highlight degree is obtained through the gray information of the algorithm. When screening and fusing texture images, if there is a large jump in the gray change of the same area in different images, it may be a highlight area, and then combined with the RT corresponding to this image to judge whether it is shooting directly at the object, so as to select which perspective image of this highlight area should be used for texturing.

[0036] According to the second embodiment of the present invention, a handheld scanner is provided. The handheld scanner includes a texture camera 10, a first black-and-white camera 21 and a second black-and-white camera 22, and the first black-and-white camera 21 and the second black-and-white camera 22 are arranged at intervals; a laser projector 30, and the texture camera 10 and the first black-and-white camera 21 are respectively arranged on both sides of the laser projector 30. In addition, it further includes a speckle projector 40, and the speckle projector 40 is arranged at intervals with the laser projector 30.

[0037] Among them, the laser projector, the first black-and-white camera and the second black-and-white camera construct a binocular stereo vision system. The handheld scanner includes a calculation module, in which the internal and external parameters of the laser projector, the first black-and-white camera and the second black-and-white camera, and the internal and external parameters of the laser projector, the texture camera and the binocular stereo vision system are pre-stored.

[0038] When the handheld scanner performs scanning, it performs three-dimensional scanning and texture scanning on the object to be scanned through the speckle projector 40 and the texture camera 10. This scanning mode is the second scanning mode. Then, for the parts that need to present details or cannot be obtained due to darkness or reflection, supplementary scanning is performed through the mode in the first embodiment, that is, while the handheld scanner performs laser scanning, it obtains texture through the texture camera. The algorithm screens the three-dimensional point cloud according to the obtained data, and primarily retains the point cloud of the laser. After the point cloud fusion is completed, the texture images are screened and fused from the texture maps obtained from the two scanning modes according to the shooting angle and the highlight degree of the image, so as to obtain the texture image of the overall point cloud.

[0039] Specifically, during the screening and fusion process, screening is performed first followed by fusion. During screening, judgment is made based on the degree of gray-scale edge skipping and whether the image is directly facing the object. During the fusion process between the point cloud obtained by the speckle pattern and the point cloud obtained by the laser pattern, it is necessary to unify the coordinate systems of the two scanning modes. The coordinate system can be unified based on fiducial points or features. When there is an overlap in the area, the point cloud obtained by the laser pattern is preferred. The point clouds of multiple scanning modes and the color image are fused and output, ensuring that for objects with dark or reflective surfaces, three-dimensional data and color information can be directly and quickly obtained without powder spraying treatment.

[0040] A handheld scanner according to a third embodiment of the present invention, the scanner includes a first scanning unit 60, a second scanning unit 70, and a handheld unit 80. The first scanning unit 60 and the second scanning unit 70 are respectively disposed at both ends of the handheld unit 80.

[0041] Among them, a laser projector 30 and a first black-and-white camera 21 are disposed on the first scanning unit 60. The laser projector 30 and the first black-and-white camera 21 are adjacently disposed. A speckle projector 40 and a second black-and-white camera 22 are disposed on the second scanning unit 70.

[0042] As Figure 1 shown, the main body of the handheld unit 80 is columnar, and its two ends are respectively connected to the first scanning unit 60 and the second scanning unit 70. Among them, a laser projector 30 is disposed on the end surface of one end of the first scanning unit 60, and a first black-and-white camera 21 is disposed on one side of the laser projector 30. A speckle projector 40 is disposed on the end surface of one end of the second scanning unit 70, a second black-and-white camera 22 is disposed on one side of the speckle projector 40, and a texture camera 10 is disposed on the other side of the speckle projector 40.

[0043] The texture camera 10 and the second black-and-white camera 22 are respectively disposed on both sides of the speckle projector 40, and the shooting angles are relatively close, which can increase the overlapping area of the two cameras and can color the point cloud data (i.e., add texture) to the greatest extent. In addition, the first black-and-white camera 21 and the second black-and-white camera 22 are respectively disposed on the first scanning unit 60 and the second scanning unit 70, and there is a handheld unit 80 between the first scanning unit 60 and the second scanning unit 70. Therefore, there is a certain interval between the first black-and-white camera 21 and the second black-and-white camera 22, which can improve the accuracy of the scanned data. Preferably, there is an emission angle A between the emission optical paths of the first black-and-white camera 21 and the second black-and-white camera 22. Among them, the emission angle A is preferably between 5° and 20°, which can further improve the accuracy of the scanned data.

[0044] In a fourth embodiment of the present invention, the handheld scanner further includes a fill light.

[0045] Specifically, the handheld scanner further includes: a first fill light 51, and there are multiple first fill lights 51, which are arranged around the first black-and-white camera 21. Arranging multiple first fill lights 51 around the first black-and-white camera 21 aims to make the fiducial points made of reflective materials reflect more clearly onto the camera when light hits them, which can minimize the interference from other environments, better highlight the fiducial points, and create convenient conditions for subsequent recognition. In this embodiment, the handheld scanner further includes a second fill light 52, and there are multiple second fill lights 52, which are arranged around the second black-and-white camera 22.

[0046] Preferably, the handheld scanner further includes a third fill light 53, and there are multiple third fill lights 53, which are arranged around the texture camera. Arranging a fill light around the texture camera can exclude the interference of ambient light and better reflect the original color information of the object.

[0047] In the fifth embodiment of the present invention, the scanner includes a first scanning unit 60, a second scanning unit 70, and a handheld unit 80. The first scanning unit 60 and the second scanning unit 70 are respectively arranged at both ends of the handheld unit 80. The handheld scanner includes a texture camera 10, a first black-and-white camera 21, and a second black-and-white camera 22. The first black-and-white camera 21 and the second black-and-white camera 22 are arranged at intervals; it further includes a laser projector 30. The texture camera 10 and the first black-and-white camera 21 are respectively arranged on both sides of the laser projector 30. The handheld scanner further includes a speckle projector 40, and the speckle projector 40 is arranged at intervals from the laser projector 30. The laser projector 30 and the first black-and-white camera 21 are arranged on the first scanning unit 60, and the laser projector 30 is adjacent to the first black-and-white camera 21. The speckle projector 40 and the second black-and-white camera 22 are arranged on the second scanning unit 70.

[0048] Preferably, the scanner further includes a third black-and-white camera 23, which is arranged on the first scanning unit 60 and is located on both sides of the laser projector 30 respectively with the first black-and-white camera 21. With three relatively dispersed black-and-white cameras, the recognition accuracy of the scanner for three-dimensional objects can be further improved.

[0049] The handheld scanner of the present application can obtain color information in real time while performing laser scanning, and can also perform data fusion on scanning data in different modes, thereby greatly expanding the applicable range of the scanner.

[0050] In the fifth embodiment of the present invention, a scanning method for a handheld scanner is provided. This scanning method is applied to the handheld scanner described in any of the above embodiments, specifically as Figure 2 shown, and the scanning method includes:

[0051] Step S101: Project a laser beam onto the surface of the object to be measured using a laser projector, and synchronously acquire the laser line images modulated by the surface of the object to be measured using a first black-and-white camera and a second black-and-white camera. Among them, the laser line images are used for three-dimensional reconstruction of the object to be measured.

[0052] Step S102: Stop the laser projector from projecting the laser beam onto the surface of the object to be measured, and acquire the texture image of the surface of the object to be measured using a texture camera; among them, the texture image is used for texture reconstruction of the object to be measured.

[0053] Step S103: Based on the internal and external parameters of the first black-and-white camera and the second black-and-white camera, perform point cloud reconstruction on the laser line images acquired by the first black-and-white camera and the second black-and-white camera; and perform texture mapping on the point cloud using the texture image to obtain the final three-dimensional scanned image.

[0054] Specifically, performing texture mapping on the point cloud using the texture image to obtain the final three-dimensional scanned image is achieved through the following steps: Step S1031: Based on the internal and external parameters of the first black-and-white camera, the second black-and-white camera, and the texture camera, determine the correspondence between each pixel in the texture image and each point in the point cloud; Step S1032: According to the correspondence between each pixel in the texture image and each point in the point cloud, perform a texture mapping operation on the point cloud to determine the final three-dimensional scanned image.

[0055] Further, determining the correspondence between each pixel in the texture image and each point in the point cloud based on the internal and external parameters of the first black-and-white camera, the second black-and-white camera, and the texture camera is achieved through the following steps: First, based on the internal and external parameters of the first black-and-white camera, the second black-and-white camera, and the texture camera, determine the correspondence between each pixel in the laser line images acquired by the first black-and-white camera and the second black-and-white camera and each pixel in the texture image acquired by the texture camera; Second, according to the correspondence between each pixel in the laser line image and the texture image and the correspondence between each pixel in the laser line image and each point in the point cloud, determine the correspondence between each pixel in the texture image and each point in the point cloud.

[0056] That is, based on the internal and external parameters of the black-and-white camera and the texture camera, determine the correspondence between each pixel in the laser line images and the texture image taken successively; according to the correspondence between each pixel in the laser line image and the texture image and the correspondence between each pixel in the laser line image and each point in the point cloud, determine the correspondence between each point in the point cloud and each pixel in the texture image.

[0057] Further, according to the correspondence between each pixel in the texture map and each point in the point cloud, performing a texturing operation on the point cloud to determine the final 3D scanned image is achieved through the following steps: splicing and processing the current frame of point cloud and the previous frame of point cloud; and determining the first region of the current frame of point cloud and the texturing image of the first region based on at least the texture information of the texture map corresponding to the current frame of point cloud and the texture information of the texture map corresponding to the previous frame of point cloud. At this time, continue to process the point clouds of all frames using the above steps to obtain the final 3D scanned image.

[0058] Among them, determining the first region of the current frame of point cloud based on at least the texture information of the texture map corresponding to the current frame of point cloud and the texture information of the texture map corresponding to the previous frame of point cloud means: determining the first region of the current frame of point cloud based on at least the gray-scale change between the texture map corresponding to the current frame of point cloud and the texture map corresponding to the previous frame of point cloud, where the gray-scale change of the first region satisfies a preset condition.

[0059] Among them, determining the texturing image of the first region based on the texture information of the texture map corresponding to the current frame of point cloud and the texture information of the texture map corresponding to the previous frame of point cloud means: comparing multiple texture maps corresponding to the first region of the current frame of point cloud to determine the target texture map obtained by taking a frontal view of the corresponding position of the first region on the object to be measured; using the target texture map to perform a texturing operation on the first region to obtain the texturing image of the first region.

[0060] The texture mapping of the point cloud and the texture fusion mapping between multiple frames of point clouds can be processed in real-time during scanning or after scanning is completed. That is, texture mapping can be performed for each acquired texture map and texture fusion mapping can be performed with the previous frame of texture map, or texture fusion mapping can be performed at once after multiple texture maps are acquired.

[0061] It should be noted that: in the case of performing texture fusion mapping at once after multiple texture maps are acquired, the front and rear frames of point clouds are both spliced and processed. The corresponding points in the current frame of point cloud correspond to both the texture information of the corresponding pixels in the previous frame of texture map and the texture information of the corresponding pixels in the next frame of texture map. At this time, the "determining the first region of the current frame of point cloud based on at least the gray-scale change between the texture map corresponding to the current frame of point cloud and the texture map corresponding to the previous frame of point cloud" described above can be: determining the first region of the current frame of point cloud based on the gray-scale change between the texture map corresponding to the current frame of point cloud, the texture map corresponding to the previous frame of point cloud, and the texture map corresponding to the next frame of point cloud.

[0062] For example: If there are large jumps in the grayscale changes on the texture map, the jump area is very likely to be the highlight area. At this time, the rotation and translation matrix corresponding to the texture map (i.e., RT) is combined to determine whether the texture map is taken directly facing the object, and then determine whether to use this texture map for texture mapping of the highlight area. Preferably, multiple texture maps corresponding to the highlight area (isotopic points) can also be selected for comparison to select which perspective image to use for texture mapping of this highlight area.

[0063] In addition, it should also be noted that: through the circuit design in the handheld scanner, the time interval between using the first black-and-white camera and the second black-and-white camera to obtain the laser line map modulated by the surface of the object to be measured and using the texture camera to obtain the texture map of the surface of the object to be measured is lower than the preset threshold. That is, in the conventional technology, it is very difficult to ensure synchronous acquisition of laser data. However, in this application, through circuit design and optimization of exposure parameters, the laser acquisition and texture acquisition are completed within a very small time difference, solving the dilemma of the prior art.

[0064] Finally, an explanation is given for the scanning method of the handheld scanner provided in this embodiment: The scanning method provided in this embodiment is the laser scanning mode of the handheld scanner, and its main work is as follows: The laser projects a laser line map onto the surface of the object to be measured, and the first black-and-white camera and the second black-and-white camera synchronously obtain the laser line map modulated by the surface of the object to be measured and transmit it to the computer for three-dimensional reconstruction; the laser stops working, and the texture camera obtains the texture map of the surface of the object to be measured and transmits it to the computer for texture reconstruction; the laser line map obtained by the first black-and-white camera and the laser line map obtained by the second black-and-white camera are used for point cloud reconstruction based on the internal and external parameters of the two preset black-and-white cameras; finally, the texture map is used for texture mapping of the point cloud to obtain the final three-dimensional scanned image.

[0065] In the sixth embodiment of the present invention, a scanning method for a handheld scanner is provided. This scanning method is applied to the handheld scanner described in any of the above embodiments, specifically as Figure 3 shown, and the scanning method includes:

[0066] Step S1: Use a speckle projector to project a speckle pattern onto the surface of the object to be measured, and use the first black-and-white camera and the second black-and-white camera to synchronously obtain the target speckle pattern modulated by the surface of the object to be measured, where the target speckle pattern is used for three-dimensional reconstruction of the object to be measured;

[0067] Step S2: Stop the speckle projector from projecting the speckle pattern onto the surface of the object to be measured, and use the texture camera to obtain the texture map of the surface of the object to be measured; where the texture map is used for texture reconstruction of the object to be measured;

[0068] Step S3: Based on the internal and external parameters of the first black-and-white camera and the second black-and-white camera, perform point cloud reconstruction on the target speckle patterns obtained by the first black-and-white camera and the second black-and-white camera; and use the texture map to perform texture mapping on the point cloud to obtain the final 3D scan image.

[0069] It should be noted that in this embodiment, steps S1 and S2 in the above scanning method are alternately executed; or, in this embodiment, step S1 in the above scanning method is executed multiple times, and then step S2 is executed.

[0070] Finally, an explanation is given for the scanning method of the handheld scanner provided in this embodiment: The scanning method provided in this embodiment is the speckle scanning mode of the handheld scanner, and its main work is as follows: The first black-and-white camera and the second black-and-white camera synchronously acquire the speckle patterns modulated by the surface of the object to be measured and transmit them to the computer for 3D reconstruction; the speckle projector stops working, and the texture camera acquires the texture map of the surface of the object to be measured and transmits it to the computer for texture reconstruction (it should be emphasized that in the speckle scanning mode, it can be to acquire one speckle pattern and then acquire one texture map, or it can be to acquire several speckle patterns and then acquire one texture map); the speckle patterns acquired by the first black-and-white camera and the second black-and-white camera are used for point cloud reconstruction based on the internal and external parameters of the preset two black-and-white cameras; finally, the texture map is used to perform texture mapping on the point cloud to obtain the final 3D scan image.

[0071] In the seventh embodiment of the present invention, a scanning method of a handheld scanner is provided. This scanning method is applied to the handheld scanner described in any of the above embodiments, specifically as Figure 4 shown, the scanning method includes:

[0072] Step S301: Use a speckle projector to project a speckle pattern onto the surface of the object to be measured, and use the first black-and-white camera and the second black-and-white camera to synchronously acquire the target speckle pattern modulated by the surface of the object to be measured, where the target speckle pattern is used for 3D reconstruction of the object to be measured.

[0073] Step S302: Stop the speckle projector from projecting the speckle pattern onto the surface of the object to be measured, and use a laser projector to project a laser beam onto the surface of the object to be measured, and use the first black-and-white camera and the second black-and-white camera to synchronously acquire the laser line pattern modulated by the surface of the object to be measured, where the laser line pattern is used for 3D reconstruction of the object to be measured.

[0074] Step S303: Stop the laser projector from projecting the laser beam onto the surface of the object to be measured, and use the texture camera to acquire the texture map of the surface of the object to be measured; where the texture map is used for texture reconstruction of the object to be measured.

[0075] Step S304: Based on the internal and external parameters of the first black-and-white camera and the second black-and-white camera, perform point cloud reconstruction on the target speckle patterns obtained by the first black-and-white camera and the second black-and-white camera; and perform point cloud reconstruction on the laser line patterns obtained by the first black-and-white camera and the second black-and-white camera.

[0076] Step S305: Perform screening processing on the first point cloud obtained by point cloud reconstruction of the target speckle patterns and the second point cloud obtained by point cloud reconstruction of the laser line patterns to obtain the target point cloud; and perform texture mapping on the target point cloud using the texture map to obtain the final 3D scan image.

[0077] Finally, an explanation is given for the scanning method of the handheld scanner provided in this embodiment: The scanning method provided in this embodiment is a combined mode of laser scanning and speckle scanning of the handheld scanner.

[0078] In the eighth embodiment of the present invention, an example is given for the application mode of the handheld scanner described in any of the above embodiments:

[0079] The handheld scanner is composed of at least two scanning methods. Here, laser scanning and speckle scanning are taken as examples. The handheld scanner includes no less than 2 black-and-white cameras, a texture camera, a corresponding ring-shaped fill light group, a laser, and a speckle projector.

[0080] Speckle scanning first mode: Speckle projector + 2 black-and-white cameras + texture camera + corresponding fill light.

[0081] Laser scanning second mode: Laser projector + 2 black-and-white cameras + texture camera + corresponding fill light.

[0082] Among them, the first operation process: Hold the scanner, perform 3D and texture scanning in the first-mode speckle scanning, and then perform supplementary scanning on the parts that need to be presented in detail or cannot be obtained due to dark color or reflection using the second-mode laser scanning. The algorithm performs screening on the 3D point cloud according to the obtained data, primarily retaining the point cloud obtained by laser scanning; after the point cloud fusion is completed, texture image screening and fusion are performed from the texture maps obtained in the first and second modes according to the shooting angle and the highlight degree of the image, so as to obtain the texture image of the overall point cloud.

[0083] Among them, the second operation process: Hold the scanner, directly perform second-mode scanning on high-reflective or dark objects to obtain the texture at the same time. After the point cloud fusion is completed, texture image screening and fusion are performed according to the shooting angle and the highlight degree of the image, so as to obtain the texture image of the overall point cloud.

[0084] At this time, it can be known that the handheld scanner in this embodiment can achieve the following technical effects: 1. It can obtain the color information of the corresponding point cloud in real time during laser scanning; 2. It can fuse and output the point clouds and color texture images of multiple scanning modes, ensuring that three-dimensional data and color information can be directly and quickly obtained without powder spraying treatment for objects with dark or reflective surfaces.

[0085] In the above embodiments of the present invention, the speckle projector projects a speckle pattern, and the laser projector projects a laser, preferably a laser stripe pattern.

[0086] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0087] In the above embodiments of the present invention, the descriptions of each embodiment have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0088] The above is only the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A hand-held scanner, characterized in that, Comprising: A texture camera (10); A first black-and-white camera (21) and a second black-and-white camera (22), the first black-and-white camera (21) and the second black-and-white camera (22) being spaced apart; A laser projector (30), the texture camera (10) and the first black-and-white camera (21) being respectively disposed on both sides of the laser projector (30); The handheld scanner is used to perform the following steps: Project laser light onto the surface of the object to be measured by the laser projector (30), and synchronously acquire the laser line pattern modulated by the surface of the object to be measured using the first black-and-white camera (21) and the second black-and-white camera (22), wherein the laser line pattern is used for three-dimensional reconstruction of the object to be measured; Stop the laser projector (30) from projecting laser light onto the surface of the object to be measured, and use the texture camera (10) to acquire the texture map of the surface of the object to be measured; wherein the texture map is used for texture reconstruction of the object to be measured; Based on the internal and external parameters of the first black-and-white camera (21) and the second black-and-white camera (22), perform point cloud reconstruction on the laser line patterns acquired by the first black-and-white camera (21) and the second black-and-white camera (22); Based on the internal and external parameters of the first black-and-white camera (21), the second black-and-white camera (22) and the texture camera (10), determine the correspondence between each pixel in the texture map and each point in the point cloud; Process the point clouds of all frames using the following steps to obtain the final three-dimensional scanned image: Perform stitching processing on the current frame point cloud and the previous frame point cloud; and, at least based on the gray-scale change of the texture maps corresponding to the current frame point cloud and the previous frame point cloud, determine the first region of the current frame point cloud, compare the multiple texture maps corresponding to the first region, determine the target texture map for frontal shooting of the corresponding position of the first region on the object to be measured, use the target texture map to perform texture mapping processing on the first region, and determine the texture mapping image of the first region.

2. The handheld scanner according to claim 1, wherein Further comprising: A speckle projector (40), the speckle projector (40) being spaced apart from the laser projector (30).

3. The hand-held scanner according to claim 2, wherein Further comprising at least one of the following: A first scanning unit (60), the laser projector (30) and the first black-and-white camera (21) being disposed on the first scanning unit (60), the laser projector (30) and the first black-and-white camera (21) being adjacent to each other; A second scanning unit (70), the speckle projector (40) and the second black-and-white camera (22) being disposed on the second scanning unit (70); And, in the case where the handheld scanner includes the first scanning unit (60) and the second scanning unit (70), the handheld scanner further includes: a handheld part (80), the first scanning unit (60) and the second scanning unit (70) being respectively disposed at both ends of the handheld part (80).

4. The hand-held scanner according to claim 1, characterized in that, There is an exit angle A between the exit light paths of the first black-and-white camera (21) and the second black-and-white camera (22), wherein the exit angle A is between 5° and 20°.

5. The hand-held scanner according to claim 1, characterized in that, Further comprising at least one of the following: The first fill light (51), where there are multiple first fill lights (51), and the multiple first fill lights (51) are arranged around the first black and white camera (21); The second fill light (52), where there are multiple second fill lights (52), and the multiple second fill lights (52) are arranged around the second black and white camera (22); The third fill light (53), where there are multiple third fill lights (53), and the multiple third fill lights (53) are arranged around the texture camera (10).

6. The hand-held scanner according to claim 1, wherein It further includes: A third black and white camera (23), and the third black and white camera (23) is spaced apart from both the first black and white camera (21) and the second black and white camera (22).

7. A scanning method for a handheld scanner, characterized in that, The scanning method is applied to the handheld scanner described in any one of the above claims 1 - 6, where the scanning method includes: Using a laser projector (30) to project laser light onto the surface of the object to be measured, and using the first black and white camera (21) and the second black and white camera (22) to synchronously obtain the laser line images modulated by the surface of the object to be measured, where the laser line images are used for three - dimensional reconstruction of the object to be measured; Stop the laser projector (30) from projecting laser light onto the surface of the object to be measured, and use the texture camera (10) to obtain the texture image of the surface of the object to be measured; where the texture image is used for texture reconstruction of the object to be measured; Based on the internal and external parameters of the first black and white camera (21) and the second black and white camera (22), perform point cloud reconstruction on the laser line images obtained by the first black and white camera (21) and the second black and white camera (22); Based on the internal and external parameters of the first black and white camera (21), the second black and white camera (22), and the texture camera (10), determine the correspondence between each pixel in the texture image and each point in the point cloud; Use the following steps to process the point clouds of all frames to obtain the final three - dimensional scan image: Perform stitching processing on the current frame point cloud and the previous frame point cloud; and at least based on the gray - scale change of the texture images corresponding to the current frame point cloud and the previous frame point cloud, determine the first area of the current frame point cloud, compare the multiple texture images corresponding to the first area, determine the target texture image for frontal shooting of the corresponding position of the first area on the object to be measured, and use the target texture image to perform texture mapping processing on the first area to determine the texture - mapped image of the first area.

8. The scanning method according to claim 7, wherein Based on the internal and external parameters of the first black and white camera (21), the second black and white camera (22), and the texture camera (10), determining the correspondence between each pixel in the texture image and each point in the point cloud includes: Based on the internal and external parameters of the first black and white camera (21), the second black and white camera (22), and the texture camera (10), determine the correspondence between each pixel in the laser line images obtained by the first black and white camera (21) and the second black and white camera (22) and each pixel in the texture image obtained by the texture camera (10); Determine the correspondence between each pixel in the texture map and each point in the point cloud according to the correspondence between each pixel in the laser line map and the texture map and the correspondence between each pixel in the laser line map and each point in the point cloud.

9. The scanning method according to claim 7, wherein The gray-scale change of the first region satisfies a preset condition.

10. The scanning method according to claim 7, characterized in that, Through the circuit design in the handheld scanner, the time interval between obtaining the laser line map modulated by the surface of the object to be measured using the first black-and-white camera (21) and the second black-and-white camera (22) and obtaining the texture map of the surface of the object to be measured using the texture camera (10) is lower than a preset threshold.

11. A scanning method for a handheld scanner, characterized in that, The scanning method is applied to the handheld scanner described in any one of the above claims 2-6, wherein the scanning method includes: Step S1: Project a speckle pattern onto the surface of the object to be measured using a speckle projector (40), and synchronously obtain a target speckle pattern modulated by the surface of the object to be measured using the first black-and-white camera (21) and the second black-and-white camera (22), wherein the target speckle pattern is used for three-dimensional reconstruction of the object to be measured; Step S2: Stop the speckle projector (40) from projecting the speckle pattern onto the surface of the object to be measured, and use the texture camera (10) to obtain the texture map of the surface of the object to be measured; wherein the texture map is used for texture reconstruction of the object to be measured; Step S3: Based on the internal and external parameters of the first black-and-white camera (21) and the second black-and-white camera (22), perform point cloud reconstruction on the target speckle pattern obtained by the first black-and-white camera (21) and the second black-and-white camera (22); Based on the internal and external parameters of the first black-and-white camera (21), the second black-and-white camera (22), and the texture camera (10), determine the correspondence between each pixel in the texture map and each point in the point cloud; Use the following steps to process the point clouds of all frames to obtain the final three-dimensional scanned image: Perform stitching processing on the current frame point cloud and the previous frame point cloud; and, at least based on the gray-scale change of the texture maps corresponding to the current frame point cloud and the previous frame point cloud, determine the first region of the current frame point cloud, compare the multiple texture maps corresponding to the first region, determine the target texture map for frontal shooting at the corresponding position of the first region on the object to be measured, and use the target texture map to perform texture mapping processing on the first region to determine the texture mapped image of the first region.

12. The scanning method according to claim 11, wherein, Steps S1 and S2 in the scanning method are alternately executed; or, step S1 in the scanning method is executed multiple times, and then step S2 is executed.

13. A scanning method for a handheld scanner, characterized in that, The scanning method is applied to the handheld scanner described in any one of the above claims 2-6, wherein the scanning method includes: Project a speckle pattern onto the surface of the object to be measured using a speckle projector (40), and synchronously obtain a target speckle pattern modulated by the surface of the object to be measured using the first black-and-white camera (21) and the second black-and-white camera (22), wherein the target speckle pattern is used for three-dimensional reconstruction of the object to be measured; Stop the speckle projector (40) from projecting a speckle pattern onto the surface of the object to be measured, and use the laser projector (30) to project a laser beam onto the surface of the object to be measured, and use the first black-and-white camera (21) and the second black-and-white camera (22) to synchronously obtain the laser line pattern modulated by the surface of the object to be measured, wherein the laser line pattern is used for three-dimensional reconstruction of the object to be measured; Stop the laser projector (30) from projecting a laser beam onto the surface of the object to be measured, and use the texture camera (10) to obtain the texture map of the surface of the object to be measured; wherein the texture map is used for texture reconstruction of the object to be measured; Based on the internal and external parameters of the first black-and-white camera (21) and the second black-and-white camera (22), perform point cloud reconstruction on the target speckle patterns obtained by the first black-and-white camera (21) and the second black-and-white camera (22); and perform point cloud reconstruction on the laser line patterns obtained by the first black-and-white camera (21) and the second black-and-white camera (22); Perform screening processing on the first point cloud obtained by point cloud reconstruction of the target speckle pattern and the second point cloud obtained by point cloud reconstruction of the laser line pattern to obtain the target point cloud; Based on the internal and external parameters of the first black-and-white camera (21), the second black-and-white camera (22) and the texture camera (10), determine the correspondence between each pixel in the texture map and each point in the point cloud; Process the point clouds of all frames by the following steps to obtain the final three-dimensional scanned image: Perform stitching processing on the current frame point cloud and the previous frame point cloud; and at least based on the gray change of the texture map corresponding to the current frame point cloud and the previous frame point cloud, determine the first area of the current frame point cloud, compare the multiple texture maps corresponding to the first area, determine the target texture map for frontal shooting at the corresponding position of the first area on the object to be measured, use the target texture map to perform texture mapping processing on the first area, and determine the texture mapped image of the first area.

Images