Three-dimensional scanning method, three-dimensional scanning system, electronic device and computer device

Abstract

The application relates to a three-dimensional scanning method, a three-dimensional scanning system, an electronic device and a computer device, wherein the three-dimensional scanning method comprises the following steps: performing three-dimensional reconstruction on a group of current frame scanning images to obtain a current frame first point cloud, and performing feature point identification and feature point matching on the group of current frame scanning images based on a feature pattern to obtain a current frame second point cloud; determining a conversion relationship between the current frame first point cloud and a world coordinate system to obtain a current frame conversion relationship; finally, performing point cloud splicing based on a latest conversion relationship set and a latest first point cloud set, and performing point cloud fusion based on the latest conversion relationship set and a latest second point cloud set to obtain point cloud model data of a target object. The application combines two kinds of point cloud data with different uses to complete point cloud splicing and fusion, thereby realizing accurate point cloud model reconstruction based on a single feature pattern and improving the model reconstruction effect under single feature pattern projection.

Description

Technical Field

[0001] This application relates to the field of 3D scanning, and in particular to 3D scanning methods, 3D scanning systems, electronic devices, and computer equipment. Background Technology

[0002] Structured light 3D reconstruction is used for acquiring 3D shapes. It typically involves projecting specially coded structured light patterns onto an object and using a camera to capture the reflected light to reconstruct the object's 2D shape. Among these methods, speckle structured light reconstruction is commonly used to achieve this process. Speckle is a phenomenon where light intensity distribution is uneven due to reflection, refraction, and scattering on an object's surface. In structured light 3D reconstruction, the speckle phenomenon can be used to obtain information about minute deformations on the object's surface, thereby enabling 3D reconstruction of the object's shape.

[0003] Currently, the reconstruction of point cloud models by directly projecting a single type of feature pattern and then directly reconstructing the point cloud model based on the scanned data collected from the projected feature pattern results in poor reconstruction quality of the object's point cloud model.

[0004] There is currently no effective solution to the problem of poor model reconstruction results when using feature pattern projection based on a single type in related technologies. Summary of the Invention

[0005] This embodiment provides a three-dimensional scanning method, a three-dimensional scanning system, an electronic device, and a computer equipment to solve the problem of poor model reconstruction effect based on single-type feature pattern projection in related technologies.

[0006] Firstly, this embodiment provides a three-dimensional scanning method, including:

[0007] A three-dimensional scan of the target object is performed to obtain a set of current frame scan images of the target object; wherein, the current frame scan images contain feature patterns projected onto the surface of the target object;

[0008] Three-dimensional reconstruction is performed on the set of current frame scan images to obtain the first point cloud of the current frame. Based on the feature pattern, feature points are identified and matched on the set of current frame scan images to reconstruct the second point cloud of the current frame. The first point cloud of the current frame is updated to the first point cloud set, and the second point cloud of the current frame is updated to the second point cloud set.

[0009] Based on the point cloud registration algorithm, the transformation relationship between the first point cloud of the current frame and the world coordinate system is determined, the transformation relationship of the current frame is obtained, and the transformation relationship of the current frame is updated to the transformation relationship set;

[0010] Obtain a set of next-frame scan images of the target object and update the first point cloud set, the second point cloud set, and the transformation relationship set as new current-frame scan images;

[0011] After updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set, and point cloud fusion is performed based on the latest transformation relationship set and the latest second point cloud set to obtain the point cloud model data of the target object.

[0012] In some embodiments, the step of performing 3D reconstruction on the set of current frame scan images to obtain the first point cloud of the current frame includes:

[0013] Based on a stereo vision matching algorithm, a three-dimensional reconstruction is performed on the set of current frame scan images to obtain the first point cloud of the current frame.

[0014] In some embodiments, after updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set, and point cloud fusion is performed based on the latest transformation relationship set and the latest second point cloud set to obtain the point cloud model data of the target object, including:

[0015] After updating the first point cloud set, the second point cloud set, and the transformation relationship set, global registration is performed based on the latest first point cloud set, and the transformation relationship set is corrected based on the global registration result.

[0016] Point cloud stitching is performed based on the corrected set of transformation relationships and the latest first point cloud set, and point cloud fusion is performed based on the corrected set of transformation relationships and the latest second point cloud set to obtain the point cloud model data of the target object.

[0017] In some embodiments, after updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set, and point cloud fusion is performed based on the latest transformation relationship set and the latest second point cloud set to obtain the point cloud model data of the target object, including:

[0018] After updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set. Global point cloud fusion is then performed based on the latest first point cloud set, the latest second point cloud set, and the latest transformation relationship set to obtain the point cloud model data of the target object.

[0019] In some embodiments, after updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set, and global point cloud fusion is performed based on the latest first point cloud set, the latest second point cloud set, and the latest transformation relationship set to obtain the point cloud model data of the target object, including:

[0020] After updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set.

[0021] Based on the weights assigned to the first point cloud set and the second point cloud set, the latest first point cloud set and the latest second point cloud set, and the latest transformation relationship set, a weighted global point cloud fusion is performed to obtain the point cloud model data of the target object.

[0022] In some of these embodiments, the feature pattern on the surface of the target object is formed based on projection from a single projector.

[0023] Secondly, this embodiment provides a three-dimensional scanning system, including: an image acquisition unit, a processor, and a projector; wherein the projector and the image acquisition unit are communicatively connected to the processor;

[0024] The projector is used to project a feature pattern onto the surface of a target object;

[0025] The image acquisition device is used to acquire a scanned image of the surface of the target object containing the feature pattern;

[0026] The processor is used to execute the three-dimensional scanning method described in the first aspect.

[0027] Fourthly, this embodiment provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the three-dimensional scanning method described in the first aspect above.

[0028] Fifthly, this embodiment provides a storage medium storing a computer program that, when executed by a processor, implements the three-dimensional scanning method described in the first aspect above.

[0029] Compared with related technologies, this embodiment provides a three-dimensional scanning method, a three-dimensional scanning system, an electronic device, and a computer equipment. The 3D scanning method first performs a 3D scan of the target object to obtain a set of current frame scan images of the target object. These current frame scan images contain feature patterns projected onto the surface of the target object. Next, a 3D reconstruction is performed on the set of current frame scan images to obtain the first point cloud of the current frame. Based on the feature patterns, feature points are identified and matched in the set of current frame scan images to reconstruct the second point cloud of the current frame. The first point cloud of the current frame is updated to the first point cloud set, and the second point cloud of the current frame is updated to the second point cloud set. Based on a point cloud registration algorithm, the transformation relationship between the first point cloud of the current frame and the world coordinate system is determined, obtaining the current frame transformation relationship, which is then updated to the transformation relationship set. A set of next frame scan images of the target object is obtained and used as new current frame scan images to update the first point cloud set, the second point cloud set, and the transformation relationship set. Finally, after updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set, and point cloud fusion is performed based on the latest transformation relationship set and the latest second point cloud set to obtain the point cloud model data of the target object. It combines point cloud data of two different purposes to complete point cloud stitching and fusion. Even based on a single feature pattern projection, it can achieve accurate reconstruction of detailed features, thereby achieving accurate reconstruction of point cloud models based on a single feature pattern and improving the model reconstruction effect under single feature pattern projection.

[0030] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0031] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0032] Figure 1 This is a hardware structure block diagram of the terminal of the three-dimensional scanning method in this embodiment;

[0033] Figure 2 This is a flowchart of the three-dimensional scanning method in this embodiment;

[0034] Figure 3a This is a schematic diagram of a feature pattern in this embodiment;

[0035] Figure 3b This is another schematic diagram of the feature pattern in this embodiment;

[0036] Figure 4 This is a flowchart of another three-dimensional scanning method in this embodiment;

[0037] Figure 5 This is a schematic diagram of the structure of the three-dimensional scanning system in this embodiment. Detailed Implementation

[0038] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0039] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0040] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the 3D scanning method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0041] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the 3D scanning method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the aforementioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0042] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0043] This embodiment provides a three-dimensional data processing method. Figure 2 This is a flowchart of the three-dimensional data processing method in this embodiment, such as... Figure 2 As shown, the process includes the following steps:

[0044] Step S210: Perform a three-dimensional scan on the target object to obtain a set of current frame scan images of the target object; wherein, the current frame scan images contain feature patterns projected onto the surface of the target object.

[0045] This process involves simultaneously imaging the target object using at least two image acquisition devices to obtain a set of current frame scan images. A single imaging operation can acquire a set of scan images of the target object within the same frame, including images from different image acquisition devices. Understandably, if the target object is simultaneously imaged multiple times using at least two image acquisition devices, multiple frames of scan images can be obtained, with scan images belonging to the same frame forming a set. Specifically, when imaging the target object, a projector projects a feature pattern onto the surface of the target object, and then each image acquisition device acquires a surface image of the target object containing the feature pattern, thus obtaining the aforementioned scan images. The projector can be an infrared projector, such as a vertical-cavity surface-emitting laser (VCSEL) capable of projecting speckle patterns. Alternatively, the projector can be other visible light projectors. When using a VCSEL as the projector, a monochrome camera can be used as the image acquisition device; when using other visible light projectors, a color camera can be used. For example, a VCSEL is used in conjunction with two monochrome cameras to perform 3D reconstruction of a target object. By simultaneously triggering the VCSEL and the two monochrome cameras, two grayscale images are obtained in a single imaging operation; these two grayscale images constitute a set of scanned images. The feature pattern contained in the scanned images can be a speckle pattern. Specifically, when obtaining the first point cloud of the current frame through 3D reconstruction, the feature pattern projected onto the target object needs to address the matching problem between images from different image acquisition devices. Furthermore, the feature pattern is also used for feature recognition to perform feature reconstruction. Therefore, the aforementioned feature pattern needs to possess both the function of assisting in image matching between different image acquisition devices and the function of facilitating feature recognition to complete feature reconstruction. For example, based on binocular vision matching, when acquiring the first point cloud of the current frame, the left and right cameras need to use the feature pattern to complete image matching; furthermore, when acquiring the second point cloud of the current frame, it is necessary to identify and match the feature points in the feature pattern to reconstruct the second point cloud of the current frame. It is understood that feature patterns possessing both of these functions are applicable to this embodiment.

[0046] For example, the aforementioned feature pattern can be a dot-shaped pattern, a pattern with clear corners, or a pattern with clear edges, or other patterns with fine features. It is understood that those skilled in the art can select other types of projectors, image acquisition devices, and feature patterns based on actual application scenarios, and this embodiment does not specifically limit these selections.

[0047] Figure 3a This is a schematic diagram of a feature pattern in this embodiment. Figure 3b This is a schematic diagram of another feature pattern in this embodiment. For example... Figure 3aAs shown, the feature patterns exhibit relatively fine corner and edge features. For example, Figure 3a The corner point is indicated by the hollow circle 'a', and the edge is indicated by the dashed line segment 'b'. For example... Figure 3b As shown, the feature pattern can be a dot-shaped pattern, and box c represents one type of extractable dot feature (for ease of display). Figure 3b (The image is the actual feature pattern after inversion).

[0048] Step S220: Perform 3D reconstruction on a set of current frame scan images to obtain the first point cloud of the current frame, and perform feature point identification and feature point matching on a set of current frame scan images based on feature patterns to reconstruct the second point cloud of the current frame. Update the first point cloud of the current frame to the first point cloud set, and update the second point cloud of the current frame to the second point cloud set.

[0049] This process involves stitching together first point clouds from different frames and fusing second point clouds from different frames to obtain a reconstructed point cloud model of the target object. Specifically, a set of current frame images can be 3D reconstructed to obtain the first point cloud corresponding to the current frame scan image. Then, feature points are identified and matched based on feature patterns to reconstruct the second point cloud corresponding to the current frame scan image. Different 3D reconstruction methods can be used for different feature patterns to obtain the aforementioned first point cloud. For example, when using a speckle pattern as the feature pattern, the first point cloud can be reconstructed based on the speckle pattern.

[0050] For a set of target images acquired by an image acquisition device during a single imaging process, feature points can be identified based on the characteristic patterns contained within them. The center point of the characteristic pattern can be pre-selected as the feature point, or one or more other points of the characteristic pattern can be selected, such as edge points, diagonal points, etc. For example, when using a VCSEL as a projector to project a speckle pattern in the shape of speckles onto the surface of a target object, based on the identification results of the speckle centers in each set of target images obtained from a monochrome camera, a speckle point cloud is reconstructed as a second point cloud. Based on this, this step, using scanned images from the same frame, can extract point cloud data for two different purposes.

[0051] The aforementioned first point cloud set is formed by extracting first point clouds from different frame scan images. When the image acquisition device performs a single imaging of the target object, a set of scan images of the same frame is obtained. Based on this set of scan images, 3D reconstruction is performed to obtain a set of first point clouds. After multiple imaging of the target object, multiple sets of first point clouds are obtained, and these multiple sets of first point clouds form the aforementioned first point cloud set. Similarly, multiple sets of second point clouds form the aforementioned second point cloud set. It should be noted that, since the positional relationship between multiple image acquisition devices is pre-calibrated, when extracting first and second point clouds from a set of scan images of the same frame, the resulting set of first point clouds and set of second point clouds have the same coordinate system.

[0052] Step S230: Based on the point cloud registration algorithm, determine the transformation relationship between the first point cloud of the current frame and the world coordinate system, obtain the transformation relationship of the current frame, and update the transformation relationship of the current frame to the transformation relationship set.

[0053] Since the first point cloud and the second point cloud in the current frame share the same coordinate system, the transformation relationship between the first point cloud and the world coordinate system is equivalent to the transformation relationship between the second point cloud and the world coordinate system. Grouping the first point clouds corresponding to the same scanned image frame into one group and the second point clouds corresponding to the same scanned image frame into another group, there is a one-to-one correspondence between the first point clouds in the first point cloud set and the second point clouds in the second point cloud set in the coordinate system. The transformation relationship between each group of first point clouds and the world coordinate system is consistent with the transformation relationship between the corresponding groups of second point clouds and the world coordinate system. For example, based on point cloud registration, if the transformation relationship between the first point cloud and the world coordinate system in the current frame is rt1, then the transformation relationship between the second point cloud and the world coordinate system in the current frame is also rt1.

[0054] Step S240: Obtain a set of next frame scan images of the target object and update the first point cloud set, the second point cloud set, and the transformation relationship set as new current frame scan images.

[0055] That is, a set of next-frame scan images obtained by synchronous imaging from at least two image acquisition devices is acquired, the first point cloud is extracted from it, and the transformation relationship is solved based on the point cloud registration. The second point cloud is also extracted and updated accordingly in the first point cloud set, the transformation relationship set, and the second point cloud set. Thus, the first point cloud set contains the first point clouds from different frame scan images, the second point cloud set contains the second point clouds from different frame scan images, and the transformation relationship set contains the transformation relationships solved based on the first point clouds from different frame scan images.

[0056] Step S250: After updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set, and point cloud fusion is performed based on the latest transformation relationship set and the latest second point cloud set to obtain the point cloud model data of the target object.

[0057] Specifically, once the data volume of the first and second point cloud sets reaches a preset data acquisition threshold, or once the number of frames of the acquired scanned images reaches a preset frame count threshold, the update of the first and second point cloud sets and the transformation relationship set is confirmed to be complete. At this point, point cloud stitching is performed using the latest first point cloud set and the latest transformation relationship set, and point cloud fusion is performed using the latest second point cloud set and the latest transformation relationship set. Based on point cloud stitching and fusion, a reconstructed point cloud model of the target object in the same coordinate system is obtained. This point cloud model data is the three-dimensional spatial model of the target object. Based on the first and second point cloud sets of the target object, combined with the transformation relationship set, accurate reconstruction of the three-dimensional spatial model of the target object can be achieved. Thus, even based on the projection of a single feature pattern, a point cloud model with good reconstruction results can be obtained.

[0058] Compared to related technologies that directly utilize the projection of a single feature pattern, the resulting point cloud model reconstruction is of poor quality. This embodiment can achieve accurate point cloud model reconstruction based on two different types of point cloud data while using a single feature pattern projection, thus improving the reconstruction effect of the point cloud model.

[0059] Steps S210 to S230 involve: performing a 3D scan on the target object to obtain a set of current frame scan images; wherein the current frame scan images contain feature patterns projected onto the surface of the target object; performing 3D reconstruction on the set of current frame scan images to obtain the first point cloud of the current frame; identifying and matching feature points on the set of current frame scan images based on the feature patterns to reconstruct the second point cloud of the current frame; updating the first point cloud of the current frame to the first point cloud set; updating the second point cloud of the current frame to the second point cloud set; determining the transformation relationship between the first point cloud of the current frame and the world coordinate system based on the point cloud registration algorithm to obtain the current frame transformation relationship; updating the current frame transformation relationship to the transformation relationship set; acquiring a set of next frame scan images of the target object as new current frame scan images to update the first point cloud set, the second point cloud set, and the transformation relationship set; after completing the update of the first point cloud set, the second point cloud set, and the transformation relationship set, performing point cloud stitching based on the latest transformation relationship set and the latest first point cloud set, and performing point cloud fusion based on the latest transformation relationship set and the latest second point cloud set to obtain the point cloud model data of the target object. It achieves accurate point cloud model reconstruction based on two types of point cloud data, improving the reconstruction effect of point cloud models under single feature pattern projection.

[0060] In one embodiment, based on the above step S220, performing three-dimensional reconstruction on a set of current frame scan images to obtain the first point cloud of the current frame may include:

[0061] Based on a stereo vision matching algorithm, a set of scanned images of the current frame is reconstructed in three dimensions to obtain the first point cloud of the current frame.

[0062] In this process, a single imaging operation acquires a set of scanned images within the same frame. 3D reconstruction is then performed on this set of scanned images to extract the first point cloud. This requires at least two image acquisition devices to simultaneously image the target object; therefore, a set of scanned images includes at least two images. Different 3D reconstruction methods can be selected depending on the projected feature pattern. For example, when the feature pattern projected onto the target object's surface is a speckle pattern, a speckle reconstruction algorithm can be used to extract the first point cloud.

[0063] In another embodiment, based on the above step S220, the identification and matching of feature points in a set of current frame scan images based on feature patterns to reconstruct the second point cloud of the current frame may include:

[0064] In this embodiment, feature points are extracted and matched from a set of current frame scan images to reconstruct a second point cloud. For example, a speckle pattern can be used as the feature pattern, and the center of the speckle can be used as the feature point. The center of the speckle can be extracted and matched from a set of current frame scan images to reconstruct a speckle point cloud as the second point cloud. This embodiment, based on the extraction and matching of fine features from feature images, can provide accurate basic data to improve the accuracy and detail of subsequent point cloud model reconstruction, thereby improving the effect of point cloud model reconstruction.

[0065] Specifically, in one embodiment, after updating the first point cloud set, the second point cloud set, and the transformation relation set, point cloud stitching is performed based on the latest transformation relation set and the latest first point cloud set, and point cloud fusion is performed based on the latest transformation relation set and the latest second point cloud set to obtain the point cloud model data of the target object, which may include:

[0066] After updating the first point cloud set, the second point cloud set, and the transformation relationship set, global registration is performed based on the latest first point cloud set, and the transformation relationship set is corrected based on the global registration result. Point cloud stitching is performed based on the corrected transformation relationship set and the latest first point cloud set, and point cloud fusion is performed based on the corrected transformation relationship set and the latest second point cloud set to obtain the point cloud model data of the target object.

[0067] To eliminate potential misalignments during the scanning process and obtain a more accurate set of transformation relationships, global registration of the point cloud can be performed using a first point cloud set. Any general global registration method can be used to perform this registration; this embodiment does not specify a particular global registration process. It is understood that the purpose of using global registration in this embodiment is to further improve the accuracy of the transformation relationship set. Therefore, if the transformation relationship set is determined to be relatively accurate, subsequent point cloud model reconstruction can be successfully completed even without global registration.

[0068] Furthermore, in one embodiment, based on the above step S230, after updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set, and point cloud fusion is performed based on the latest transformation relationship set and the latest second point cloud set to obtain the point cloud model data of the target object, which may include:

[0069] After updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set. Then, global point cloud fusion is performed based on the latest first point cloud set, the latest second point cloud set, and the latest transformation relationship set to obtain the point cloud model data of the target object.

[0070] In this embodiment, when reconstructing the point cloud model data of the target object, a first point cloud set is used for stitching, and a second point cloud set is used for feature fusion. Based on point cloud stitching and feature fusion, point cloud model data containing high-detail features of the target object is obtained. Specifically, point cloud stitching can be performed based on the first point cloud set and the transformation relation set, and point cloud fusion can be performed based on the first point cloud set, the second point cloud set, and the transformation relation set to finally obtain the point cloud model data. Based on this, this embodiment can achieve accurate and detailed point cloud model reconstruction even when using a single VCSEL, thereby improving the reconstruction effect of point cloud models under single feature pattern projection.

[0071] In one embodiment, after updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set. Global point cloud fusion is then performed based on the latest first point cloud set, the latest second point cloud set, and the latest transformation relationship set to obtain the point cloud model data of the target object. This may include:

[0072] After updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set. According to the weights assigned to the first and second point cloud sets, the latest first and second point cloud sets, and the latest transformation relationship set, weighted global point cloud fusion is performed to obtain the point cloud model data of the target object.

[0073] Specifically, different weights can be assigned to the first and second point cloud sets based on the desired reconstruction effect in the actual application scenario. When a more obvious feature pattern display effect is required, the weight of the second point cloud set can be set to 1, and the weight of the first point cloud set can be set to 0. When a more obvious planar point cloud display effect is required, the weight of the first point cloud set can be increased, and the weight of the second point cloud set can be decreased. Based on this, this embodiment can achieve accurate and detailed reconstruction of a 3D spatial model based on the reconstruction of planar point clouds and fine features.

[0074] In one embodiment, the feature pattern on the surface of the target object is formed by projection from a single projector. When using multiple projectors to project the feature pattern, the coordination between different projectors needs to be considered to avoid feature pattern overlap. Furthermore, feature patterns projected by multiple projectors will complicate the matching and recognition steps between scanned images. Therefore, this embodiment uses a single projector for feature pattern projection, which avoids introducing too many devices during 3D scanning, thus reducing equipment costs; it also eliminates the need to consider feature pattern overlap issues in scenarios with multiple projectors, reduces the complexity of image matching and recognition, and thus simplifies the overall 3D scanning process.

[0075] Figure 4 This is a flowchart of another three-dimensional scanning method in this embodiment, such as... Figure 4 As shown, the three-dimensional scanning method includes the following steps:

[0076] Step S401: Obtain two grayscale images of the target object as a set of current frame scan images; wherein, the speckle projector and two black and white cameras are triggered simultaneously to obtain two grayscale images, and the speckle projector projects a dotted speckle pattern.

[0077] Step S402: Perform speckle reconstruction on the two grayscale images to obtain the first point cloud of the current frame; the first point cloud is a planar point cloud.

[0078] Step S403: Identify and match the center of the spots in the two grayscale images to reconstruct the second point cloud of the current frame; wherein the first point cloud of the current frame and the second point cloud of the current frame are in the same coordinate system.

[0079] Step S404: Based on the point cloud registration algorithm, calculate the transformation relationship between the first point cloud in the current frame and the world coordinate system; at this time, the transformation relationship between the second point cloud in the current frame and the world coordinate system is the same as the transformation relationship between the first point cloud in the current frame and the world coordinate system.

[0080] Step S405: Repeat steps S401 to S404 to obtain a set of first point clouds formed by multiple sets of first point clouds, a set of second point clouds formed by multiple sets of second point clouds, and a set of transformation relationships formed by multiple sets of transformation relationships; wherein, based on steps S401 to S404, planar point clouds and speckle point clouds are extracted from other frames of scanned images, and the first point clouds and second point clouds of each frame are obtained respectively, and the transformation relationship between the first point clouds of each frame and the world coordinate system is obtained, thereby obtaining the above-mentioned set of transformation relationships.

[0081] Step S406: Perform global registration of the point cloud using the first point cloud set; wherein, based on global registration, misalignment of the point cloud set can be eliminated, and the accuracy of the transformation relationship set can be improved.

[0082] Step S407: Point cloud stitching is performed using the first point cloud set and the transformation relation set. Global point cloud fusion is then performed based on the first point cloud set, the second point cloud set, and the transformation relation set to obtain point cloud model data. Different weights are assigned to the first and second point cloud sets during the fusion process.

[0083] This embodiment also provides a three-dimensional data processing apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that perform a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0084] This embodiment also provides a three-dimensional scanning system. Figure 5 This is a schematic diagram of the structure of the three-dimensional scanning system 50 in this embodiment, as shown below. Figure 5 As shown, the 3D scanning system 50 includes: a projector 52, an image acquisition unit 54, and a processor 56; wherein the projector 52 and the image acquisition unit 54 are communicatively connected to the processor 56; the projector 52 is used to project a feature pattern onto the surface of a target object; the image acquisition unit 54 is used to acquire a target image containing the feature pattern on the surface of the target object; and the processor 56 is used to execute the 3D scanning method provided in any of the above embodiments.

[0085] The projector 52 can be an infrared projector, such as a VSCEL, or other visible light projectors. The image acquisition unit 54 can be at least two, specifically a monochrome camera or a color camera. When the projector 52 is an infrared projector, the image acquisition unit 54 is a monochrome camera; when the projector 52 is a visible light projector, the image acquisition unit 54 is a color camera. The number of projectors 52 is not limited; one or more projectors 52 can be configured according to the needs of the actual application scenario.

[0086] This embodiment also provides an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0087] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0088] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0089] S1, Perform a 3D scan on the target object to obtain a set of current frame scan images of the target object; wherein, the current frame scan images contain feature patterns projected onto the surface of the target object;

[0090] S2, perform 3D reconstruction on a set of current frame scan images to obtain the first point cloud of the current frame, and perform feature point identification and feature point matching on a set of current frame scan images based on feature patterns to reconstruct the second point cloud of the current frame, update the first point cloud of the current frame to the first point cloud set, and update the second point cloud of the current frame to the second point cloud set.

[0091] S3, based on the point cloud registration algorithm, determines the transformation relationship between the first point cloud of the current frame and the world coordinate system, obtains the transformation relationship of the current frame, and updates the transformation relationship of the current frame to the transformation relationship set;

[0092] S4, obtain a set of next frame scan images of the target object, and use them as new current frame scan images to update the first point cloud set, the second point cloud set, and the transformation relationship set;

[0093] S5. After updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set, and point cloud fusion is performed based on the latest transformation relationship set and the latest second point cloud set to obtain the point cloud model data of the target object.

[0094] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0095] Furthermore, in conjunction with the three-dimensional data processing methods provided in the above embodiments, this embodiment can also provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements any of the three-dimensional data processing methods described in the above embodiments.

[0096] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0097] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0098] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0099] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0100] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A three-dimensional scanning method, characterized in that, include: A three-dimensional scan of the target object is performed to obtain a set of current frame scan images of the target object; wherein, the current frame scan images contain feature patterns projected onto the surface of the target object; Based on the stereo vision matching algorithm, the set of current frame scan images are reconstructed in three dimensions to obtain the first point cloud of the current frame. Based on the feature pattern, the set of current frame scan images are used to identify and match feature points to reconstruct the second point cloud of the current frame. The first point cloud of the current frame is updated to the first point cloud set, and the second point cloud of the current frame is updated to the second point cloud set. Based on the point cloud registration algorithm, the transformation relationship between the first point cloud of the current frame and the world coordinate system is determined, the transformation relationship of the current frame is obtained, and the transformation relationship of the current frame is updated to the transformation relationship set; Obtain a set of next-frame scan images of the target object and update the first point cloud set, the second point cloud set, and the transformation relationship set as new current-frame scan images; After updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set, and point cloud fusion is performed based on the latest transformation relationship set and the latest second point cloud set to obtain the point cloud model data of the target object, including: After updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set. Based on the weights assigned to the first point cloud set and the second point cloud set, the latest first point cloud set and the latest second point cloud set, and the latest transformation relationship set, a weighted global point cloud fusion is performed to obtain the point cloud model data of the target object.

2. The three-dimensional scanning method according to claim 1, characterized in that, After updating the first point cloud set, the second point cloud set, and the transformation relationship set, point cloud stitching is performed based on the latest transformation relationship set and the latest first point cloud set, and point cloud fusion is performed based on the latest transformation relationship set and the latest second point cloud set to obtain the point cloud model data of the target object, including: After updating the first point cloud set, the second point cloud set, and the transformation relationship set, global registration is performed based on the latest first point cloud set, and the transformation relationship set is corrected based on the global registration result. Point cloud stitching is performed based on the corrected set of transformation relationships and the latest first point cloud set, and point cloud fusion is performed based on the corrected set of transformation relationships and the latest second point cloud set to obtain the point cloud model data of the target object.

3. The three-dimensional scanning method according to any one of claims 1 to 2, characterized in that, The feature pattern on the surface of the target object is formed by projection from a single projector.

4. A three-dimensional scanning system, characterized in that, include: An image acquisition unit, a processor, and a projector; wherein the projector and the image acquisition unit are communicatively connected to the processor; The projector is used to project a feature pattern onto the surface of a target object; The image acquisition device is used to acquire a scanned image of the surface of the target object containing the feature pattern; The processor is used to execute the three-dimensional scanning method according to any one of claims 1 to 3.

5. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the three-dimensional scanning method according to any one of claims 1 to 3.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the three-dimensional scanning method according to any one of claims 1 to 3.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the three-dimensional scanning method according to any one of claims 1 to 3.

Images