Three-dimensional scanning methods, systems, devices, and media

Abstract

Embodiments of the present application disclose a three-dimensional scanning method, system, device and medium. The method comprises: obtaining a computer-aided model of a scanning object, wherein the computer-aided model comprises a plurality of segmentation regions, and each segmentation region defines a corresponding scanning resolution; obtaining real-time scanning data comprising the scanning object captured by a scanning device; registering the real-time scanning data with the computer-aided model to determine one or more target segmentation regions corresponding to the real-time scanning data from the plurality of segmentation regions; and generating at least a partial mesh of the scanning object based on the real-time scanning data, wherein a mesh resolution of the at least a partial mesh is determined by one or more scanning resolutions of the one or more target segmentation regions. The method of the embodiments of the present application can improve scanning efficiency while ensuring the quality of the reconstructed mesh model.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to three-dimensional scanning methods, systems, devices and media. Background Technology

[0002] In the fields of inspection and digitization, high-precision 3D scanning typically requires significant time and computational resources. Related methods employ uniform resolution scanning, which leads to wasted scanning resources in simple geometric areas while potentially resulting in insufficient resolution in complex feature regions. Summary of the Invention

[0003] In a first aspect, embodiments of this application provide a three-dimensional scanning method, including: Obtain a computer-aided model of the scanned object, wherein the computer-aided model includes multiple segmented regions, and each segmented region defines a corresponding scanning resolution; Acquire real-time scan data, including the object being scanned, captured by the scanning device; The real-time scanning data is registered with the computer-aided model to determine one or more target segmentation regions corresponding to the real-time scanning data from the plurality of segmentation regions; and At least a portion of the mesh of the scanned object is generated based on the real-time scan data, wherein the mesh resolution of the at least portion of the mesh is determined by one or more scan resolutions of the one or more target segmentation regions.

[0004] In some embodiments, acquiring a computer-aided model of the scanned object includes: Obtain the initial auxiliary model of the scanned object; Extract the feature boundaries of the initial auxiliary model, and perform mesh normal vector clustering on the initial auxiliary model to obtain multiple mesh sets; Using the feature boundaries as constraints, the initial auxiliary model is divided into multiple first regions based on the adjacent grids in each of the grid sets; The initial auxiliary model is divided into multiple second regions, where the area of ​​each region is greater than a first area threshold, and the area of ​​each region is less than a second area threshold. Determine the scanning resolution corresponding to each of the second regions to obtain candidate 3D reference models; The candidate 3D reference model is spatially partitioned using a balanced octree to obtain the computer-aided model.

[0005] In some embodiments, the step of extracting the feature boundaries of the initial auxiliary model and performing mesh normal vector clustering on the initial auxiliary model to obtain multiple mesh sets includes: The feature boundary of the initial auxiliary model is extracted based on the dihedral angle of the mesh patch corresponding to the edge information of the initial auxiliary model; Based on the normal vectors of each mesh patch in the initial auxiliary model, the mesh patches are clustered to obtain the multiple mesh sets.

[0006] In some embodiments, segmenting regions in the plurality of first regions whose areas are greater than a first area threshold includes: Calculate the average curvature and Gaussian curvature of each grid vertex in the region; The region is segmented based on the average curvature, the Gaussian curvature, and the curvature threshold.

[0007] In some embodiments, determining the scanning resolution corresponding to each of the second regions includes: Obtain the user's annotation operations for each of the second regions, and determine the scanning resolution corresponding to each of the second regions; Alternatively, a scanning resolution may be assigned to each of the second regions based on the curvature of the corresponding region.

[0008] In some embodiments, registering the real-time scanning data with the computer-aided model to determine one or more target segmentation regions corresponding to the real-time scanning data from the plurality of segmentation regions includes: The scanning feature vector of the real-time scanning data is extracted using a machine learning model; Based on the scan feature vector and the region feature vector of each segmented region in the computer-aided model, one or more candidate regions are determined from the multiple segmented regions according to feature similarity, wherein the region feature vector is extracted by the machine learning model; The real-time scanning data is iteratively matched with the one or more candidate regions, and the candidate region with the smallest registration error is determined as the target segmentation region.

[0009] In some embodiments, determining one or more candidate regions from the plurality of segmented regions based on feature similarity, according to the scanned feature vector and the region feature vectors of each segmented region in the computer-aided model, includes: Based on the scan feature vector and the feature vector of each region, the feature similarity between the real-time scan data and each segmented region is determined. Multiple segmented regions are filtered based on the feature similarity and similarity threshold; Hierarchical clustering and deduplication of hierarchical relationships are performed on the segmented regions obtained after screening to obtain one or more candidate regions.

[0010] In some embodiments, after registering the real-time scan data with the computer-aided model and determining one or more target segmentation regions corresponding to the real-time scan data from the plurality of segmentation regions, the method further includes: Based on the movement trend of the scanning device, the extended area is determined in the computer-aided model; The next frame of the real-time scanning data is obtained, and the consistency of the next frame of the scanning data and the extended area is checked until it is determined that the scanning data of the scanning device and the computer-aided model have been registered.

[0011] In some embodiments, the conditions for determining that the registration of the scanning data from the scanning device with the computer-aided model is complete include: The scanned data has reached a first coverage threshold in terms of coverage of key feature regions in the computer-aided model. Alternatively, the boundary closure of the scanned data has reached the closure threshold.

[0012] Secondly, embodiments of this application also provide a three-dimensional scanning system, including: The scanning device is configured to capture real-time scan data of the object being scanned; and A computing device includes a memory and instructions stored in the memory, which, when executed by one or more processors, cause the computing device to perform the aforementioned three-dimensional scanning method.

[0013] Thirdly, embodiments of this application also provide a scanning device, which includes a memory and instructions stored in the memory, wherein when the instructions are executed by one or more processors, the scanning device enables the above-described three-dimensional scanning method.

[0014] Fourthly, embodiments of this application also provide a storage medium storing a computer program, the computer program including instructions that, when executed by a processor, can implement the above-described three-dimensional scanning method.

[0015] This application provides a 3D scanning method, system, device, and medium. Before scanning the object, a computer-aided model of the object is pre-acquired. This computer-aided model includes multiple segmented regions, each with a corresponding scanning resolution, thereby achieving predictive global planning of scanning resources. After capturing real-time scanning data of the object, this application determines one or more target segmented regions corresponding to the real-time scanning data from the computer-aided model, and generates a mesh corresponding to the real-time scanning data based on the scanning resolution of the one or more target segmented regions. Different scanning resolutions are pre-allocated for different scanning positions to ensure the reconstruction quality of the mesh model.

[0016] Meanwhile, during the scanning process, after acquiring the real-time scanning data of the object captured by the scanning device, the real-time scanning data is registered with the computer-aided model. This process realizes the allocation of scanning resolution, eliminating the need for high-resolution scanning of the entire object and large-scale data processing, reducing the computational load during the scanning process, shortening the scanning time, and thus improving scanning efficiency. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a three-dimensional scanning method provided for an exemplary embodiment of this application; Figure 2 A schematic diagram of a sub-process of a three-dimensional scanning method provided for an exemplary embodiment of this application; Figure 3 Another schematic diagram of a sub-process of the three-dimensional scanning method provided for an exemplary embodiment of this application; Figure 4 A schematic block diagram of a three-dimensional scanning system provided for an exemplary embodiment of this application; Figure 5 A schematic block diagram of a scanning device provided for an exemplary embodiment of this application; Figure 6 A schematic block diagram of a computing device provided for an exemplary embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0021] It should also be understood that the terminology used in this application specification is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this application specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0022] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0023] This application provides three-dimensional scanning methods, systems, devices, and media.

[0024] The subject executing the three-dimensional scanning method can be the scanning device or three-dimensional scanning system provided in the embodiments of this application. The three-dimensional scanning system includes a scanning device and a computing device.

[0025] When the executing entity is a scanning device, the scanning device includes a memory and instructions stored in the memory. When the instructions are executed by one or more processors, the scanning device enables the scanning device to implement the three-dimensional scanning method provided in any exemplary embodiment of this application. When the executing entity is a 3D scanning system, the scanning device in the system is configured to capture real-time scanning data of the scanned object. The computing device includes a memory and instructions stored in the memory. When the instructions are executed by one or more processors, the computing device enables the computing device to implement the 3D scanning method provided in any exemplary embodiment of this application. The computing device can be a terminal or a server. The terminal can be a wearable device, computer, mobile phone, tablet computer, or laptop computer, etc.

[0026] Figure 1 This is a flowchart illustrating the three-dimensional scanning method provided in an embodiment of this application. Figure 1As shown, the method includes the following steps S110-S140.

[0027] S110. Obtain a computer-aided model of the scanned object, wherein the computer-aided model includes multiple segmented regions, and each segmented region defines a corresponding scanning resolution.

[0028] In this embodiment, the computer-aided model is divided into multiple segmented regions, and each segmented region is defined with a corresponding scanning resolution. Based on the geometric features of the scanned object and the actual scanning requirements, high scanning resolution can be pre-allocated to key feature regions and low scanning resolution can be pre-allocated to non-key feature regions. This fundamentally solves the technical defects of traditional methods, such as wasted scanning resources in non-key regions and insufficient scanning resolution in key regions. It effectively ensures the overall accuracy and quality of the model after scanning and reconstruction, and ensures that the model can accurately reflect the key geometric features of the scanned object.

[0029] For example, each segmented region is labeled with a corresponding scan resolution, which is used to determine the grid resolution of the grid corresponding to the generated real-time scan data.

[0030] It is understood that the objects to be scanned described in this article may be, for example, complete machines, accessories, industrial facilities, building facilities, works of art, or other suitable objects, and this article does not impose any limitations on the objects to be scanned.

[0031] In some embodiments, please refer to Figure 2 Specifically, step S110 includes: S1101. Obtain the initial auxiliary model of the scanned object.

[0032] The computer-aided model is a model obtained by dividing the initial auxiliary model into multiple levels. The initial auxiliary model can be a CAD model of the scanned object or an initial 3D model of the scanned object. The initial 3D model is a 3D reconstruction model obtained by the scanning device performing a low-precision full-area scan of the target object.

[0033] S1102. Extract the feature boundaries of the initial auxiliary model and perform mesh normal vector clustering on the initial auxiliary model to obtain multiple mesh sets.

[0034] In this embodiment, the feature boundaries of the initial auxiliary model and the initial auxiliary model can be extracted and split into multiple mesh sets through the following steps: the feature boundaries of the initial auxiliary model are extracted based on the dihedral angles of the mesh patches corresponding to the edge information of the initial auxiliary model; the mesh patches are clustered based on the normal vectors of each mesh patch in the initial auxiliary model to obtain the multiple mesh sets.

[0035] Specifically, based on the existing edge information of the initial auxiliary model, the dihedral angles of the corresponding mesh patches and the preset dihedral angle threshold are used to extract the feature boundaries of the initial auxiliary model. The adjacent edges of mesh patches with dihedral angles greater than the dihedral angle threshold are determined as feature edges. The closed boundary formed by connecting all feature edges is the feature boundary of the initial auxiliary model. Furthermore, the K-means clustering algorithm is used to cluster the normal vectors of all mesh patches in the initial auxiliary model to obtain multiple mesh sets.

[0036] S1103. Using the feature boundary as a constraint, the initial auxiliary model is divided into multiple first regions according to the adjacent grids in each of the grid sets.

[0037] In this embodiment, step S1102 groups all faces with the same or similar orientations into one class based on normal vector clustering. This operation may group non-adjacent faces with the same orientation into one class. To eliminate this situation, this embodiment uses feature boundaries as constraints and introduces a region growing algorithm to merge adjacent mesh faces in the mesh set and split non-adjacent mesh faces, thereby dividing the initial auxiliary model into multiple first regions. This ensures that each segmented region has clear geometric semantics and provides an accurate basis for subsequent balanced octree space partitioning.

[0038] For example, geometric semantics can include planes, cylinders, spheres, etc.

[0039] S1104. Segment the regions in the multiple first regions whose area is greater than a first area threshold, and cluster the regions in the multiple first regions whose area is less than a second area threshold, dividing the initial auxiliary model into multiple second regions, wherein the first area threshold is greater than the second area threshold.

[0040] This embodiment can segment a region in the first region whose area is greater than a first area threshold based on the following steps: calculating the average curvature and Gaussian curvature of each grid vertex in the region; segmenting the region according to the average curvature, the Gaussian curvature and the curvature threshold, wherein the average curvature reflects the curvature of the grid vertex surface and the Gaussian curvature reflects the concavity and convexity of the grid vertex surface.

[0041] Specifically, regions in the first region with an area greater than a first area threshold are divided into high curvature regions, medium curvature regions, and low curvature regions according to curvature thresholds, which include high curvature thresholds, medium curvature thresholds, and low curvature thresholds. For example, a high curvature region can be a region where the average curvature or Gaussian curvature of the mesh vertices is greater than the high curvature threshold, corresponding to key features of the part (such as grooves, chamfers, and hole edges), and a high scanning resolution can be predefined; a medium curvature region can be a region where the average curvature and Gaussian curvature of the mesh vertices are between the high curvature threshold and the low curvature threshold, corresponding to transition parts of the part (such as the junction of a plane and a chamfer), and a medium scanning resolution can be predefined; a low curvature region can be a region where the average curvature and Gaussian curvature of the mesh vertices are both less than the low threshold, corresponding to non-critical areas of the part such as simple planes and outer cylindrical surfaces, and a low scanning resolution can be predefined.

[0042] In some examples, after regions larger than the first area threshold are classified, morphological operations are used to smooth the segmentation boundaries, thereby dividing regions larger than the first area threshold into geometrically defined sub-regions.

[0043] In some examples, clustering is performed on regions in the first region whose area is smaller than the second area threshold to avoid increased registration complexity caused by over-segmentation.

[0044] S1105. Determine the scanning resolution corresponding to each of the second regions to obtain the candidate three-dimensional reference model.

[0045] In some examples, the scanning resolution of the second region can be determined by the following steps: obtaining the user's annotation operations for each of the second regions and determining the scanning resolution corresponding to each of the second regions; or, assigning a scanning resolution to each of the second regions based on the curvature of the region corresponding to the second region.

[0046] In this embodiment, the user can mark the resolution of each second region in the interactive interface according to the scanning requirements of the scanned object, so as to determine the scanning resolution corresponding to each second region, or automatically allocate the scanning resolution according to the curvature of the region.

[0047] Specifically, when allocating scanning resolution to the second region based on the region curvature corresponding to the second region, the scanning resolution can be allocated based on the average curvature and Gaussian curvature. For example, the average curvature and Gaussian curvature of each second region are calculated. Regions with average curvature or Gaussian curvature greater than the high curvature threshold are allocated high scanning resolution. Regions with average curvature and Gaussian curvature between the high curvature threshold and the low curvature threshold are allocated medium scanning resolution. Regions with average curvature and Gaussian curvature both less than the low threshold are allocated low scanning resolution.

[0048] In some examples, the second region after the assigned scan resolution described herein is the segmented region described herein.

[0049] The exemplary embodiments in this paper provide two resolution determination methods: annotation and automatic allocation, which take into account both personalized scanning needs and the efficiency of automated resolution allocation, and are suitable for different scanning scenarios (such as high-precision customized scanning and batch standardized scanning).

[0050] S1106. The candidate 3D reference model is spatially partitioned based on a balanced octree to obtain the computer-aided model.

[0051] For example, using the bounding box of the candidate 3D reference model as the root node, a balanced octree is used for spatial recursive subdivision. The subdivision termination condition is set to n times the scanner's scanning range (such as 2 times or other suitable multiples). Finally, a computer-aided model containing multi-level spatial indexes is obtained. The segmented regions of this model are bound to the spatial indexes, which facilitates rapid matching in the future.

[0052] S120. Acquire real-time scanning data, including the object being scanned, captured by the scanning device.

[0053] For example, the scanning device may be the exemplary scanning device described herein, and the real-time scanning data may be point cloud data generated using an image captured by the scanning device, which may contain basic information such as the three-dimensional coordinates and normal vectors of the points.

[0054] S130. Register the real-time scanning data with the computer-aided model to determine one or more target segmentation regions corresponding to the real-time scanning data from the plurality of segmentation regions.

[0055] After obtaining the real-time scanning data, this embodiment determines the target segmentation region through real-time registration. Each frame of real-time scanning data can correspond to one or more target segmentation regions. That is, different scanning resolutions can be used for mesh reconstruction at different positions of the same frame of scanning data. In some embodiments, please refer to Figure 3 Specifically, step S130 includes: S1301. Extract the scanning feature vector of the real-time scanning data through a machine learning model.

[0056] The machine learning model in this embodiment is a model obtained after training on multiple training samples. The training samples are the point cloud data, normal vectors and edge information of each sample segmentation region in the computer-aided sample model. The trained machine learning model can quickly extract feature vectors of any scanned data or segmentation region.

[0057] In some examples, the machine learning model can be a model fine-tuned based on a small number of training samples.

[0058] S1302. Based on the scan feature vector and the region feature vector of each segmented region in the computer-aided model, determine one or more candidate regions from the multiple segmented regions according to feature similarity, wherein the region feature vector is extracted by the machine learning model.

[0059] In this embodiment, the regional feature vectors of each segmented region in the computer-aided model are calculated by a machine learning model. Then, one or more candidate regions are determined from multiple segmented regions based on feature similarity. Specifically, based on the scan feature vector and each of the regional feature vectors, the feature similarity between the real-time scan data and each of the segmented regions is determined. Multiple segmented regions are filtered based on the feature similarity and a similarity threshold. The segmented regions obtained after filtering are subjected to hierarchical clustering and hierarchical relationship deduplication to obtain one or more candidate regions.

[0060] Specifically, segmented regions with feature similarity greater than a similarity threshold are filtered out. Since regions with similar features and overlapping spatial levels may still exist after filtering, to avoid mismatches and duplicate calculations in subsequent point cloud matching, such as using ICP matching, this embodiment performs hierarchical clustering and hierarchical relationship deduplication on the segmented regions obtained after filtering. Specifically, hierarchical clustering includes the following steps: Using feature similarity (the similarity of regional feature vectors of each segmented region) as the clustering basis, hierarchical clustering is performed on the segmented regions obtained after screening: calculate the pairwise feature similarity between all regions after screening, and group regions with feature similarity higher than the clustering threshold into the same cluster; adopt a bottom-up hierarchical clustering method, first merge the two most similar regions into one cluster, and then merge the cluster with regions with close similarity, and finally form different clusters.

[0061] Hierarchical clustering groups segmented regions with highly similar features into one category, avoiding subsequent ICP matching of each region in the same category (only preliminary matching is needed for each cluster to further narrow down the scope), while also avoiding mismatches between regions in the same category.

[0062] Deduplication of hierarchical relationships includes the following steps: In this embodiment, the computer-aided model has been spatially partitioned using a balanced octree. Each segmented region is bound to a multi-level spatial index and has a clear spatial hierarchy. For each cluster after hierarchical clustering, the spatial hierarchy of each segmented region within the cluster is analyzed, and duplicate regions with "parent-child hierarchy" or "inclusion relationship" are removed. That is, the region within the cluster that best fits the real-time scanning data and has no overlap is retained, while the included or overlapping regions are deleted. Eliminating overlapping and duplicate regions in spatial hierarchy ensures that each candidate region is spatially independent and free from interference, avoiding redundant calculations during subsequent iterative closest point (ICP) matching. It also further identifies the region that best matches the spatial location of the real-time scan data, improving the accuracy and efficiency of fine matching.

[0063] In this embodiment, a Top-K candidate set is generated for fine matching (iterative nearest point matching). For example, the top 3 regions with the highest similarity are determined as candidate regions, and coarse matching of feature similarity is completed through step S1302.

[0064] S1303. Perform iterative nearest point matching between the real-time scanning data and the one or more candidate regions, and determine the candidate region with the smallest registration error as the target segmentation region.

[0065] Specifically, the final registration error after matching all candidate regions with the real-time scanning data by ICP is collected. All registration errors are compared and sorted, and the candidate region with the smallest registration error is selected and determined as the target segmentation region corresponding to the real-time scanning data.

[0066] This embodiment employs a multi-stage registration method involving machine learning model feature extraction, coarse matching, and ICP matching. First, coarse matching narrows the matching range, then fine matching determines the precise target, solving the problems of excessive computational load and slow registration speed in global fine matching. Furthermore, the use of a machine learning model to extract high-dimensional feature vectors accurately captures the deep geometric features of point cloud data, improving the accuracy of coarse matching, reducing the number of candidate regions, and further reducing the computational load of fine matching. Moreover, this embodiment uses the minimum registration error of ICP matching as the criterion for determining the target segmentation region, ensuring the accuracy of the registration results and ensuring a precise correspondence between real-time scanning data and the segmentation region of the computer-aided model.

[0067] S140. Generate at least a portion of the mesh of the scanned object based on the real-time scan data, wherein the mesh resolution of the at least portion of the mesh is determined by one or more scan resolutions of the one or more target segmentation regions.

[0068] After determining the target segmentation region corresponding to the real-time scanning data, this embodiment determines the grid resolution based on one or more scanning resolutions of one or more target segmentation regions, and then generates at least a portion of the grid of the scanned object based on the grid resolution.

[0069] In some embodiments, after determining one or more target segmentation regions corresponding to the real-time scanning data from the plurality of segmentation regions, the method further includes: determining an extended region in the computer-aided model according to the movement trend of the scanning device; obtaining the next frame of scanning data of the real-time scanning data; performing a consistency check on the next frame of scanning data and the extended region until it is determined that the scanning data of the scanning device and the computer-aided model have been registered.

[0070] In some examples, the conditions for determining that the scanning data of the scanning device and the computer-aided model have been registered include: the coverage of the key feature regions in the computer-aided model by the scanned data reaches a first coverage threshold; or, the boundary closure of the scanned data reaches a closure threshold.

[0071] This step is a dynamic tracking and integrity verification stage in the registration process. After locking the target segmentation region corresponding to a single frame of real-time scan data, the movement path of the scanning device is predicted, the scanning range is expanded, and consistency verification of multiple frames of data ensures that the entire scanning process gradually covers the key areas of the computer-aided model, ultimately achieving full-domain registration and increasing the robustness of the registration project.

[0072] It is understood that, before scanning the object, this application's embodiments pre-obtain a computer-aided model of the object. This computer-aided model includes multiple segmented regions, and each segmented region has a predefined corresponding scanning resolution, thereby achieving predictive global planning of scanning resources. Compared to the uniform resolution global scanning mode used in traditional 3D scanning methods, after capturing real-time scanning data of the object, this application determines one or more target segmented regions corresponding to the real-time scanning data from the computer-aided model, and generates a mesh corresponding to the real-time scanning data based on the scanning resolution of the one or more target segmented regions. Different scanning resolutions are pre-allocated for different scanning positions, ensuring the reconstruction quality of the mesh model.

[0073] Meanwhile, during the scanning process, after acquiring the real-time scanning data of the object captured by the scanning device, the real-time scanning data is registered with the computer-aided model. This process achieves targeted allocation of scanning resolution, eliminating the need for high-resolution scanning of the entire object and large-scale data processing, reducing the computational load during the scanning process, shortening the scanning time, and thus improving scanning efficiency.

[0074] Figure 4 This is a schematic block diagram of a three-dimensional scanning system provided in an embodiment of this application. Figure 4 As shown, corresponding to the above-described three-dimensional scanning method, this application also provides a three-dimensional scanning system. The three-dimensional scanning system includes a scanning device and a computing device, wherein: The scanning device is configured to capture real-time scan data of the object being scanned. A computing device is configured to acquire a computer-aided model of a scanned object, wherein the computer-aided model includes multiple segmented regions, and each segmented region defines a corresponding scan resolution; acquire real-time scan data of the scanned object captured by a scanning device; register the real-time scan data with the computer-aided model to determine one or more target segmented regions corresponding to the real-time scan data from the multiple segmented regions; and generate at least a portion of a mesh of the scanned object based on the real-time scan data, wherein the mesh resolution of the at least a portion of the mesh is determined by one or more scan resolutions of the one or more target segmented regions.

[0075] In some embodiments, when the computing device performs the step of acquiring the computer-aided model of the scanned object, it is specifically used for: Obtain an initial auxiliary model of the scanned object; extract the feature boundaries of the initial auxiliary model, and perform mesh normal vector clustering on the initial auxiliary model to obtain multiple mesh sets; Using the feature boundaries as constraints, the initial auxiliary model is divided into multiple first regions based on the adjacent grids in each of the grid sets; The initial auxiliary model is divided into multiple second regions, where the area of ​​each region is greater than a first area threshold, and the area of ​​each region is less than a second area threshold. The scanning resolution corresponding to each of the second regions is determined to obtain a candidate 3D reference model; the candidate 3D reference model is spatially partitioned based on a balanced octree to obtain the computer-aided model.

[0076] In some embodiments, when the computing device performs the steps of extracting the feature boundaries of the initial auxiliary model and performing mesh normal vector clustering on the initial auxiliary model to obtain multiple mesh sets, it is specifically used for: The feature boundary of the initial auxiliary model is extracted based on the dihedral angle of the mesh patch corresponding to the edge information of the initial auxiliary model; Based on the normal vectors of each mesh patch in the initial auxiliary model, the mesh patches are clustered to obtain the multiple mesh sets.

[0077] In some embodiments, when the computing device performs the step of segmenting regions in a plurality of first regions whose areas are greater than a first area threshold, it is specifically used for: Calculate the average curvature and Gaussian curvature of each grid vertex in the region; The region is segmented based on the average curvature, the Gaussian curvature, and the curvature threshold.

[0078] In some embodiments, when the computing device performs the step of determining the scanning resolution corresponding to each of the second regions, it is specifically used for: Obtain the user's annotation operations for each of the second regions, and determine the scanning resolution corresponding to each of the second regions; Alternatively, a scanning resolution may be assigned to each of the second regions based on the curvature of the corresponding region.

[0079] In some embodiments, when the computing device performs the step of registering the real-time scan data with the computer-aided model and determining one or more target segmentation regions corresponding to the real-time scan data from the plurality of segmentation regions, it is specifically used for: The scanning feature vector of the real-time scanning data is extracted using a machine learning model; Based on the scan feature vector and the region feature vector of each segmented region in the computer-aided model, one or more candidate regions are determined from the multiple segmented regions according to feature similarity, wherein the region feature vector is extracted by the machine learning model; The real-time scanning data is iteratively matched with the one or more candidate regions, and the candidate region with the smallest registration error is determined as the target segmentation region.

[0080] In some embodiments, when the computing device performs the step of determining one or more candidate regions from the plurality of segmented regions based on feature similarity according to the scan feature vector and the region feature vectors of each segmented region in the computer-aided model, it is specifically used for: Based on the scan feature vector and the feature vector of each region, the feature similarity between the real-time scan data and each segmented region is determined. Multiple segmented regions are filtered based on the feature similarity and similarity threshold; Hierarchical clustering and deduplication of hierarchical relationships are performed on the segmented regions obtained after screening to obtain one or more candidate regions.

[0081] In some embodiments, after the computing device performs the step of registering the real-time scan data with the computer-aided model and determining one or more target segmentation regions corresponding to the real-time scan data from the plurality of segmentation regions, it is further configured to: Based on the movement trend of the scanning device, the extended area is determined in the computer-aided model; The next frame of the real-time scanning data is obtained, and the consistency of the next frame of the scanning data and the extended area is checked until it is determined that the scanning data of the scanning device and the computer-aided model have been registered.

[0082] In some embodiments, the conditions for determining that the registration of the scanning data from the scanning device with the computer-aided model is complete include: The scanned data has reached a first coverage threshold in terms of coverage of key feature regions in the computer-aided model. Alternatively, the boundary closure of the scanned data has reached the closure threshold.

[0083] It is understood that the 3D scanning system provided in this application pre-acquires a computer-aided model of the object before scanning it. This computer-aided model includes multiple segmented regions, and each segmented region has a predefined corresponding scanning resolution, thereby realizing predictive global planning of scanning resources. Compared with the uniform resolution global scanning mode used in traditional 3D scanning methods, after capturing the real-time scanning data of the object, this application determines one or more target segmented regions corresponding to the real-time scanning data from the computer-aided model, and generates a mesh corresponding to the real-time scanning data based on the scanning resolution of the one or more target segmented regions. Different scanning resolutions are pre-allocated for different scanning positions to ensure the reconstruction quality of the mesh model.

[0084] Meanwhile, during the scanning process, after the 3D scanning system acquires the real-time scanning data of the object captured by the scanning device, it performs registration processing with the computer-aided model. This process achieves targeted allocation of scanning resolution, eliminating the need for high-resolution scanning of the entire object and extensive data processing, reducing the computational load during the scanning process, shortening the scanning time, and thus improving scanning efficiency.

[0085] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned 3D scanning system and each unit can be referred to the corresponding description in the foregoing method embodiments. For the sake of convenience and brevity, it will not be repeated here.

[0086] Figure 5A schematic diagram of an exemplary scanning device 50 is shown. In the depicted embodiment, the scanning device 50 includes a frame structure 51 and an imaging module 52 located on the frame structure 51. The imaging modules 52 can be arranged side by side such that the fields of view of each imaging module at least partially overlap. In some embodiments, the imaging module 52 may include three cameras, namely a first camera 521, a second camera 522, and a third camera 524. The imaging module 52 may also include a light projector 523, which may include a light source, a collimating lens, and a diffractive optical element. The light source is configured to emit a light beam toward the lens. The light beam, collimated by the collimating lens, propagates to the diffractive optical element, which replicates the focused light beam to form a speckle pattern or fringe pattern and projects it onto the scanned object / area. The speckle image or fringe pattern is reflected back from the scanned object / area and acquired by the imaging module 52 for further processing by a processing chip (not shown) to obtain three-dimensional information of the scanned object / area. In other embodiments, the light projector 523 may be an image projector, such as a digital micromirror device, a liquid crystal display projector, or an organic electroluminescent display projector.

[0087] In some embodiments, the light projector 523 may include a single light source, such as a light source emitting infrared light, white light, blue light, or other visible monochromatic light. In other embodiments, the light projector 523 is configured to emit light with wavelengths between 405 nm and 1100 nm. In still other embodiments, the light projector 523 may include two or three identical light sources, such as two or three light sources emitting infrared light. Alternatively, the light projector 523 may include two or three different light sources, such as a first light source emitting infrared light and a second light source emitting white light, or a first light source emitting infrared light, a second light source emitting white light, and a third light source emitting blue light. The two or three identical light sources may be part of the same light projector 523 or may be implemented as separate units (e.g., in an additional light projector unit), and similarly, the two or three different light sources may be part of the same light projector 523 or may be implemented as separate units (e.g., in an additional light projector unit).

[0088] In some embodiments, the imaging module 52 may also include another light projector (not shown), such as a speckle pattern projector, a stripe pattern projector, or an image projector.

[0089] The first camera 521 and the third camera 524 are typically monochrome (e.g., black and white) cameras, and will depend on the type of light source(s) used in the light projector(s) 523. In some embodiments, the first camera 521 and the third camera 524 may be monochrome, visible spectrum, or near-infrared cameras, and the light projector 523 may be an infrared or near-infrared light projector. The first camera 521 and the third camera 524 may use any suitable shutter technology, including but not limited to rolling shutters, global shutters, mechanical shutters, and optical liquid crystal display (LCD) shutters. In some embodiments, the second camera 522 may be a color camera (also known as a texture camera). The texture camera may use any suitable shutter technology, including but not limited to rolling shutters, global shutters, mechanical shutters, and optical liquid crystal display (LCD) shutters. In some embodiments, the first camera 521, the second camera 522, and the third camera 524 may have similar configurations to improve matching confidence and speed. In other embodiments, the imaging module 52 may also include a fourth camera, such that the scanning device includes three monochrome cameras and one color camera. In a further embodiment, the imaging module can also use a single camera to capture reflected light and color textures, omitting a second (and third and / or fourth) camera.

[0090] like Figure 5 As shown, a first camera 521, a second camera 522, a light projector 523, and a third camera 524 can be located side-by-side on one surface of the frame structure 51, spaced apart from each other, and all facing directly forward of the surface. In some examples, the first camera 521 has a first field of view facing the front region of the surface, the second camera 522 has a second field of view facing the front region of the surface, the third camera 524 has a third field of view facing the front region of the surface, and the light projector 523 has a projected field of view facing the front region of the surface. In some examples, the first field of view and the projected field of view at least partially overlap, the second field of view and the projected field of view at least partially overlap, the third field of view and the projected field of view at least partially overlap, and the first field of view, the second field of view and the third field of view at least partially overlap.

[0091] A data connection (such as USB, serial communication connection) between the scanning device 50 and one or more computer processors (not shown) allows the transmission of data collected by the first camera 521, the second camera 522, and the third camera 524, enabling it to be processed to derive 3D measurements of the surface of the scanned object / object. The one or more computer processors may be implemented in a remote computing system (electronic device), or alternatively, may be part of the scanning device 50 itself.

[0092] For example, light projector 523 includes a single light projector unit, or it may have two or more light projector units. The light projector unit can be configured to project visible or invisible light, coherent or incoherent light. In some embodiments, the light projector unit may include one or more light sources consisting of lasers (e.g., vertical cavity surface emitter (VCSEL), edge emitter (EEL), solid-state lasers, semiconductor lasers, and / or one or more LEDs (or OLEDs)).

[0093] A light projector unit can be configured to project a structured light pattern consisting of multiple light sheets arranged side-by-side. When the light sheets are projected onto the surface of an object, they can appear as elongated light stripes. These elongated light stripes are non-intersecting and, in some embodiments, can be substantially parallel to each other, while in others, they can intersect each other. In some embodiments, the light sheets can also appear as dense dots or spots, such as a collection of dots or spots of different sizes. In some embodiments, the light projector unit can be a programmable light projector unit capable of projecting more than one light pattern. For example, the light projector unit can be configured to project different structured line patterns. In some embodiments, the light projector unit can emit light with wavelengths between 405 nm and 1100 nm.

[0094] In some examples, using a first camera 521 and a third camera 524, two images of an object can be captured simultaneously. Image processing can be applied, for example, to a computational method implemented by one or more processors, or to a computational method implemented, for example, by an electronic device, to derive a 3D measurement of the surface of the scanned object / object.

[0095] In some examples, a second camera 522 can be used to capture the texture of an object while the first camera 521 and the second camera 522 are capturing images of the object, and the texture can be applied to a computational method, for example, implemented by one or more processors, or to a computational method, for example, implemented by an electronic device, to map onto a 3D measurement of the surface of the scanned object / object.

[0096] In some examples, using a membrane / film with bandpass filter functionality fixed to the lens of a camera (e.g., first camera 521, second camera 522, third camera 524) can match the wavelength of the projector units (multiple), which can help reduce light source interference from ambient light and other projector units.

[0097] In some examples, a calibration plate, such as a piece or a set of calibration plates whose true geometric distance values ​​have been measured in advance using high-precision methods such as photogrammetry, is used to measure the intrinsic and extrinsic parameters of the first camera 521, the second camera 522, and the third camera 524. The measurement process typically involves a series of continuous image acquisitions using the scanning device 50 after adjusting the calibration plate to different positions, and the calculation of the spatial position and orientation of the first camera 521, the second camera 522, and the third camera 524 by identifying the positions of reference (marked) points / regions / lines in the calibration images, thereby completing the calibration of the intrinsic and extrinsic parameters of the first camera 521, the second camera 522, and the third camera 524.

[0098] The aforementioned computing device can be implemented as a computer program, which can, for example... Figure 6 It runs on the computing device shown.

[0099] Please see Figure 6 , Figure 6 This is a schematic block diagram of a computing device provided in an embodiment of this application. The computing device 600 can be a terminal or a server.

[0100] See Figure 6 The computing device 600 includes a processor 602, a memory, and a network interface 605 connected via a system bus 601. The memory may include a non-volatile storage medium 603 and internal memory 604.

[0101] The non-volatile storage medium 603 may store an operating system 6031 and a computer program 6032. The computer program 6032 includes program instructions that, when executed, cause the processor 602 to perform a three-dimensional scanning method.

[0102] The processor 602 provides computing and control capabilities to support the operation of the entire computing device 600.

[0103] The internal memory 604 provides an environment for the execution of the computer program 6032 in the non-volatile storage medium 603. When the computer program 6032 is executed by the processor 602, the processor 602 can perform a three-dimensional scanning method.

[0104] This network interface 605 is used for network communication with other devices. Those skilled in the art will understand that... Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computing device 600 to which the present application is applied. The specific computing device 600 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0105] The processor 602 is used to run a computer program 6032 stored in the memory to perform the following steps: Obtain a computer-aided model of the scanned object, wherein the computer-aided model includes multiple segmented regions, and each segmented region defines a corresponding scanning resolution; Acquire real-time scan data, including the object being scanned, captured by the scanning device; The real-time scanning data is registered with the computer-aided model to determine one or more target segmentation regions corresponding to the real-time scanning data from the plurality of segmentation regions; and At least a portion of the mesh of the scanned object is generated based on the real-time scan data, wherein the mesh resolution of the at least portion of the mesh is determined by one or more scan resolutions of the one or more target segmentation regions.

[0106] It should be understood that, in the embodiments of this application, the processor 602 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0107] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0108] Therefore, this application also provides a storage medium. This storage medium can be a storage medium that stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor performs the three-dimensional scanning method provided in the embodiments of this application.

[0109] The storage medium can be any storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0110] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0111] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0112] The steps in the methods of this application embodiment can be adjusted, merged, or deleted according to actual needs. The units in the apparatus of this application embodiment can be merged, divided, or deleted according to actual needs. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0113] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computing device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0114] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A three-dimensional scanning method, comprising: Obtain a computer-aided model of the scanned object, wherein the computer-aided model includes multiple segmented regions, and each segmented region defines a corresponding scanning resolution; Acquire real-time scan data, including the object being scanned, captured by the scanning device; The real-time scanning data is registered with the computer-aided model to determine one or more target segmentation regions corresponding to the real-time scanning data from the plurality of segmentation regions; and At least a portion of the mesh of the scanned object is generated based on the real-time scan data, wherein the mesh resolution of the at least portion of the mesh is determined by one or more scan resolutions of the one or more target segmentation regions.

2. The method according to claim 1, wherein, The computer-aided model for acquiring the scanned object includes: Obtain the initial auxiliary model of the scanned object; Extract the feature boundaries of the initial auxiliary model, and perform mesh normal vector clustering on the initial auxiliary model to obtain multiple mesh sets; Using the feature boundaries as constraints, the initial auxiliary model is divided into multiple first regions based on the adjacent grids in each of the grid sets; The initial auxiliary model is divided into multiple second regions, where the area of ​​each region is greater than a first area threshold, and the area of ​​each region is less than a second area threshold. Determine the scanning resolution corresponding to each of the second regions to obtain candidate 3D reference models; The candidate 3D reference model is spatially partitioned using a balanced octree to obtain the computer-aided model.

3. The method according to claim 2, wherein, The process involves extracting the feature boundaries of the initial auxiliary model and performing mesh normal vector clustering on the initial auxiliary model to obtain multiple mesh sets, including: The feature boundary of the initial auxiliary model is extracted based on the dihedral angle of the mesh patch corresponding to the edge information of the initial auxiliary model; Based on the normal vectors of each mesh patch in the initial auxiliary model, the mesh patches are clustered to obtain the multiple mesh sets.

4. The method according to claim 2, wherein, The segmentation of regions in the plurality of first regions whose area is greater than a first area threshold includes: Calculate the average curvature and Gaussian curvature of each grid vertex in the region; The region is segmented based on the average curvature, the Gaussian curvature, and the curvature threshold.

5. The method according to claim 2, wherein, Determining the scanning resolution corresponding to each of the second regions includes: Obtain the user's annotation operations for each of the second regions, and determine the scanning resolution corresponding to each of the second regions; Alternatively, a scanning resolution may be assigned to each of the second regions based on the curvature of the corresponding region.

6. The method according to claim 1, wherein, The step of registering the real-time scanning data with the computer-aided model and determining one or more target segmentation regions corresponding to the real-time scanning data from the plurality of segmentation regions includes: The scanning feature vector of the real-time scanning data is extracted using a machine learning model; Based on the scan feature vector and the region feature vector of each segmented region in the computer-aided model, one or more candidate regions are determined from the multiple segmented regions according to feature similarity, wherein the region feature vector is extracted by the machine learning model; The real-time scanning data is iteratively matched with the one or more candidate regions, and the candidate region with the smallest registration error is determined as the target segmentation region.

7. The method according to claim 6, wherein, The step of determining one or more candidate regions from the multiple segmented regions based on feature similarity, according to the scanned feature vector and the region feature vectors of each segmented region in the computer-aided model, includes: Based on the scan feature vector and the feature vector of each region, the feature similarity between the real-time scan data and each segmented region is determined. Multiple segmented regions are filtered based on the feature similarity and similarity threshold; Hierarchical clustering and deduplication of hierarchical relationships are performed on the segmented regions obtained after screening to obtain one or more candidate regions.

8. The method according to claim 1, wherein, After registering the real-time scanning data with the computer-aided model and determining one or more target segmentation regions corresponding to the real-time scanning data from the plurality of segmentation regions, the method further includes: Based on the movement trend of the scanning device, the extended area is determined in the computer-aided model; The next frame of the real-time scanning data is obtained, and the consistency of the next frame of the scanning data and the extended area is checked until it is determined that the scanning data of the scanning device and the computer-aided model have been registered.

9. The method according to claim 8, wherein, The conditions for determining that the scanning data of the scanning device and the computer-aided model have been successfully registered include: The scanned data has reached a first coverage threshold in terms of coverage of key feature regions in the computer-aided model. Alternatively, the boundary closure of the scanned data has reached the closure threshold.

10. A three-dimensional scanning system, comprising: The scanning device is configured to capture real-time scan data of the object being scanned. as well as A computing device includes a memory and instructions stored in the memory, which, when executed by one or more processors, cause the computing device to perform the three-dimensional scanning method as described in any one of claims 1-9.

11. A scanning device, comprising: A memory and instructions stored in the memory, wherein the instructions, when executed by one or more processors, cause the scanning device to perform the three-dimensional scanning method as described in any one of claims 1-9.

12. A storage medium storing a computer program, the computer program including instructions that, when executed by a processor, cause the processor to perform the three-dimensional scanning method as described in any one of claims 1-9.

Images