Real-time registration method and device of high-density point cloud, scanning equipment and storage medium

Abstract

The invention belongs to the technical field of optical three-dimensional scanning, and relates to a real-time registration method for high-density point clouds, which comprises the following steps: acquiring two groups of sparse point cloud data generated by three-dimensional scanning in an oral cavity, inputting the point cloud data into a constructed efficient registration model, extracting multi-dimensional feature descriptions of a source point cloud and a target point cloud, and constructing a registration model; the point cloud registration method comprises the following steps of: obtaining a point pair set, calculating a feature distance to form an initial point pair set, performing spatial consistency analysis on the initial point pair set, rejecting outlier pairs to obtain a cluster point pair subset, and finally performing point cloud registration calculation on the cluster point pair subset to complete point cloud registration. The invention further relates to a high-density point cloud real-time registration device, scanning equipment and a storage medium. According to the technical scheme provided by the invention, the registration precision and stability can be improved, and meanwhile, the requirement of the intraoral scanner on the real-time performance is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of optical three-dimensional scanning, and more particularly to a real-time registration method and device for high-density point clouds, a scanning device and a storage medium. BACKGROUND

[0002] With the continuous progress of medical digitization technology, the field of oral prosthetic restoration is undergoing profound changes. Traditional impression technology is gradually being replaced by direct technology, i.e., digital impression technology, which has become the mainstream technology for making prostheses.

[0003] The optical scanning instrument on which the digital impression technology depends is the intraoral scanner (IOS). With the widespread popularity of this technology, the impression making process has been shortened, and the time cost of doctors and patients has also been reduced. However, with the increasing competition in the oral scanning market, the real-time requirement for intraoral scanning instruments is becoming increasingly stringent. Patients and operating doctors demand immediate output, expecting the instrument to present the intraoral dental arch completely, accurately and in real time on the display during the scanning process.

[0004] To further reduce time consumption, researchers have tried to optimize the processing of fringe images. However, practice shows that this optimization method can only reduce the overall time consumption to a limited extent. This is because the most time-consuming part of the entire process is the registration of point clouds. When the dental point cloud acquires sufficient overall features, the fringe image processing will soon encounter a bottleneck.

[0005] Due to the small space in the oral cavity, most intraoral scanning instruments currently use small and high-precision digital projection devices as projection equipment. When reconstructing the digital model of the entire dental arch, a large amount of three-dimensional dental data from multiple perspectives needs to be collected by the scanning instrument and then spliced. In order to quickly form a complete three-dimensional impression reconstruction, registration with the generated point cloud must be completed in real time.

[0006] At present, most instruments on the market use a combination of RANSAC (Random Sample Consensus) and ICP (Iterative Closest Point) algorithms for point cloud registration. This algorithm combination can generate a complete and high-precision dental arch, but it has obvious drawbacks, i.e., it takes a long time and cannot meet the urgent demands of patients and operating doctors in terms of real-time performance.

[0007] Therefore, how to develop a point cloud registration method that can improve matching accuracy while ensuring real-time registration in view of the particularity of the oral environment has become a technical problem to be solved. SUMMARY

[0008] The purpose of the present application is to provide a high-density point cloud real-time registration method, device, scanning equipment and storage medium, to solve the technical problem that the existing point cloud registration method cannot balance matching accuracy and registration real-time performance.

[0009] In a first aspect, a high-density point cloud real-time registration method is provided, which adopts the technical solution described as follows:

[0010] Obtain two groups of sparse point cloud data generated by intraoral three-dimensional scanning, the point cloud data including a source point cloud and a target point cloud;

[0011] Input the point cloud data into a constructed efficient registration model;

[0012] Extract multi-dimensional feature descriptions of the source point cloud and the target point cloud, and generate an initial point pair set based on the multi-dimensional feature descriptions;

[0013] Perform spatial consistency analysis on the initial point pair set to generate a high-dimensional point pair set with high-dimensional geometric spatial features;

[0014] Remove outlier point pairs of the high-dimensional point pair set to obtain a cluster point pair subset;

[0015] Perform point cloud registration calculation on the cluster point pair subset to obtain a point cloud registration result, and generate a complete intraoral three-dimensional digital model according to the point cloud registration result.

[0016] In a second aspect, a high-density point cloud real-time registration device is provided, which adopts the technical solution described as follows:

[0017] An acquisition module is configured to acquire two groups of sparse point cloud data generated by intraoral three-dimensional scanning, the point cloud data including a source point cloud and a target point cloud;

[0018] An input module is configured to input the point cloud data into a constructed efficient registration model;

[0019] A feature matching module is configured to extract multi-dimensional feature descriptions of the source point cloud and the target point cloud, and generate an initial point pair set based on the multi-dimensional feature descriptions;

[0020] A spatial characteristic processing module is configured to perform spatial consistency analysis on the initial point pair set to generate a high-dimensional point pair set with high-dimensional geometric spatial features;

[0021] An outlier point removal module is configured to remove outlier point pairs of the high-dimensional point pair set to obtain a cluster point pair subset;

[0022] The optimal solution module is configured to perform rigid transformation matrix calculation on the cluster point pair subset to obtain a point cloud registration result, and generate a complete intraoral three-dimensional digital model according to the point cloud registration result.

[0023] In a third aspect, a scanning device is provided, which employs the technical solutions described below.

[0024] Embodiments of the present application provide a scanning device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the real-time registration method of high-density point clouds of any one of the first aspect when executing the computer program.

[0025] In a fourth aspect, a computer-readable storage medium is provided, which employs the technical solutions described below.

[0026] The computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions are executed by a processor to implement the steps of the real-time registration method of high-density point clouds.

[0027] Compared with the prior art, the present application has the following beneficial effects:

[0028] The present application provides a real-time registration method of high-density point clouds, which includes inputting the obtained point cloud data into a constructed efficient registration model, extracting multi-dimensional feature descriptions of source point clouds and target point clouds, generating an initial point pair set based on the multi-dimensional feature descriptions, performing spatial consistency analysis on the initial point pair set to generate a high-dimensional point pair set with high-dimensional geometric space features, eliminating outlier point pairs of the high-dimensional point pair set to obtain a cluster point pair subset, performing point cloud registration calculation on the cluster point pair subset to obtain a point cloud registration result, and generating a complete intraoral three-dimensional digital model according to the point cloud registration result; the present application generates an initial point pair set by extracting multi-dimensional feature descriptions of source point clouds and target point clouds, which improves the reliability of feature matching and optimizes the calculation efficiency; then, the spatial consistency analysis helps to improve the expression ability of features and ensures that the spatial information of point pairs is fully utilized, thereby improving the accuracy and reliability of data processing; next, the elimination of outlier point pairs can more accurately reflect the geometric consistency between point clouds and enhance feature consistency, thereby improving registration accuracy; finally, the point cloud registration calculation on the cluster point pair subset can improve the accuracy and stability of point pair matching results and effectively ensure registration quality, therefore, the point cloud registration by the efficient registration model effectively improves the quality and speed of intraoral dense point cloud registration with low signal-to-noise ratio, improves registration accuracy and stability, and meets the real-time performance requirements of intraoral scanners. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the solutions in the present application, the drawings needed to be used in the embodiments will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0030] Figure 1 is an exemplary system architecture diagram in which the present application can be applied;

[0031] Figure 2 is a flow chart of one embodiment of the real-time registration method of high-density point clouds according to the present application;

[0032] Figure 3 is a flow chart of step S205 of the real-time registration method of high-density point clouds provided by another embodiment of the present application

[0033] Figure 4 is a structural schematic diagram of one embodiment of the real-time registration device of high-density point clouds according to the present application;

[0034] Figure 5 is a structural schematic diagram of one embodiment of the scanning device according to the present application;

[0035] Figure 6 is a result curve of different evaluation indexes of three comparison methods in coarse and fine registration of point clouds under the presence or absence of noise;

[0036] Figure 7 is a point cloud coincidence situation diagram of three registration models in the coarse registration stage under the absence of noise;

[0037] Figure 8 is a fitting degree comparison diagram of 3D-SCG and RANSAC two registration models on the same part surface edge. DETAILED DESCRIPTION

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the specification herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application; the description and claims of the present application as well as the above drawings, illustrate and set forth rather than limit the present application; the terms "comprise", "comprising", "include", "including" and "has" and their variants as used herein are intended to cover a non-exclusive inclusion; the terms "first", "second" and the like in the description and claims of this application and above drawings are used for distinguishing between similar objects and not necessarily for describing a specific sequential or chronological order.

[0039] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0040] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0041] like Figure 1 As shown, system architecture 100 may include terminal device 101, network 102, and server 103. Terminal device 101 may be a laptop 1011, tablet 1012, or mobile phone 1013. Network 102 is used as a medium to provide a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables.

[0042] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various communication client applications can be installed on terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0043] Terminal device 101 can be various electronic devices with a display screen and support web browsing. In addition to laptops 1011, tablets 1012, or mobile phones 1013, terminal device 101 can also be e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers.

[0044] Server 103 can be a server that provides various services, such as a backend server that provides support for the pages displayed on terminal device 101.

[0045] It should be noted that the real-time registration method for high-density point clouds provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the real-time registration device for high-density point clouds is generally set in the server / terminal device.

[0046] It should be understood thatFigure 1 The number of terminal devices, networks and servers in the system is only illustrative. Depending on the implementation needs, there can be any number of terminal devices, networks and servers.

[0047] With reference to Figure 2 , a flowchart of one embodiment of a real-time registration method for high-density point clouds according to the present application is shown, comprising the following steps:

[0048] Step S201, obtaining two groups of sparse point cloud data generated by intraoral three-dimensional scanning, the point cloud data including source point cloud and target point cloud.

[0049] The point cloud data can be obtained by scanning the inside of the patient's mouth with an intraoral scanner (IOS). Specifically, when the scanner moves inside the mouth, it can collect three-dimensional spatial information of structures such as teeth and gums inside the mouth, thereby generating point cloud data. The point density is low (usually point density <1000 points / cm 2 ), which is called sparse point cloud.

[0050] Point cloud data is a data form used to represent the surface information of objects in three-dimensional space. It is composed of a series of discrete points, each point containing its coordinates in three-dimensional space (usually X, Y, Z values of three dimensions). These points can accurately describe the shape, contour and texture of the object surface. In the context of oral scanning, point cloud data can reflect detailed information about the shape, size, position of teeth, and the morphology of gums, etc. Point cloud data includes source point cloud (SRC) and target point cloud (TGT), where the source point cloud refers to the point cloud data obtained from a certain perspective or scanning position during oral scanning, which is the reference point cloud in the subsequent point cloud registration process; the target point cloud refers to the point cloud data obtained from another perspective or scanning position during oral scanning, which needs to be registered with the source point cloud. In the point cloud registration process, the target point cloud will be matched and aligned with the source point cloud to achieve complete reconstruction of the three-dimensional model of the entire oral cavity.

[0051] Step S202, inputting the point cloud data into the constructed efficient registration model.

[0052] The high-efficiency registration model is a high-efficiency registration model (3D-Spatial Consistent Grouping, 3D-SCG) for real-time registration of high-density point clouds of intraoral 3D scanning, comprising a feature matching module (Matching module), a spatial characteristic processing module (Tooth Net module), an outlier removal module (Outlier Removal module) and an optimal solution module, which can improve the real-time performance of the scanner and further reduce the production time of the digital impression.

[0053] In step S203, multi-dimensional feature descriptions of the source point cloud and the target point cloud are extracted, and an initial point pair set is generated based on the multi-dimensional feature descriptions.

[0054] In the embodiment, the feature matching module of the high-efficiency registration model analyzes the features of the two groups of sparse point cloud data (the source point cloud and the target point cloud), extracts multi-dimensional feature descriptions, and according to the multi-dimensional feature descriptions and the feature distance, the nearest two points between the point clouds are combined into a related point pair to form an initial point pair set.

[0055] Further, the above step of extracting multi-dimensional feature descriptions of the source point cloud and the target point cloud and generating an initial point pair set based on the multi-dimensional feature descriptions comprises:

[0056] For the source point cloud and the target point cloud, a fast point feature histogram algorithm is used to extract multi-dimensional feature descriptions of each point;

[0057] The feature distance between each point in the source point cloud and each point in the target point cloud is calculated;

[0058] For each point in the source point cloud, according to the multi-dimensional feature description, the point with the smallest feature distance in the target point cloud is searched, a related point pair is formed, and all related point pairs are integrated into an initial point pair set.

[0059] A fast point feature histogram (FPFH) algorithm is used to extract multi-dimensional feature descriptions of each point in the source point cloud and the target point cloud. The multi-dimensional feature description is the FPFH feature, which is used to capture the local geometric features of the point, and is generated by counting the three-dimensional geometric relationship between the point and its neighborhood points, including the angle between the normal vectors, the distance from the point to the tangent plane and the angle between the two normal vectors in the tangent plane.

[0060] The coordinates of each point S in the source point cloud SRC are obtained, and the feature distance between each point T in the target point cloud TGT and each point S is calculated, and the calculation formula is as follows:

[0061]

[0062] Wherein, S (x,y) represents the coordinates of point S; T(x,y) coordinates of point T; inverse coordinates of point T.

[0063] According to the calculated feature distance between point S and each point T, the point T with the minimum feature distance to point S is retrieved, the two are associated with a point pair, and all associated point pairs are integrated into an initial point pair set Corr, in the following specific way:

[0064] Corr={(S (x,y) ,T (x,y) )|S (x,y) ∈SRC fpfh , T (x,y) ∈TGT fpfh};

[0065] wherein SRC fpfn and TGT fpfh represent that the points S and T come from SRC and TGT with FPFH features.

[0066] For example, the process of obtaining the initial point pair set from the source point cloud and the target point cloud can be understood as a key link for the preliminary matching of intraoral scanning data. Assuming that the point cloud data of the upper and lower teeth is scanned, the source point cloud represents the upper teeth, and the target point cloud represents the lower teeth. The FPFH feature of each point is extracted by the FPFH algorithm, and the nearest point pair is selected according to the spatial distance between two points. This method can quickly establish the corresponding relationship between points and points, laying a foundation for subsequent registration. Such processing helps to reduce the interference of irrelevant point pairs and improve the accuracy of matching.

[0067] Extracting multi-dimensional feature descriptions of the source point cloud and the target point cloud by the FPFH algorithm and calculating the feature distance to form the initial point pair set can improve the reliability of feature matching, enhance the robustness of the algorithm, and optimize the calculation efficiency.

[0068] In step S204, the spatial consistency of the initial point pair set is analyzed to generate a high-dimensional point pair set with high-dimensional geometric spatial features.

[0069] In this embodiment, a Tooth Net module (spatial characteristic processing module) is designed based on the spatial characteristics of the point cloud, taking point pairs as the minimum input unit, combining the spatial consistency between nearby neighbor point pairs, and generating high-dimensional geometric spatial features of point pairs.

[0070] Further, the step of analyzing the spatial consistency of the initial point pair set to generate a high-dimensional point pair set with high-dimensional geometric spatial features includes:

[0071] Taking each associated point pair in the initial point pair set as an input unit, the dimension of the associated point pair is adjusted to a preset high-dimensional space.

[0072] generate a spatial consistency parameter according to spatial consistency between neighborhood point pairs of the associated point pair;

[0073] perform feature conversion on the associated point pair through the spatial consistency parameter to obtain a high-dimensional point pair set with high-dimensional geometric space features.

[0074] Specifically, the initial point pair set is preliminarily processed, and the spatial position information of each associated point pair in the initial point pair set Corr is obtained, where the point pair composed of two points can be expressed as a point pair (s, t) with a dimension of (N, 6), where s ∈ SRC and t ∈ TGT. The dimension of the associated point pair is converted to a preset high-dimensional space, that is, the associated point pair (s, t) is converted from a dimension of (N, 6) to a dimension of (N, 128). The relative position relationship between the point pairs is constructed based on the spatial distribution characteristics of the neighborhood point pairs to generate the spatial distribution features of the associated point pair. Based on the spatial distribution features of the associated point pair, the local consistency information of each associated point pair in the high-dimensional geometric space is further extracted, and the local geometric constraint condition d st of each point pair is determined by comparing the position relationship between the neighborhood point pairs to form the local consistency description of the point pair, that is, the spatial consistency parameter τ. The high-dimensional geometric space features of each associated point pair in the point pair set are refined based on the spatial consistency parameter τ of the point pair to generate the high-dimensional geometric space feature representation of each point pair, and the high-dimensional geometric space feature representation of each point pair is integrated into a high-dimensional point pair set.

[0075] wherein the spatial consistency parameter τ and the local geometric constraint condition d st are calculated as follows:

[0076]

[0077] d st =|||s i -s j ||-||t i -t j |||;

[0078] wherein (s i , t i ) represents the i-th associated point pair; σ d is set to 0.01, representing the associated point pair; [·] + represents the maximum value compared with 0.

[0079] By analyzing the spatial characteristics of the point pair, the expression ability of the features can be improved, the spatial information of the point pair can be fully utilized, and the accuracy and reliability of data processing can be significantly improved.

[0080] Step S205, remove outlier pairs of the high-dimensional point pair set to obtain a cluster point pair subset.

[0081] An outlier removal module is designed for the outlier pairs of the high-dimensional point pair set. The point pairs are regarded as nodes, and the relationship between the point pairs is regarded as a graph, so as to determine the outlier pairs and remove them.

[0082] Further, referring to Figure 3 As shown in the above, the step of removing the outlier pairs of the high-dimensional point pair set to obtain a cluster point pair subset includes:

[0083] Step S301, calculate the confidence of each point pair in the high-dimensional point pair set.

[0084] In the outlier removal module, the confidence of each point pair in the high-dimensional point pair set is calculated using a multi-layer neural network. In some embodiments, the multi-layer neural network is trained and can use a multi-layer perceptron, including an input layer, multiple attention layers, and an output layer, wherein each attention layer includes a self-attention layer, a feedforward network layer, a normalization layer, and a residual connection layer.

[0085] Step S302, use a non-maximum suppression algorithm to screen the uniformly distributed matching point pairs according to the confidence to form an initial point pair subset.

[0086] The non-maximum suppression algorithm is used to preliminarily screen the point pairs according to the confidence to obtain candidate point pairs. Specifically, each point pair in the high-dimensional point pair set is sorted in descending order of confidence, and each point pair is processed in turn. If the confidence of the current point pair is higher than the confidence of all other point pairs in its preset neighborhood radius, the point pair is retained, otherwise it is suppressed (i.e. deleted).

[0087] To avoid missing detection at a single scale, multiple neighborhood radii can be set to perform non-maximum suppression, and the union of high-confidence and locally unique point pairs is finally retained.

[0088] The local density of the candidate point pairs (such as the number of point pairs per unit volume) is calculated, and if the density of a certain region is too high (such as exceeding threshold T1) or too low (such as below threshold T2, T2

[0089] Finally, a set of matching point pairs with high confidence, uniform spatial distribution, and is output as an initial point pair subset Corr’.

[0090] Step S303, use a nearest neighbor algorithm to retrieve a preset number of nearest neighbor point pairs for each point pair in the initial point pair subset to generate an index sequence.

[0091] In the embodiment, the neighbor algorithm adopts KNN (K-Nearest Neighbors) algorithm. Assuming that the initial point pair subset Corr' contains N point pairs, the KNN algorithm (setting K as 30) is used to search 30 nearest point pairs for each point pair in the initial point pair subset Corr' in the initial point pair set Corr according to the generated 128-dimensional high-dimensional features, which can effectively reduce the size of the matrix M (from N*N to 30*30) and the matching range of the point pairs (from N pairs to 30 pairs), and form an index sequence with dimensions (N, 30, 128).

[0092] In step S304, the corresponding points are extracted from the source point cloud and the target point cloud according to the index sequence to form a target point cloud subset.

[0093] Specifically, the corresponding points of the index sequence are found from the source point cloud SRC and the target point cloud TGT with the original size and the FPFH feature description to form a target point cloud subset Corr''.

[0094] In step S305, a two-dimensional adaptive matrix is constructed according to the target point cloud subset, and the point pairs with the correlation degree not meeting the condition are removed based on the two-dimensional adaptive matrix to obtain a cluster point pair subset.

[0095] In the embodiment, the Spectral Matching (SM) algorithm is used to calculate the target point cloud subset Corr'' to further optimize the point pair matching.

[0096] Further, a two-dimensional adaptive matrix is constructed for the target point cloud subset, each element of the two-dimensional adaptive matrix representing the correlation degree between the point pairs in the source point cloud and the target point cloud; the point pairs in the target point cloud subset are screened according to the correlation degree, and the point pairs with the correlation degree lower than a preset threshold are removed to obtain a correlation point pair subset; the weighted values of the point pairs in the correlation point pair subset are calculated, the point pair with the largest weighted value is selected as an internal connection point pair, and a final cluster point pair subset after removing outliers is generated.

[0097] Specifically, a two-dimensional adaptive matrix M is constructed, each element in the two-dimensional adaptive matrix M representing the correlation degree of all point pairs in the source point cloud SRC and the target point cloud TGT, all point pairs are taken as a list L, and the correlation degree of all point pairs in M(a, b) is calculated.

[0098]

[0099]

[0100] Wherein, the node a represents each point pair in the list L; M(a, b) represents the correlation degree, i.e. the connection between the nodes; D represents the distance between the two point pairs; σ d is 0.01, representing an internal connection point pair.

[0101] If the difference between the point pairs is too large, i.e. the correlation degree is lower than the preset threshold, the point pairs are removed from the list L, and only the point pairs meeting the limit condition are retained to form a subset of correlated point pairs.

[0102] The weighted values of the point pairs in the subset of correlated point pairs are calculated, and the maximum value is selected as the final weighted value λ of the point pair, and the higher the weighted value is, the greater the possibility of the in-line point pair is.

[0103] λ = argmax (M (a, b)).

[0104] The finally generated subset of clustered point pairs after removing outliers represents the maximum value of the nearest neighbor points of each point in the initial point pair subset Corr', i.e. the point pair has high correlation.

[0105] By constructing a two-dimensional adaptive matrix and removing point pairs with low correlation, the geometric consistency between point clouds can be more accurately reflected, the feature consistency is enhanced, and the registration accuracy is improved.

[0106] In step S206, point cloud registration calculation is performed on the subset of clustered point pairs to obtain a point cloud registration result.

[0107] In the embodiment, the optimal solution module effectively reduces the time consumption of calculating the rotation matrix parameters by replacing multiple iterative solutions with the calculation of point pairs in the existing region.

[0108] The optimal solution module mainly performs point cloud registration calculation on the subset of clustered point pairs. When performing point cloud registration calculation, the subset of clustered point pairs is subjected to two rigid transformation matrix calculations. Compared with using all matching points to calculate the rigid transformation matrix, using a set of point pairs with high reliability as the matrix solution range can more quickly complete the solution of the optimal rigid transformation matrix.

[0109] Further, the maximum value of the nearest neighbor points is used as the constructed weighted value to perform the first rigid transformation matrix calculation on the subset of clustered point pairs to generate a preliminary rigid transformation set. The rigid transformation parameter set with the minimum error is selected from the preliminary rigid transformation set as the optimal rigid transformation parameter set. The second rigid transformation matrix calculation is performed on the subset of clustered point pairs by using the optimal rigid transformation parameter set to determine the optimal rigid transformation matrix. The source point cloud and the target point cloud are registered according to the optimal rigid transformation matrix, and a point cloud registration result is output.

[0110] The point pair subset is preliminarily processed, and the nearest neighbor point information of each point is extracted therefrom to construct a feature set containing the maximum value of the nearest neighbor points, and the feature set is taken as the basis for subsequent calculation to generate corresponding weight data for each point. The weight data constructed from the maximum value of the nearest neighbor points is used to perform first rigid transformation matrix calculation on the point pair subset, specifically, the maximum value of the nearest neighbor points is taken as the weight of the least square method, and a set of rigid transformation parameters is generated for each point by the least square method to form a preliminary rigid transformation parameter set. The error of the rigid transformation parameter of each point is evaluated from the preliminary rigid transformation parameter set, and the transformation parameter with the minimum error is selected to form an optimal rigid transformation parameter set. The optimal rigid transformation parameter set is used to perform second rigid transformation matrix calculation on the point pair subset, a spatial restriction condition between the point pairs is added, and a final rigid transformation matrix, i.e., an optimal rigid transformation matrix, is determined. The source point cloud and the target point cloud are registered according to the optimal rigid transformation matrix to generate a complete intraoral three-dimensional digital model.

[0111] In some optional implementations, the step of selecting the rigid transformation parameter with the minimum error from the preliminary rigid transformation set as the optimal rigid transformation parameter set includes:

[0112] A rigid transformation parameter function is constructed according to the point pairs in the point pair subset, and the rigid transformation parameter function is used to calculate a rotation matrix and a translation vector;

[0113] The point pair subset is subjected to weighted error calculation by the rigid transformation parameter function to generate an error value set;

[0114] The parameter corresponding to the minimum error is selected from the error value set as the optimal rigid transformation parameter set.

[0115] The optimal rigid transformation parameter function is as follows:

[0116]

[0117] wherein R represents an initial rotation matrix of the rigid transformation; t represents an initial translation vector of the rigid transformation; R' represents a rotation matrix of the rigid transformation with the minimum error; t' represents a translation vector of the rigid transformation with the minimum error; and n represents the nth point pair in Corr'.

[0118] The optimal rigid transformation parameter set is formed by selecting the parameter with the minimum error, which ensures the reliability of the transformation parameter and provides a guarantee for further accurate calculation.

[0119] In the embodiment, the spatial restriction condition is added as the basis for judging the in-line point pairs according to the point pairs in the point pair subset, and the maximum value is selected as the optimal rigid transformation matrix, which is as follows:

[0120]

[0121] wherein γ represents a spatial restriction condition, which can be set according to actual conditions.

[0122] Through twice rigid transformation matrix calculation on the cluster point pair subset, the accuracy and stability of the point pair matching result are significantly improved, and especially in the registration task of complex geometric shapes, the registration quality can be effectively guaranteed.

[0123] The real-time registration method of high-density point clouds of the application can be executed by a digital dental impression instrument, and the point cloud registration is performed through an efficient registration model, which effectively improves the quality and speed of the registration of the dense point cloud with low signal-to-noise ratio in the mouth, improves the registration accuracy and stability, and guarantees the real-time performance requirement of the intraoral scanner.

[0124] Further referring to Figure 4 , as an implementation of the method shown in Figure 2 , the application provides an embodiment of a real-time registration device for high-density point clouds, which corresponds to the method embodiment shown in Figure 2 , and the device can be specifically applied to various electronic devices.

[0125] As shown in Figure 3 , the real-time registration device for high-density point clouds 400 comprises an acquisition module 401, an input module 402, a feature matching module 403, a spatial characteristic processing module 404, an outlier elimination module 405, and an optimal solution module 406. Among them:

[0126] The acquisition module 401 is used for acquiring two groups of sparse point cloud data generated by intraoral three-dimensional scanning, and the point cloud data comprises source point cloud and target point cloud;

[0127] The input module 402 is used for inputting the point cloud data into the constructed efficient registration model;

[0128] The feature matching module 403 is used for extracting multi-dimensional feature descriptions of the source point cloud and the target point cloud, and generating an initial point pair set based on the multi-dimensional feature descriptions;

[0129] The spatial characteristic processing module 404 is used for performing spatial consistency analysis on the initial point pair set to generate a high-dimensional point pair set with high-dimensional geometric spatial characteristics;

[0130] The outlier elimination module 405 is used for eliminating outlier point pairs of the high-dimensional point pair set to obtain a cluster point pair subset;

[0131] The optimal solution module 406 is used for performing point cloud registration calculation on the cluster point pair subset to obtain a point cloud registration result, and generating a complete intraoral three-dimensional digital model according to the point cloud registration result.

[0132] The feature matching module, the spatial characteristic processing module, the outlier elimination module, and the optimal solution module are components of the efficient registration model, and the efficient registration model is part of the real-time registration device 400 for high-density point clouds.

[0133] The real-time registration device 400 for high-density point clouds performs point cloud registration through the efficient registration model, effectively improves the quality and speed of registration of intraoral low signal-to-noise ratio dense point clouds, improves the registration accuracy and stability, and meets the real-time performance requirements of the intraoral scanner.

[0134] In some optional implementations, the feature matching module 403 is further configured to:

[0135] extract a multi-dimensional feature description of each point by using a fast point feature histogram algorithm for the source point cloud and the target point cloud;

[0136] calculate a feature distance between each point in the source point cloud and each point in the target point cloud;

[0137] for each point in the source point cloud, retrieve a point with the smallest feature distance in the target point cloud according to the multi-dimensional feature description, form a corresponding point pair, and integrate all the corresponding point pairs into the initial point pair set.

[0138] In some optional implementations, the spatial characteristic processing module 404 is further configured to:

[0139] use each corresponding point pair in the initial point pair set as an input unit, adjust the dimension of the corresponding point pair to a preset high-dimensional space;

[0140] generate a spatial consistency parameter according to the spatial consistency between the neighborhood point pairs of the corresponding point pair;

[0141] perform feature conversion on the corresponding point pair through the spatial consistency parameter to obtain a high-dimensional point pair set with high-dimensional geometric spatial characteristics.

[0142] In some optional implementations, the outlier elimination module 405 includes:

[0143] a confidence calculation submodule configured to calculate a confidence of each point pair in the high-dimensional point pair set;

[0144] a non-maximum suppression submodule configured to filter out uniformly distributed matching point pairs according to the confidence by using a non-maximum suppression algorithm to form an initial point pair subset;

[0145] a near-neighbor calculation submodule configured to retrieve a preset number of near-neighbor point pairs for each point pair in the initial point pair subset by using a near-neighbor algorithm to generate an index sequence;

[0146] an extracting submodule configured to extract corresponding points from the source point cloud and the target point cloud according to the index sequence to form a target point cloud subset;

[0147] a removing submodule configured to construct a two-dimensional adaptive matrix according to the target point cloud subset, remove point pairs that do not meet a condition based on the two-dimensional adaptive matrix, and obtain a cluster point pair subset.

[0148] In some optional implementations of the embodiment, the removing submodule is further configured to:

[0149] construct a two-dimensional adaptive matrix for the target point cloud subset, each element of the two-dimensional adaptive matrix representing a degree of association between point pairs in the source point cloud and the target point cloud;

[0150] screen the point pairs in the target point cloud subset according to the degree of association, remove point pairs with a degree of association lower than a preset threshold, and obtain an associated point pair subset;

[0151] calculate a weighted value for the point pairs in the associated point pair subset, select a point pair with a maximum weighted value as an inlier point pair, and generate a final cluster point pair subset after removing outliers.

[0152] In some optional implementations, the optimal solution module 406 includes:

[0153] a first rigid transformation submodule configured to perform first rigid transformation matrix calculation on the cluster point pair subset by using the weight constructed by the maximum value of the nearest neighbor points, and generate a preliminary rigid transformation set;

[0154] a selecting submodule configured to select a rigid transformation parameter set with a minimum error from the preliminary rigid transformation set as a preferred rigid transformation parameter set;

[0155] a second rigid transformation submodule configured to perform second rigid transformation matrix calculation on the cluster point pair subset by using the preferred rigid transformation parameter set, and determine an optimal rigid transformation matrix;

[0156] a registration submodule configured to register the source point cloud and the target point cloud according to the optimal rigid transformation matrix, and output a point cloud registration result.

[0157] In some optional implementations of the embodiment, the selecting submodule is further configured to:

[0158] construct a rigid transformation parameter function according to the point pairs in the cluster point pair subset, the rigid transformation parameter function being used to calculate a rotation matrix and a translation vector;

[0159] The weighted error calculation is performed on the cluster point pair subset using the rigid body transformation parameter function to generate an error value set.

[0160] The parameter corresponding to the minimum error is selected from the set of error values ​​and used as the preferred rigid body transformation parameter set.

[0161] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0162] To address the aforementioned technical problems, this application also provides a scanning device. Please refer to the following for details. Figure 5 , Figure 5 This is a basic structural block diagram of the scanning device in this embodiment.

[0163] Scanning device 5 includes: at least one processor 51 Figure 7 Only one is shown in the diagram), memory 52, and computer program 53 stored in memory 52 and executable on at least one processor 51; when processor 51 executes computer program 53, it implements the steps in the various method embodiments described above.

[0164] The scanning device may include, but is not limited to, a processor 51 and a memory 52. ​​Those skilled in the art will understand that... Figure 7 This is merely an example of a scanning device and does not constitute a limitation on the scanning device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input / output devices, network access devices, buses, etc.

[0165] The processor 51 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0166] The memory 52 can be an internal storage unit of the scanning device 5, such as a hard disk or a memory of the scanning device, in some embodiments. The memory 52 can also be an external storage device of the scanning device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the scanning device, in other embodiments. Further, the memory 52 can include both an internal storage unit and an external storage device of the scanning device. The memory 52 is used to store an operating system, an application program, a BootLoader, data, and other programs, such as program codes of computer programs, etc. The memory 52 can also be used to temporarily store data that has been output or is to be output.

[0167] For example, the computer program 53 can be divided into one or more modules / units, which are stored in the memory 52 and executed by the processor 51 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 53 in the scanning device 5.

[0168] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0169] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above described functions. Each functional unit and module in the embodiments can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or software. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0170] The foregoing integrated units, if implemented in the form of software function units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on such understanding, all or part of the processes in the foregoing embodiment methods can be completed by instructing relevant hardware through a computer program, which can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the various method embodiments described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms. The computer-readable medium includes any entity or device capable of carrying the computer program code to the device / terminal equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0171] The application also provides another implementation, that is, to provide a computer-readable storage medium, the computer-readable storage medium stores computer-readable instructions, the computer-readable instructions can be executed by at least one processor, so that the at least one processor executes the steps of the real-time registration method of high-density point cloud as described above.

[0172] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, of course, they can also be realized by hardware, but in many cases the former is a better implementation. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a number of instructions to make a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) execute the methods described in various embodiments of the present application.

[0173] The content of the present application will be described more specifically and further illustrated by combining with specific embodiments, but these embodiments are by no means limiting to the present application.

[0174] Embodiment

[0175] The point cloud registration model PCR-Net used in this embodiment is also based on deep learning as 3D-SCG. A variety of different databases are used for comparison, and the rotation angle error (RE) and translation error (TE) evaluation indexes are added to further evaluate the final rotation matrix obtained by these models. The classic point cloud databases Bunny and ModelNet40 are combined with the tooth point cloud database, and the presence of noise is also increased, that is, the proportion of outliers in the point cloud is increased. Among them, the Bunny database and the ModelNet40 database are similar to the intraoral point cloud, and are both dense point clouds.

[0176] Three registration schemes will be used to compare the registration effect under the angles of point cloud size, density and the presence of noise, as shown in FIG. 6. Figure 6 The three registration schemes are RANSAC algorithm, PCR-Net model and the efficient registration model (3D-SCG) of the present application.

[0177] In the case where no noise is applied, RANSAC can obtain a better result, and it can be found that although there is a gap in time consumption, the other indexes are almost the same as the results of the 3D-SCG scheme. PCR-Net consumes more time than 3D-SCG, although it is a deep learning model, but since its main process needs to update the rotation matrix through multiple network iterations, the iterative process increases the calculation amount of registration and the accuracy is slightly inferior to the RANSAC algorithm.

[0178] In order to verify the robust performance of each model in the noise environment, Gaussian noise is applied to all the source point clouds used. Figure 6 The curve data shown in FIG. 7 shows the accurate registration results completed by the three models in the presence or absence of noise. The RANSAC algorithm is obviously less capable in the noise environment than the network structure model, and the 3D-SCG model still maintains high accuracy and low time consumption. In different types of point cloud data registration, it is always the model with the highest accuracy and the lowest time consumption.

[0179] Therefore, even in the noise environment, 3D-SCG can still obtain better registration performance; it is not difficult to find from the registration of large-scale point clouds that the deep learning model is significantly better than the RANSAC algorithm in time consumption. From the perspective of noise resistance, it is found that the deep learning model has better noise resistance.

[0180] Figure 7 In the case where there is no noise, the point cloud coincidence in the coarse registration stage of the three registration models. As can be seen from FIG. 8, the results are consistent with the first curve in FIG. 6, 3D-SCG has the highest degree of fit, followed by RANSAC algorithm, and PCR has the most obvious deviation. Figure 6 ​

[0181] To observe the 3D-SCG and RANSAC model in detail in the coarse registration of dense point cloud, we zoom in on the same part to view the surface edge fitting, as shown in FIG. 6. Figure 8 As shown in FIG. 6, the left side is the RANSAC model registration result, and the right side is the 3D-SCG model registration result. As can be seen from the figure, at the leftmost edge of the point cloud, it can be seen that the RANSAC algorithm still has a certain degree of deviation, which is obviously not as smooth as the 3D-SCG.

[0182] To verify the effectiveness and robustness of the real-time registration method of high-density point cloud proposed in the present application, comparative experiments were conducted on multiple public data sets and multiple registration methods. At the same time, high-density large and small scale point cloud data sets each consisting of 4 groups and real dental point cloud data sets were scanned and made by using a mouth scanner, which were used to train and test the real-time matching feasibility of the 3D-SCG model. Compared with the traditional registration scheme RANSAC+ICP, the overall time consumption of the method of the present application was reduced by about 80% under the premise of ensuring the registration accuracy. At the same time, the matching failure rate in the registration process was reduced by 33%.

[0183] In summary, the real-time registration method of high-density point cloud for intraoral 3D scanning proposed in the present application effectively improves the quality and speed of the registration of dense point cloud with low signal-to-noise ratio in the mouth, and ensures the real-time performance requirements of the intraoral scanner.

[0184] Obviously, the above-described embodiments are only part of the embodiments of the present application, not all the embodiments, and the preferred embodiments of the present application are given in the drawings, but do not limit the patent scope of the present application. The present application can be realized in many different forms, and on the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or make equivalent replacements to some technical features. Any equivalent structure made by using the contents of the specification and drawings, directly or indirectly applied to other related technical fields, is also within the scope of the patent protection of the present application.

Claims

1. A real-time registration method for high-density point clouds, characterized in that, Includes the following steps: Two sets of sparse point cloud data generated by intraoral 3D scanning are acquired, the point cloud data including source point cloud and target point cloud; The point cloud data is input into the constructed high-efficiency registration model; Extract multidimensional feature descriptions from the source point cloud and the target point cloud, and generate an initial set of point pairs based on the multidimensional feature descriptions; Perform spatial consistency analysis on the initial set of point pairs to generate a high-dimensional set of point pairs with high-dimensional geometric spatial characteristics; By removing outlier point pairs from the high-dimensional point pair set, a clustered point pair subset is obtained; Point cloud registration calculations are performed on the subset of clustered points to obtain point cloud registration results, and a complete three-dimensional digital model of the oral cavity is generated based on the point cloud registration results.

2. The real-time registration method for high-density point clouds according to claim 1, characterized in that, The step of extracting multidimensional feature descriptions of the source point cloud and the target point cloud, and generating an initial set of point pairs based on the multidimensional feature descriptions, includes: For the source point cloud and the target point cloud, a fast point feature histogram algorithm is used to extract multidimensional feature descriptions for each point; Calculate the feature distance between each point in the source point cloud and each point in the target point cloud; For each point in the source point cloud, based on the multidimensional feature description, the point with the smallest feature distance in the target point cloud is retrieved, forming an associated point pair, and all the associated point pairs are integrated into the initial point pair set.

3. The real-time registration method for high-density point clouds according to claim 1, characterized in that, The step of performing spatial consistency analysis on the initial set of point pairs to generate a high-dimensional set of point pairs with high-dimensional geometric spatial characteristics includes: Using each associated point pair in the initial point pair set as an input unit, the dimension of the associated point pair is adjusted to a preset high-dimensional space; Based on the spatial consistency between neighboring point pairs of the associated point pair, a spatial consistency parameter is generated; The associated point pairs are transformed using the spatial consistency parameter to obtain the high-dimensional point pair set with high-dimensional geometric spatial features.

4. The real-time registration method for high-density point clouds according to claim 1, characterized in that, The step of removing outlier point pairs from the high-dimensional point pair set to obtain a clustered subset of point pairs includes: Calculate the confidence score of each point pair in the high-dimensional point pair set; A non-maximum suppression algorithm is used to filter uniformly distributed matching point pairs based on the confidence level to form an initial subset of point pairs; The nearest neighbor algorithm is used to retrieve a preset number of nearest neighbor pairs for each point pair in the initial subset of point pairs, and an index sequence is generated. Based on the index sequence, corresponding points are extracted from the source point cloud and the target point cloud to form a subset of the target point cloud; A two-dimensional adaptation matrix is ​​constructed based on the target point cloud subset. Point pairs that do not meet the correlation conditions are removed based on the two-dimensional adaptation matrix to obtain a clustered point pair subset.

5. The real-time registration method for high-density point clouds according to claim 4, characterized in that, The steps of constructing a two-dimensional adaptation matrix based on the target point cloud subset, and removing point pairs whose correlation does not meet the condition based on the two-dimensional adaptation matrix to obtain a clustered point pair subset include: For the target point cloud subset, a two-dimensional adaptation matrix is ​​constructed, where each element of the two-dimensional adaptation matrix represents the correlation degree between point pairs in the source point cloud and the target point cloud; The point pairs in the target point cloud subset are filtered according to the correlation degree, and point pairs with a correlation degree lower than a preset threshold are removed to obtain a subset of related point pairs. Calculate the weighted value of the point pairs in the subset of associated point pairs, select the point pair with the largest weighted value as the inline point pair, and generate the final subset of clustered point pairs after removing outliers.

6. The real-time registration method for high-density point clouds according to claim 1, characterized in that, The step of performing point cloud registration calculation on the subset of clustered point pairs to obtain the point cloud registration result includes: Using the weights constructed from the maximum values ​​of the nearest neighbors, the first rigid body transformation matrix is ​​calculated for the subset of clustered points to generate a preliminary rigid body transformation set. Select the rigid body transformation parameters with the smallest error from the preliminary rigid body transformation set as the preferred rigid body transformation parameter set; Using the preferred set of rigid body transformation parameters, a second rigid body transformation matrix calculation is performed on the cluster point pair subset to determine the optimal rigid body transformation matrix; Based on the optimal rigid body transformation matrix, the source point cloud and the target point cloud are registered, and the point cloud registration result is output.

7. The real-time registration method for high-density point clouds according to claim 6, characterized in that, The step of selecting the rigid body transformation parameters with the smallest error from the preliminary rigid body transformation set as the preferred rigid body transformation parameter set includes: Based on the point pairs in the clustered point pair subset, a rigid body transformation parameter function is constructed, which is used to calculate the rotation matrix and translation vector. The weighted error calculation is performed on the cluster point pair subset using the rigid body transformation parameter function to generate an error value set. The parameter corresponding to the minimum error is selected from the set of error values ​​and used as the preferred rigid body transformation parameter set.

8. A real-time registration device for high-density point clouds, characterized in that, include: The acquisition module is used to acquire two sets of sparse point cloud data generated by a three-dimensional scan of the oral cavity, wherein the point cloud data includes a source point cloud and a target point cloud. The input module is used to input the point cloud data into the constructed high-efficiency registration model; The feature matching module is used to extract multi-dimensional feature descriptions of the source point cloud and the target point cloud, and generate an initial set of point pairs based on the multi-dimensional feature descriptions; The spatial characteristic processing module is used to perform spatial consistency analysis on the initial set of point pairs to generate a high-dimensional set of point pairs with high-dimensional geometric spatial characteristics. The outlier removal module is used to remove outlier pairs from the high-dimensional point pair set to obtain a clustered subset of point pairs. The optimal solution module is used to perform point cloud registration calculations on the clustered point pairs subset, obtain point cloud registration results, and generate a complete three-dimensional digital model of the oral cavity based on the point cloud registration results.

9. A scanning device, characterized in that, The system includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the real-time registration method for high-density point clouds as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the real-time registration method for high-density point clouds as described in any one of claims 1 to 7.

Images