Data processing method, point cloud data registration method, device and intraoral scanning device

Abstract

The present invention discloses a data processing method, a point cloud data registration method, a device and an intraoral scanning device. The registration method comprises: obtaining a first rotation and translation matrix according to a point cloud to be registered and a target surface to be registered; transforming the point cloud to be registered using the first rotation and translation matrix to obtain a first transformed point cloud, and updating the point cloud to be registered using the first transformed point cloud; judging whether the number of point cloud transformations meets a first preset condition; if so, obtaining an approximation rotation and translation matrix according to the first rotation and translation matrix using an approximation algorithm, and obtaining an approximation transformed point cloud by multiple transformations using the approximation rotation and translation matrix until a second preset condition is met; judging whether the registration is completed according to the point cloud to be registered and the target surface to be registered, and if not, returning to the step of obtaining the first rotation and translation matrix of the point cloud to be registered and the target surface to be registered, until the registration is completed. The method improves the point-to-surface registration speed by registering the point cloud to be registered with the target surface to be registered.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional scanning technology, and in particular to a point cloud data processing method, a point cloud data registration method, a point cloud data registration device and an intraoral scanning device. Background Art

[0002] The object's 3D model reconstruction process includes point cloud reconstruction, point cloud registration, point cloud fusion, surface reconstruction, etc. In the related technology, the above registration process is usually achieved by point-to-point registration and point-to-surface registration. Among them, point-to-point registration is faster and can be used for 3D reconstruction in real-time scanning. For example, the current intraoral scanning equipment uses point-to-point registration to achieve registration during oral cavity scanning, thereby achieving the above 3D reconstruction process.

[0003] However, since the positions of the points are discrete, the accuracy of calculating the corresponding points is relatively low. Compared with the point-to-point registration method, the point-to-plane registration method has higher accuracy, but the registration speed is relatively low, which to some extent affects the scope of application of point-to-plane registration. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems in the related art to a certain extent. To this end, the first object of the present invention is to provide a point cloud data registration method, which improves the point-to-surface registration speed by continuously transforming the point cloud to be registered multiple times using an approximate rotation and translation matrix during the registration process of the point cloud data and the target surface.

[0005] The second objective of the present invention is to provide a data processing method.

[0006] The third objective of the present invention is to provide a point cloud data registration device.

[0007] A fourth objective of the present invention is to provide an intraoral scanning device.

[0008] To achieve the above-mentioned purpose, the first aspect of the present invention proposes a point cloud data registration method, including: obtaining a first rotation and translation matrix based on the point cloud to be registered and the target surface to be registered; using the first rotation and translation matrix to transform the point cloud to be registered to obtain a first transformed point cloud, and using the first transformed point cloud to update the point cloud to be registered; judging whether the current number of point cloud transformations meets a first preset condition; if so, using an approximation algorithm to obtain an approximation rotation and translation matrix based on the first rotation and translation matrix, using the approximation rotation and translation matrix to continuously transform the point cloud to be registered multiple times to obtain an approximation transformed point cloud until a second preset condition is met, and using the approximation transformed point cloud to update the point cloud to be registered; judging whether the registration is completed based on the point cloud to be registered and the target surface to be registered, if not, returning to the step of obtaining the first rotation and translation matrix based on the point cloud to be registered and the target surface to be registered until the registration is completed.

[0009] According to the point cloud data registration method of the embodiment of the present invention, a first rotation and translation matrix is ​​obtained through the point cloud to be registered and the target surface to be registered, the point cloud to be registered is transformed by using the first rotation and translation matrix to obtain a first transformed point cloud, and the point cloud to be registered is updated by using the first transformed point cloud, and then it is judged whether the current number of point cloud transformations meets the first preset condition. If so, an approximation rotation and translation matrix is ​​obtained according to the first rotation and translation matrix by using an approximation algorithm, and the point cloud to be registered is transformed continuously for multiple times by using the approximation rotation and translation matrix to obtain an approximated transformed point cloud until the second preset condition is met, and the point cloud to be registered is updated by using the approximation transformed point cloud, and then it is judged whether the registration is completed according to the point cloud to be registered and the target surface to be registered, thereby effectively improving the point-to-surface registration speed.

[0010] To achieve the above object, a second aspect of the present invention provides a data processing method, comprising:

[0011] According to the point cloud to be registered and the surface that has been registered and fused, the target surface to be registered is cut out; according to the point cloud data registration method described above, registration is performed to obtain the registered point cloud, the registered point cloud is fused to the target surface to be registered, a new surface that has been registered and fused is obtained, the point cloud to be registered is updated, and the step of cutting out the target surface to be registered according to the point cloud to be registered and the surface that has been registered and fused is returned until all the point cloud data are involved in the registration and fusion.

[0012] According to the data processing method of the embodiment of the present invention, based on the point cloud to be registered and the surface that has been registered and fused, the target surface to be registered is cut out, and then the registration is performed through the above-mentioned point cloud data registration method to obtain the registered point cloud, and the registered point cloud is fused to the target surface to be registered to obtain a new surface that has been registered and fused, and the point cloud to be registered is updated, and the step of cutting out the target surface to be registered based on the point cloud to be registered and the surface that has been registered and fused is returned until all the point cloud data participate in the registration and fusion. This method can effectively improve the point-to-surface registration speed.

[0013] To achieve the above-mentioned purpose, the third aspect of the present invention proposes a point cloud data registration device, including a memory, a processor and a computer program stored in the memory. When the computer program is executed by the processor, the above-mentioned point cloud data registration method is implemented.

[0014] According to the point cloud data registration device of the embodiment of the present invention, the point cloud data registration method mentioned above realizes the accurate matching of the point cloud to be registered and the target surface to be registered, thereby improving the point-to-surface registration speed.

[0015] In order to achieve the above-mentioned purpose, a fourth aspect of the present invention proposes an intraoral scanning device, including the above-mentioned point cloud data registration device.

[0016] According to the intraoral scanning device of the embodiment of the present invention, the point cloud data registration device thereon realizes the accurate matching of the point cloud to be registered and the target surface to be registered through the above-mentioned point cloud data registration method, thereby improving the point-to-surface registration speed.

[0017] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flow chart of a point cloud data registration method according to an embodiment of the present invention;

[0019] Figure 2 A flowchart of a point cloud data registration process according to a specific example of the present invention;

[0020] Figure 3 is a structural block diagram of an intraoral scanning device according to an embodiment of the present invention;

[0021] Figure 4 is an overall flow chart of a point cloud data registration method according to a specific example of the present invention;

[0022] Figure 5 A preprocessing flow chart of a point cloud data registration method according to a specific example of the present invention;

[0023] Figure 6 A first-level judgment flow chart of a point cloud data registration method according to a specific example of the present invention;

[0024] Figure 7 A point-to-surface registration flow chart of a point cloud data registration method according to a specific example of the present invention;

[0025] Figure 8 A schematic diagram of a curved surface constructed after real-time scanning according to a specific example of the present invention;

[0026] Fig. 9 The figure is a schematic diagram of post-processing optimization of a curved surface according to a specific example of the present invention. DETAILED DESCRIPTION

[0027] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0028] The data processing method, point cloud data registration method, device and intraoral scanning equipment of the embodiments of the present invention are described below with reference to the accompanying drawings.

[0029] An embodiment of the present invention provides a data processing method, the method comprising:

[0030] (1) Based on the point cloud to be registered and the surface that has been fused, the target surface to be registered is segmented.

[0031] The intraoral scanning device can obtain point cloud data during scanning, and the point cloud data includes multiple point clouds. During the registration process, the multiple point clouds are traversed and used as multiple point clouds to be registered. During the later surface reconstruction, the surface reconstruction software corresponding to the intraoral scanning device can obtain the point cloud data with the highest registration degree based on the multiple point clouds to be registered, and reconstruct the initial registered and fused surface based on the point cloud data, that is, the initial target surface to be registered. Of course, it is also possible to select a point cloud from the obtained point cloud data in other ways and perform surface reconstruction based on the point cloud to obtain the initial registered and fused surface and the initial target surface to be registered.

[0032] In the subsequent cycle process, the surface that has completed the registration and fusion can be segmented according to the surrounding grid of the point cloud to be registered to obtain the segmented target surface to be registered.

[0033] (2) According to the point cloud data registration method, the registration point cloud and the target surface to be registered are registered to obtain the registered point cloud data. When the point cloud to be registered and the target surface to be registered are matched, the registered point cloud data are fused to the target surface to be registered to obtain a new registered and fused surface. The point cloud to be registered is updated, and the surface based on the point cloud to be registered and the registered and fused surface is returned to segment the target surface to be registered until all the point cloud data are involved in the registration and fusion.

[0034] After obtaining the target surface to be registered, the other point clouds are then registered with the target surface to be registered, and when the registration conditions are met, the remaining point clouds are fused with the target surface to be registered. As the above fusion process is repeated, the registered and fused surface continues to grow larger, thereby achieving the construction of the target 3D model. The registered and fused surface obtained in this step includes the surface that was registered and fused last time and the part that is fused to the current surface to be registered.

[0035] In specific implementation, whether the registration is successful can be determined based on whether the distance between the registered point cloud and the target surface to be registered is less than a preset threshold. If the distance is less than or equal to the preset threshold, the registration is successful. If the distance is greater than the preset threshold, the registration is unsuccessful. The distance can be an example of a pair of registered points between the registered point cloud and the target surface to be registered in the following text, and the preset threshold can be a preset registration distance threshold.

[0036] In this embodiment, after obtaining the point cloud to be registered and the surface that has completed the registration and fusion, the target surface to be registered is segmented according to the enclosing grid of the point cloud to be registered to obtain the segmented target surface to be registered, which can effectively reduce the amount of data processing tasks and improve the registration efficiency of the point cloud to be registered and the target surface to be registered.

[0037] The point cloud data registration method in this embodiment can be implemented according to any of the following point cloud data registration methods.

[0038] Figure 1 FIG. 1 is a flow chart of a point cloud data registration method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0039] S101, obtaining a first rotation and translation matrix according to the point cloud to be registered and the target surface to be registered.

[0040] Before obtaining the first rotation and translation matrix according to the point cloud to be registered and the target surface to be registered, the point cloud to be registered and the target surface to be registered are first obtained. The point cloud to be registered and the target surface to be registered can be determined according to the above method.

[0041] Wherein, obtaining the first rotation and translation matrix according to the point cloud to be registered and the target surface to be registered may include segmenting the target surface to be registered according to each point in the point cloud to be registered to obtain a target sub-surface corresponding to each point, and then obtaining a registration point pair according to each point in the point cloud to be registered and its corresponding target sub-surface, and then obtaining the first rotation and translation matrix according to the registration point pair. Figure 2 As shown, the segmented target surface to be registered can be segmented according to each point in the point cloud to be registered to obtain the target sub-surface corresponding to each point.

[0042] In one embodiment of the present invention, segmenting the target surface to be registered according to each point in the point cloud to be registered to obtain the target sub-surface corresponding to each point may include: generating a three-dimensional bounding box around each point in the point cloud to be registered; segmenting the target surface to be registered according to the three-dimensional bounding box of each point to obtain the target sub-surface corresponding to each point.

[0043] Specifically, in the process of registering the point cloud to be registered and the target surface to be registered, the segmented target surface to be registered can be further segmented. As an example, a 5*5*5mm-sized segment can be generated around each point in the point cloud to be registered. 3 Or 9*9*9mm 3 The three-dimensional bounding box of the point cloud to be registered is then re-segmented according to the three-dimensional bounding box of each point, so as to obtain the target sub-surface corresponding to each point. In this way, the registration accuracy of the point cloud to be registered and the target surface to be registered can be improved.

[0044] In one embodiment of the present invention, a target sub-surface is obtained based on triangular mesh points, and obtaining a pair of registration points of the point cloud to be registered based on each point in the point cloud to be registered and its corresponding target sub-surface may include: projecting each point in the point cloud to be registered on each triangular mesh of the corresponding target sub-surface to obtain a corresponding projection point; calculating the distance between each point in the point cloud to be registered and the corresponding projection points to obtain a plurality of first distances; and obtaining a pair of registration points of the point cloud to be registered based on the plurality of first distances corresponding to each point in the point cloud to be registered.

[0045] Specifically, after the surface to be registered obtained according to the triangular mesh points is segmented, the target sub-surface obtained may include multiple triangular mesh surfaces. In this embodiment, each point in the point cloud to be registered, such as point P, may be projected to each triangular mesh surface of the corresponding target sub-surface, such as G[j] (j=0, 1, ..., m), and it is determined whether the corresponding projection point falls within the triangular mesh surface. If so, the distance D[i] from the point P to its projection point is saved, wherein the method for obtaining the projection point may adopt the method of the sum of interior angles, the same direction method, the area method, the centroid method, etc.

[0046] Among them, obtaining the registration point pair of the point cloud to be registered according to the multiple first distances corresponding to each point in the point cloud to be registered may include: for each point in the point cloud to be registered, selecting the projection point corresponding to the first distance with the smallest value from the corresponding multiple first distances to form a point pair with the point to obtain a first point pair; discarding the point pair in the first point pair whose distance is greater than the first preset distance to obtain a second point pair; and obtaining the registration point pair of the point cloud to be registered according to the second point pair.

[0047] Specifically, taking point P as an example, among the multiple first distances corresponding to point P and its corresponding projection point, such as D[1], D[2], D[3], ..., D[i], D[5] has the smallest value. Then the projection point corresponding to D[5], such as P5, is taken to form the first point pair with P. Since there are multiple points in the point cloud to be registered, multiple first point pairs can be obtained. Figure 2 As shown, after obtaining multiple first point pairs, the point pairs whose midpoints in the first point pairs are at a distance from the projection point greater than a first preset distance (such as 0.243-0.3 mm) can be discarded to obtain second point pairs, and the registration point pairs of the point cloud to be registered are obtained based on the second point pairs.

[0048] In this embodiment, the accuracy of the registration point pairs is improved by selecting the projection point with the minimum distance between the point in the point cloud to be registered and its projection point to form the first point pair, and discarding the points in the first point pair with too large distance between them and their projection points, thereby improving the registration accuracy of the point cloud to be registered and the target surface to be registered.

[0049] In one embodiment of the present invention, the second point pair includes a first sub-point cloud and a second sub-point cloud, the points in the first sub-point cloud are all points in the point cloud to be registered, and the points in the second sub-point cloud are all points on the target surface to be registered, wherein the registration point pair of the point cloud to be registered is obtained based on the second point pair, including: determining a first zero point and a second zero point, wherein the coordinate value of the first zero point is the average value of the coordinate values ​​of all points in the first sub-point cloud, and the coordinate value of the second zero point is the average value of the coordinate values ​​of all points in the second sub-point cloud; obtaining a first registration point based on a weighted distance calculation between each point in the first sub-point cloud and the first zero point, obtaining a second registration point based on a weighted distance calculation between each point in the second sub-point cloud and the second zero point, and pairing the first registration point with the second registration point into a registration point pair.

[0050] The above weighted distance can be calculated according to the following formula:

[0051] Weighted value dw[i]=(D-distance[i])*T[i], where distance[i] is the minimum distance of the corresponding point projections, T[i] is the weighting coefficient, and D is the threshold, that is, the first preset distance, such as 0.243-0.3 mm.

[0052] R1[i]=(P1[i]–L1)*dw[i], i=0……N, N is the number of point pairs, R1 is the first registration point, P1 is the first sub-point cloud, and L1 is the first zero point.

[0053] R2[i]=(P2[i]–L2)*dw, i=0……N, N is the number of point pairs, R2 is the second registration point, P2 is the first sub-point cloud, and L2 is the first zero point.

[0054] It can be understood that the second point pair obtained is composed of points in the point cloud to be registered and its projection points, wherein the projection points are points on the triangular mesh surface of the target sub-surface. Since there are multiple points in the point cloud to be registered, the second point pair includes the first sub-point cloud and the second sub-point cloud, wherein the points in the first sub-point cloud are all points in the point cloud to be registered, and the points in the second sub-point cloud are all points on the target sub-surface, i.e., the target surface to be registered. For example, the first sub-point cloud is A[X]={P1, P2, P3,..., Px}, and the second sub-point cloud is B[X]={P11, P21, P31,..., Px1}, wherein Px1 is the projection point of Px, and the first sub-point cloud A[X] and the second sub-point cloud B[X] can constitute x second point pairs such as (Px, Px1).

[0055] Further, when obtaining the registration point pair of the point cloud data according to the second point pair, the first zero point P0 and the second zero point P0 of the first sub-point cloud A[X] and the second sub-point cloud B[X] can be obtained respectively according to the points P1, P2, P3, ..., Px in the first sub-point cloud A[X] and the points P11, P21, P31, ..., Px1 in the second sub-point cloud B[X]. ’ , and then calculate the weighted distance between each point P1, P2, P3, ..., Px in the first sub-point cloud A[X] and the first zero point P0 to obtain the first registration point A ’ [X] = {P10, P20, P30, ..., Px0}, and the points P11, P21, P31, ..., Px1 in the second sub-point cloud B[X] are connected to the second zero point P0 ’ The weighted distance calculation obtains the second registration point B ’ [X] = {P110, P210, P310, ..., Px10}, thus obtaining the registration point pair A ’ [X]-B ’ [X], such as (Px0, Px10).

[0056] In this embodiment, the registration point pairs are obtained by performing weighted distance calculations between each point in the first sub-point cloud and the second sub-point cloud and the first zero point and the second zero point, respectively, so that the influence factors of points with closer corresponding points are larger, thereby improving the registration efficiency.

[0057] S102, transform the point cloud to be registered using the first rotation and translation matrix to obtain a first transformed point cloud, and use the first transformed point cloud to update the point cloud to be registered. ’ [X]-B ’ [X], obtain the first rotation and translation matrix T1 through the quaternion method, singular value decomposition method and other methods, and then transform the first sub-point cloud A[X] in the point cloud to be registered through the first rotation and translation matrix T1 to obtain the first transformed point cloud Q[X], and use the first transformed point cloud Q[X] to update the first sub-point cloud A[X] in the point cloud to be registered.

[0058] S103, determining whether the current point cloud transformation times meets a first preset condition.

[0059] The current point cloud transformation times satisfying the first preset condition may include the current point cloud transformation times being less than or equal to a first preset value (such as 1). If the current point cloud transformation times exceeds the preset times (such as 5), the approximation step is skipped, that is, S104 is skipped.

[0060] S104, if satisfied, using an approximation algorithm to obtain an approximation rotation and translation matrix according to the first rotation and translation matrix, using the approximation rotation and translation matrix to continuously transform the point cloud to be registered multiple times to obtain an approximation transformation point cloud, until the second preset condition is met, and using the approximation transformation point cloud to update the point cloud to be registered.

[0061] Specifically, refer to Figure 2 As shown, if the current number of point cloud transformations meets the first preset condition, such as the updated point cloud to be registered, that is, the first transformed point cloud Q[X], is obtained by transforming the point cloud to be registered once, that is, the first transformation, an approximation algorithm can be used to iterate the first rotation and translation matrix T1 to obtain the approximate rotation and translation matrix T.

[0062] Among them, using the approximation algorithm to obtain the approximate rotation and translation matrix T according to the first rotation and translation matrix T1 may include obtaining the rotation angle (θx, θy, θz) and the translation h according to the first rotation and translation matrix T1, and then calculating the first average distance dMean of the point cloud to be registered before updating and the second average distance dMean′ of the first transformed point cloud Q[X], wherein the first average distance dMean of the point cloud to be registered is the average value of the distances of all point pairs in the registration point pairs, and the second average distance of the first transformed point cloud Q[X] is the average value of the distances between the points in the registration point pairs on the first transformed point cloud Q[X] and the corresponding points on the target surface to be registered, and then the approximate rotation and translation matrix T is obtained according to the first average distance dMean, the second average distance dMean′, the rotation angle (θx, θy, θz) and the translation h.

[0063] Among them, obtaining the approximate rotation transformation matrix T based on the first average distance dMean, the second average distance dMean′, the rotation angle (θx, θy, θz) and the translation h may include calculating the ratio of the second average distance dMean′ to the first average distance dMean; multiplying the ratio by the rotation angle (θx, θy, θz) and the translation h to obtain the approximate rotation translation matrix T.

[0064] Further, using the approximate rotation and translation matrix T to continuously transform the point cloud to be registered multiple times to obtain the approximate transformed point cloud until the second preset condition is met may include: using the approximate rotation and translation matrix T to transform the updated point cloud to be registered to obtain a second transformed point cloud; calculating the second average distance of the second transformed point cloud, and judging whether the second average distance is less than the second average distance calculated last time, the second average distance of the second transformed point cloud is the average value of the distance between the point on the second transformed point cloud and the corresponding point on the target surface to be registered in the registration point pair; if less than, updating the point cloud to be registered using the second transformed point cloud, and returning to the step of transforming the point cloud to be registered using the approximate rotation and translation matrix to obtain the second transformed point cloud; if greater than or equal to, the second preset condition is met.

[0065] For example, the first transformed point cloud Q[X] is transformed using the approximate rotation and translation matrix T to obtain the second transformed point cloud Q[X] ’ , then calculate the second transformed point cloud Q[X] ’ The second average distance dMean′ is then determined to be less than the second average distance dMean′ calculated last time. If it is less than, the second transformation point cloud Q[X] is transformed using the approximate rotation and translation matrix T. ’ Transform to obtain the updated point cloud to be registered, calculate the second average distance dMean′ of the updated point cloud to be registered, and then determine whether the second average distance dMean′ is less than the second average distance dMean′ calculated last time. If the second average distance dMean′ is greater than or equal to the second average distance dMean′ calculated last time, it indicates that the second average distance dMean′ of the updated point cloud to be registered is no longer reduced, and the updated point cloud to be registered meets the second preset condition.

[0066] S105, judging whether the registration is completed according to the point cloud to be registered and the target surface to be registered; if not, returning to the step of obtaining the first rotation and translation matrix according to the point cloud to be registered and the target surface to be registered until the registration is completed.

[0067] Among them, judging whether the registration is completed according to the point cloud to be registered and the target surface to be registered includes:

[0068] Determine whether the distance of the registration point pair is less than a preset registration distance threshold, and determine whether the current point cloud transformation times is equal to a preset transformation times threshold;

[0069] If the distance between the registration point pairs is less than the preset registration distance threshold, or the number of current point cloud transformations is equal to the preset transformation number threshold, the registration is determined to be completed; otherwise, the registration is determined to be incomplete.

[0070] This step can be used to determine whether the distance between the updated point cloud to be registered and the surface to be registered is small enough or exceeds the set number of iterations. If the conditions are met, the registration is completed.

[0071] Specifically, the updated point cloud to be registered can be updated to the approximate transformation point cloud, and then it is determined whether the dMean′ at this time is less than the set threshold. If it is less than, it means that the point cloud to be registered and the target surface to be registered meet the registration conditions, and the registration is completed. Alternatively, it is determined whether the number of iterations is greater than the set number of iterations. If it is greater, it means that the point cloud to be registered and the target surface to be registered do not meet the registration conditions, and the registration is completed. If both of the above conditions are not met, the iteration is entered, and then the first rotation and translation matrix is ​​obtained according to the updated point cloud to be registered and the target surface to be registered, and then the updated point cloud to be registered is transformed using the first rotation and translation matrix until the transformed point cloud to be registered and the target surface to be registered meet the registration conditions or exceed the number of iterations.

[0072] According to the point cloud data registration method of the embodiment of the present invention, a first rotation and translation matrix is ​​obtained through the point cloud to be registered and the target surface to be registered, and the first rotation and translation matrix is ​​used to transform the point cloud to be registered to obtain a first transformed point cloud to update the point cloud to be registered, and then it is judged whether the current number of point cloud transformations meets the first preset condition. If so, an approximation algorithm is used to obtain an approximation rotation and translation matrix based on the first rotation and translation matrix, and the approximation rotation and translation matrix is ​​used to continuously transform the point cloud to be registered multiple times to obtain an approximation transformed point cloud until the second preset condition is met, and the approximation transformed point cloud is used to update the point cloud to be registered, and then it is judged whether the registration is completed based on the point cloud to be registered and the target surface to be registered, thereby effectively improving the point-to-surface registration speed.

[0073] Furthermore, the present invention also proposes a data processing method, comprising: cutting out a target surface to be registered according to the point cloud to be registered and the surface that has been registered and fused; performing registration according to the above-mentioned point cloud data registration method to obtain a registered point cloud, fusing the registered point cloud to the target surface to be registered to obtain a new surface that has been registered and fused, updating the point cloud to be registered, returning to the step of cutting out the target surface to be registered according to the point cloud to be registered and the surface that has been registered and fused, until all point cloud data participate in the registration and fusion.

[0074] According to the data processing method of the embodiment of the present invention, based on the point cloud to be registered and the surface that has been registered and fused, the target surface to be registered is cut out, and then the registration is performed through the above-mentioned point cloud data registration method to obtain the registered point cloud, and the registered point cloud is fused to the target surface to be registered to obtain a new surface that has been registered and fused, and the point cloud to be registered is updated, and the step of cutting out the target surface to be registered based on the point cloud to be registered and the surface that has been registered and fused is returned until all the point cloud data participate in the registration and fusion. This method can effectively improve the point-to-surface registration speed.

[0075] Furthermore, the present invention also proposes a point cloud data registration device 100, comprising a memory 101, a processor 102 and a computer program stored in the memory 101, and when the computer program is executed by the processor 102, the above-mentioned point cloud data registration method is implemented.

[0076] According to the point cloud data registration device of the embodiment of the present invention, the point cloud to be registered and the target surface to be registered are accurately matched through the above-mentioned point cloud data registration method, thereby improving the point-to-surface registration speed. Furthermore, the present invention also proposes an intraoral scanning device, referring to Figure 3 As shown, the intraoral scanning device 1000 includes the above-mentioned point cloud data registration device 100.

[0077] According to the intraoral scanning device of the embodiment of the present invention, the point cloud data registration device thereon realizes the precise matching of the point cloud to be registered and the target surface to be registered through the above-mentioned point cloud data registration method, thereby improving the point-to-surface registration speed.

[0078] According to the above embodiment, a preferred example is set, which will be described in detail below.

[0079] During the scanning process, there are often errors in the position of the 3D model data due to errors in the data itself or errors in registration and fusion. If the next registration and fusion is performed in chronological order, the error of the previous one will be superimposed. The best registered spatial position is selected as the first point cloud, and the neighboring point cloud (second point cloud) in the spatial position is found for registration, and the matched point cloud is fused to the first surface until the second point cloud is completely traversed. Then the position with the best registration and fusion result is selected as the new first point cloud, and the neighboring point cloud in the spatial position is continued to be found for registration and fusion. The above steps are repeated until all point clouds are registered and fused to the first surface.

[0080] If the registration effect is not good, there may be a problem with the original data. The original data is affected by many factors, such as reflection, light transmission, movement, and excess soft tissue (such as the tongue), which lead to errors in stripe extraction. The depth position of the reconstructed point is wrong. This error will be slowly smoothed out during the registration, so the best registered point cloud is selected to create the initial first surface. Compared with other initial point clouds and initial surfaces, the superposition of errors can be reduced and the registration accuracy can be improved. Combined with the above-mentioned problems in real-time scanning, the advantage of this method is that it combines spatial information and registration fusion accuracy. On the one hand, the spatial position with the highest accuracy is selected as the first point cloud through the registration fusion return result, which effectively avoids the increasingly inaccurate position accuracy caused by the superposition of errors of multiple point clouds. On the other hand, the registration fusion of multiple neighborhood point clouds effectively smooths out the superposition of the large errors of a single original point cloud caused by jitter during the scanning process, hardware jumps, etc. on the overall result.

[0081] The overall process is as follows Figure 4 As shown, the label They correspond to the primary judgment, secondary judgment and tertiary judgment in the following text respectively. Before making the primary judgment, secondary judgment and tertiary judgment, preprocessing is performed first.

[0082] The pre-processing process is as follows Figure 5 As shown, the preprocessing steps are as follows:

[0083] (1) During the real-time scanning process, each point cloud (such as the i-th point cloud Points[i]), the corresponding rotation and translation matrix (such as the i-th rotation and translation matrix TransFromRealTime[i]), and the registration return value of each point cloud (that is, the average value of the distance between the corresponding points of the point cloud and the overall point cloud after the registration is completed, the smaller the value, the better the registration and the more accurate the position, such as the registration return value of the i-th point cloud is DistRealTime[i]) can be obtained in chronological order.

[0084] After the registration is completed, the average value of the distance between the corresponding points of the point cloud and the overall point cloud is taken. The overall point cloud refers to all the point clouds that have completed the registration before the point cloud is registered. The average value of the distance refers to the sum of the distances from each point in the point cloud to its corresponding point in the overall point cloud divided by the number of points in the point cloud. Scanning will continue and new point clouds will continue to participate in the registration.

[0085] It should be noted that the first point cloud is not actually involved in the registration. Registration requires two sets of point clouds (existing point cloud and overall point cloud). There is only one set of point clouds in the first point cloud, so the return value of the first point cloud is 10000 (a set maximum value). The point cloud collected for a single jaw can reach about 2000 points, and this return value is only for selecting the initial position, so it does not matter if the first point cloud is not considered as the initial position.

[0086] (2) Points[i] is transformed into Points′[i] according to the corresponding rotation and translation matrix. The centroid of Points′[i] is obtained to obtain the centroid coordinates pCenter[i], and the MC grid bounding box of Points′[i] is obtained to obtain the bounding grid pGrid[i]. Registration fusion is based on the idea of ​​MC surface reconstruction, which not only requires the location of the point, but also requires the topological relationship for subsequent MC surface reconstruction. Therefore, the MC surface can be directly obtained based on the location of the triangular grid points and their topological information, without the need for complete surface reconstruction.

[0087] (3) Calculate the neighborhood point cloud group of each point cloud: If the distance between the centroid coordinates of a point cloud is less than 5 mm and the overlap of the surrounding grid is greater than 50%, it is a neighborhood point cloud. The neighborhood point clouds are numbered according to the order of the overlap. For example, if the neighborhood point cloud group of a point cloud has N neighborhood point clouds, they are numbered 0, 1, 2...N-1 from large to small according to the overlap.

[0088] The barycentric coordinate distance refers to the distance between the barycentric coordinates of Points[i] and Points[j], and the overlap of the enclosing grid refers to the overlap of pGrid[i] and pGrid[j].

[0089] (4) Select the corresponding point cloud with the smallest DistRealTime[i] as the initial first point cloud, mark the point cloud attribute as False, and mark the other point clouds as True. The first point cloud is reconstructed into the first surface through MC surface reconstruction.

[0090] It should be noted that, through preprocessing, a neighborhood point cloud group (second point cloud) of each point cloud (first point cloud) can be obtained.

[0091] Furthermore, the first-level judgment process is as follows Figure 6 As shown, the first-level judgment steps are as follows:

[0092] Find the second point cloud (neighborhood point cloud) numbered 0 of the first point cloud, and determine whether the point cloud attribute is True. If so, enter the processing module. If not, determine the next second point cloud numbered 1 until all second point clouds are traversed.

[0093] Among them, the processing module steps are as follows:

[0094] Perform the corresponding rotation and translation matrix TransFromRealTime[i] on the second point cloud to obtain the transformed third point cloud. Then perform point-to-surface registration on the third point cloud and the first surface. Then determine whether the third point cloud and the first surface match. If so, perform the registration matrix transformation on the third point cloud to obtain the fourth point cloud, and calculate the average distance between the fourth point cloud and the nearest corresponding point of the first surface, which is Dist[i], and merge the fourth point cloud MC into the first surface, and mark the attribute of the second point cloud as False.

[0095] If not, proceed to determine the second point cloud with the next number.

[0096] It should be noted that, through the first-level judgment, the neighborhood point cloud group of the first point cloud can be registered and fused, thereby effectively smoothing out the superimposed influence of the large error of a single original point cloud caused by jitter in the scanning process, hardware jump, etc. on the overall result of a single image (second point cloud).

[0097] Furthermore, the second-level judgment steps are as follows: After the neighboring point clouds (second point clouds) of the current first point cloud are traversed and processed, enter the second-level judgment, find the corresponding point cloud with valid Dist and the smallest value in the point cloud group, use it as the first point cloud, and set its Dist to invalid. Then return to the first-level judgment to traverse the neighborhood until no valid Dist corresponding point cloud is found, and enter the third-level judgment. The specific judgment steps are as follows:

[0098] Determine whether there is a point cloud with a point cloud attribute of True. If not, all point clouds are registered and fused to generate the final 3D surface model. If yes, randomly select a point cloud with a point cloud attribute of True as the second point cloud, perform point-to-surface registration, and determine whether the registration result meets the registration conditions. If yes, mark the point cloud attribute of the current second point cloud as False, then fuse it to the first surface, and use it as the first point cloud, and return to the first level judgment for neighborhood traversal. If not, re-enter the third level judgment and select other point clouds with point cloud attributes of True.

[0099] Reference Figure 7 , the point-to-surface registration process used in the above process is as follows:

[0100] (1) According to the bounding grid of the point cloud, a part of the first surface (i.e., the second surface) is segmented to participate in the point-to-surface registration calculation.

[0101] Among them, the first surface is gradually enlarged through multiple registration and fusion, but each step of registration and fusion does not require the participation of all the data of the first surface. For example, if the new point cloud is at the position of the right molar, there is no need to calculate the surface of the left molar, otherwise it will only increase the amount of calculation.

[0102] (2) A small bounding box is formed for each point on the point cloud, and the second surface is divided into third surfaces according to the small bounding box. That is, each point on the point cloud corresponds to a third surface, and the entire point cloud corresponds to a third surface group.

[0103] The small bounding box can be selected according to a threshold value, and the threshold value generally selects a bounding box of 5*5*5 or 9*9*9.

[0104] (3) Project the point P on the point cloud onto each triangular mesh surface G[j] (j = 0, 1, ..., m) of the corresponding third surface to determine whether the projection point falls within the triangular mesh. If so, save the distance D[i] from the point on the point cloud to the projection point and the projection point coordinates Pro[i] (i = 0, 1, ..., n), where n < = m. After each registration, the overall point cloud will be fused. This process is the existing algorithm based on the TSDF fusion of the MC surface. Whether there is a triangle in each mesh, and the relationship between the triangle and the triangle vertex (i.e., the point on the edge of the mesh) is always determined, and the topological relationship is always attached, so which three points form a three-mesh surface is determined.

[0105] The projection point can be obtained by using the interior angle sum method, the same direction method, the area method, the centroid method, etc.

[0106] (4) Select the point with the smallest D[i] as the corresponding point of point P, and remove the point pairs with a distance greater than 0.245 mm (considered to be misregistration points). Finally, the corresponding point pairs are formed: A[i] (valid point on the point cloud) - B[i] (valid point on the third surface).

[0107] (5) Calculate the average coordinates of the two point clouds A[i] and B[i] respectively, and use the average points Point_a and Point_b as the zero points of the two point clouds. Recalculate the point cloud coordinates based on the weighted distance between each point and Point_a and Point_b, that is, the corresponding distance multiplied by the corresponding weight value, as the new corresponding point pair A′[i]-B′[i]. Then, use the quaternion method, singular value decomposition method, etc. to obtain the rotation and translation matrix Transform.

[0108] Wherein, the weighted value dw[i]=(D-distance[i])*T[i], where distance is the minimum distance obtained in step (4), T is the weighting coefficient, and D can be a threshold value determined based on the registration accuracy, similar test results or experience, i.e., the first preset distance, such as 0.243-0.3 mm.

[0109] It should be noted that the purpose of calculating the weighted distance is to control the influence of corresponding points on the rotation and translation matrix, so that the influence factor of points with closer corresponding points is greater, thereby speeding up the number of registration iterations.

[0110] (6) Use the rotation and translation matrix to transform the point cloud and determine whether the distance between the point pairs is small enough to meet the matching conditions. If so, the registration is complete. Otherwise, repeat steps (2), (3), (4), (5), and (6) until the matching conditions are met or the set number of iterations is exceeded.

[0111] (7) When performing the rotation and translation matrix transformation for the first time, that is, when the number of iterations is the first time, additional operations are required, as follows: The rotation angle (θx, θy, θz) is calculated through the rotation and translation matrix Transform, and the approximation coefficient is calculated based on the average distance dMean of the point pairs before the rotation transformation and the average distance dMean′ after the rotation transformation: △ = dMean′ / dMean, and the approximation coefficient is multiplied by the rotation angle (θx, θy, θz) and the translation amount (Tx, Ty, Tz) to obtain the inverse rotation and translation matrix ΔTransform. Then, the transformed point cloud is transformed with the rotation and translation matrix ΔTransform for multiple times until dMean′ no longer decreases.

[0112] The purpose of the first rotation and translation processing is to iterate quickly. In actual projects, it is found that multiple iterations are required to match the point cloud and the first surface, that is, the iteration speed is too slow. Through the first rotation and translation matrix transformation processing, the point cloud can quickly approach the first surface, reduce the number of iterations, and speed up the alignment speed.

[0113] like Figure 8 and Fig. 9 As shown in the figure, a set of actual scanning data results is given. After real-time scanning, the accuracy is 0.098817mm on average and 0.092013mm on standard deviation. After sorting and optimization, the accuracy is improved to 0.041527mm on average and 0.041207mm on standard deviation. Therefore, the accuracy of the overall model can be effectively improved.

[0114] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0115] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0116] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0117] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0118] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A point cloud data registration method, It is characterized in that The following steps are involved: Obtain a first rotation and translation matrix according to the point cloud to be registered and the target surface to be registered; Transforming the point cloud to be registered using the first rotation and translation matrix to obtain a first transformed point cloud, and updating the point cloud to be registered using the first transformed point cloud; Determine whether the current point cloud transformation times meets the first preset condition; If the condition is satisfied, an approximation rotation and translation matrix is ​​obtained according to the first rotation and translation matrix by using an approximation algorithm, and the point cloud to be registered is transformed multiple times by using the approximation rotation and translation matrix to obtain an approximation transformation point cloud until the second preset condition is satisfied, and the point cloud to be registered is updated by using the approximation transformation point cloud; Determine whether the registration is completed based on the point cloud to be registered and the target surface to be registered. If not completed, returning to the step of obtaining the first rotation and translation matrix according to the point cloud to be registered and the target surface to be registered until the registration is completed; The method of obtaining an approximated rotation and translation matrix according to the first rotation and translation matrix by using an approximation algorithm includes: Obtaining a rotation angle and a translation amount according to the first rotation and translation matrix; Calculate the first average distance of the point cloud to be registered before updating and the second average distance of the first transformed point cloud, wherein the first average distance of the point cloud to be registered before updating is the average value of the distances between the points in the registration point pairs on the point cloud to be registered and the corresponding points on the target surface to be registered, and the second average distance of the first transformed point cloud is the average value of the distances between the points in the registration point pairs on the first transformed point cloud and the corresponding points on the target surface to be registered; An approximate rotation and translation matrix is ​​obtained according to the first average distance, the second average distance, the rotation angle and the translation amount.

2. The point cloud data registration method according to claim 1, It is characterized in that The step of obtaining a first rotation and translation matrix according to the point cloud to be registered and the target surface to be registered includes: According to each point in the point cloud to be registered, the target surface to be registered is segmented to obtain the target sub-surface corresponding to each point; According to each point in the point cloud to be registered and its corresponding target sub-surface, a registration point pair is obtained; A first rotation and translation matrix is ​​obtained according to the registration point pair.

3. The point cloud data registration method according to claim 1, It is characterized in that The method of using the approximate rotation and translation matrix to continuously transform the point cloud to be registered multiple times to obtain the approximate transformed point cloud until the second preset condition is met includes: The approximate rotation and translation matrix is ​​used to transform the point cloud to be registered to obtain a second transformed point cloud; Calculating a second average distance of the second transformed point cloud, and determining whether the second average distance is less than the second average distance calculated last time, the second average distance of the second transformed point cloud being an average of distances between points on the second transformed point cloud and corresponding points on the target surface to be registered in the registration point pair; If it is less than, then the point cloud to be registered is updated by using the second transformed point cloud, and the step of transforming the point cloud to be registered by using the approximate rotation and translation matrix to obtain the second transformed point cloud is returned; If it is greater than or equal to, the second preset condition is met.

4. The point cloud data registration method according to claim 1, It is characterized in that The step of obtaining an approximate rotation transformation matrix according to the first average distance, the second average distance, the rotation angle and the translation amount includes: calculating a ratio of the second average distance to the first average distance; Multiply this ratio by the rotation angle and translation to obtain the approximate rotation and translation matrix.

5. The point cloud data registration method according to any one of claims 2 to 4, It is characterized in that The method of segmenting the target surface to be registered according to each point in the point cloud to be registered to obtain the target sub-surface corresponding to each point includes: Generate a three-dimensional bounding box around each point in the point cloud to be registered; The target surface to be registered is segmented according to the three-dimensional bounding box of each point to obtain the target sub-surface corresponding to each point.

6. The point cloud data registration method according to any one of claims 2 to 4, It is characterized in that The target sub-surface is obtained according to the triangular mesh points, wherein the registration point pair of the point cloud to be registered is obtained according to each point in the point cloud to be registered and its corresponding target sub-surface, including: Project each point in the point cloud to be registered onto each triangular mesh of the corresponding target subsurface to obtain the corresponding projection point; Calculate the distance between each point in the point cloud to be registered and the corresponding projection points to obtain multiple first distances; According to the multiple first distances corresponding to each point in the point cloud to be registered, a pair of registration points of the point cloud to be registered is obtained.

7. The point cloud data registration method according to claim 6, It is characterized in that The step of obtaining a pair of registration points of the point cloud to be registered according to a plurality of first distances corresponding to each point in the point cloud to be registered comprises: For each point in the point cloud to be registered, a projection point corresponding to the first distance with the smallest value is selected from the corresponding multiple first distances to form a point pair with the point, so as to obtain a first point pair; Discard the point pairs in the first point pair whose distance is greater than the first preset distance, and obtain the second point pair; A registration point pair of the point cloud to be registered is obtained according to the second point pair.

8. The point cloud data registration method according to claim 7, It is characterized in that The second point pair includes a first sub-point cloud and a second sub-point cloud, the points in the first sub-point cloud are all points in the point cloud to be registered, and the points in the second sub-point cloud are all points on the target surface to be registered, wherein the registration point pair of the point cloud to be registered is obtained according to the second point pair, including: Determine a first zero point and a second zero point, wherein a coordinate value of the first zero point is an average value of coordinate values ​​of all points in the first sub-point cloud, and a coordinate value of the second zero point is an average value of coordinate values ​​of all points in the second sub-point cloud; A first registration point is obtained by calculating the weighted distance between each point in the first sub-point cloud and the first zero point, and a second registration point is obtained by calculating the weighted distance between each point in the second sub-point cloud and the second zero point. The first registration point and the second registration point are paired into a registration point pair.

9. The point cloud data registration method according to claim 1, It is characterized in that The current point cloud transformation times meets the first preset condition, including: The current point cloud transformation times is less than or equal to the first preset value.

10. The point cloud data registration method according to claim 2, It is characterized in that Determine whether the registration is complete based on the point cloud to be registered and the target surface to be registered, including: Determine whether the distance of the registration point pair is less than a preset registration distance threshold, and determine whether the current point cloud transformation times is equal to a preset transformation times threshold; If the distance between the registration point pairs is less than the preset registration distance threshold, or the number of current point cloud transformations is equal to the preset transformation number threshold, the registration is determined to be completed; otherwise, the registration is determined to be incomplete.

11. A data processing method, It is characterized in that The method comprises: According to the point cloud to be registered and the surface that has been registered and fused, the target surface to be registered is cut out; The point cloud data registration method according to any one of claims 1 to 10 is used to register the point cloud to be registered and the target surface to be registered to obtain a registered point cloud; When the point cloud to be registered is matched with the target surface to be registered, the registered point cloud is fused to the target surface to be registered to obtain a new registered and fused surface, the point cloud to be registered is updated, and the step of segmenting the target surface to be registered based on the point cloud to be registered and the registered and fused surface is returned until all the point cloud data participate in the registration and fusion.

12. A point cloud data registration device, comprising a memory, a processor, and a computer program stored in the memory, wherein when the computer program is executed by the processor, the point cloud data registration method according to any one of claims 1 to 10 is implemented.

13. An intraoral scanning device, It is characterized in that It comprises the point cloud data registration device as described in claim 12.

Images