Hierarchical tooth point cloud completion method and system combined with image assistance

Abstract

The invention discloses a hierarchical tooth point cloud completion method and system combined with image assistance, and relates to the technical field of medical image processing, and the method comprises the steps: obtaining a to-be-completed point cloud entity image, employing an encoder to extract structural features of the to-be-completed point cloud entity image to obtain structural feature vectors, and recording the structural feature vectors as tooth image features; obtaining a to-be-complemented point cloud, extracting three-dimensional structure features of the point cloud by adopting a double-layer architecture of an encoder, fusing tooth image features to obtain a feature vector set containing bimodal structure information, and recording the feature vector set as to-be-complemented features; and sampling the to-be-complemented features, and performing high-fine granularity processing to obtain a complemented point cloud. According to the method, the structure prior information carried by the image and the three-dimensional information of the incomplete point cloud are fully combined, and the detail reduction and structure reconstruction capability in the whole point cloud complementing process is optimized, so that the tooth point cloud complementing quality is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to an image-assisted hierarchical tooth point cloud completion method and system. Background Art

[0002] With the development of computer-aided design technology, digital dentistry has received widespread attention and achieved remarkable achievements. The structural complexity of 3D tooth point clouds and the highly refined characteristics of tooth structure place extremely high demands on the precise capture of details, which poses a great challenge to fine and accurate tooth restorations. Intra-Oral Scanners (IOSs) are increasingly becoming an important tool in this field for generating accurate digital imprints of tooth structures. However, due to the limitations of sensor resolution and occlusion problems, the point cloud data collected by IOS often have sparseness and incompleteness, which is manifested in missing areas and loss of three-dimensional details. These defects significantly affect the accuracy and reliability of subsequent diagnosis and treatment, highlighting the importance of point cloud completion technology in improving data resolution and completeness to further optimize diagnosis and treatment performance and reliability.

[0003] The Chinese invention patent with publication number CN112967219B discloses a two-stage tooth point cloud completion method and system based on a deep learning network. The method is divided into two stages: in the first stage, the MSN network predicts a complete but coarse-grained point cloud based on the incomplete tooth point cloud; in the second stage, the coarse-grained predicted point cloud is fused with the input point cloud through sampling algorithm and residual connection to generate a uniformly distributed fine-grained point cloud, and finally the tooth point cloud is completed.

[0004] However, 3D tooth point clouds have more structural complexity, characterized by depressions, protrusions, and grooves, which are significantly different from common general object datasets such as KITTI, PCN, and ShapeNet-ViPC. The highly refined nature of tooth structure places extremely high demands on the accurate capture of details. In addition, changes in dental arch shape, number of teeth, and the existence of missing data areas further increase the difficulty of completion. Existing methods only rely on 3D shape prior knowledge, and due to their single-modal design, they are weak in dealing with refined and heterogeneous tooth scenes.

[0005] Therefore, in the field of tooth point cloud completion, there is an urgent need for a method that uses deep learning to combine tooth images to assist in hierarchical tooth point cloud completion, so as to make full use of the structural prior information carried by tooth images, thereby improving the accuracy and robustness of point cloud completion, which is suitable for high-quality tooth point cloud generation in low-cost, low-resolution sensor scenarios. Summary of the invention

[0006] To solve the above problems, the present invention mentions a hierarchical tooth point cloud completion method combined with image assistance. This method introduces image information assistance in the tooth point cloud completion process, effectively solving the problem that the existing technology only relies on 3D shape prior knowledge, resulting in poor performance when processing tooth refinement and heterogeneous scenes. The present invention fully combines the structural prior information carried by the image and the three-dimensional information of the incomplete point cloud, optimizes the detail restoration and structure reconstruction capabilities in the point cloud completion process, and thus significantly improves the quality of tooth point cloud completion.

[0007] The technical solution of the present invention comprises the following steps:

[0008] S1. Obtain a point cloud entity image to be completed, and use an encoder to extract structural features from the point cloud entity image to be completed to obtain a structural feature vector, which is recorded as a tooth image feature;

[0009] S2, obtaining a point cloud to be completed, using a double-level architecture of an encoder to extract three-dimensional structural features of the point cloud, and fusing tooth image features to obtain a feature vector set containing bimodal structural information, recorded as features to be completed, wherein the encoder includes a point encoder and a graph encoder;

[0010] S3: Sample the features to be completed and process them in a highly granular manner to obtain a completed point cloud.

[0011] Preferably, the specific content of S2 is:

[0012] S201, using a point encoder and a graph encoder to extract features from the point cloud to be completed to obtain point space feature 1 and graph space feature 1, performing element-by-element addition fusion and position significant fusion on the point space feature 1 and the graph space feature 1 to obtain a level 1 point cloud structure feature;

[0013] S202, fusing the level 1 point cloud structure features with the tooth image features to obtain fusion features 1;

[0014] S203, using a point encoder to extract the point space feature 1 to obtain a point space feature 2, using a graph encoder to extract the fusion feature 1 to obtain a graph space feature 2, performing element-by-element addition fusion and position significant fusion on the point space feature 2 and the graph space feature 2 to obtain a level 2 point cloud structure feature;

[0015] The level 2 point cloud structure features are passed through a graph encoder to obtain global shape features;

[0016] The level 2 point cloud structural features and global shape features constitute the features to be completed.

[0017] Preferably, the fusion of dental image features in S2 uses an image auxiliary module;

[0018] The image auxiliary module is composed of a cross-attention mechanism layer, a self-attention mechanism layer and a position saliency fusion module;

[0019] The cross-attention mechanism layer is used to receive the level 1 point cloud structure features and tooth image features and fuse them to extract the complementary features F. gpi , F gpi The fusion extraction expression is:

[0020]

[0021] Among them, F 3D and F img They are the level 1 point cloud structure features and tooth image features, N a is the number of columns of the tensor of dental image features, β(·), γ(·) and θ(·) are feature mapping functions, softmax is the Softmax activation function, It is the matrix transpose operation;

[0022] The self-attention mechanism layer is used to receive complementary features F gpi And further screen to obtain important features F g ' pi , important feature F g ' pi The filter expression is:

[0023]

[0024] Among them, β′(·), γ′(·) and θ′(·) are feature mapping functions, Performs a matrix transpose operation.

[0025] Preferably, the position saliency fusion module is used to add point cloud position information to the features obtained by element-by-element addition and fusion of point space features and graph space features in the level 1 point cloud structure features and the level 2 point cloud structure features, respectively, and the important features, and the expression is:

[0026] F out =MLPs(softmax(MLPs(Q-K+δ))(V+δ))+F in ;

[0027] Among them, F in is the input feature of the position saliency fusion module; Q, K, V are in The feature encoding after three independent fully connected layers, F out is the output feature of the position saliency fusion module, and MLPs is a multi-layer perceptron feature mapping function;

[0028]

[0029] Among them, ρ is the fully connected position encoding function, p i The input point cloud The coordinates of a point in Yes i exist The nearest neighbor area obtained by KNN clustering strategy, p j is a point in the aforementioned neighboring area, δ is the position information code, i is P in , j is the serial number of each point in the aforementioned neighboring area N(i).

[0030] Preferably, the features to be completed are sampled and processed in a highly fine-grained manner, and the specific content of the completed point cloud is obtained as follows:

[0031] S301, upsampling the features to be completed to generate coarse and fine granularity completion point cloud P 0 ;

[0032] S302: Complete the point cloud P for coarse and fine granularity 0 Downsampling is performed to obtain N 0 points, and concatenate them with the input point cloud to be completed to obtain a concatenated point cloud, and downsample the concatenated point cloud to N 1 points, recorded as seed point cloud P c ;

[0033] S303, seed point cloud P c The generated point cloud P is obtained by high granularity 1 , and the point cloud P will be generated 1 The upsampling process is repeated, and the i-th upsampling point cloud is recorded as the generated point cloud P i , until after n upsampling processes, the final completed point cloud P is obtained n .

[0034] This application proposes a hierarchical tooth point cloud completion system combined with image assistance, constructs a tooth image feature extraction module to extract the structural prior information contained in the image; constructs a point cloud feature extraction module, adopts a two-level architecture of point graph combined analysis, extracts the three-dimensional structural features of the point cloud to be completed, and fuses the structural prior information carried in the image through the image assistance module; the fused features to be completed are generated through the point cloud generation module, and finally a dense and complete tooth point cloud completion is achieved. The present invention innovatively introduces tooth images as auxiliary information, which significantly improves the precision and accuracy of point cloud completion, and is particularly suitable for high-quality tooth point cloud generation in low-cost, low-resolution sensor scenarios.

[0035] Specifically, a hierarchical tooth point cloud completion system combined with image assistance includes:

[0036] Tooth image feature extraction module: obtain the point cloud entity image to be completed, use the image encoder to extract the structural features of the point cloud entity image to be completed to obtain the structural feature vector, which is recorded as the tooth image feature;

[0037] Point cloud feature extraction module: The encoder uses a dual-level architecture to extract the three-dimensional structural features of the point cloud, and fuses the tooth image features to obtain a feature vector set containing dual-modal structural information, which is recorded as the feature to be completed. The encoder includes a point encoder and a graph encoder.

[0038] Point cloud generation module: The point cloud generation module includes several sampling units, which are built based on UpTransformer. The point cloud generation module is used to sample the features to be completed and process them in a highly granular manner to obtain the completed point cloud;

[0039] Supervised training module: supervise the training of the tooth image feature extraction module, point cloud feature extraction module and point cloud generation module, and optimize the parameters of each module to ensure the accuracy and quality of the completed point cloud.

[0040] Preferably, the process of supervised training by the supervised training module is as follows:

[0041] The real complete point cloud P corresponding to the point cloud to be completed gt Generate a point cloud P for each upsampling unit i , seed point cloud P c To supervise, specifically, the following loss function is used for supervision:

[0042]

[0043]

[0044]

[0045]

[0046] in, is the final loss function; X and Y represent point sets, respectively, x∈X and y∈Y represent the point coordinates in the point set; P i ′ and P c ′ indicates that P gt After downsampling, the point cloud P i and P c The real point cloud with the same number of points; P represents the input point cloud to be completed, P n Indicates the final completed point cloud, L CD1 Used to measure the similarity between X and Y, L completion Used to measure the completion effect, L preservation A measure of shape retention.

[0047] Preferably, the processing content of the point cloud generation module includes:

[0048] Upsample the features to be completed to generate coarse and fine granularity completion point cloud P 0 ;

[0049] Completing the point cloud P for coarse and fine granularity 0 Downsampling is performed to obtain N 0 points, and concatenate them with the input point cloud to be completed to obtain a concatenated point cloud, and downsample the concatenated point cloud to N 1 points, recorded as seed point cloud P c ;

[0050] For the seed point cloud P c Input the upsampling unit to obtain the generated point cloud P 1 , and the point cloud P will be generated 1 As the input of the next upsampling unit, the upsampling process is repeated, and the output point cloud of the i-th upsampling unit is recorded as the generated point cloud P i , until after n upsampling units, the final completed point cloud P is obtained n

[0051] In summary, compared with traditional technologies, the present invention is a hierarchical tooth point cloud completion method and system combined with image assistance. The present invention introduces tooth images as auxiliary information, integrates tooth image features and point cloud features, and makes full use of the structural prior information in the image, which significantly improves the fineness and accuracy of point cloud completion. It effectively solves the problem that the existing method only relies on 3D shape prior and is difficult to handle tooth refinement and heterogeneity. It is suitable for high-quality tooth point cloud generation in low-cost, low-resolution sensor scenarios.

[0052] The technical method of the present invention is further described in detail below through the drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is an overall block diagram of the hierarchical tooth point cloud completion method combined with image assistance of the present invention;

[0054] Figure 2 The following are the visualization effects of tooth point cloud completion using different methods. DETAILED DESCRIPTION

[0055] The technical method of the present invention is further described below by means of the accompanying drawings and embodiments. It should be noted that unless otherwise specifically stated, the relative arrangement of the components and steps, numerical expressions and numerical values ​​described in these embodiments do not limit the scope of the present application.

[0056] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.

[0057] Technologies, systems, and devices known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, systems, and devices should be considered part of the specification.

[0058] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0059] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0060] like Figure 1 As shown, a hierarchical tooth point cloud completion method combined with image assistance includes:

[0061] S1. Obtain a point cloud entity image to be completed, and use an encoder to extract structural features from the point cloud entity image to be completed to obtain a structural feature vector, which is recorded as a tooth image feature;

[0062] S2, obtaining a point cloud to be completed, using a double-level architecture of an encoder to extract three-dimensional structural features of the point cloud, and fusing tooth image features to obtain a feature vector set containing bimodal structural information, recorded as features to be completed, wherein the encoder includes a point encoder and a graph encoder;

[0063] Furthermore, the specific content of S2 is:

[0064] S201, using a point encoder and a graph encoder to extract features from the point cloud to be completed to obtain point space feature 1 and graph space feature 1, performing element-by-element addition fusion and position significant fusion on the point space feature 1 and the graph space feature 1 to obtain a level 1 point cloud structure feature;

[0065] S202, fusing the level 1 point cloud structure features with the tooth image features to obtain fusion features 1;

[0066] S203, using a point encoder to extract the point space feature 1 to obtain a point space feature 2, using a graph encoder to extract the fusion feature 1 to obtain a graph space feature 2, performing element-by-element addition fusion and position significant fusion on the point space feature 2 and the graph space feature 2 to obtain a level 2 point cloud structure feature;

[0067] The level 2 point cloud structure features are passed through a graph encoder to obtain global shape features;

[0068] The level 2 point cloud structural features and global shape features constitute the features to be completed.

[0069] Furthermore, the process of fusing tooth image features described in S2 uses an image auxiliary module;

[0070] The image auxiliary module is composed of a cross-attention mechanism layer, a self-attention mechanism layer and a position saliency fusion module;

[0071] The cross-attention mechanism layer is used to receive the level 1 point cloud structure features and tooth image features and fuse them to extract the complementary features F. gpi , F gpi The fusion extraction expression is:

[0072]

[0073] Among them, F 3D and F img They are the level 1 point cloud structure features and tooth image features, N a is the number of columns of the tensor of dental image features, β(·), γ(·) and θ(·) are feature mapping functions, softmax is the Softmax activation function, It is the matrix transpose operation;

[0074] The self-attention mechanism layer is used to receive complementary features F gpi And further screen to obtain important features F g ' pi , important feature F g ' pi The filter expression is:

[0075]

[0076] Among them, β′(·), γ′(·) and θ′(·) are feature mapping functions, Performs a matrix transpose operation.

[0077] Furthermore, the position saliency fusion module is used to respectively add and fuse the features of the point space features and the image space features in the level 1 point cloud structure features and the level 2 point cloud structure features, and the important features F g ' pi Adding point cloud location information, the expression is:

[0078] F out =MLPs(softmax(MLPs(Q-K+δ))(V+δ))+F in ;

[0079] Among them, F inThe input features of the position saliency fusion module are the features of the point cloud structure features at level 1 and the midpoint space features and the image space features at level 2, and the important features F g ' pi ; Q, K, V are in F in The feature encoding after three independent fully connected layers, F out is the output feature of the position saliency fusion module, and MLPs is a multi-layer perceptron feature mapping function;

[0080]

[0081] Among them, ρ is the fully connected position encoding function, p i The input point cloud The coordinates of a point in Yes i exist The nearest neighbor area obtained by KNN clustering strategy, p j is a point in the aforementioned neighboring area, δ is the position information code, i is P in The serial number of each point in the , j is the aforementioned neighboring area The sequence numbers of the points in .

[0082] S3: Sample the features to be completed and process them in a highly granular manner to obtain a completed point cloud.

[0083] Furthermore, the features to be completed are sampled and processed in a highly granular manner, and the specific content of the completed point cloud is obtained as follows:

[0084] S301, upsampling the features to be completed to generate coarse and fine granularity completion point cloud P 0 ;

[0085] S302: Complete the point cloud P for coarse and fine granularity 0 Downsampling is performed to obtain N 0 points, and concatenate them with the input point cloud to be completed to obtain a concatenated point cloud, and downsample the concatenated point cloud to N 1 points, recorded as seed point cloud P c ;

[0086] S303, seed point cloud P c The generated point cloud P is obtained by high granularity 1 , and the point cloud P will be generated 1 The upsampling process is repeated, and the i-th upsampling point cloud is recorded as the generated point cloud P i , until after n upsampling processes, the final completed point cloud P is obtained n .

[0087] A hierarchical tooth point cloud completion system combined with image assistance, comprising:

[0088] Tooth image feature extraction module: obtain the point cloud entity image to be completed, use the image encoder to extract the structural features of the point cloud entity image to be completed to obtain the structural feature vector, which is recorded as the tooth image feature;

[0089] Point cloud feature extraction module: The encoder's dual-level architecture is used to extract the three-dimensional structural features of the point cloud, and the tooth image features are fused to obtain a feature vector set containing dual-modal structural information, which is recorded as the feature to be completed. The encoder includes a point encoder and a graph encoder. The feature vector set of the dual-modal structural information contains the point cloud structural characteristics and the structural prior characteristics of the image.

[0090] Point cloud generation module: The point cloud generation module includes several sampling units, which are built based on UpTransformer. The point cloud generation module is used to sample the features to be completed and process them in a highly granular manner to obtain a completed point cloud;

[0091] Supervised training module: supervise the training of the tooth image feature extraction module, point cloud feature extraction module and point cloud generation module, and optimize the parameters of each module to ensure the accuracy and quality of the completed point cloud.

[0092] Furthermore, the supervised training module supervises the training process as follows:

[0093] The real complete point cloud P corresponding to the point cloud to be completed gt Generate a point cloud P for each upsampling unit i , seed point cloud P c To supervise, specifically, the following loss function is used for supervision:

[0094]

[0095]

[0096]

[0097]

[0098] in, is the final loss function; X and Y represent point sets, respectively, x∈X and y∈Y represent the point coordinates in the point set; P i ′ and P c ′ indicates that P gt After downsampling, the point cloud P i and P c The real point cloud with the same number of points; P represents the input point cloud to be completed, P n Indicates the final completed point cloud, Used to measure the similarity between X and Y. Used to measure the completion effect, A measure of shape retention.

[0099] The processing content of the point cloud generation module includes:

[0100] Upsample the features to be completed to generate coarse and fine granularity completion point cloud P 0 ;

[0101] Completing the point cloud P for coarse and fine granularity 0 Downsampling is performed to obtain N 0 points, and concatenate them with the input point cloud to be completed to obtain a concatenated point cloud, and downsample the concatenated point cloud to N 1 points, recorded as seed point cloud P c ;

[0102] For the seed point cloud P c Input the upsampling unit to obtain the generated point cloud P 1 , and the point cloud P will be generated 1 As the input of the next upsampling unit, the upsampling process is repeated, and the output point cloud of the i-th upsampling unit is recorded as the generated point cloud P i , until after n upsampling units, the final completed point cloud P is obtained n .

[0103] In order to fully verify the effectiveness of the hierarchical tooth point cloud completion method combined with image assistance proposed in this paper, this paper conducted a comparative experiment with the existing mainstream methods on the FDDI dataset. Table 1 shows the scores of each method in Chamfer distance (CD) and EarthMover'sDistance (EMD) related indicators; Figure 2 The visualization effect of tooth point cloud completion under different methods is presented.

[0104] It can be seen from the experimental results that the method of the present invention is significantly better than the existing methods in terms of CD and EMD indicators, indicating that it has obvious advantages in the accuracy and shape fidelity of point cloud completion. Figure 2 The visualization effect in further verifies that the image-assisted strategy can better preserve the detailed features of the tooth structure, thereby significantly improving the effect and quality of tooth point cloud completion.

[0105] Table 1 Comparison of tooth point cloud completion prediction performance

[0106] method <![CDATA[CDL1×10 3 ]]> <![CDATA[CDL2×10 3 ]]> <![CDATA[EMD×10 3 ]]> PointAttN 369 560 495 AnchorFormer 437 573 1071 AdaPoinTr 348 398 1071 SnowflakeNet 271 227 1030 Seedformer 255 179 1291 TUCNet 222 194 775 CRA-PCN 221 120 274 ViPC 288 225 393 Method of the present invention 211 107 259

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical method of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical method to deviate from the spirit and scope of the technical method of the present invention.

Claims

1. A hierarchical tooth point cloud completion method combined with image assistance, characterized in that: The following steps are involved: S1. Obtain a point cloud entity image to be completed, and use an encoder to extract structural features from the point cloud entity image to be completed to obtain a structural feature vector, which is recorded as a tooth image feature; S2, obtaining a point cloud to be completed, using an encoder to extract the three-dimensional structural features of the point cloud, and fusing the tooth image features to obtain a feature vector set containing bimodal structural information, recorded as features to be completed, wherein the encoder includes a point encoder and a graph encoder; S3: Sample the features to be completed and process them in a highly granular manner to obtain a completed point cloud.

2. The image-assisted hierarchical tooth point cloud completion method according to claim 1, characterized in that: The specific contents of S2 are: S201, using a point encoder and a graph encoder to extract features from the point cloud to be completed to obtain point space feature 1 and graph space feature 1, performing element-by-element addition fusion and position significant fusion on the point space feature 1 and the graph space feature 1 to obtain a level 1 point cloud structure feature; S202, fusing the level 1 point cloud structure features with the tooth image features to obtain fusion features 1; S203, using a point encoder to extract the point space feature 1 to obtain a point space feature 2, using a graph encoder to extract the fusion feature 1 to obtain a graph space feature 2, performing element-by-element addition fusion and position significant fusion on the point space feature 2 and the graph space feature 2 to obtain a level 2 point cloud structure feature; The level 2 point cloud structure features are passed through a graph encoder to obtain global shape features; The level 2 point cloud structural features and global shape features constitute the features to be completed.

3. The image-assisted hierarchical tooth point cloud completion method according to claim 2, characterized in that: The fusion of dental image features described in S2 uses an image-assisted module; The image auxiliary module is composed of a cross-attention mechanism layer, a self-attention mechanism layer and a position saliency fusion module; The cross-attention mechanism layer is used to receive the level 1 point cloud structure features and tooth image features and fuse them to extract the complementary features F. gpi , F gpi The fusion extraction expression is: Among them, F 3D and F img They are the level 1 point cloud structure features and tooth image features, N a is the number of columns of the tensor of dental image features, β(·), γ(·) and θ(·) are feature mapping functions, softmax is the Softmax activation function, and T is the matrix transpose operation. The self-attention mechanism layer is used to receive complementary features F gpi And further screen to obtain important features F g ' pi , important feature F g ' pi The filter expression is: Among them, β′(·), γ′(·) and θ′(·) are feature mapping functions, and T is the matrix transpose operation.

4. The image-assisted hierarchical tooth point cloud completion method according to claim 3, characterized in that: The position saliency fusion module is used to respectively combine the features of the point space features and the image space features in the level 1 point cloud structure features and the level 2 point cloud structure features, and the important features F g ' pi Adding point cloud location information, the expression is: F out =MLPs(softmax(MLPs(Q-K+δ))(V+δ))+F in ; Among them, F in is the input feature of the position saliency fusion module; Q, K, V are in The feature encoding after three independent fully connected layers, F out is the output feature of the position saliency fusion module, and MLPs is a multi-layer perceptron feature mapping function; δ=ρ(p i -p j ),j∈N(i); Among them, ρ is the fully connected position encoding function, p i is the input point cloud P in The coordinates of a point in p, N(i) is i In P in The nearest neighbor area obtained by KNN clustering strategy, p j is a point in the aforementioned neighboring area, δ is the position information code, i is P in , j is the serial number of each point in the aforementioned neighboring area N(i).

5. The image-assisted hierarchical tooth point cloud completion method according to claim 4, characterized in that: The features to be completed are sampled and processed in a highly fine-grained manner to obtain the specific content of the completed point cloud: S301, upsampling the features to be completed to generate a coarse and fine granularity completion point cloud P0; S302, down-sample the coarse and fine granularity completion point cloud P0 to obtain N0 points, and splice it with the input point cloud to be completed to obtain a spliced ​​point cloud, and down-sample the spliced ​​point cloud to N1 points, which is recorded as the seed point cloud P c ; S303, for the seed point cloud P c The generated point cloud P1 is obtained by high granularity, and the generated point cloud P1 is cyclically upsampled. The i-th upsampled point cloud is recorded as the generated point cloud P i , until after n upsampling processes, the final completed point cloud P is obtained n .

6. A hierarchical tooth point cloud completion system combined with image assistance, characterized in that: include: Tooth image feature extraction module: obtain the point cloud entity image to be completed, use the image encoder to extract the structural features of the point cloud entity image to be completed to obtain the structural feature vector, which is recorded as the tooth image feature; Point cloud feature extraction module: The encoder's dual-level architecture is used to extract the three-dimensional structural features of the point cloud, and the tooth image features are fused to obtain a feature vector set containing dual-modal structural information, which is recorded as the feature to be completed. The encoder includes a point encoder and a graph encoder. The extracted feature vector contains the point cloud structural characteristics and the structural prior characteristics of the image. Point cloud generation module: The point cloud generation module includes several sampling units, which are built based on UpTransformer. The point cloud generation module is used to sample the features to be completed and process them in a highly granular manner to obtain a completed point cloud; Supervised training module: supervise the training of the tooth image feature extraction module, point cloud feature extraction module and point cloud generation module, and optimize the parameters of each module to ensure the accuracy and quality of the completed point cloud.

7. The image-assisted hierarchical tooth point cloud completion system according to claim 6, characterized in that: The process of supervised training module supervised training is as follows: The real complete point cloud P corresponding to the point cloud to be completed gt Generate a point cloud P for each upsampling unit i , seed point cloud P c To supervise, specifically, the following loss function is used for supervision: in, is the final loss function; X and Y represent point sets, respectively, x∈X and y∈Y represent the point coordinates in the point set; P i ′ and P c ′ indicates that P gt After downsampling, the point cloud P i and P c The real point cloud with the same number of points; P represents the input point cloud to be completed, P n Indicates the final completed point cloud, Used to measure the similarity between X and Y. Used to measure the completion effect, A measure of shape retention.

8. The image-assisted hierarchical tooth point cloud completion system according to claim 6, characterized in that: The processing content of the point cloud generation module includes: Upsample the features to be completed to generate coarse and fine granularity completion point cloud P0; The coarse-grained point cloud P0 is downsampled to obtain N0 points, and then spliced ​​with the input point cloud to be completed to obtain a spliced ​​point cloud. The spliced ​​point cloud is downsampled to N1 points, which is recorded as the seed point cloud P c ; For the seed point cloud P c The input upsampling unit is highly granularized to obtain the generated point cloud P1, and the generated point cloud P1 is used as the input of the next upsampling unit, and the upsampling process is performed cyclically. The output point cloud of the i-th upsampling unit is recorded as the generated point cloud P i , until after n upsampling units, the final completed point cloud P is obtained n .

Images