Dental three-dimensional model local optimization method and system based on multi-modal fusion

Abstract

The invention discloses a dental three-dimensional model local optimization method and system based on multi-modal fusion, and belongs to the field of three-dimensional model modeling. According to the method, a source model (CT model) and a target model (mouth scanning model) are constructed, an initial corresponding relation is established by utilizing manual point selection, after coarse registration is completed, the corresponding position of each point is optimized by adopting fine registration based on triangular surface projection, and finally, a local area in the target model is embedded into the source model, so that unification of structural integrity and local precision is realized.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional modeling, and in particular relates to a method and system for local optimization of dental three-dimensional models based on multimodal fusion. Background Art

[0002] In the fields of 3D modeling and medicine, creating high-precision 3D models of local structures is of great significance. Currently, commonly used medical imaging methods include cone beam computed tomography (CBCT) and intraoral scanning (IOS). CBCT can provide complete oral structural information, including the jaw and tooth roots, but its spatial resolution is low. Intraoral scanning technology, on the other hand, can capture detailed geometric morphology of crowns and occlusal surfaces with submillimeter accuracy, making it suitable for denture design and occlusal analysis. However, its coverage is limited and is often accompanied by local occlusion, distortion, or data loss.

[0003] Therefore, directly using single-modality data often fails to meet the dual requirements of completeness and accuracy for dental models. To resolve this contradiction, researchers have proposed fusing oral scan data with CBCT data, using 3D registration algorithms to align multimodal models, and obtaining high-quality, complete 3D models by replacing or optimizing local regions. However, current methods still have the following problems:

[0004] 1. Unstable registration: There are scale differences, positioning errors, and data missing between CBCT and IOS, which often lead to decreased registration accuracy.

[0005] 2. Inflexible local replacement: Existing algorithms are mostly based on global registration, which makes it difficult to accurately control the optimization range of local areas.

[0006] 3. Poor fusion quality: Problems such as cracks, overlaps, or inconsistent normals often occur between the replacement area and the overall model, affecting subsequent processing and use.

[0007] Therefore, there is an urgent need for a three-dimensional local optimization method that integrates high-precision oral scan models and low-precision CBCT models, which can accurately control the replacement area, achieve seamless fusion and improve the overall model quality. Summary of the Invention

[0008] The present invention aims to address existing challenges and provide a method and system for local optimization of dental 3D models based on multimodal fusion. This method enables seamless replacement and fusion of tooth regions within a 3D model, and is suitable for clinical scenarios such as oral digital modeling, virtual implant planning, and denture design.

[0009] In order to achieve the above-mentioned object of the invention, the present invention specifically adopts the following technical solutions:

[0010] In a first aspect, the present invention provides a method for local optimization of a dental three-dimensional model based on multimodal fusion, which comprises the following steps:

[0011] S1. Perform 3D reconstruction on the user's original CBCT image data to obtain a low-precision 3D model containing alveolar bone and root structure information and use it as the source model;

[0012] S2. Obtaining the user's oral scan data and performing three-dimensional modeling to obtain a high-precision model containing details of the crown and occlusal surface and using it as the target model;

[0013] S3. Preprocess the source model and the target model separately, and unify the coordinate systems and units of the two models;

[0014] S4. Select multiple anatomical feature points on the preprocessed source model, and select anatomical feature points on the preprocessed target model that semantically correspond to the anatomical feature points of the preprocessed source model. The two semantically corresponding anatomical feature points form an initial feature point pair. Then, with each anatomical feature point as the center, a neighborhood point set corresponding to each anatomical feature point is constructed using a neighborhood expansion strategy;

[0015] S5. Performing an initial rigid registration based on the initial feature point pairs selected in step S4 to obtain a coarsely registered source model; wherein the initial rigid registration process estimates optimal rigid transformation parameters for aligning the source model to the target model by minimizing the sum of squared Euclidean distances between corresponding anatomical feature points in the source model and the target model;

[0016] S6. Performing fine registration on the coarsely registered source model to make it highly consistent with the local geometric structure of the target model, thereby obtaining a finely registered source model;

[0017] S7. Obtain the local area on the precisely aligned source model that needs to be replaced by the target model, find the area corresponding to the local area in the target model and use it as the replacement area, crop the replacement area from the target model and embed it into the precisely aligned source model, and simultaneously perform topological repair, normal recalculation, and smoothing on the fusion boundary to eliminate seams and achieve seamless fusion. Finally, output a high-quality three-dimensional dental model with complete structure and enhanced local accuracy.

[0018] Based on the above solution, each step can be implemented in the following preferred specific manner.

[0019] As a preference for the first aspect above, in step S4, the specific method of constructing a neighborhood point set using a neighborhood expansion strategy is as follows: the selected anatomical feature point is used as the center of the spherical neighborhood, the radius of the spherical neighborhood is preset, and after the spherical neighborhood is formed, all points falling within the spherical neighborhood are retrieved on the model where the selected anatomical feature point is located using a spatial index structure to form a neighborhood point set corresponding to each anatomical feature point, and all anatomical feature points are traversed. Finally, the anatomical feature points and their neighborhood point set in the preprocessed source model are used as the source point set for registration, and the corresponding anatomical feature points and their neighborhood point set in the preprocessed target model are used as the target point set for registration.

[0020] As a preferred embodiment of the first aspect, in step S5, the specific steps of the initial rigid registration are as follows:

[0021] S51. For each point in the registered source point set, search for the point with the smallest Euclidean distance to it in the registered target point set to form a set of coarse registration feature point pairs. Estimate the optimal rotation matrix and translation vector using all constructed coarse registration feature point pairs based on the least squares method, and use the optimal rotation matrix and translation vector as the optimal rigid body transformation parameters.

[0022] S52. Apply the optimal rigid body transformation parameters to preliminarily align the source model to the target model coordinate system to obtain the roughly aligned source model.

[0023] As a preferred embodiment of the first aspect, in step S5, the specific steps of the initial rigid registration are as follows:

[0024] S61. For each anatomical feature point in the source model after coarse registration, find the triangular surface with the smallest orthogonal projection error with the anatomical feature point in the triangular surface mesh of the target model, and orthogonally project the anatomical feature point onto the triangular surface to obtain the corresponding projection point;

[0025] S62. Each anatomical feature point in the coarsely registered source model and its corresponding projection point form a matching point pair, and then construct a registration error term. The registration error terms obtained from all matching point pairs are summed as the total registration error. The total registration error is minimized through an iterative optimization method to obtain the precise rigid body transformation parameters.

[0026] S63. Apply precise rigid body transformation parameters to precisely align the source model to the target model coordinate system to obtain a precisely aligned source model.

[0027] Furthermore, the functional form of the total registration error E(R, t) is:

[0028]

[0029] Where p i represents the i-th anatomical feature point in the source model after coarse registration; p′i Indicates p i The corresponding projection point; t represents the translation vector in the precise rigid body transformation parameters; R represents the rotation matrix in the precise rigid body transformation parameters; ‖·‖ 2 represents the L2 norm; ‖Rp i +tp′ i ‖ 2 Represents matching point pairs (p i ,p′ i ) corresponds to the registration error term.

[0030] As a preference of the first aspect above, the radius is an integer between 2 mm and 5 mm.

[0031] As a preferred embodiment of the first aspect, the iterative optimization method is a Gauss-Newton method or a Loevenberg-Marquardt method.

[0032] In a second aspect, the present invention provides a dental three-dimensional model local optimization system based on multimodal fusion, which comprises:

[0033] The model building module is used to perform three-dimensional reconstruction on the user's CBCT raw image data to obtain a low-precision three-dimensional model containing alveolar bone and root structure information and use it as the source model; obtain the user's oral scan data and perform three-dimensional modeling to obtain a high-precision model containing crown and occlusal surface details and use it as the target model;

[0034] The preprocessing module is used to preprocess the source model and the target model respectively, and unify the coordinate systems and units of the two models;

[0035] The feature point selection, matching and neighborhood enhancement module is used to select multiple anatomical feature points on the preprocessed source model, and select anatomical feature points on the preprocessed target model that semantically correspond to the anatomical feature points of the preprocessed source model. The two semantically corresponding anatomical feature points form an initial feature point pair, and then, with each anatomical feature point as the center, a neighborhood point set corresponding to each anatomical feature point is constructed using a neighborhood expansion strategy;

[0036] A coarse registration module is used to perform initial rigid registration based on the selected initial feature point pairs to obtain a coarsely registered source model. The initial rigid registration process estimates the optimal rigid transformation parameters for aligning the source model to the target model by minimizing the sum of squared Euclidean distances between corresponding anatomical feature points in the source model and the target model.

[0037] The fine registration module is used to perform fine registration on the coarsely registered source model to make it highly consistent with the local geometric structure of the target model, thus obtaining the finely registered source model.

[0038] The local area replacement and fusion module is used for obtaining a local area on a source model which needs to be replaced by a target model after accurate registration, finding a corresponding area in the target model and taking the corresponding area as a replacement area, cutting and embedding the replacement area from the target model into the source model after accurate registration, and performing topological repair, normal recalculation and smoothing processing on a fusion boundary to eliminate a joint and realize seamless fusion, so that a high-quality three-dimensional dental model with complete structure and enhanced local precision is finally output.

[0039] In a third aspect, the present application provides a computer program product, comprising computer programs / instructions, which, when executed by a processor, can realize the multi-modal fusion based local optimization method of a dental three-dimensional model according to any one of the above first aspect.

[0040] In a fourth aspect, the present application provides a computer readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the multi-modal fusion based local optimization method of a dental three-dimensional model according to any one of the above first aspect is realized.

[0041] In a fifth aspect, the present application provides a computer electronic device, comprising a memory and a processor.

[0042] The memory is used for storing a computer program.

[0043] The processor is used for realizing the multi-modal fusion based local optimization method of a dental three-dimensional model according to any one of the above first aspect when the computer program is executed.

[0044] Compared with the prior art, the present application has the following beneficial effects:

[0045] The present invention discloses a local optimization method for dental three-dimensional models based on multimodal fusion. In view of the problems of low resolution of existing CBCT models, local missing of oral scan data, and insufficient stability in the multimodal registration process, the present invention proposes a combination of coarse registration and fine registration strategy based on triangular surface projection, and constructs a robust feature point pair set through manual point selection and neighborhood expansion, thereby enhancing the accuracy and stability of local structure alignment. The fine registration method based on triangular mesh projection proposed in the present invention can significantly improve the local registration accuracy between CBCT data and oral scan data. On this basis, through the local model embedding and boundary fusion mechanism, the problems of gaps, overlaps and normal inconsistencies that may occur in the local replacement process are effectively solved, and the natural transition and seamless integration between multimodal models are achieved. In addition, the present invention supports users to flexibly specify or automatically identify local areas to be optimized, which can effectively avoid redundant processing of non-critical areas and improve the overall modeling efficiency. This method overcomes the problems of low local registration accuracy, unnatural replacement, and poor fusion quality in existing technologies, taking into account the structural integrity and local geometric accuracy of the model. It is widely applicable to various clinical application scenarios such as denture design, virtual implantation, and occlusion analysis, and provides an efficient, stable and reliable three-dimensional model optimization solution for oral digital modeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flow chart of the steps of the present invention;

[0047] Figure 2 Schematic diagram of the CBCT data model obtained by three-dimensional reconstruction in this embodiment;

[0048] Figure 3 Schematic diagram of the oral scan data model obtained by three-dimensional modeling in this embodiment;

[0049] Figure 4 This is a schematic diagram of anatomical feature point selection and matching in this embodiment;

[0050] Figure 5 Schematic diagram of constructing matching point pairs by orthogonal projection in this embodiment;

[0051] Figure 6 Schematic diagram of the source model after precise registration in this embodiment;

[0052] Figure 7 This is a schematic diagram of the high-quality three-dimensional dental model finally generated in this embodiment. DETAILED DESCRIPTION

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more clearly understood, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in the various embodiments of the present invention can be combined accordingly without conflicting with each other.

[0054] The present invention provides a method for local optimization of dental three-dimensional models based on multimodal fusion. This method combines rigid registration with a projection-based fine registration strategy, enabling seamless replacement of high-precision local regions. The core concept is to construct a source model (CT model) and a target model (oral scan model), establish an initial correspondence using manually selected points, and after completing a coarse registration, use a fine registration based on triangulated surface projection to optimize the corresponding position of each point. Finally, the local region in the target model is embedded in the source model, achieving the unity of structural integrity and local accuracy.

[0055] like Figure 1 As shown, in a preferred implementation of the present invention, the above-mentioned method for local optimization of a dental three-dimensional model based on multimodal fusion includes the following steps S1 to S7. The specific implementation process is described in detail below.

[0056] 1) CBCT data modeling

[0057] S1. Perform 3D reconstruction on the user's original CBCT image data to obtain a low-precision 3D model containing alveolar bone and root structure information and use it as the source model.

[0058] It should be noted that, in the present invention, the CBCT data model (i.e., source model) obtained by three-dimensional reconstruction is as follows: Figure 2 As shown, the model has good overall structural coherence and is suitable as a full-mouth reference frame.

[0059] 2) Oral scan data modeling

[0060] S2. Obtain the oral scan data of the user and perform three-dimensional modeling to obtain a high-precision model including details of the crown and occlusal surface and use it as the target model.

[0061] It should be noted that, in the present invention, the oral scan data model (i.e., target model) obtained by three-dimensional modeling is as follows: Figure 3 As shown, this model provides detailed surface geometry and serves as a reference model for optimization.

[0062] 3) Data preprocessing

[0063] S3. Preprocess the source model and the target model separately, and unify the coordinate systems and units of the two models.

[0064] It should be noted that, in the present invention, the pre-processing operations performed on the source model and the target model include but are not limited to noise reduction, mesh simplification, and normal recalculation.

[0065] 4) Feature point selection, matching, and neighborhood enhancement

[0066] S4. Select multiple anatomical feature points on the preprocessed source model, and select anatomical feature points on the preprocessed target model that semantically correspond to the anatomical feature points of the preprocessed source model. An initial feature point pair is formed by the two semantically corresponding anatomical feature points. Then, with each anatomical feature point as the center, a neighborhood point set corresponding to each anatomical feature point is constructed using a neighborhood expansion strategy.

[0067] It should be noted that, in the present invention, the specific method of constructing a neighborhood point set using a neighborhood expansion strategy is as follows: the selected anatomical feature point is used as the center of the spherical neighborhood, the radius of the spherical neighborhood is preset, and after the spherical neighborhood is formed, the spatial index structure is used to retrieve all points that fall into the spherical neighborhood on the model where the selected anatomical feature point is located, forming a neighborhood point set corresponding to each anatomical feature point, traversing all anatomical feature points, and finally using the anatomical feature points and their neighborhood point set in the preprocessed source model as the source point set for registration, and using the corresponding anatomical feature points and their neighborhood point set in the preprocessed target model as the target point set for registration.

[0068] Therefore, the neighborhood expansion strategy avoids the registration failure problem caused by single-point error, while improving the accuracy and stability of the model's local structure alignment.

[0069] It should be noted that, in this embodiment, Figure 4 As shown in the figure, a small number of anatomical feature points with semantic correspondences, such as key points such as the cusp, incisal edge, and occlusal fossa, are manually or automatically selected on the source and target models to construct a set of feature point pairs. On this basis, to improve the robustness of the registration and the accuracy of local structure matching, a neighborhood expansion strategy is further adopted. With each anatomical feature point as the center, a spatial index structure (such as a KD tree) retrieves neighborhood points and constructs a neighborhood point set within a certain radius (such as 2mm to 5mm). This enhances the descriptive power of the feature point pair set and captures local shape features. These neighborhood points can then be used as extended input for the registration data to form weighted or equally weighted source and target point sets.

[0070] 5) Rough registration

[0071] S5. Perform an initial rigid registration based on the initial feature point pairs selected in step S4 to obtain a coarsely registered source model; wherein the initial rigid registration process estimates an optimal rigid body transformation parameter for aligning the source model to the target model by minimizing the sum of squared Euclidean distances between corresponding anatomical feature points in the source model and the target model.

[0072] It should be noted that, in the present invention, the specific steps of initial rigid registration are as follows:

[0073] S51. For each point in the registered source point set, search for the point with the smallest Euclidean distance to it in the registered target point set to form a set of coarse registration feature point pairs. Estimate the optimal rotation matrix and translation vector using all constructed coarse registration feature point pairs based on the least squares method, and use the optimal rotation matrix and translation vector as the optimal rigid body transformation parameters.

[0074] S52. Apply the optimal rigid body transformation parameters to preliminarily align the source model to the target model coordinate system to obtain the roughly aligned source model.

[0075] It should be noted that the coarse registration method of the present invention has the characteristics of high computational efficiency and strong robustness. It can effectively deal with the scale differences and initial position deviations between multimodal data, and provide good initial alignment results for subsequent fine registration.

[0076] 6) Precise registration based on triangular projection

[0077] S6. Perform fine registration on the coarsely registered source model to make it highly consistent with the local geometric structure of the target model, thereby obtaining a finely registered source model.

[0078] It should be noted that, in the present invention, based on the coarse registration result, the source model is further refined and registered. The specific steps are as follows:

[0079] S61. For each anatomical feature point p in the source model after coarse registration i , find the triangular surface with the smallest orthogonal projection error with the anatomical feature point in the triangular surface mesh of the target model, and orthogonally project the anatomical feature point onto the triangular surface to obtain the corresponding projection point p′ i .

[0080] In this embodiment, if Figure 5 As shown, the selected anatomical feature point p i , and its corresponding projection point is p′ i .

[0081] S62. Each anatomical feature point in the source model after rough registration and its corresponding projection point constitutes a matching point pair (p i ,p′ i), which is used to construct the registration error term. The registration error terms obtained from all matching point pairs are summed up as the total registration error. The total registration error is minimized through an iterative optimization method to obtain the precise rigid body transformation parameters.

[0082] It should be noted that, in this embodiment, the functional form of the total registration error E(R, t) is:

[0083]

[0084] Where p i represents the i-th anatomical feature point in the source model after coarse registration; p′ i Indicates p i The corresponding projection point; t represents the translation vector in the precise rigid body transformation parameters; R represents the rotation matrix in the precise rigid body transformation parameters; ‖·‖ 2 represents the L2 norm; ‖Rp i +tp′ i ‖ 2 Represents matching point pairs (p i ,p′ i ) corresponds to the registration error term.

[0085] It should be noted that, in this embodiment, the iterative optimization method may adopt the Gauss-Newton method or the Levenberg-Marquardt method (LM method).

[0086] S63. Apply precise rigid body transformation parameters to precisely align the source model to the target model coordinate system to obtain a precisely aligned source model.

[0087] In this embodiment, the source model after precise registration is as follows: Figure 6 shown.

[0088] 7) Local area replacement and fusion

[0089] S7. Obtain the local area (such as the crown area) on the precisely aligned source model that needs to be replaced by the target model, find the area corresponding to the local area in the target model and use it as the replacement area, crop the replacement area from the target model and embed it into the precisely aligned source model, and simultaneously perform topological repair, normal recalculation, and smoothing on the fusion boundary to eliminate seams and achieve seamless fusion, and finally output a high-quality three-dimensional dental model with complete structure and enhanced local accuracy.

[0090] In this embodiment, the high-quality three-dimensional dental model finally generated is as follows: Figure 7 shown.

[0091] It should also be noted that the multimodal fusion-based local optimization method for a dental 3D model in the above-mentioned embodiment can essentially be executed via a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a multimodal fusion-based local optimization system for a dental 3D model, corresponding to the multimodal fusion-based local optimization method for a dental 3D model in the above-mentioned embodiment, comprising:

[0092] The model building module is used to perform three-dimensional reconstruction on the user's CBCT raw image data to obtain a low-precision three-dimensional model containing alveolar bone and root structure information and use it as the source model; obtain the user's oral scan data and perform three-dimensional modeling to obtain a high-precision model containing crown and occlusal surface details and use it as the target model;

[0093] The preprocessing module is used to preprocess the source model and the target model respectively, and unify the coordinate systems and units of the two models;

[0094] The feature point selection, matching and neighborhood enhancement module is used to select multiple anatomical feature points on the preprocessed source model, and select anatomical feature points on the preprocessed target model that semantically correspond to the anatomical feature points of the preprocessed source model. The two semantically corresponding anatomical feature points form an initial feature point pair, and then, with each anatomical feature point as the center, a neighborhood point set corresponding to each anatomical feature point is constructed using a neighborhood expansion strategy;

[0095] A coarse registration module is used to perform initial rigid registration based on the selected initial feature point pairs to obtain a coarsely registered source model. The initial rigid registration process estimates the optimal rigid transformation parameters for aligning the source model to the target model by minimizing the sum of squared Euclidean distances between corresponding anatomical feature points in the source model and the target model.

[0096] The fine registration module is used to perform fine registration on the coarsely registered source model to make it highly consistent with the local geometric structure of the target model, thus obtaining the finely registered source model.

[0097] The local area replacement and fusion module is used to obtain the local area on the precisely aligned source model that needs to be replaced by the target model, find the area corresponding to the local area in the target model and use it as the replacement area, crop the replacement area from the target model and embed it into the precisely aligned source model, and at the same time perform topological repair, normal recalculation and smoothing on the fusion boundary to eliminate seams and achieve seamless fusion, and finally output a high-quality three-dimensional dental model with complete structure and enhanced local accuracy.

[0098] It is understood that the multimodal fusion-based local optimization method for a dental 3D model described in S1-S7 above can essentially be implemented via a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer program product corresponding to the multimodal fusion-based local optimization method for a dental 3D model provided in the above embodiment. The computer program product includes a computer program / instructions that, when executed by a processor, implements the multimodal fusion-based local optimization method for a dental 3D model as described in the above embodiment.

[0099] Similarly, based on the same inventive concept, another preferred embodiment of the present invention further provides a computer electronic device corresponding to the dental three-dimensional model local optimization method based on multimodal fusion provided in the above embodiment, which includes a memory and a processor;

[0100] The memory is used to store computer programs;

[0101] The processor is configured to implement the dental three-dimensional model local optimization method based on multimodal fusion in the above embodiment when executing the computer program.

[0102] Furthermore, the logic instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention.

[0103] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the dental three-dimensional model local optimization method based on multimodal fusion provided in the above embodiment, and the storage medium stores a computer program. When the computer program is executed by the processor, it can implement the dental three-dimensional model local optimization method based on multimodal fusion in the above embodiment.

[0104] It is understood that the storage medium may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage medium may be any medium capable of storing program code, such as a USB flash drive, a mobile hard drive, a magnetic disk, or an optical disk.

[0105] It is understandable that the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0106] It should also be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the various embodiments provided in this application, the division of steps or modules in the system and method is only a logical function division. In actual implementation, there may be other division methods, for example, multiple modules or steps can be combined or integrated together, and a module or step can also be split.

[0107] The embodiment described above is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Persons skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent substitution or equivalent transformation falls within the scope of protection of the present invention.

Claims

1. A dental 3D model local optimization method based on multimodal fusion, characterized in that: The following steps are involved: S1. Perform 3D reconstruction on the user's original CBCT image data to obtain a low-precision 3D model containing alveolar bone and root structure information and use it as the source model; S2. Obtaining the user's oral scan data and performing three-dimensional modeling to obtain a high-precision model containing details of the crown and occlusal surface and using it as the target model; S3. Preprocess the source model and the target model separately, and unify the coordinate systems and units of the two models; S4. Select multiple anatomical feature points on the preprocessed source model, and select anatomical feature points on the preprocessed target model that semantically correspond to the anatomical feature points of the preprocessed source model. The two semantically corresponding anatomical feature points form an initial feature point pair. Then, with each anatomical feature point as the center, a neighborhood point set corresponding to each anatomical feature point is constructed using a neighborhood expansion strategy; S5. Performing an initial rigid registration based on the initial feature point pairs selected in step S4 to obtain a coarsely registered source model; wherein the initial rigid registration process estimates optimal rigid transformation parameters for aligning the source model to the target model by minimizing the sum of squared Euclidean distances between corresponding anatomical feature points in the source model and the target model; S6. Performing fine registration on the coarsely registered source model to make it highly consistent with the local geometric structure of the target model, thereby obtaining a finely registered source model; S7. Obtain the local area on the precisely aligned source model that needs to be replaced by the target model, find the area corresponding to the local area in the target model and use it as the replacement area, crop the replacement area from the target model and embed it into the precisely aligned source model, and simultaneously perform topological repair, normal recalculation, and smoothing on the fusion boundary to eliminate seams and achieve seamless fusion. Finally, output a high-quality three-dimensional dental model with complete structure and enhanced local accuracy.

2. A dental three-dimensional model local optimization method based on multimodal fusion according to claim 1, characterized in that: In step S4, the specific method of constructing a neighborhood point set using the neighborhood expansion strategy is as follows: the selected anatomical feature point is used as the center of the spherical neighborhood, the radius of the spherical neighborhood is preset, and after the spherical neighborhood is formed, the spatial index structure is used to retrieve all points that fall into the spherical neighborhood on the model where the selected anatomical feature point is located to form a neighborhood point set corresponding to each anatomical feature point, and all anatomical feature points are traversed. Finally, the anatomical feature points and their neighborhood point set in the preprocessed source model are used as the source point set for registration, and the corresponding anatomical feature points and their neighborhood point set in the preprocessed target model are used as the target point set for registration.

3. The method for local optimization of a dental three-dimensional model based on multimodal fusion according to claim 2, characterized in that: In step S5, the specific steps of initial rigid registration are as follows: S51. For each point in the registered source point set, search for the point with the smallest Euclidean distance to it in the registered target point set to form a set of coarse registration feature point pairs. Estimate the optimal rotation matrix and translation vector using all constructed coarse registration feature point pairs based on the least squares method, and use the optimal rotation matrix and translation vector as the optimal rigid body transformation parameters. S52. Apply the optimal rigid body transformation parameters to preliminarily align the source model to the target model coordinate system to obtain the roughly aligned source model.

4. The method for local optimization of a dental three-dimensional model based on multimodal fusion according to claim 3, characterized in that: In step S5, the specific steps of initial rigid registration are as follows: S61. For each anatomical feature point in the source model after coarse registration, find the triangular surface with the smallest orthogonal projection error with the anatomical feature point in the triangular surface mesh of the target model, and orthogonally project the anatomical feature point onto the triangular surface to obtain the corresponding projection point; S62. Each anatomical feature point in the coarsely registered source model and its corresponding projection point form a matching point pair, and then construct a registration error term. The registration error terms obtained from all matching point pairs are summed as the total registration error. The total registration error is minimized through an iterative optimization method to obtain the precise rigid body transformation parameters. S63. Apply precise rigid body transformation parameters to precisely align the source model to the target model coordinate system to obtain a precisely aligned source model.

5. The method for local optimization of a dental three-dimensional model based on multimodal fusion according to claim 2, characterized in that: The radius is an integer between 2 mm and 5 mm.

6. The method for local optimization of a dental three-dimensional model based on multimodal fusion according to claim 4, characterized in that: The iterative optimization method is the Gauss-Newton method or the Loevenberg-Marquardt method.

7. A dental three-dimensional model local optimization system based on multimodal fusion, characterized in that: include: The model building module is used to perform three-dimensional reconstruction on the user's CBCT raw image data to obtain a low-precision three-dimensional model containing alveolar bone and root structure information and use it as the source model; obtain the user's oral scan data and perform three-dimensional modeling to obtain a high-precision model containing crown and occlusal surface details and use it as the target model; The preprocessing module is used to preprocess the source model and the target model respectively, and unify the coordinate systems and units of the two models; The feature point selection, matching and neighborhood enhancement module is used to select multiple anatomical feature points on the preprocessed source model, and select anatomical feature points on the preprocessed target model that semantically correspond to the anatomical feature points of the preprocessed source model. The two semantically corresponding anatomical feature points form an initial feature point pair, and then, with each anatomical feature point as the center, a neighborhood point set corresponding to each anatomical feature point is constructed using a neighborhood expansion strategy; A coarse registration module is used to perform initial rigid registration based on the selected initial feature point pairs to obtain a coarsely registered source model. The initial rigid registration process estimates the optimal rigid transformation parameters for aligning the source model to the target model by minimizing the sum of squared Euclidean distances between corresponding anatomical feature points in the source model and the target model. The fine registration module is used to perform fine registration on the coarsely registered source model to make it highly consistent with the local geometric structure of the target model, thus obtaining the finely registered source model. The local area replacement and fusion module is used to obtain the local area on the precisely aligned source model that needs to be replaced by the target model, find the area corresponding to the local area in the target model and use it as the replacement area, crop the replacement area from the target model and embed it into the precisely aligned source model, and at the same time perform topological repair, normal recalculation and smoothing on the fusion boundary to eliminate seams and achieve seamless fusion, and finally output a high-quality three-dimensional dental model with complete structure and enhanced local accuracy.

8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, it can implement the dental three-dimensional model local optimization method based on multimodal fusion as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by the processor, the method for local optimization of a dental three-dimensional model based on multimodal fusion according to any one of claims 1 to 6 is implemented.

10. A computer electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is configured to implement the dental three-dimensional model local optimization method based on multimodal fusion as described in any one of claims 1 to 6 when executing the computer program.

Images