Three-dimensional stomatognathic model reconstruction system based on multi-modal data fusion

Abstract

The invention discloses a three-dimensional stomatognathic model reconstruction system based on multi-modal data fusion. The system comprises a multi-modal data input unit, a template deformation reconstruction unit, a registration fusion unit, a multi-source data integration unit and an output unit. Through fusion processing of a CBCT image, an oral cavity vision measurement model and facial scanning data, a body deformation algorithm is adopted to couple biomechanical characteristics to realize craniojaw template deformation, and a non-rigid ICP algorithm is combined for dynamic regulation and control to realize facial template adaptation. A deep neural network is innovatively constructed to segment CBCT gingival data, the CBCT gingival data is fused with an oral cavity vision measurement model, and high-precision tooth reconstruction is realized by applying a differential geometry multi-scale curvature field segmentation and adversarial edge optimization technology. Through a composite registration strategy combining adaptive rigid registration and non-rigid registration, an occlusal plane constraint mechanism and an orbital curvature extreme point matching algorithm are innovatively introduced, finally, multi-source data high-precision registration fusion is realized, and a three-dimensional oral-jaw system model with anatomical structure integrity and clinical precision can be generated.

Description

Technical Field: The present invention belongs to the field of computer-aided biomedical engineering, and particularly relates to a three-dimensional oral and maxillofacial model reconstruction system for multimodal data fusion. Background Art: Currently, multi-source data acquisition means are generally adopted in the digital oral diagnosis and treatment workflow, including obtaining the three-dimensional morphology of the dental arch by an oral vision measuring instrument, recording the dynamic characteristics of soft tissues by a facial scanner, and reconstructing the bony anatomical structure by cone beam computed tomography (CBCT). There are significant technical bottlenecks in the three-dimensional integration process of cross-source heterogeneous data in traditional methods: 1) The significant differences in accuracy and acquisition coordinate systems of different modality data lead to misalignment in fusion; 2) Existing multimodal registration algorithms lack a stable biometric space mapping mechanism and are difficult to maintain precise biomechanical matching in the topological relationship of the dental occlusion-jaw-craniofacial complex, thus restricting the accurate implementation of personalized treatment plan planning, treatment prognosis simulation, and interdisciplinary data interoperability in clinical practice.

[0003] The present invention optimizes the technology for key problems in multimodal oral and maxillofacial three-dimensional reconstruction. Existing individualized modeling methods mainly have problems such as data integrity defects and insufficient registration reliability. The present invention proposes a three-dimensional oral and maxillofacial model reconstruction system for multimodal data. In the first stage, an adaptive template deformation algorithm based on anatomical topology constraints is used. By introducing a geometric reasoning mechanism of biomechanical characteristics, high-precision repair of the maxillofacial defect area is achieved, and morphological noise filtering of multi-source scan data is synchronously completed. In the second stage, a composite feature registration technology guided by the occlusal plane is adopted. By fusing the occlusal functional characteristics of teeth and craniofacial anatomical landmark points, a registration framework with dual anatomical-functional constraints is constructed to ensure the accurate spatial relationship mapping of the jawbone, facial soft tissues, and tooth units in three-dimensional space. This solution breaks through the limitations of traditional single-modal registration and realizes a composite oral and maxillofacial model with a reconstruction accuracy matching the clinical biomechanical requirements, providing a reliable basic support for digital orthodontics and implant restoration. Summary of the Invention: The purpose of the present invention is to provide a method for a three-dimensional oral and maxillofacial model reconstruction system for multimodal data, to solve problems such as a large amount of noise and incompleteness in the current model.

[0005] The present invention is implemented as follows: S1. The input unit uses CBCT, an oral vision measuring instrument, and a facial scanner to obtain head CT images, an oral vision measurement model, and a facial model, and prepares a template model of the standard oral and maxillofacial and facial soft and hard tissues.

[0006] S2. Fuse the CBCT teeth and the oral vision measurement crown model, and output a high-precision tooth surface by combining rigid and non-rigid registration algorithms.

[0007] The specific content of S2 is as follows: S21: The CBCT tooth reconstruction unit uses deep learning technology to segment CBCT and reconstruct a tooth model. First, preprocess and enhance the CBCT data. Ensure the consistency of the image data by standardizing the voxel spacing and gray level range. At the same time, adopt a hybrid data augmentation strategy: at the frequency domain level, combine the fast Fourier transform to achieve cross-case fusion of low-frequency components and high-frequency noise injection; in the spatial domain, apply elastic deformation with dynamic constraints and local rotation to simulate real occlusal differences. Secondly, construct a three-dimensional segmentation network that combines residual attention and dilated convolution. The network architecture is based on 3D UNet. A channel-spatial dual attention mechanism is embedded in the encoder to improve the ability to recognize small tooth features. The decoder uses progressive dilated convolution to capture multi-scale context information. During training, optimize the model parameters through a weighted cross-entropy and Dice joint loss function. In the third stage, optimize the tooth model reconstruction. Introduce an adversarial training mechanism to construct an edge-sensitive discriminator. Achieve precise contour correction by dynamically adjusting the gradient difference between the edges generated by the segmentation network and the real annotation. At the same time, use multi-scale cubic B-spline interpolation to fuse feature maps of different resolutions, and combine the marching cubes algorithm to generate a topologically continuous crown-root integrated model. Finally, complete the end-to-end system integration and verification. Achieve clinical-level reconstruction efficiency through a GPU-accelerated inference pipeline, and use the three-dimensional Dice coefficient, surface root mean square error, and subjective scores of oral surgeons to quantitatively evaluate the reconstruction accuracy.

[0008] S22: The tooth segmentation unit of the oral visual measurement model optimizes the extraction of tooth feature lines through curvature feature analysis and three-dimensional morphological processing, and achieves accurate crown segmentation based on topological verification. First, construct a curvature tensor field through local cubic surface fitting, and use the ridge point / valley point determination criterion to identify initial feature points: for the mesh edge , if it meets the extreme coefficient sign condition and the principal curvature sorting constraint , then interpolate to generate candidate feature points; use feature intensity

[0009] to filter out weak features and retain high-confidence feature lines. Subsequently, map the feature lines to the original mesh and perform a three-dimensional morphological dilation operation. Expand the feature vertex set through a spherical structuring element to close the broken area:

[0010] where is the set of discrete feature line vertices, is the radius of the spherical kernel. Further adopt an improved thinning algorithm to extract the skeleton of the feature region: calculate the vertex neighborhood crossing number

[0011] Iteratively delete non-endpoints and meet "Simple" vertices; perform chamfer smoothing on the skeleton through geodesic projection to ensure that the error is less than the average edge length Finally, verify the biomedical width of the closed-loop feature line

[0012] where is the gingival boundary, and the single tooth crown is segmented along the feature line, and its topological integrity index

[0013] Verify the surface closure

[0014] S23: The registration deformation unit adopts adaptive multimodal registration and improved ICP deformation optimization technology to achieve high-precision tooth modeling through exponential weight fusion and differential geometric constraints. Register and deform the teeth reconstructed by CBCT and the teeth segmented by the oral visual measurement model. First, derive the adaptive search radius based on the geometric feature parameters of the diagonal distance of the three-dimensional model bounding box, and its mathematical expression is

[0015] Based on this, calculate the fast point feature histogram of the three-dimensional point cloud of the source and target models. By constructing a matching cost function based on the spatial distribution characteristics

[0016] where is the curvature similarity weight, and the optimal rigid body transformation parameters are iteratively solved by combining the random sample consensus robust estimator. In this process, the Levenberg-Marquardt optimization algorithm is used to synchronously optimize the rotation matrix R and the translation vector T to satisfy

[0017] The convergence condition

[0018] In the non-rigid optimization part, a two-layer optimization model is constructed in the improved iterative closest point framework. The surface layer uses the k-d tree accelerated nearest neighbor search algorithm to establish the correspondence, and its time complexity is optimized to . The underlying deformation field modeling introduces the thin plate spline energy function

[0019] And couple the curvature consistency constraint term to form a composite objective function. Set the dynamic convergence criterion

[0020] Achieve the balance between computational efficiency and accuracy

[0021] In the data fusion stage, establish a parametric mapping function in the feature-sensitive area

[0022] Among them, the mixing weights Adopt an exponential decay model

[0023] Perform spatial modulation. In the fusion boundary region, construct a continuity constraint condition by calculating the L2 norm difference of the second-order differential attributes of the surface, ensuring that the geometric transition meets the morphological requirements of

[0024] Finally, adopt the Hausdorff distance calculation method based on spatial binning statistics and the voxelized Dice similarity coefficient as the quantitative verification index.

[0025] S3. Separate and reconstruct the cranial / maxillofacial hard tissues and facial soft tissues through a biomechanical driving algorithm, and adapt to the anatomical structure of the target individual The specific content of S3 is as follows: S31: The cranial-maxillofacial hard tissue template deformation unit is divided into two multi-modal global registration and elastic deformation systems. In the global registration stage, an improved iterative closest point - principal component analysis algorithm is adopted. First, establish the initial pose through the principal component axis matching. Specifically, calculate the covariance matrix of the template vertices and the CT voxel centers, enumerate four groups of possible principal component analysis axis correspondence relationships (including the permutations and combinations of the three coordinate axes and mirror transformations), and then perform ICP optimization with an adaptive scaling factor: by minimizing an energy function in the form of

[0026] (where u is the global scaling factor, d is the translation vector, p represents the rotated point, corresponding to the point q in the target shape), synchronously optimize the 7-degree-of-freedom parameters. This strategy selects the configuration with the minimum Hausdorff distance through a multi-candidate verification mechanism, laying the foundation for subsequent non-rigid deformation. In the local deformation stage, construct a volume-preserving elastic model to achieve sub-voxel-level fine registration: based on tetrahedral mechanics theory, design a composite energy equation No specific content here, please check the original text

[0027] Among them, the corresponding terms establish the vertex-voxel association through the binary projection matrix P, and the Laplacian matrix L in the Dirichlet energy term is weighted by the tetrahedral volume, effectively maintaining the biomechanical characteristics. Introduce a dynamic relaxation mechanism: initially set to perform strong regularization constraints. When the median vertex displacement between consecutive iterations < 0.1 voxel size, Exponential decay (halving each time) forms a 9-level progressive optimization hierarchy. For the vertices inside the cranial cavity, a penetration optimization strategy is developed - detecting the template vertices entering the CT skull through the ray method, setting them to zero in their corresponding item weight matrix W, and deforming only through the Dirichlet energy transfer between tetrahedrons.

[0028] S32: In the facial soft tissue template deformation unit, the facial soft tissue template model undergoes non-rigid deformation based on the facial scan model. For the spatial alignment problem between the facial scan (polygonal mesh data) and CBCT (voxel data), a dual registration strategy based on feature points is adopted. First, a biological coordinate system composed of bilateral tragus points and nasion is established, and initial rough registration is achieved by manually annotating 12 anatomical landmark points (including bony landmarks such as infraorbital foramen and mental foramen, and soft tissue landmarks such as the tip of the nose and lip margin). Subsequently, an improved iterative closest point algorithm is applied, introducing a curvature constraint condition on the basis of the traditional point-to-plane matching: calculating the Gaussian curvature of the sampled points, and setting the point pairs with a curvature difference threshold ≤ 0.15 mm⁻¹ to participate in the iteration. The algorithm uses a kd-tree to accelerate the search, and performs singular value decomposition (SVD) to optimize the rigid body transformation matrix in each round of iteration. The convergence condition is set as the root mean square error change of 10 consecutive iterations < 0.1 mm.

[0029] After completing the spatial registration, a hierarchical fusion architecture is designed to process different data characteristics. For the surface geometric data, vertex coloring mapping technology is adopted: encoding the bone density value of CBCT into the value channel of the HSV color space, and performing channel fusion with the RGB texture information of the facial scan. For the internal structure, a hybrid data structure based on octree is developed: retaining the original CBCT voxels in the critical regions of bone tissue (such as alveolar process, temporomandibular joint, etc.), and converting the soft tissue region into an adaptive subdivision mesh. A cross-modal interpolation algorithm is specially designed to achieve smooth joining within a 5-mm transition zone at the registration boundary using the moving least squares method. The weight function selects a bicubic polynomial kernel to ensure continuous gradient.

[0030] S4. Optimize the spatial consistency of the multi-source model based on occlusal constraints and bio-markers to ensure the three-dimensional matching of the cranio-maxillofacial-dentition-soft tissue. Synthesize multi-modal data and verification results, and output a functional three-dimensional oral and maxillofacial model. Specifically, first, construct a spatial rigid constraint framework based on the occlusal plane, register the hard tissue model of the jaw and the actual occlusal data through the singular value decomposition algorithm, and strictly limit the rigid transformation to ensure a reasonable biological movement trajectory. Secondly, optimize the global alignment accuracy through anatomical markers, extract the mutation points of the orbital margin curvature and the symmetric features of the nasal base as spatial anchors, and perform elastic matching of the soft tissue and hard tissue in combination with the thin plate spline deformation algorithm. Finally, achieve full-modal fusion through the coupling of multiple biomechanical factors, establish the tooth contact constraint relationship based on the alveolar morphology, drive the dynamic attachment of the muscles using the linear mixed skinning algorithm, and synchronously verify the geometric consistency (such as the jaw-tooth distance, surface continuity) and functional simulation adaptability (such as bite force response) of the model through iterative optimization to ensure the authenticity and reliability of the model in anatomical structure and dynamic mechanics.

[0031] Compared with the prior art, the beneficial effects of this solution are as follows: (1) Innovatively propose a multi-modal anatomical topology fusion architecture, and achieve sub-millimeter-level spatial integration of the cranio-maxillofacial complex through two-stage rigid registration (occlusal plane constraint + feature point guidance). Compared with the traditional single-point matching method, the alignment accuracy of anatomical markers is improved, and the efficiency of clinical diagnosis and treatment planning is optimized.

[0032] (2) Adopt an improved non-rigid deformation algorithm (elastic boundary constraint + thin plate spline regularization) to simultaneously maintain the micro-geometric accuracy of the tooth crown surface, the biomechanical characteristics of the jaw bone, and the facial soft tissue continuity during the registration process, providing a high-fidelity data basis for constructing a digital virtual patient with multi-physical field coupling. Description of the Drawings: In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0034] Figure 1 It is a schematic diagram of the overall process of the multi-modal three-dimensional oral and maxillofacial model reconstruction system of the present invention. Detailed Embodiments: To construct a three-dimensional oral and maxillofacial model based on multimodal data and enable those skilled in the art to better understand the solution of the present invention, the technical solution of the present invention will be further described in detail below in conjunction with the embodiments and accompanying drawings of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0036] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0037] As Figure 1 shown, a three-dimensional oral and maxillofacial model reconstruction system for multimodal data.

[0038] S1. Use a CBCT, an oral vision measuring instrument, and a facial scanner to obtain head CT images, intraoral occlusion models, and facial models, and prepare template models of the skull, jawbone, and facial soft and hard tissues.

[0039] S2. Fuse the CBCT tooth roots and the oral scanned tooth crown models, and combine rigid and non-rigid registration algorithms to output a high-precision tooth surface.

[0040] S21. Use deep learning technology to segment the CBCT and reconstruct the tooth model. The implementation of this step can be divided into four main stages. First, perform CBCT data preprocessing and enhancement to ensure the consistency of image data by standardizing the voxel spacing and gray level range. At the same time, adopt a hybrid data enhancement strategy: at the frequency domain level, combine the fast Fourier transform to achieve cross-case fusion of low-frequency components and high-frequency noise injection, and apply dynamically constrained elastic deformation and local rotation in the spatial domain to simulate real occlusion differences. Secondly, construct a three-dimensional segmentation network that combines residual attention and dilated convolution. The network architecture is based on 3D UNet. A channel-spatial dual attention mechanism is embedded in the encoder to improve the recognition ability of small tooth features, and the decoder uses progressive dilated convolution to capture multi-scale context information. During training, optimize the model parameters through a weighted cross-entropy and Dice joint loss function. In the third stage, perform tooth model reconstruction optimization. Introduce an adversarial training mechanism to construct an edge-sensitive discriminator, and achieve precise contour correction by dynamically adjusting the gradient difference between the edges generated by the segmentation network and the real annotations. At the same time, use multi-scale cubic B-spline interpolation to fuse feature maps of different resolutions, and combine the marching cubes algorithm to generate a topologically continuous crown-root integrated model. Finally, complete end-to-end system integration and verification. Achieve clinical-level reconstruction efficiency through a GPU-accelerated inference pipeline, and quantitatively evaluate the reconstruction accuracy using three-dimensional Dice coefficients, surface root mean square errors, and subjective scores of oral surgeons.

[0041] S22. Tooth segmentation of the oral visual measurement model. Based on the differential geometric properties and topological skeleton analysis of the three-dimensional dental and jaw model, this step realizes the extraction of the closed-loop feature lines between teeth with medical-grade high precision. The specific implementation process is as follows: First, construct a curvature tensor field through local cubic surface fitting, and use the ridge point / valley point determination criterion to identify the initial feature points: for the mesh edge , if the extreme coefficient sign condition and the principal curvature sorting constraint are satisfied, then interpolate to generate candidate feature points; use the feature strength

[0042] to filter out weak features and retain high-confidence feature lines. Subsequently, map the feature lines to the original mesh and perform a three-dimensional morphological dilation operation to expand the feature vertex set through a spherical structure element to close the fracture region:

[0043] where is the set of discrete feature line vertices, is the radius of the spherical kernel. Further, use an improved thinning algorithm to extract the skeleton of the feature region: calculate the vertex neighborhood crossing number

[0044] iteratively delete non-endpoint and "simple" vertices that satisfy ; perform chamfer smoothing on the skeleton through geodesic projection to ensure that the error is less than the average edge length . Finally, verify the biomedical width of the closed-loop feature line

[0045] where is the gingival boundary, and segment the single tooth crown along the feature line, and its topological integrity index

[0046] verifies the surface closure.

[0047] S23. Fusion of the teeth in the oral visual measurement model and the teeth reconstructed by CBCT. This step consists of three parts: feature matching, non-rigid optimization, and data fusion. In the feature matching part, first, derive an adaptive search radius based on the geometric feature parameters of the diagonal distance of the three-dimensional model bounding box, and its mathematical expression is

[0048] Based on this, calculate the fast point feature histogram of the three-dimensional point cloud of the source and target models. By constructing a matching cost function based on the spatial distribution features

[0049] where is the curvature similarity weight, and the optimal rigid body transformation parameters are iteratively solved by combining with the random sample consensus robust estimator. In this process, the Levenberg-Marquardt optimization algorithm is used to synchronously optimize the rotation matrix R and the translation vector T to satisfy

[0050] the convergence condition.

[0051] In the non-rigid optimization part, a two-layer optimization model is constructed in the improved iterative closest point framework. The surface layer uses the k-d tree accelerated nearest neighbor search algorithm to establish the corresponding relationship, and its time complexity is optimized to . The underlying deformation field modeling introduces the thin plate spline energy function

[0052] and couples the curvature consistency constraint term to form a composite objective function. Set the dynamic convergence criterion

[0053] to achieve the balance between computational efficiency and accuracy.

[0054] In the data fusion stage, a parametric mapping function

[0055] is established in the feature sensitive area, where the mixing weight adopts the exponential decay model

[0056] for spatial modulation. In the fusion boundary region, the continuity constraint condition is constructed by calculating the L2 norm difference of the second-order differential attribute of the surface to ensure that the geometric transition satisfies

[0057] the morphological requirements. Finally, the Hausdorff distance calculation method based on spatial binning statistics and the voxelized Dice similarity coefficient are used as the quantitative verification indicators.

[0058] S3. Separate and reconstruct the cranial / jaw hard tissues and facial soft tissues through the biomechanical driving algorithm to adapt to the anatomical structure of the target individual.

[0059] S31. The skull and jawbone template model undergoes non-rigid deformation based on CBCT. This step requires preparing bimodal volume data as input, one being a high-quality tetrahedral template mesh of a healthy skull and the other being voxelized skull data from the patient's CT scan. This step is divided into a multimodal global registration and a local deformation stage. In the global registration stage, the initial spatial alignment of the template mesh and the patient's CT data is achieved. First, the principal axis correspondence of the two-modal data is established through principal component analysis (PCA): Calculate the covariance matrix of the template mesh vertices and the CT voxel center points, and enumerate four groups of possible PCA axis correspondences including axis permutations and mirror transformations, covering the symmetry and direction differences of the anatomical structure. Subsequently, for each candidate axis configuration, an improved iterative closest point (ICP) optimization is performed to synchronously optimize the 7-degree-of-freedom parameters including the global scaling factor , translation vector , and rotation matrix, minimizing the energy function

[0060] where is the rotated coordinate of the template vertex, and is the corresponding point of the target CT data. Through a multi-candidate verification mechanism, multiple rounds of optimization iterations are performed for each initialization, and the best registration result is selected based on the Hausdorff distance, significantly improving the initial alignment accuracy.

[0061] In the local deformation stage, based on the global registration, a volume-preserving elastic model is constructed based on tetrahedral mechanics, and sub-voxel-level deformation is achieved using a composite energy equation:

[0062] where the first term associates the template vertices with the CT voxels through a binary projection matrix , the Laplacian matrix of the second term maintains smoothness according to the tetrahedral volume weighted mechanical properties, and the third term introduces an inertia constraint to prevent oscillation. The dynamic relaxation mechanism adjusts the regularization strength according to the iteration convergence degree: In the initial stage, a high weight constrains the overall deformation. When the median vertex displacement is lower than the preset threshold, is gradually exponentially decayed, forming a hierarchical optimization framework and gradually releasing the degrees of freedom for local detail adjustment.

[0063] To avoid the template vertices penetrating into the interior of the patient's CT skull, ray casting is used to detect the vertices to be corrected, and a weight matrix Set its weight in the projection term to zero so that its deformation only depends on the Dirichlet energy transfer between tetrahedrons. At the same time, use the weighted Laplacian term to ensure that the deformation energy in the non-penetrating area diffuses naturally through the mesh topology, maintaining the continuity and biomechanical rationality of the anatomical structure. The optimization process iteratively solves the energy minimization problem to finally achieve high-precision non-rigid registration between the template mesh and the patient's skull.

[0064] S32. The facial template model undergoes non-rigid deformation based on the facial scan model. For the spatial alignment problem between the facial scan (polygonal mesh data) and CBCT (voxel data), a dual registration strategy based on feature points is adopted. First, establish a biological coordinate system composed of bilateral tragus points and nasion point, and achieve initial rough registration by manually marking 12 anatomical landmark points (including bony landmarks such as infraorbital foramen and mental foramen, and soft tissue landmarks such as the tip of the nose and lip margin). Subsequently, apply the improved iterative closest point algorithm, and introduce a curvature constraint condition on the basis of the traditional point-to-plane matching: calculate the Gaussian curvature of the sampling points (neighborhood with a radius of 3 mm), and set the point pairs with a curvature difference threshold ≤ 0.15 mm⁻¹ to participate in the iteration. The algorithm uses kd-tree to accelerate the search, and performs singular value decomposition (SVD) to optimize the rigid body transformation matrix in each round of iteration. The convergence condition is set as the root mean square error change of 10 consecutive iterations < 0.02 mm.

[0065] After completing the spatial registration, design a hierarchical fusion architecture to process different data characteristics. For surface geometric data, use vertex coloring mapping technology: encode the bone density value of CBCT as the value channel in the HSV color space, and perform channel fusion with the RGB texture information of the facial scan. For internal structures, develop an octree-based hybrid data structure: retain the original CBCT voxels (resolution of 0.2 mm³) in the critical areas of bone tissue (alveolar process, temporomandibular joint, etc.), and convert the soft tissue area into an adaptive subdivision mesh. Specially design a cross-modal interpolation algorithm, and use the moving least squares method to achieve smooth joining within a 5-mm transition zone at the registration boundary. The weight function selects a bicubic polynomial kernel to ensure continuous gradient.

[0066] S4. Calibrate each model of the stomatognathic system. First, construct a spatial rigid constraint framework based on the occlusal plane, register the jaw hard tissue model with the actual occlusal data through the singular value decomposition algorithm, and strictly limit the rigid transformation to ensure a reasonable biological movement trajectory. Secondly, optimize the global alignment accuracy through anatomical landmark points, extract the orbital margin curvature mutation points and the nasal base symmetry features as spatial anchor points, and perform elastic matching of soft tissue and hard tissue in combination with the thin plate spline deformation algorithm. Finally, achieve full-modal fusion through the coupling of multiple biomechanical elements, establish tooth contact constraint relationships based on the alveolar morphology, drive the dynamic attachment of muscles using the linear blend skinning algorithm, and verify the geometric consistency (such as the jaw-teeth distance, surface continuity) and functional simulation adaptability (such as bite force response) of the model through iterative optimization to ensure the authenticity and reliability of the model in anatomical structure and dynamic mechanics.

[0067] The above has introduced in detail a three-dimensional stomatognathic model reconstruction system for multimodal data provided by an embodiment of the present invention. Specific examples are used herein to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A three-dimensional oral and maxillofacial model reconstruction system for multimodal data, characterized in that, The system includes a multi-modal data input unit, a template deformation reconstruction unit, a model registration and fusion unit, a multi-source data calibration unit, and an output unit for the 3D model of the oro-maxillofacial system; among them, the multi-modal data includes: oral photos, X-ray films, CBCT image data, oral visual measurement models, facial scan models, 3D oro-maxillofacial template models, etc., and the template deformation reconstruction unit includes a cranial and jaw hard tissue template deformation reconstruction unit and a facial soft tissue template deformation reconstruction unit; the fusion unit of the CBCT reconstructed teeth and the oral visual measurement crown model includes a tooth segmentation unit of the oral visual measurement model, a CBCT tooth reconstruction unit, and a registration and deformation unit.

2. The three-dimensional oral and maxillofacial model reconstruction system for multimodal data fusion according to claim 1, characterized in that The input unit is used to read and parse oral image data, oral visual measurement models, facial scan models, standard oro-maxillofacial template models, etc.; among them, the template deformation reconstruction unit deforms the standard template model into the oro-maxillofacial target model to be reconstructed; the template deformation reconstruction unit realizes the accurate deformation mapping of the template model to the CBCT target tissue through the coupling of volume energy conservation constraints and biomechanical characteristics; the facial template deformation reconstruction unit realizes the precise adaptation of the facial template to the target model through the coupling of dynamic stiffness weight regulation and multi-scale deformation fields.

3. The three-dimensional oral and maxillofacial model reconstruction system for multimodal data according to claim 2, characterized in that, After the operation of the template deformation reconstruction unit, the output model is a three-dimensional model of hard and soft tissues such as the skull, jaws, and face with two-dimensional manifold characteristics, and the corresponding data of the cranial and jaw surface model unit and the facial surface model unit are the input data of the multi-source data calibration unit.

4. A three-dimensional oral and maxillofacial model reconstruction system for multimodal data fusion according to claim 1, characterized in that, The fusion unit of the CBCT reconstructed teeth and the oral visual measurement crown model first reconstructs the tooth surface model in the CBCT image, and then uses a non-rigid deformation registration algorithm to deform with the oral visual measurement crown surface model as the target to achieve the fusion of the CBCT reconstructed teeth and the oral visual measurement crown model; the fusion unit of the CBCT reconstructed teeth and the oral visual measurement crown model includes a tooth segmentation unit of the oral visual measurement model, a CBCT tooth reconstruction unit, and a registration and deformation unit.

5. A three-dimensional oral and maxillofacial model reconstruction system for multimodal data fusion according to claim 4, characterized in that, The tooth segmentation unit of the oral visual measurement model constructs a grid multi-scale curvature field through differential geometry algorithms and realizes accurate tooth segmentation by fusing curvature-driven fuzzy clustering; the CBCT tooth reconstruction unit constructs a three-dimensional segmentation network that fuses residual attention and dilated convolution, adopts a data augmentation strategy that combines frequency domain enhancement and geometric distortion, and combines adversarial edge optimization and multi-scale spline interpolation to realize tooth model reconstruction; the registration and fusion unit realizes rigid body registration through adaptive point fast feature histogram feature matching and random sample consensus robust alignment, completes non-rigid registration using an iterative registration algorithm and thin plate spline deformation optimization, and combines exponential decay hybrid mapping and dual medical index verification to achieve high-precision fusion of multi-modal dental and jaw models.

6. The three-dimensional oral and maxillofacial model reconstruction system for multimodal data according to claim 5, characterized in that, After the operations of the tooth segmentation unit of the oral visual measurement model, the CBCT tooth reconstruction unit, and the registration and deformation unit, an accurate tooth surface model is output, and the corresponding data of the tooth model unit with an accurate crown are the input data of the model alignment unit.

7. A three-dimensional oral and maxillofacial model reconstruction system for multimodal data according to claim 6, characterized in that, The multi-source data calibration unit realizes the alignment of the jawbone model based on the rigid registration constrained by the occlusal plane, the improved spatial registration based on the extreme points of the orbital margin curvature and the topologically invariant points of the nasal base, and the fusion of the full-element head model including the three-dimensional data of the skull-jaw-dentition-soft tissue, and finally reconstructs a three-dimensional oral and maxillofacial model based on multi-modal data.

Images