Dimensionality reduction adaptive multimodal image registration method, device, medium and system

Abstract

The application discloses a dimension reduction adaptive multi-modal image registration method and device, medium and system, and belongs to the field of medical image processing, and comprises the following steps: CBCT data dimension reduction and dentition region labeling, CBCT dentition segmentation based on deep learning, IOS model preprocessing and dentition segmentation, rough registration, fine registration and the like; the application improves registration accuracy and efficiency, and has high clinical popularization value.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and more specifically, to a method, apparatus, medium, and system for dimensionality reduction adaptive multimodal image registration. Background Technology

[0002] In digital dental implantology, multimodal registration of cone-beam computed tomography (CBCT) and intraoral scanning (IOS) models is a key technology for surgical planning, surgical guide design, and postoperative evaluation. CBCT can provide information on deep anatomical structures such as tooth roots, alveolar bone, and nerve canals, but it suffers from high radiation dose, limited spatial resolution, and metal artifacts. IOS models, on the other hand, offer high-resolution details of the crown surface but cannot display root and bone tissue information. Therefore, precise registration and fusion of these two technologies can complement each other, constructing a digital model containing complete crown morphology and root information, providing a data foundation for precise implantation.

[0003] Automatic registration of cone-beam computed tomography (CBCT) and intraoral scan (IOS) models is a key technology in the digital workflow of oral and maxillofacial surgery, dental implantology, and orthodontics. It aims to fuse data from two different modalities to overcome their respective limitations and achieve high-precision 3D dental and jaw reconstruction. CBCT provides comprehensive information on tooth roots and jawbones, but its depiction of crown surface details is limited and it is susceptible to metal artifacts. In contrast, IOS can capture tooth surface details with high precision, but it cannot acquire root and bone data, and may introduce cumulative stitching errors during full arch scanning. Therefore, effective registration of CBCT and IOS data is a necessary step in constructing a complete and accurate dental and jaw model. Currently, automatic registration methods are mainly divided into two categories: traditional methods based on geometric features and artificial intelligence methods based on deep learning.

[0004] Traditional geometric feature-based methods typically employ a two-stage process of "coarse registration + fine registration." First, coarse registration is performed by extracting feature descriptors from the point cloud (such as Fast Point Feature Histograms (FPFH), 4-Point Congruent Sets (4PCS), etc.) or utilizing prior information such as dental arch curves to initially align the two models. Then, fine registration is performed using the Iterative Closest Point (ICP) algorithm or its variants. This involves iterative optimization to find the optimal rigid body transformation between the two point sets, minimizing the registration error. The core of these methods lies in utilizing the geometric characteristics of the point cloud data itself for matching. FPFH (Fast Point Feature Histograms) is a local feature descriptor based on the geometric information of a point and its neighborhood, capable of capturing surface shape changes in the point cloud. The 4PCS (4-Point Congruent Sets) algorithm estimates the initial transformation by finding the consistency set of four coplanar points in the two point clouds. The ICP (Iterative Closest Point) algorithm and its variants are commonly used methods for fine registration; they optimize the transformation matrix by repeatedly calculating the nearest point pairs and minimizing the distance between them until convergence. Traditional methods have the advantage of not relying on large amounts of training data and having relatively low computational resource requirements. For example, surface registration-based methods achieve soft tissue registration with IOS by creating a soft tissue model of CBCT scans. The method proposed by Zhai et al. utilizes covariance descriptors and non-cooperative game theory for TLS point cloud registration, and works effectively even in the presence of noise, outliers, and varying degrees of overlap. Guo et al. proposed a point cloud registration method based on multi-local feature matching, which improves registration accuracy by studying the influence of feature values ​​and descriptors (including corresponding radii of curvature) of point cloud sets at different scales on reducing FPFH misregistration. Despite these advantages, traditional methods also have significant drawbacks. First, the performance of the ICP algorithm is highly dependent on the initial position of coarse registration. When the initial alignment is poor, the algorithm is prone to converge to a local optimum, leading to registration failure. Second, these methods are sensitive to outliers and noise. Common metal artifacts, anatomical variations, and morphological differences between different modalities in CBCT data can lead to errors in point cloud correspondence, significantly reducing the registration accuracy of the ICP algorithm. Piao et al. evaluated the impact of different registration methods and dental restorations on the registration time and accuracy of CBCT and IOS data, finding that the proportion of restorations affects registration accuracy. Furthermore, when patients have edentulous areas, malocclusion, or numerous metal restorations, the reliability of geometric feature extraction and matching decreases, making stable registration difficult and lacking robustness.

[0005] In recent years, deep learning-based registration methods have significantly improved the automation and robustness of registration by learning the mapping relationship between CBCT and IOS through end-to-end network models. These methods typically utilize convolutional neural networks for feature extraction, correspondence matching, or direct regression of transformation parameters, enabling them to handle more complex clinical scenarios. Deep learning methods, by learning complex nonlinear mappings, can automatically extract high-level features from raw image data, thus achieving more robust registration. For example, the Deep Dental Multimodal Fusion (DDMF) framework is an automated multimodal fusion framework that uses deep learning models to reconstruct high-fidelity 3D dental bone structures. This framework first preprocesses CBCT and IOS data, then performs registration through a deep learning model, and finally fuses the data to generate a high-quality 3D model. Kim et al. proposed a single-tooth image fusion method that seamlessly integrates CBCT and IOS data, fusing crowns from IOS and roots from CBCT into a single image. Ren et al. achieved high-precision tooth reconstruction using automated multimodal fusion technology; their method not only combines the advantages of CBCT and IOS but also solves problems such as stitching errors during data fusion. ACSwinNet is a deep learning-based rigid registration method for accurate registration of head and neck CT-CBCT images in image-guided radiotherapy. Such methods also show potential in handling non-rigid deformations. For example, research has explored deep learning-based deformable image registration for organ deformation during radiotherapy.

[0006] Deep learning-based methods also face challenges. First, training deep learning models requires a large amount of precisely labeled paired data. However, manually performing high-precision registration and annotation of CBCT and IOS is a labor-intensive task, requiring repeated operations by professionals. Due to factors such as patient privacy, there is currently a lack of large-scale, precisely labeled public datasets of CBCT-IOS pairings, which severely limits the development and clinical translation of algorithms. Second, CBCT voxel data and IOS point cloud data have high-dimensionality characteristics. Directly inputting them into deep learning networks for inference consumes a large amount of computational resources, resulting in low registration efficiency and difficulty in meeting real-time clinical needs. For example, the artificial intelligence system proposed by Liu et al. focuses on the automatic segmentation of CBCT images of oral surgery-related tissues. Its image preprocessing is based on data distribution histograms and can adaptively process CBCT images with different parameters. While this preprocessing step helps improve segmentation accuracy, it also increases the computational burden. Finally, existing models are quite sensitive to real-world clinical variations outside the training data (such as different scanning devices, different reconstruction parameters, personalized anatomical structures, etc.), and the transferability and stability of the models need to be improved. For example, while models such as SynthSeg demonstrate robustness to changes in contrast and resolution in brain MRI segmentation, this generalization ability remains a challenge in oral multimodal registration.

[0007] Both traditional methods based on geometric features and artificial intelligence methods based on deep learning share some common problems with current CBCT-IOS automatic registration technology: Sensitivity to metal artifacts: In clinical practice, patients often have metal restorations (such as porcelain crowns, implants, etc.). CBCT images produce physical effects such as beam hardening, scattering, and photon starvation, leading to stripes and shadows in metal artifact areas, interfering with the accurate identification of anatomical structures. Existing methods experience a significant decrease in registration accuracy as the degree of metal artifacts increases. Although some metal artifact reduction (MAR) algorithms exist, such as context-loss-based artifact removal methods or synthetic CT (sCT) methods, these methods may blur fine structures while removing artifacts, affecting the final registration accuracy. Poor anatomical adaptability: Different patients have individual differences in dental arch morphology, edentulism, and alveolar bone conditions. Existing methods struggle to adaptively handle these anatomical variations, resulting in unstable registration performance. This limits its application in complex clinical cases to some extent. For example, in CBCT mandibular bone overlays of orthodontic patients, voxel-based and surface-based registration methods show varying performance in terms of accuracy, reliability, and efficiency. High data acquisition and annotation costs: whether it's manual parameter adjustment in traditional methods to avoid local optima or large-scale data annotation in deep learning methods, both require significant manpower and time, increasing the barrier to entry for these technologies. This is similar to the challenges faced in the current field of medical image registration, namely the lack of high-quality "real label" deformation fields for supervised learning. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and provide a dimension-reduced adaptive multimodal image registration method, device, medium and system, which improves registration accuracy and efficiency and has high clinical application value.

[0009] The objective of this invention is achieved through the following solution: A dimension reduction adaptive multimodal image registration method includes the following steps: Three-dimensional CBCT data is used to generate two-dimensional MIP images. The dental arch regions in the two-dimensional MIP images are labeled to obtain curves that separate the upper and lower dental arches. The labeled data is used to train a deep learning network to establish a mapping model from the two-dimensional MIP images to the curves that separate the upper and lower teeth and the endpoints of the dental arches. The 2D MIP image generated from the CBCT data to be registered is input into the trained deep learning network to predict the curves separating the upper and lower dentitions and the endpoint positions of the dentitions. Based on the curves and endpoint positions, the dentition region is segmented in the 3D CBCT data to obtain the CBCT dentition point cloud model. Furthermore, the spatial coordinate system of the IOS model is determined and the dentition region is segmented, and the gingiva and soft tissue are removed to obtain the IOS dentition point cloud model. The obtained CBCT dental arch point cloud model and the IOS dental arch point cloud model are aligned in spatial coordinates to obtain preliminary alignment registration results. For the pre-aligned CBCT dentition point cloud model and IOS dentition point cloud model, the curvature variance of the neighboring region is sampled to obtain a high-neighbor curvature variance sampling point set and a low-neighbor curvature variance sampling point set. Then, the high-neighbor curvature variance sampling point set is registered, and then the point set combining the high-neighbor curvature variance sampling point set and the low-neighbor curvature variance sampling point set is registered to obtain the final transformation matrix. The transformation matrix is ​​then applied to the original IOS model to complete the final registration.

[0010] Furthermore, the step of generating a two-dimensional MIP image from three-dimensional CBCT data specifically includes the following sub-steps: encoding the maximum HU value of each pixel along the y-axis using the maximum density projection algorithm, and projecting it from the midsagittal plane along the positive and negative x-axis directions respectively to generate two side-view MIP images, one on the left and one on the right. The generated MIP images are uniformly adjusted to a fixed pixel size, scaled proportionally horizontally, and filled with selected color pixels vertically.

[0011] Furthermore, the deep learning network adopts an improved DenseNet121 network structure, specifically including two independent branch networks, which are used to predict the dental arch contour curve and endpoint position, respectively.

[0012] Furthermore, the training of the deep learning network specifically includes the following sub-steps: Step 1051: Design an improved DenseNet121 network structure: adopt a dual-branch output structure, share the DenseNet121 backbone network, and branch out two independent fully connected layers in the last layer: a curve prediction branch and an endpoint prediction branch; the output of the curve prediction branch corresponds to the predicted y-coordinate value of each column of the MIP image; the output of the endpoint prediction branch corresponds to the (x,y) coordinates of the four endpoints; the DenseNet121 backbone network contains multiple dense blocks, each dense block contains multiple convolutional layers, and dense connections are used between layers; configure the network parameters. Step 1052, set training parameters; Step 1053: Train the model until the loss function converges; during the training process, save the model parameters once every set number of rounds, and finally select the model with the smallest loss on the validation set as the final model. Step 1054, Curve post-processing optimization, performs fixed-window smoothing optimization on the output of the curve prediction branch.

[0013] Furthermore, the training parameters include: Batch size: batchSize=10; Training epochs = 1300; The initial learning rate is set to initialLearnRatio = 0.01. The Adam optimizer is used, with MSELoss as the loss function; The fixed window smoothing optimization specifically fixes the window length to 17, marks points that exceed the neighborhood average by ±1.2 standard deviations as outliers, and replaces the outliers with the neighborhood average.

[0014] Furthermore, before segmenting the dentition region from the 3D CBCT data, the process includes the step of determining a reconstruction threshold based on the CBCT data. Step 1071: Read all voxel HU values ​​of CBCT data, construct HU value set H, statistically analyze the HU value distribution, and calculate the cumulative frequency; Step 1072: Calculate the dynamic threshold using the following formula: dHU = min{h ∈ H | |{k : h_k ≤ h}|>0.98N}; in: N is the total number of voxels; |{k : h_k ≤ h}| represents the number of voxels whose HU value is less than or equal to h; min{…} represents the minimum HU value that satisfies the condition; This dynamic threshold corresponds to the 98th percentile of the HU value distribution, ensuring that areas with bone density, such as the crown, are included, while excluding low-density areas, including air and soft tissue. Step 1073: Using dHU as the reconstruction threshold, voxels with HU values ​​greater than or equal to dHU in the CBCT data are marked as tooth regions for subsequent three-dimensional reconstruction.

[0015] Furthermore, the step of segmenting the dental arch region in the 3D CBCT data to obtain a CBCT dental arch point cloud model specifically includes the following sub-steps: Based on deep learning prediction results and dynamic reconstruction thresholds, precise segmentation of dental arch regions is achieved in 3D CBCT data, specifically including: Step 1061: Limiting the search range, using the four endpoints predicted by deep learning to determine the search range for the dental arch: x-axis range: [min(endpoint 1_x, endpoint 3_x), max(endpoint 2_x, endpoint 4_x)]; y-axis range: The entire y-axis range; z-axis range: determined by the position of the tooth cusp; when searching upwards, only the first high-density voxel encountered is checked. Step 1062: Maximum gradient search along the axis. A maximum gradient search method along the axis is proposed, specifically extracting CBCT data layer by layer along the x-axis, and searching column by column along the y-axis for each layer: For each slice with a fixed x-axis, scan from top to bottom along the y-axis; Calculate the gradient of HU values ​​of adjacent voxels: Gradient(y) = |HU(y+1) - HU(y)|; The location of the maximum gradient corresponds to the boundary of the tooth crown surface; Record the y-coordinate of this position as the boundary point of the tooth row; Step 1063: Extract the crown region. Search along the z-axis for the position of the first high-density voxel as the cusp, extend downwards within a set range, and retain all voxels within that range. Step 1064: Perform fixed-window smoothing optimization on the extracted tooth row boundaries to eliminate local outliers; Step 1065: Convert the segmented dental arch region voxels into a 3D point cloud format. Each voxel corresponds to a point and contains (x, y, z) coordinate information. The point cloud data is used for subsequent registration.

[0016] Furthermore, the spatial coordinate system determines the axial direction using Principal Component Analysis (PCA) and the horizontal direction using the minimum bounding rectangle. Specifically, PCA is used to obtain three principal axes, with the principal axis having the smallest eigenvalue corresponding to the shortest direction, which is taken as the z-axis. A slice of a set number of pixels is taken along the z-axis from the cusp to the gingiva, and the minimum bounding rectangle of the point cloud of the slice is calculated to determine the x-axis and y-axis directions.

[0017] Furthermore, the IOS model is segmented into dental arch regions, specifically including the following sub-steps: using the endpoints predicted by the deep learning network to determine the range of the anterior and posterior dental arches, extracting points within the range of the maximum / minimum z-coordinate vertices along the z-axis, and removing gingival tissue.

[0018] Further, the step of sampling the curvature variance of neighboring regions to obtain a set of sampling points with high neighbor curvature variance and a set of sampling points with low neighbor curvature variance specifically includes the following sub-steps: Step 1111, for each point p in the point cloud i Calculate the average principal curvature of each point in its neighborhood; Step 1112, calculate the CVN value for each point using the following formula: ; in: p i Points in a point cloud; N is the number of neighboring points; C j For p i The average principal curvature of the j-th neighboring point; avg(C j ) represents the mean of the principal curvatures of all neighboring points; Step 1113: Perform the same double-threshold sampling on both the CBCT point cloud and the IOS point cloud to ensure that the sampled point sets of the two point clouds have a corresponding relationship; the double-threshold sampling specifically includes: Set two CVN thresholds and perform two sampling operations: High proximity curvature variance sampling point set: Using the first CVN threshold, points with CVN values ​​greater than or equal to the first CVN threshold are extracted. These points correspond to the non-smooth areas on the crown surface and are used as key feature points for registration. Low Neighbor Curvature Variance Sampling Point Set: Using the second CVN threshold, points with CVN values ​​less than the second CVN threshold are extracted. These points correspond to smooth regions. The characteristic of CVN values ​​reflecting the curvature of local regions is used to exclude metal artifact points.

[0019] Furthermore, the step of aligning the obtained CBCT dental point cloud model with the IOS dental point cloud model in terms of spatial coordinate axes specifically includes the following sub-steps: using the spatial coordinate system of the CBCT data as a reference, the spatial coordinate system of the IOS model is aligned with it through translation and rotation.

[0020] Furthermore, the registration of the high-neighbor curvature variance sampling point set specifically employs the iterative nearest-neighbor algorithm to register the high-neighbor curvature variance sampling point set.

[0021] Furthermore, the registration of the point set combining high-neighbor curvature variance sampling and low-neighbor curvature variance sampling specifically employs the generalized iterative nearest-point algorithm to register the point set combining high-neighbor curvature variance sampling and low-neighbor curvature variance sampling.

[0022] Furthermore, obtaining the final transformation matrix specifically includes the following sub-steps: fixing the CBCT model position as reference data, moving and rotating the IOS model; recording the transformation matrix of the two registrations, and finally applying it to the original IOS model.

[0023] A dimension-reduction adaptive multimodal image registration device includes a MIP projection module, a data annotation module, a data augmentation module, a deep learning training module, a dental arch segmentation module, a dynamic reconstruction module, an IOS preprocessing module, a coarse registration module, a CVN sampling module, an ICP registration module, a GICP registration module, a transformation application module, and a result output module. This device is used to run the dimension-reduction adaptive multimodal image registration method as described in any of the preceding claims, and: The MIP projection module is used to project 3D CBCT data to generate 2D MIP images. The data annotation module is used to annotate the dental arch contours and endpoints of MIP images; The data augmentation module is used to perform random translation augmentation on labeled images; The deep learning training module is used to train deep learning networks to predict curves and endpoints. The dentition segmentation module is used to segment dentition in 3D CBCT based on prediction results; The dynamic reconstruction module is used to dynamically determine the reconstruction threshold; The IOS preprocessing module is used to remove soft tissue from the gums and determine the spatial coordinate system. The coarse registration module is used to align the coordinate axes of the CBCT and IOS models. The CVN sampling module is used to sample the curvature variance of the neighborhood region. The ICP registration module is used to achieve fine registration with a set of sampling points with high neighbor curvature variance; The GICP registration module is used to achieve fine registration with the high and low neighbor curvature variance sampling point set; The Transform Application module is used to apply the transformation matrix to the original IOS model.

[0024] Furthermore, it also includes a CBCT data acquisition module, used to legally acquire CBCT raw data in DICOM format.

[0025] Furthermore, it also includes an iOS data acquisition module, used to legally acquire iOS model data in STL format.

[0026] Furthermore, it also includes a result output module, which is used to output the registered fusion model.

[0027] A computer device, characterized in that the computer device includes a processor and a memory, wherein the memory stores a computer program that, when loaded by the processor, executes the method described in any of the preceding claims.

[0028] A computer-readable storage medium storing a computer program that, when loaded by a processor, executes the method described in any of the preceding claims.

[0029] The beneficial effects of this invention include: (1) This invention achieves fully automated processing through MIP dimensionality reduction, deep learning automatic segmentation and two-stage automatic registration, which significantly improves registration efficiency and shortens preoperative planning time; while ensuring high efficiency, it achieves registration accuracy comparable to advanced commercial platforms.

[0030] (2) The present invention uses the maximum density projection MIP algorithm to reduce the dimensionality of three-dimensional CBCT data to two dimensions, simplifying the complex three-dimensional segmentation task into two-dimensional image annotation, reducing the annotation workload, eliminating the need for time-consuming three-dimensional point cloud registration and annotation by professional technicians; reducing the training data requirements and technical threshold, thereby reducing data dependence and annotation costs, and facilitating clinical translation.

[0031] (3) This invention effectively identifies and filters metal artifacts in CBCT data by using the adjacent region curvature variance CVN sampling technology. High CVN threshold sampling extracts feature areas such as marginal ridges and incisal ridges on the crown surface, providing stable feature points for registration; low CVN threshold sampling excludes abnormal points in the metal artifact area, avoiding erroneous correspondence caused by artifacts, thereby effectively resisting metal artifact interference and improving clinical robustness.

[0032] (4) This invention enhances the adaptability to different anatomical variations of patients through multiple technical features. Among them, the dynamic HU adjustment algorithm determines the reconstruction threshold based on the CBCT data of each patient, which solves the problem of inaccurate reconstruction with fixed threshold under different equipment and different scanning parameters. The improved dentition segmentation method combines the endpoint range predicted by deep learning and the maximum gradient search along the axis, which can accurately segment the upper and lower dentition. Even in complex cases such as missing teeth and malocclusion, it can still effectively extract the crown region. The fixed window smoothing optimization performs post-processing on the deep learning prediction curve, corrects outliers, and improves the continuity and accuracy of the segmentation results. Thus, it can adapt to personalized anatomical structures and improve the generalization ability of the algorithm.

[0033] (5) This invention eliminates the need for manual intervention throughout the entire process, from CBCT data import, dental segmentation, coarse registration to fine registration, thus eliminating errors caused by differences in operator experience during manual registration. Compared with semi-automatic methods, this invention eliminates the need for manual selection of initial points or adjustment of parameters, avoiding the problem of ICP getting trapped in local optima due to poor initial positions. Compared with existing deep learning methods, this invention reduces computational complexity through dimensionality reduction and feature sampling, can run quickly on ordinary desktop computers, has good hardware compatibility, and thus can achieve full automation, reducing human error.

[0034] (6) The present invention adopts a two-stage fine registration strategy of ICP registration of high neighbor curvature variance sampling point set and GICP registration of high and low neighbor curvature variance sampling point set, which takes into account both local feature preservation and overall structure alignment, thereby improving registration stability and consistency.

[0035] (7) While ensuring sub-millimeter accuracy, the present invention achieves minute-level registration efficiency and does not require operation by professional technicians. It can be directly embedded into the digital implantation workflow. The present invention can serve as an effective alternative to manual registration and provide efficient and accurate data fusion support for preoperative planning, surgical guide design, postoperative evaluation and other aspects. It has significant clinical application value. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a general flowchart of the method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the deep learning network training process for dental arch region recognition in the method of this embodiment of the invention; Figure 3 This is a flowchart of the two-stage registration and alignment process in the method of this embodiment of the invention. Detailed Implementation

[0038] All features disclosed in all embodiments of this specification, or steps in all methods or processes implied in the disclosure, may be combined and / or extended or replaced in any way, except for mutually exclusive features and / or steps.

[0039] In a preferred embodiment, a multimodal image automatic registration scheme for cone-beam computed tomography (CBCT) and intraoral scan (IOS) models for dental implantation is particularly disclosed, including a method and corresponding apparatus.

[0040] As a first aspect of the present invention, a dimension reduction adaptive multimodal image registration method is provided, comprising the following steps: Step S1: CBCT data dimensionality reduction and dental arch region annotation: Project the 3D CBCT data along a specified direction (maximum density projection algorithm can be used) to generate a 2D MIP image; annotate the dental arch region in the 2D MIP image to obtain the curves separating the upper and lower dental arches; it should be noted that, in specific implementation, the annotated data is used to train a deep learning network to establish a mapping model from the 2D MIP image to the location of the dental arch region.

[0041] Deep learning-based CBCT dental segmentation: Generate a 2D MIP image from the CBCT data to be registered (the maximum density projection algorithm can be used), input it into a trained deep learning network, and predict the curves that separate the upper and lower dental arches and the endpoint positions of the dental arches; based on the curves and endpoint positions, segment the dental arch region in the 3D CBCT data to obtain a CBCT dental arch point cloud model.

[0042] IOS Model Preprocessing and Tooth Arch Segmentation: The spatial coordinate system of the IOS model is determined and the tooth arch region is segmented. The gingiva and soft tissue are removed to obtain the IOS tooth arch point cloud model. It should be noted that in the specific implementation, the spatial coordinate system is determined by the principal component analysis (PCA) method to determine the axis and by the minimum bounding rectangle to determine the horizontal direction.

[0043] Step S2: Coarse registration: Align the obtained CBCT dental point cloud model with the IOS dental point cloud model in terms of spatial coordinate axes, so that the origin and coordinate axis direction of the two are consistent, and obtain the preliminary alignment registration result.

[0044] Step S3: Fine Registration Based on Curvature Sampling: The CBCT dentistry point cloud model and the IOS dentistry point cloud model after initial alignment are sampled for the curvature variance of neighboring regions to obtain high-neighbor curvature variance sampling point sets and low-neighbor curvature variance sampling point sets. First, the high-neighbor curvature variance sampling point sets are registered, and then the point set combining the high-neighbor curvature variance and low-neighbor curvature variance sampling is registered to obtain the final transformation matrix. The transformation matrix is ​​then applied to the original IOS model to complete automatic registration. It should be noted that in the specific implementation, the iterative nearest-point algorithm is used to register the high-neighbor curvature variance sampling point set, and then the generalized iterative nearest-point algorithm is used to register the point set combining the high-neighbor curvature variance and low-neighbor curvature variance sampling.

[0045] In the above method embodiment, the MIP algorithm is used to reduce the dimensionality of three-dimensional data to two-dimensional in terms of data processing, transforming the three-dimensional segmentation problem into a two-dimensional annotation problem, reducing the difficulty of data acquisition and annotation, and avoiding the problem of heavy computational burden and the need for three-dimensional annotation when directly processing three-dimensional CBCT voxel data.

[0046] In the above-described method embodiment, the curvature variance sampling of the neighboring region is adopted. By setting high and low neighbor curvature variance sampling point sets with different thresholds, metal artifact interference can be effectively identified and filtered out. This solves the problems of existing technologies that rely on intensity-based similarity analysis and are sensitive to artifacts, or that use methods such as RANSAC but have low computational efficiency.

[0047] The above method embodiment proposes a dynamic HU adjustment algorithm dHU, which automatically determines the reconstruction threshold based on individual patient data. This can improve adaptability to different devices and solve the problem that CBCT reconstruction using a fixed HU threshold is difficult to adapt to different device parameters.

[0048] In the above method embodiment, the use of endpoint-limited range + maximum gradient search along the axis and fixed window smoothing optimization improves the segmentation accuracy and avoids the problem that tooth row segmentation methods rely on a single segmentation curve to search along the z-axis, making it difficult to separate the upper and lower teeth when they are close together.

[0049] In the above method embodiment, after PCA determines the z-axis, the minimum bounding rectangle is calculated to determine the x and y axis directions, which improves the initial alignment accuracy and solves the problem of inaccurate direction determination when relying solely on PCA or feature point matching.

[0050] In the above method implementation scheme, a two-stage strategy is adopted, which is to first register by sampling high neighbor curvature variance and then register by combining high and low neighbor curvature variance sampling. This approach takes into account both edge feature preservation and overall registration accuracy, and proposes a fine registration strategy, thus avoiding the problems of single ICP or GICP algorithms.

[0051] In other embodiments based on the above-described method of the present invention, additional features regarding the deep learning network are as follows: The deep learning network employs an improved DenseNet121 network structure, including two independent branch networks, respectively used to predict the DE curves separating the upper and lower dental arches and the four endpoint positions of the dental arches. The training parameters of the DenseNet121 network are: batchSize=10, epochs=1300, initialLearnRatio=0.01, using the Adam optimizer and MSELoss as the loss function. Fixed-window smoothing optimization is performed on the curve results predicted by the deep learning network, with a window length of 17. Points exceeding the neighborhood mean ±1.2 standard deviations are marked as outliers and corrected.

[0052] In other embodiments based on the above-described method of the present invention, a further design scheme for CBCT data preprocessing is as follows: The maximum density projection algorithm encodes the maximum HU value of each pixel along the y-axis direction, projects it from the midsagittal plane along the positive and negative x-axis directions respectively, generating two lateral MIP images, one on the left and one on the right. The generated MIP images are uniformly adjusted to a size of 512×512 pixels, scaled proportionally horizontally, and filled with black pixels vertically. The calculation formula for the dynamic HU adjustment algorithm dHU is: dHU = min{h∈H | |{k:h_k≤h}|>0.98N}, that is, the minimum HU value greater than 98% of voxels in the patient data is taken as the reconstruction threshold.

[0053] In other embodiments based on the above-described method of the present invention, a further design scheme for IOS model processing is provided: the method for determining the spatial coordinate system of the IOS model includes: obtaining three principal axes using principal component analysis, with the shortest principal axis as the z-axis (vertical direction); taking 4 pixels along the z-axis from the cusp to the gingiva, and calculating the minimum bounding rectangle of the slice to determine the x-axis and y-axis directions. The IOS model dentition segmentation includes: determining the anterior and posterior dental arch range using four endpoints predicted by deep learning, extracting points within a 7mm range from the maximum / minimum z-coordinate vertex along the z-axis, and removing gingival tissue.

[0054] In other embodiments based on the above-described method of the present invention, a further design scheme for sampling the curvature variance of the neighboring region is as follows: The formula for calculating the curvature variance of the neighboring region is: S_pi = 1 / N ∑(Cj - avg(Cj))^2, where pi is a point in the point cloud, N is the number of neighboring points, and Cj is the average principal curvature of the j-th neighboring point. The high neighbor curvature variance sampling point set is sampled using a high CVN threshold (0.0001) to extract the non-smooth regions (marginal ridges and incisal ridges) on the crown surface; the low neighbor curvature variance sampling point set is sampled using a low CVN threshold (0.00001) to exclude interference from metal artifacts.

[0055] In other embodiments based on the above-described method of the present invention, a further design scheme for the registration process is as follows: coarse registration includes: using the spatial coordinate system of the CBCT data as a reference, aligning the spatial coordinate system of the IOS model with it through translation and rotation. Fine registration includes: fixing the position of the CBCT model as reference data, moving and rotating the IOS model; recording the transformation matrix of the two registrations, and finally applying it to the original IOS model.

[0056] in, Figure 1 The entire process from CBCT data input to final registration result output is demonstrated, including key steps such as MIP projection, data annotation, data augmentation, DenseNet training, dental segmentation and reconstruction, IOS model preprocessing, coarse registration, and fine registration. Figure 2 The training process of a deep learning network for dental arch region recognition is demonstrated. Figure 3 The detailed workflow from CBCT and IOS model preprocessing to final fine registration is demonstrated, including dentition segmentation, coarse registration, CVN sampling, ICP registration, and GICP registration.

[0057] As a second aspect of the present invention, a dimension-reduction adaptive multimodal image registration device is provided, particularly a dimension-reduction adaptive CBCT-IOS automatic registration device based on deep learning, comprising the following modules: a CBCT data acquisition module, a MIP projection module, a data annotation module, a data augmentation module, a deep learning training module, a dental arch segmentation module, a dynamic reconstruction module, an IOS data acquisition module, an IOS preprocessing module, a coarse registration module, a CVN sampling module, an ICP registration module, a GICP registration module transformation application module, and a result output module.

[0058] The CBCT data acquisition module acquires the patient's raw CBCT data in DICOM format.

[0059] The MIP projection module projects 3D CBCT data to generate 2D MIP images.

[0060] The data annotation module annotates the dental arch contours and endpoints of the MIP image.

[0061] The data augmentation module performs random translation enhancement on the labeled image.

[0062] The deep learning training module trains the DenseNet121 network to predict curves and endpoints.

[0063] Among them, the dentition segmentation module segments the dentition in 3D CBCT based on the prediction results.

[0064] The dynamic reconstruction module uses the dHU algorithm to dynamically determine the reconstruction threshold.

[0065] The IOS data acquisition module acquires IOS model data in STL format from patients.

[0066] The IOS preprocessing module removes soft tissue from the gums and determines the spatial coordinate system.

[0067] The coarse registration module aligns the coordinate axes of the CBCT and the IOS model.

[0068] The CVN sampling module performs curvature variance sampling in the vicinity region.

[0069] Among them, the ICP registration module enables fine registration of sampling point sets with high proximity curvature variance.

[0070] Among them, the GICP registration module enables fine registration of the high and low neighbor curvature variance sampling point set.

[0071] The transformation application module applies the transformation matrix to the original IOS model.

[0072] The result output module outputs the registered fusion model. More specifically, the CBCT data acquisition module obtains raw CBCT data in DICOM format from patients. The module acquires CBCT data from the Department of Implantology at West China Hospital of Stomatology. Inclusion criteria are: age 18-70 years, good oral hygiene, and scan time between February 2020 and February 2025. Exclusion criteria are: cysts, tumors, abnormal tooth morphology, missing 6 or more teeth in a single jaw, severe periodontal disease, and untreated systemic diseases. It should be noted that this embodiment collects 200 CBCT data cases, of which 180 are used for training and 20 for testing. All data were approved by the Ethics Committee of West China Hospital of Stomatology (Approval No.: WCHSIRB-D-2024-319-R1). The CBCT data is stored in DICOM format, with each dataset containing hundreds of dcm files, each file containing one tomographic image, and the image size is 512×512 pixels.

[0073] More specifically, the MIP projection module projects 3D CBCT data to generate 2D MIP images. The dimensionality reduction of the CBCT data is performed based on the maximum density projection algorithm, with the following specific steps: Step 1021: Determine the projection direction. Using the midsagittal plane of the CBCT data as a reference, project along the positive and negative x-axis directions respectively. The positive x-axis direction corresponds to the patient's right side, and the negative x-axis direction corresponds to the patient's left side.

[0074] Step 1022: Perform MIP projection. For each projection direction, calculate the maximum HU value pixel by pixel along the y-axis (coronal axis) and encode this maximum value as a pixel grayscale value.

[0075] Step 1023: Generate bilateral MIP images. Starting from the midsagittal plane, project to the left and right sides respectively to generate two lateral MIP images: Left projection image: reflects the patient's left dentition. Right projection image: reflects the patient's right dentition.

[0076] Step 1024: Image Size Normalization. All generated MIP images are uniformly resized to 512×512 pixels. The adjustment method is as follows: first, scale the images proportionally to achieve a horizontal resolution of 512 pixels, then fill any insufficient vertical space with black pixels (grayscale value 0). Ensure all images are of consistent size for easier subsequent deep learning processing.

[0077] More specifically, the data annotation module annotates the dental arch contours and endpoints of MIP images. This module allows experienced dental digitization researchers to manually annotate MIP images using the dedicated annotation tool MyLabel. Annotations include: Step 1031: Annotate the curve separating the upper and lower dentition. On the MIP image, draw a continuous curve along the crown surface of the upper and lower dentition. During annotation, constrain each x-coordinate to record only one y-value to ensure the curve is in function form for easy subsequent processing. For unannotated x-coordinates, the system can automatically find the nearest annotated x-coordinate's corresponding y-value to fill in the gap, generating a continuous path covering all x-coordinates.

[0078] Step 1032: Marking the four endpoints. Mark four key endpoints on the MIP image to determine the anteroposterior extent of the dental arch and the position of the maxilla and mandible: Endpoint 1: The most anterior point of the maxillary dentition (usually the mesial contact point of the central incisor); Endpoint 2: The last endpoint of the maxillary dentition (usually the distal surface of the last molar); Endpoint 3: The most anterior point of the mandibular dentition; Endpoint 4: The last endpoint of the mandibular dentition; Step 1033: Save the annotation results. Save the annotated curve coordinate sequence and the coordinates of the four endpoints as a JSON file. The file name should be the original image file name with the "_label.json" suffix added.

[0079] More specifically, the data augmentation module performs random translation augmentation on the labeled images. This module augments the labeled MIP images to expand the training dataset and improve the model's generalization ability, specifically including: Step 1041: Random vertical translation. Generate 50 enhanced images for each original image. The translation amount Δy is a random integer ranging from [-20, 20] pixels. After the image is translated, the pixels that were originally translated out of the image area are discarded, and the newly generated blank areas are filled with black pixels.

[0080] Step 1042: Annotation Correction. The image translation Δy is synchronously applied to the corresponding annotation data: Curve Coordinates: The y-coordinates of all points are increased by Δy. Endpoint Coordinates: The y-coordinates of the four endpoints are increased by Δy.

[0081] Step 1043: Augmentation Dataset Generation. After augmentation, the original 180 training samples generate a total of 180 × 50 + 180 = 9180 training samples. These samples are used for training subsequent deep learning models.

[0082] More specifically, the deep learning training module trains the DenseNet121 network for curve and endpoint prediction. This module uses an improved DenseNet121 network for training, with the improved network structure as follows: Figure 2 As shown, it specifically includes the following: Step 1051: Network Structure Design. A dual-branch output structure is adopted, sharing the DenseNet121 backbone network. Two independent fully connected layers branch off at the last layer: Curve Prediction Branch: Output dimension 512, corresponding to the predicted y-coordinate of each column of the MIP image. Endpoint Prediction Branch: Output dimension 8, corresponding to the (x, y) coordinates of the four endpoints.

[0083] The DenseNet121 backbone network consists of four dense blocks, each containing multiple convolutional layers connected densely between them. The network parameters are configured as follows: Initial convolutional layer: 7×7 convolution, stride 2, output channels 64; Pooling layer: 3×3 max pooling, step size 2; Dense Block 1: Combination of 6 1×1 and 3×3 convolutional layers, with a growth rate of 32%; Transition layer 1: 1×1 convolution + 2×2 average pooling; Dense Block 2: 12 convolutional layers, growth rate 32; Transition layer 2: 1×1 convolution + 2×2 average pooling; Dense Block 3: 24 convolutional layers, growth rate 32%; Transition layer 3: 1×1 convolution + 2×2 average pooling; Dense Block 4: 16 convolutional layers, growth rate 32%; Classification layer: Global average pooling + fully connected layer; Step 1052: Training parameter settings. Use the following training parameters: batchSize: 10; epochs: 1300; initialLearnRatio: 0.01 (initial learning rate); decayCriticalValue: 1000 (starting epoch of learning rate decay); decayStepsize: 50 (learning rate decay step size); decayRatio: 0.5 (learning rate decay coefficient); batchPrintNum: 500 (print the training status once every 500 batches); Optimizer: Adam; Loss function: MSELoss (mean squared error loss); Step 1053: Model Training. In this embodiment, the model is trained for 6 days on a desktop computer equipped with an NVIDIA GeForce RTX 4060 Ti GPU until the loss function converges. During training, the model parameters are saved every 50 epochs, and the model with the smallest loss on the validation set is selected as the final model.

[0084] Step 1054: Curve post-processing optimization. Perform fixed-window smoothing optimization on the output of the curve prediction branch: Window length: 17 (i.e., considering 8 points before and after the current point); Outlier detection: If the predicted value of the current point exceeds ±1.2 standard deviations of the average value of all points in the neighborhood, it is identified as an outlier. Outlier correction: Replace outliers with the neighborhood average.

[0085] More specifically, the dentition segmentation module segments the dentition in 3D CBCT based on prediction results. This module accurately segments the dentition region in 3D CBCT data based on deep learning prediction results and dynamic reconstruction thresholds, specifically including: Step 1061: Limiting the Search Range. Using the four endpoints predicted by deep learning, the search range for the dental arch is determined: x-axis range: [min(endpoint 1_x, endpoint 3_x), max(endpoint 2_x, endpoint 4_x)]; y-axis range: The entire y-axis range; z-axis range: determined by the position of the tooth cusp; when searching upwards, only the first high-density voxel encountered is checked. Step 1062: Maximum Gradient Search along the Axis. This embodiment proposes a maximum gradient search method along the axis, extracting CBCT data layer by layer along the x-axis, and searching column by column along the y-axis for each layer: For each slice with a fixed x-axis, scan from top to bottom along the y-axis; Calculate the gradient of HU values ​​of adjacent voxels: Gradient(y) = |HU(y+1) - HU(y)|; The location of the maximum gradient corresponds to the boundary of the tooth crown surface; Record the y-coordinate of this position as the boundary point of the tooth row; Step 1063: Crown region extraction. Based on dental anatomy, the crown length is generally around 8mm. To eliminate gingival interference, only the voxel within 5mm below the cusp is preserved: Determine the cusp location: Search along the z-axis for the location of the first high-density voxel as the cusp; Extend downwards by 5mm, preserving all voxels within that range.

[0086] Step 1064: Fixed window smoothing optimization. Apply the same smoothing optimization as in step 1054 to the extracted tooth row boundaries to eliminate local outliers and ensure the continuity of the segmentation results.

[0087] Step 1065: 3D Point Cloud Generation. The segmented dental arch region voxels are converted into a 3D point cloud format, with each voxel corresponding to a point containing (x, y, z) coordinate information. The point cloud data is used for subsequent registration.

[0088] More specifically, the dynamic reconstruction module uses the dHU algorithm to dynamically determine the reconstruction threshold. This module proposes a dynamic HU adjustment algorithm (self-named dHU) that automatically determines the reconstruction threshold based on each patient's CBCT data, specifically including: Step 1071: HU value statistics. Read all voxel HU values ​​from the patient's CBCT data and construct a HU value set H. Statistically analyze the HU value distribution and calculate the cumulative frequency.

[0089] Step 1072: Threshold Calculation. The dynamic threshold is calculated using the following formula: dHU = min{h ∈ H | |{k :h_k ≤ h}|>0.98N} Where: N is the total number of voxels, |{k : h_k ≤ h}| represents the number of voxels with a HU value less than or equal to h, and min{…} represents the minimum HU value that meets the condition. This threshold corresponds to the 98th percentile of the HU value distribution, ensuring that bone density areas such as tooth crowns are included, while excluding low-density areas such as air and soft tissue.

[0090] Step 1073: Threshold Application. Using dHU as the reconstruction threshold, voxels in the CBCT data with HU values ​​greater than or equal to dHU are marked as tooth regions for subsequent 3D reconstruction.

[0091] More specifically, the IOS data acquisition module acquires patient IOS model data in STL format. This module acquires the IOS model data corresponding to the CBCT scan, in STL (stereolithography) file format. Inclusion criteria are: images taken on the same day as the CBCT scan, containing the complete maxillary dentition. Data already used for model training is excluded.

[0092] More specifically, the IOS preprocessing module removes soft tissue from the gums and determines the spatial coordinate system. This module preprocesses the IOS model, removing soft tissue such as the gums and determining the spatial coordinate system, and includes the following sub-steps: Step 1091: Dental Arch Segmentation. The anterior and posterior dental arch extents are determined using four endpoints predicted by deep learning, and gingival tissue is removed. Search for the cusp position (maximum or minimum z coordinate, depending on the scanning direction) along the z-axis (tooth length axis direction); Extract all points within a 7mm range from the cusp of the tooth toward the gum line; this range corresponds to the crown area. Discard any gum and soft tissue points that extend beyond this area.

[0093] Step 1092: Determining the Spatial Coordinate System. The spatial coordinate system of the IOS model is determined using principal component analysis combined with the minimum bounding rectangle method. PCA is used to obtain three principal axes. The principal axis with the smallest eigenvalue corresponds to the shortest direction, which is used as the z-axis (vertical direction). The positive z-axis direction (pointing towards the tooth tip) is determined based on the direction of the normal vector. A slice of 4 pixels is taken along the z-axis from the cusp of the tooth towards the gingiva. This slice mainly includes the lower part of the crown and has little impact on the gingiva. Calculate the minimum bounding rectangle of the point cloud of the slice, where the longer side of the rectangle corresponds to the width direction of the dental arch and the shorter side corresponds to the thickness direction of the dental arch. The direction is determined based on the shape of the dental arch: the front part of the dental arch is narrower and the back part is wider. Based on this, the positive directions of the x-axis (mesiodistal direction) and y-axis (buctolingual direction) are determined.

[0094] Step 1093: Center point calculation. Calculate the geometric center of the segmented tooth row point cloud, which will serve as the origin reference for subsequent coarse registration.

[0095] More specifically, the coarse registration module aligns the coordinate axes of the CBCT and IOS models. This module aligns the coordinate axes of the CBCT and IOS models, specifically including: Step 1101: Determine the reference coordinate system. Use the spatial coordinate system of the CBCT data as the reference coordinate system. The CBCT data itself contains absolute spatial coordinate information (from the Image Position Patient label in the DICOM file), which can be used directly.

[0096] Step 1102: IOS Model Coordinate Transformation. Transform the IOS model from its own coordinate system to the CBCT coordinate system: Translation transformation: Translate the center point of the IOS model to the center point of the CBCT model. Rotation transformation: Rotate the z-axis of the IOS model until it is parallel to the z-axis of the CBCT, and then rotate the x and y axes until they are parallel to the x and y axes of the CBCT. Step 1103: Transformation matrix calculation. Record the rotation matrix R_rough and translation vector T_rough of the above transformation, which will be used as the initial position for subsequent fine registration.

[0097] More specifically, the ICP registration module achieves fine registration of a high-neighbor curvature variance sampling point set. This module uses the iterative nearest-neighbor algorithm to perform initial fine registration of the high-neighbor curvature variance sampling point set, specifically including: Step 1121: Input data: Reference point cloud: The set of sampling points with high neighbor curvature variance in the CBCT model; Point cloud to be registered: a set of sampling points with high neighbor curvature variance of the IOS model; Initial transformation matrix: (R_rough, T_rough) from the coarse registration module; Step 1122: ICP Algorithm Iteration. The ICP algorithm finds the optimal rigid body transformation through iterative optimization, minimizing the error between two point sets: Corresponding point matching: For each point in the point cloud to be registered, find the nearest point in the reference point cloud as the corresponding point; Transformation estimation: Based on all corresponding point pairs, calculate the rigid body transformation that minimizes the mean square error; Transformation application: Apply the calculated transformation to the point cloud to be registered; Iteration Termination: The iteration terminates when the amount of change is less than the threshold or when the maximum number of iterations is reached. Step 1123: Transformation Matrix Recording. Record the transformation matrix T_icp = (R_icp, T_icp) obtained from ICP registration, which will be used as the initial position for the next stage of registration.

[0098] More specifically, the CVN sampling module performs neighborhood curvature variance sampling. This module uses the neighborhood curvature variance (CVN) method to sample the point cloud to reduce metallic artifact interference. Step 1111: Curvature Calculation. For each point p in the point cloud... i Calculate the average principal curvature of each point in its neighborhood. The neighborhood size N is set to 20 nearest neighbors.

[0099] Step 1112, CVN value calculation. The CVN value for each point is calculated using the following formula: ; in: p i Points in a point cloud; N is the number of neighboring points; C j For p i The average principal curvature of the j-th neighboring point; avg(C j ) represents the mean of the principal curvatures of all neighboring points; The CVN value reflects the consistency of curvature in a local region: Planar or spherical regions: uniform curvature, CVN value close to 0; Feature regions such as edge ridges and shear ridges: moderate curvature variation and moderate CVN value; Metal artifact region: curvature abrupt change, extremely high CVN value.

[0100] Step 1113: Dual-threshold sampling. Set two CVN thresholds and perform two sampling operations: High Neighbor Curvature Variance Sampling Point Set: Using a CVN threshold of 0.0001, points with CVN values ​​greater than or equal to 0.0001 are extracted. These points correspond to non-smooth regions on the crown surface (marginal ridges, incisal ridges, etc.) and are key feature points for registration.

[0101] Low-neighbor curvature variance sampling point set: Using a CVN threshold of 0.00001, points with CVN values ​​less than 0.00001 are extracted. These points correspond to smooth regions, excluding metallic artifact points with extremely high CVN values.

[0102] Sampling consistency is guaranteed. The same double-threshold sampling is performed on both the CBCT point cloud and the IOS point cloud to ensure that the sampled point sets of the two point clouds have a corresponding relationship, which facilitates subsequent registration.

[0103] More specifically, the GICP registration module achieves fine registration of the point set combining high and low neighbor curvature variance samples. This module uses the generalized iterative nearest-neighbor algorithm to perform secondary fine registration on the point set combining high and low neighbor curvature variance samples, specifically including: Step 1131: Point set merging. Merge the set of sampling points with high neighbor curvature variance and the set of sampling points with low neighbor curvature variance to form a complete registration point cloud: Complete CBCT point cloud = CBCT high-neighbor curvature variance sampling point set ∪ CBCT low-neighbor curvature variance sampling point set; IOS complete point cloud = IOS high neighbor curvature variance sampling point set ∪ IOS low neighbor curvature variance sampling point set; Step 1132: GICP Algorithm Iteration. The GICP algorithm introduces a probabilistic model based on ICP, considering the local planar characteristics of the point cloud: Calculate the covariance matrix for each point to describe the local surface orientation; Covariance information is considered when matching corresponding points to improve matching accuracy; A probabilistic model is used during transformation estimation to enhance robustness to noise.

[0104] Step 1133: Transformation matrix calculation. Using the ICP registration result as the initial position, perform GICP iterative optimization to obtain the final transformation matrix T_final = (R_final, T_final).

[0105] More specifically, the transformation application module applies the transformation matrix to the original IOS model. This module applies the transformation matrix obtained from fine registration to the original IOS model, specifically including: Step 1141: Transformation Combination. Calculate the complete transformation matrix from the original IOS model to the CBCT model: T_total = T_final T_icp T_rough; where Represents the composite operation of transformation.

[0106] Step 1142: Model Transformation. Apply the transformation matrix T_total to each vertex v of the original IOS model: v' = R_total·v + T_total. This yields the transformed IOS model point cloud.

[0107] More specifically, the results output module outputs the registered fusion model. This module outputs the final registration result: Registered IOS model (transformed to CBCT coordinate system); Registration transformation matrix T_total; Registration accuracy assessment report (including registration time and registration error).

[0108] In summary, the solutions of the embodiments of the present invention can solve the following problems: First, it reduces the reliance on large-scale, accurately labeled 3D data, solves the problem that existing deep learning methods are difficult to train and clinically translate due to the lack of public datasets, simplifies the data labeling process, and lowers the threshold for technology application.

[0109] Second, it effectively resists the interference of metal artifacts, solves the problem that the accuracy of existing registration methods drops significantly when metal restorations are present, and improves the robustness of the algorithm in real clinical scenarios.

[0110] Third, improve the adaptability of the registration algorithm to personalized anatomical structures and solve the problems of existing methods being sensitive to clinical variations such as dental arch morphology and missing teeth in different patients and having unstable registration performance.

[0111] Fourth, it enables automatic registration that combines high precision and high efficiency, solving the problem that existing methods cannot simultaneously achieve both registration accuracy and processing speed. This allows it to replace the commonly used manual registration methods in clinical practice, shortening preoperative planning time and meeting real-time clinical needs.

[0112] The technical solution of the embodiments of the present invention has the following effects: 1. Significantly improves registration efficiency and shortens preoperative planning time. Specifically, the embodiment of this invention achieves fully automated processing through MIP dimensionality reduction, deep learning-based automatic segmentation, and two-stage automatic registration. Experimental results show that the dimensionality reduction adaptive registration method of this invention has an average registration time of 59±25 seconds, significantly better than the 148±20 seconds of commonly used manual registration methods (p<0.05), reducing the registration time to about one-third of the manual method. This efficiency improvement allows preoperative planning to be completed within one minute, greatly reducing the operation time for doctors and technicians and improving clinical work efficiency.

[0113] 2. Maintaining high-precision registration to meet clinical application standards. Specifically, while ensuring high efficiency, the method of this invention achieves registration accuracy comparable to advanced commercial platforms. Experimental data shows that the registration accuracy of this method is 0.218±0.064mm, which is not statistically different from the 0.240±0.061mm of the manual registration group (p>0.05). The registration results all meet the clinically recognized accuracy standard of <0.5mm, indicating that the method of this invention can be reliably applied to clinical scenarios such as dental implant surgery planning and guide design.

[0114] 3. Reduces data dependency and annotation costs, facilitating clinical translation. Specifically, the embodiment of this invention uses the Maximum Intensity Projection (MIP) algorithm to reduce the dimensionality of 3D CBCT data to 2D, simplifying the complex 3D segmentation task into 2D image annotation. The beneficial effects of this technical feature include: Reduced annotation workload: 2D annotation saves about 50% of labor costs compared to 3D annotation, eliminating the need for time-consuming 3D point cloud registration and annotation by professional technicians.

[0115] Reduced training data requirements: By using data augmentation (generating 50 augmented images for each original image), high-precision deep learning models can be trained with only a small amount of original data, solving the current problem of lacking large-scale public datasets.

[0116] Lower technical barriers: Clinical institutions can quickly adapt models using the limited data they have accumulated, which facilitates the promotion and application of the technology.

[0117] 4. Effectively combats metal artifact interference and improves clinical robustness. Specifically, the embodiment of this invention uses neighborhood curvature variance (CVN) sampling technology to effectively identify and filter metal artifacts in CBCT data: High CVN threshold (0.0001) sampling extracts feature regions such as marginal ridges and incisal ridges on the crown surface, providing stable feature points for registration; A low CVN threshold (0.00001) is used to sample and exclude outliers in the metal artifact region, thus avoiding incorrect correspondences caused by artifacts.

[0118] Experimental results show that even with metal prostheses in patients, the registration accuracy of the proposed solution remains around 0.22 mm with a standard deviation of only 0.064 mm, indicating good stability of the algorithm against artifacts. In contrast, existing commercial platforms exhibit significantly increased registration errors (standard deviation as high as 0.442 mm in the Yake Smart Group) when metal artifacts are present, and registration failures occur (2 out of 32 cases failed).

[0119] 5. Adaptive to personalized anatomical structures, improving the algorithm's generalization ability. Specifically, the embodiments of this invention enhance adaptability to different patient anatomical variations through several technical features: Dynamic HU adjustment algorithm (dHU): Automatically determines the reconstruction threshold based on each patient's CBCT data (taking the minimum HU value greater than 98% voxels), solving the problem of inaccurate reconstruction with fixed threshold under different equipment and scanning parameters.

[0120] An improved dentition segmentation method combines deep learning-predicted endpoint ranges with axial maximum gradient search to accurately segment the maxillary and mandibular dentition, effectively extracting the crown region even in complex cases such as missing teeth or malocclusion.

[0121] Fixed window smoothing optimization: Post-processing of deep learning prediction curves corrects outliers, improving the continuity and accuracy of segmentation results.

[0122] Clinical trials showed that the registration results of the proposed method remained stable in 32 different patients, with the median deviation being comparable to that of manual registration, and it performed well in both the anterior and posterior tooth regions.

[0123] 6. Achieve full-process automation and reduce human error. Specifically, the solution in this embodiment of the invention requires no manual intervention from CBCT data import, dental segmentation, coarse registration to fine registration, eliminating errors caused by differences in operator experience during manual registration. Compared with semi-automatic methods, the solution in this embodiment of the invention does not require manual selection of initial points or adjustment of parameters, avoiding the problem of ICP getting trapped in local optima due to poor initial positions. Compared with existing deep learning methods, the solution in this embodiment of the invention reduces computational complexity through dimensionality reduction and feature sampling, and can run quickly on ordinary desktop computers (NVIDIA GeForce RTX 4060 Ti), exhibiting good hardware compatibility.

[0124] 7. Improved registration stability and consistency. Specifically, the embodiment of the present invention employs a two-stage fine registration strategy (ICP registration of the high-neighbor curvature variance sampling point set + GICP registration of the high-low-neighbor curvature variance sampling point set), which balances local feature preservation and overall structural alignment. Experimental results show that the standard deviation of the registration bias of this method is 0.064 mm, comparable to 0.061 mm in the manual registration group, indicating that the registration results of the embodiment of the present invention have high consistency and repeatability.

[0125] 8. Excellent clinical application potential. Specifically, the dimension-reduced adaptive automatic registration system proposed in this embodiment of the invention achieves minute-level registration efficiency (average 59 seconds) while maintaining sub-millimeter-level accuracy (0.218 mm). Furthermore, it requires no professional technicians and can be directly integrated into digital implantation workflows. This system can serve as an effective alternative to manual registration, providing efficient and accurate data fusion support for preoperative planning, surgical guide design, and postoperative evaluation, demonstrating significant clinical application value.

[0126] The units described in the embodiments of the present invention can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0127] According to one aspect of the present invention, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described above.

[0128] In another aspect, embodiments of the present invention also provide a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

Claims

1. A dimension reduction adaptive multimodal image registration method, characterized in that, Including the following steps: Three-dimensional CBCT data is used to generate two-dimensional MIP images. The dental arch regions in the two-dimensional MIP images are labeled to obtain curves that separate the upper and lower dental arches. The labeled data is used to train a deep learning network to establish a mapping model from the two-dimensional MIP images to the curves that separate the upper and lower teeth and the endpoints of the dental arches. The two-dimensional MIP image generated from the CBCT data to be registered is input into the trained deep learning network to predict the curves separating the upper and lower dentitions and the endpoint positions of the dentitions. Based on the curve and endpoint positions, the dentition region is segmented in the 3D CBCT data to obtain the CBCT dentition point cloud model; and the spatial coordinate system is determined and the dentition region is segmented in the IOS model, and the gingiva and soft tissue are removed to obtain the IOS dentition point cloud model. The obtained CBCT dental arch point cloud model and the IOS dental arch point cloud model are aligned in spatial coordinates to obtain preliminary alignment registration results. For the pre-aligned CBCT dentition point cloud model and IOS dentition point cloud model, the curvature variance of the neighboring region is sampled to obtain the high neighbor curvature variance sampling point set and the low neighbor curvature variance sampling point set. Then, the high neighbor curvature variance sampling point set is registered, and then the point set combining the high neighbor curvature variance sampling point set and the low neighbor curvature variance sampling point set is registered to obtain the final transformation matrix. The transformation matrix is ​​applied to the original IOS model to complete the final registration.

2. The dimension reduction adaptive multimodal image registration method according to claim 1, characterized in that, The process of generating two-dimensional MIP images from three-dimensional CBCT data specifically includes the following sub-steps: encoding the maximum HU value of each pixel along the y-axis using the maximum density projection algorithm, and projecting it from the midsagittal plane along the positive and negative x-axis directions to generate two side-view MIP images, one on the left and one on the right. The generated MIP images are uniformly adjusted to a fixed pixel size, scaled proportionally horizontally, and filled with selected color pixels vertically.

3. The dimension reduction adaptive multimodal image registration method according to claim 1, characterized in that, The deep learning network adopts an improved DenseNet121 network structure, specifically including two independent branch networks, which are used to predict the dental arch contour curve and endpoint position, respectively.

4. The dimension reduction adaptive multimodal image registration method according to claim 3, characterized in that, The training of the deep learning network specifically includes the following sub-steps: Step 1051: Design an improved DenseNet121 network structure: adopt a dual-branch output structure, share the DenseNet121 backbone network, and branch out two independent fully connected layers in the last layer: a curve prediction branch and an endpoint prediction branch; the output of the curve prediction branch corresponds to the predicted y-coordinate value of each column of the MIP image; the output of the endpoint prediction branch corresponds to the (x,y) coordinates of the four endpoints; the DenseNet121 backbone network contains multiple dense blocks, each dense block contains multiple convolutional layers, and dense connections are used between layers; configure the network parameters. Step 1052, set training parameters; Step 1053: Train the model until the loss function converges; During training, the model parameters are saved once every set number of rounds, and the model with the smallest loss on the validation set is finally selected as the final model. Step 1054, Curve post-processing optimization, performs fixed-window smoothing optimization on the output of the curve prediction branch.

5. The dimension reduction adaptive multimodal image registration method according to claim 4, characterized in that, The training parameters include: Batch size: batchSize=10; Training epochs = 1300; The initial learning rate is set to initialLearnRatio = 0.

01. The Adam optimizer is used, with MSELoss as the loss function; The fixed window smoothing optimization specifically fixes the window length to 17, marks points that exceed the neighborhood average by ±1.2 standard deviations as outliers, and replaces the outliers with the neighborhood average.

6. The dimension reduction adaptive multimodal image registration method according to claim 1, characterized in that, Before segmenting the dentition region from 3D CBCT data, the following step is included: determining the reconstruction threshold based on the CBCT data: Step 1071: Read all voxel HU values ​​of CBCT data, construct HU value set H, statistically analyze the HU value distribution, and calculate the cumulative frequency; Step 1072: Calculate the dynamic threshold using the following formula: dHU = min{h ∈ H | |{k : h_k ≤ h}|>0.98N}; in: N is the total number of voxels; |{k : h_k ≤ h}| represents the number of voxels whose HU value is less than or equal to h; min{…} represents the minimum HU value that satisfies the condition; This dynamic threshold corresponds to the 98th percentile of the HU value distribution, ensuring that areas with bone density, such as the crown, are included, while excluding low-density areas, including air and soft tissue. Step 1073: Using dHU as the reconstruction threshold, voxels with HU values ​​greater than or equal to dHU in the CBCT data are marked as tooth regions for subsequent three-dimensional reconstruction.

7. The dimension reduction adaptive multimodal image registration method according to claim 6, characterized in that, The step of segmenting the dentition region from 3D CBCT data to obtain a CBCT dentition point cloud model specifically includes the following sub-steps: Based on deep learning prediction results and dynamic reconstruction thresholds, precise segmentation of dental arch regions is achieved in 3D CBCT data, specifically including: Step 1061: Limiting the search range, using the four endpoints predicted by deep learning to determine the search range for the dental arch: x-axis range: [min(endpoint 1_x, endpoint 3_x), max(endpoint 2_x, endpoint 4_x)]; y-axis range: The entire y-axis range; z-axis range: determined by the position of the tooth cusp; when searching upwards, only the first high-density voxel encountered is checked. Step 1062: Maximum gradient search along the axis. A maximum gradient search method along the axis is proposed, specifically extracting CBCT data layer by layer along the x-axis, and searching column by column along the y-axis for each layer: For each slice with a fixed x-axis, scan from top to bottom along the y-axis; Calculate the gradient of HU values ​​of adjacent voxels: Gradient(y) = |HU(y+1) - HU(y)|; The location of the maximum gradient corresponds to the boundary of the tooth crown surface; Record the y-coordinate of this position as the boundary point of the tooth row; Step 1063: Extract the crown region. Search along the z-axis for the position of the first high-density voxel as the cusp, extend downwards within a set range, and retain all voxels within that range. Step 1064: Perform fixed-window smoothing optimization on the extracted tooth row boundaries to eliminate local outliers; Step 1065: Convert the segmented dental arch region voxels into a 3D point cloud format. Each voxel corresponds to a point and contains (x, y, z) coordinate information. The point cloud data is used for subsequent registration.

8. The dimension reduction adaptive multimodal image registration method according to claim 1, characterized in that, The spatial coordinate system uses Principal Component Analysis (PCA) to determine the axial direction and the minimum bounding rectangle to determine the horizontal direction. Specifically, PCA is used to obtain three principal axes, with the principal axis with the smallest eigenvalue corresponding to the shortest direction, which is taken as the z-axis. A slice of a set number of pixels is taken along the z-axis from the cusp to the gingiva, and the minimum bounding rectangle of the point cloud of the slice is calculated to determine the x-axis and y-axis directions.

9. The dimension reduction adaptive multimodal image registration method according to claim 1, characterized in that, The IOS model is segmented into dental arch regions, which includes the following sub-steps: using the endpoints predicted by the deep learning network to determine the range of the anterior and posterior dental arches, extracting points within the range of the maximum / minimum z-coordinate vertices along the z-axis, and removing gingival tissue.

10. The dimension reduction adaptive multimodal image registration method according to claim 1, characterized in that, The step of sampling the curvature variance of neighboring regions to obtain a set of sampling points with high neighbor curvature variance and a set of sampling points with low neighbor curvature variance specifically includes the following sub-steps: Step 1111, for each point p in the point cloud i Calculate the average principal curvature of each point in its neighborhood; Step 1112, calculate the CVN value for each point using the following formula: ； in: p i Points in a point cloud; N is the number of neighboring points; C j For p i The average principal curvature of the j-th neighboring point; avg(C j ) represents the mean of the principal curvatures of all neighboring points; Step 1113: Perform the same double-threshold sampling on both the CBCT point cloud and the IOS point cloud to ensure that the sampled point sets of the two point clouds have a corresponding relationship; the double-threshold sampling specifically includes: Set two CVN thresholds and perform two sampling operations: High proximity curvature variance sampling point set: Using the first CVN threshold, points with CVN values ​​greater than or equal to the first CVN threshold are extracted. These points correspond to the non-smooth areas on the crown surface and are used as key feature points for registration. Low Neighbor Curvature Variance Sampling Point Set: Using the second CVN threshold, points with CVN values ​​less than the second CVN threshold are extracted. These points correspond to smooth regions. The characteristic of CVN values ​​reflecting the curvature of local regions is used to exclude metal artifact points.

11. The dimension reduction adaptive multimodal image registration method according to claim 8, characterized in that, The step of aligning the obtained CBCT dental point cloud model with the IOS dental point cloud model in terms of spatial coordinate axes specifically includes the following sub-steps: using the spatial coordinate system of the CBCT data as a reference, the spatial coordinate system of the IOS model is aligned with it through translation and rotation.

12. The dimension reduction adaptive multimodal image registration method according to claim 1, characterized in that, The registration of the high-neighbor curvature variance sampling point set specifically employs the iterative nearest-neighbor algorithm.

13. The dimension reduction adaptive multimodal image registration method according to claim 1, characterized in that, The registration of the point set combining high-neighbor curvature variance sampling and low-neighbor curvature variance sampling specifically employs the generalized iterative nearest-point algorithm.

14. The dimension reduction adaptive multimodal image registration method according to claim 1, characterized in that, Obtaining the final transformation matrix specifically includes the following sub-steps: fixing the CBCT model position as reference data, moving and rotating the IOS model; recording the transformation matrix of the two registrations, and finally applying it to the original IOS model.

15. A dimension-reduction adaptive multimodal image registration device, characterized in that, The device includes a MIP projection module, a data annotation module, a data augmentation module, a deep learning training module, a tooth row segmentation module, a dynamic reconstruction module, an IOS preprocessing module, a coarse registration module, a CVN sampling module, an ICP registration module, a GICP registration module, a transformation application module, and a result output module; the device is used to run the dimension reduction adaptive multimodal image registration method according to any one of claims 1 to 14, and: The MIP projection module is used to project 3D CBCT data to generate 2D MIP images. The data annotation module is used to annotate the dental arch contours and endpoints of MIP images; The data augmentation module is used to perform random translation augmentation on labeled images; The deep learning training module is used to train deep learning networks to predict curves and endpoints. The dentition segmentation module is used to segment dentition in 3D CBCT based on prediction results; The dynamic reconstruction module is used to dynamically determine the reconstruction threshold; The IOS preprocessing module is used to remove soft tissue from the gums and determine the spatial coordinate system. The coarse registration module is used to align the coordinate axes of the CBCT and IOS models. The CVN sampling module is used to sample the curvature variance of the neighborhood region. The ICP registration module is used to achieve fine registration with a set of sampling points with high neighbor curvature variance; The GICP registration module is used to achieve fine registration with the high and low neighbor curvature variance sampling point set; The Transform Application module is used to apply the transformation matrix to the original IOS model.

16. The dimension reduction adaptive multimodal image registration device according to claim 6, characterized in that, Also includes: The CBCT data acquisition module is used to legally acquire raw CBCT data in DICOM format.

17. The dimension reduction adaptive multimodal image registration device according to claim 6, characterized in that, Also includes: The iOS data acquisition module is used to legally acquire iOS model data in STL format.

18. The dimension reduction adaptive multimodal image registration device according to claim 6, characterized in that, Also includes: The results output module is used to output the registered fusion model.

19. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that, when loaded by the processor, executes the method as described in any one of claims 1 to 14.

20. A computer-readable storage medium, characterized in that, A computer program is stored in a readable storage medium, which, when loaded by a processor, executes the method as described in any one of claims 1 to 14.

Images