An IOS image and CBCT image registration method based on particle swarm and single tooth optimization
By employing particle swarm optimization and single-tooth optimization methods, combined with deep learning networks and semantic segmentation technology, the problems of low registration accuracy and poor robustness caused by resolution differences and pose inconsistencies between IOS and CBCT models were solved, achieving high-precision three-dimensional tooth registration and supporting clinical applications such as orthodontics and implantology.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GANYUE MEDICAL TECH (CHENGDU) CO LTD
- Filing Date
- 2026-01-19
- Publication Date
- 2026-06-09
AI Technical Summary
In existing technologies, IOS and CBCT models suffer from low registration accuracy and poor robustness due to differences in resolution and pose inconsistencies.
A method based on particle swarm optimization and single-tooth optimization is adopted. The particle swarm optimization algorithm is used to search and optimize the six-degree-of-freedom rigid transformation parameters. Combined with a two-stage deep learning network framework and semantic segmentation technology, global initial registration and local fine alignment are achieved. The iterative nearest point algorithm is used to minimize the local distance error and generate a high-precision three-dimensional tooth registration model.
It improves the fusion accuracy and stability of CBCT and IOS images, solves the problem of pose inconsistency between different modal data, realizes high-precision registration of multimodal 3D oral models, and supports automated registration in clinical scenarios such as orthodontics and implantology.
Smart Images
Figure CN121544676B_ABST
Abstract
Description
Technical Field
[0001] This invention pertains to image data processing technology, and more specifically, relates to a method for registering IOS images and CBCT images based on particle swarm optimization and single-tooth optimization. Background Technology
[0002] With the rapid development of digital dental technology, 3D models are increasingly widely used in clinical diagnosis and treatment, such as tooth reconstruction, orthodontic analysis, and implant design. Intraoral optical scanning (IOS), with its high resolution and radiation-free operation, can accurately acquire 3D structural information of the tooth surface (such as crown morphology and occlusal relationships), while cone-beam computed tomography (CBCT) images provide complete internal anatomical information of the teeth, roots, and bone tissue, and are widely used in implant design, orthodontic planning, and periapical diagnosis. Integrating these two key information sources and achieving high-precision alignment through registration techniques to construct a complete and accurate 3D digital model of the teeth has become an important direction for improving the accuracy and reliability of digital dental diagnosis and treatment. CBCT and IOS belong to different modalities; the former focuses on voxel intensity and internal bone structure, while the latter focuses on surface mesh representation. This presents multiple challenges, including inconsistencies in features, resolution differences, data noise, and patient posture, making accurate registration between the two difficult. Summary of the Invention
[0003] To address the issues of low registration accuracy and poor robustness between IOS and CBCT models due to resolution differences and pose inconsistencies in existing technologies, this invention proposes a registration method for IOS and CBCT images based on particle swarm optimization and single-tooth optimization, comprising the following steps:
[0004] S1. First, acquire the patient's IOS image data and CBCT image data. For the CBCT image data, read the DICOM voxel information and perform threshold segmentation to extract the hard tissue structure of the teeth and perform semantic segmentation annotation on the segmented 3D model. For the IOS image data, perform fine semantic annotation on different tooth regions to obtain segmentation labels. Subsequently, perform normalization and decentralization preprocessing operations on the IOS image data and CBCT image data to finally construct a CBCT segmentation dataset and an IOS segmentation dataset containing semantic information.
[0005] S2, Based on the convolutional neural network architecture of nnU-Net, a CBCT tooth segmentation model is constructed and trained. The CBCT tooth segmentation model takes CBCT image data as input and outputs the three-dimensional voxel semantic segmentation result of the tooth.
[0006] S3. Based on a two-stage deep learning network framework, an IOS tooth segmentation model is constructed and trained. The two-stage deep learning network framework includes a tooth centroid detection module for localization and a semantic segmentation module for fine tooth segmentation. The IOS tooth segmentation model takes IOS image data as input and outputs the semantic segmentation result of the tooth surface mesh.
[0007] S4. Using the trained CBCT tooth segmentation model and IOS tooth segmentation model, inference is performed on the CBCT image data and IOS image data of the target patient respectively to obtain CBCT tooth semantic segmentation images and IOS tooth semantic segmentation images, and their respective geometric information is extracted to generate CBCT tooth point clouds and IOS tooth point clouds.
[0008] S5. Initialize the particle swarm and establish a spatial index structure for the CBCT tooth point cloud; apply the transformation parameters represented by the particles to the IOS tooth point cloud, and calculate the fitness value between the IOS tooth point cloud and the CBCT tooth point cloud through the spatial index structure; update the individual optimal position and global optimal position of the particle swarm based on the fitness value, and update the particle state according to the individual optimal position and global optimal position; perform constraint processing on the updated particle state and iterate until the preset termination condition is met; determine the optimal transformation parameters based on the global optimal position and construct the transformation matrix to complete the initial registration of the IOS tooth point cloud and the CBCT tooth point cloud.
[0009] S6. Based on the global initial registration result obtained in S5, the semantic label information of the CBCT tooth semantic segmentation image and IOS tooth semantic segmentation image obtained in S4 is used to divide the whole dental arch point cloud into multiple independent local units based on a single tooth, and the IOS tooth local point cloud and CBCT tooth local point cloud corresponding to each tooth are extracted.
[0010] S7. Each IOS tooth local point cloud and CBCT tooth local point cloud unit is independently aligned using single-tooth iteration. By minimizing the distance error of the local correspondence, the small misalignment of a single tooth is eliminated and the position of the overall model is updated, finally obtaining a high-precision three-dimensional tooth registration model.
[0011] Furthermore, the CBCT tooth segmentation model described in step S2 is constructed based on the nnU-Net adaptive configuration framework, and the specific steps of its one-time training iteration include:
[0012] S21, The nnU-Net framework automatically configures and generates an adapted 3D U-Net network topology based on the voxel resolution, data scale, and number of categories of the input CBCT image data, serving as the basic architecture of the model.
[0013] S22, the model adopts the standard U-Net encoder-decoder structure, and its convolutional layer introduces the nnU-Net standardized configuration, including instance normalization and Leaky-ReLU activation function; the encoder extracts multi-scale semantic information through 3D convolution and progressive downsampling; the decoder restores the resolution through upsampling, and fuses shallow structural details and deep semantic features through feature skip connections between corresponding layers to improve segmentation accuracy;
[0014] S23: A batch of data-enhanced CBCT image data is read from the CBCT segmentation dataset in S1 as input and forward propagation is performed; the network output predicts the probability of each voxel belonging to a specific category through Softmax; at the same time, the network outputs the prediction results at multiple downsampling levels of the decoder to achieve deep supervision.
[0015] S24, a hybrid loss function, combines weighted cross-entropy loss. and Dice loss During training, the aforementioned hybrid loss is calculated for each output layer of the deep supervision; ultimately, the total loss of the model is calculated by combining the loss values of each layer's segmentation head with... The weighted sum is obtained by weighting the values.
[0016] Furthermore, the specific steps for one iteration of training the IOS tooth segmentation model described in step S3 include:
[0017] In S31, the IOS segmentation dataset from S1 is input into the tooth centroid detection module to predict the offset vector and distance confidence of each facet relative to the tooth center. This module adopts a PointNet++-based architecture, extracts facet features through multi-scale sampling, and outputs the offset vector and distance confidence of each facet. By moving the facets and filtering high-confidence candidate faces, the DBSCAN clustering algorithm is applied to extract the centroid of each cluster as the tooth center point.
[0018] S32, for each center point, a local region containing a fixed number of facets is cropped using the K-nearest neighbor algorithm; the distance distribution function from each facet within the local region to all tooth center points is calculated. The enhanced feature representation is then concatenated with the original coordinate features and input into the semantic segmentation module. The distance distribution function is calculated as follows:
[0019] ,
[0020] in, These are the coordinates of points on a local area; These are the coordinates of the j-th tooth center point in the set of center points; Indicates Euclidean distance;
[0021] In step S33, the IOS segmentation dataset from S1 is input into the tooth semantic segmentation module. The enhancement features of each point are separated into coordinate features and geometric enhancement features, which are then input into the coordinate feature stream and geometric enhancement feature stream, respectively. At the feature input front end, a spatial transformation network is used to learn the affine transformation matrices of the coordinate stream and the normal vector stream, respectively, to perform spatial alignment correction on the original input to eliminate the influence of pose rotation. Subsequently, learnable parameters are introduced into each layer of the encoder. The adaptive KNN strategy extracts semantic features of the current layer through a semantic encoder and dynamically calculates and fuses the similarity matrix by combining Euclidean spatial distance and semantic spatial cosine similarity.
[0022] This captures the geometric edges and semantically consistent neighborhoods, where Represents learnable parameters , Indicates Euclidean spatial distance. The semantic space cosine similarity is represented. Based on this neighborhood, the coordinate feature stream extracts local fine features through the grouped vector attention module, and the geometric enhancement feature stream aggregates neighborhood information through graph convolution and performs max pooling. Finally, the cross attention module dynamically weights and fuses the two streams by calculating the saliency weights of the features to generate cross attention features with multimodal information.
[0023] S34 first generates multi-scale cross-attention features at each stage of the cascaded encoder, and then calculates the global contribution weights of coordinate flow, geometric flow, and cross-flow. The system achieves adaptive weighted integration of multi-source features, and then performs nonlinear enhancement on the fused feature stream through a feature alignment module. At the network output, a 1×1 convolutional layer is used to transform the dimension and extract the classification score. The softmax function is used to transform it into the probability distribution of each facet belonging to a specific tooth category. Finally, the weighted negative log-likelihood loss function is used to calculate the loss value between the predicted class probability and the true segmentation label.
[0024] Furthermore, step S4 also includes the following steps:
[0025] S41, the tooth segmentation contour voxel information output by the CBCT tooth segmentation model is converted into point cloud data to obtain CBCT tooth point cloud. Specifically, the method is as follows: Read the DICOM voxel data corresponding to the CBCT 3D tooth segmentation result, obtain the 3D voxel array (Volume) and its spatial resolution; perform threshold binarization on the voxel values, where the threshold T is the voxel intensity determination threshold. When not explicitly set, if it is CT data, the default tooth threshold of 1500 HU is automatically selected; if it is a binary mask, a threshold of 0.5 is used. The specific conversion rules are as follows:
[0026]
[0027] In the formula, These are the binarized voxel values. The original voxel values are used; isosurfaces are extracted based on the marchingcubes algorithm to generate a set of triangular facets (Faces) and a set of vertices (Verts), where the vertices are the 3D coordinates of tooth surface elements; the vertex coordinates are used as point cloud data to construct a CBCT tooth point cloud set. ;
[0028] S42, convert the iOS tooth semantic segmentation results output by the iOS tooth segmentation model into point cloud data. Specifically, extract the vertices of the tooth surface mesh from the semantic segmentation results as point cloud data to form an iOS tooth point cloud set. .
[0029] Furthermore, step S5 specifically includes:
[0030] S51, Initialize Particle Swarm: Set the particle swarm size to... Each particle represents a set of six-DOF rigid transformation parameters, including three rotation Euler angles and three translation components. The position vector of each particle is randomly initialized. and velocity vector ,in For particle indexing, the initial range of the rotation angle is set to... The initial range of the translation is , where D is the Euclidean distance between the centroids of the IOS point cloud and the CBCT point cloud;
[0031] S52 employs spatial indexing technology to pre-construct a KDTree of the CBCT tooth point cloud to accelerate the search; the transformation parameters represented by each particle are applied to the IOS tooth point cloud. Query the transformed iOS tooth point cloud using KDTree Each point in the CBCT tooth point cloud Calculate the nearest neighbor distance in the transformed IOS tooth point cloud. With CBCT tooth point cloud The matching error between them is used as the fitness value. The smaller the fitness value, the better the registration effect.
[0032] S53, compare the current fitness value with the historical best value, and update the individual historical best position of each particle. and the global historical best position of the population ;
[0033] S54 updates the particle's velocity and position based on individual and global extrema; during the update process, the velocity vector is thresholded, and the rotation speed is limited to a certain value. Within the arc, the translation speed is limited to the translation range. Within this range, the updated position is subjected to boundary constraints, and the next iteration continues until the maximum number of iterations is reached or the fitness value satisfies the convergence condition.
[0034] S55, based on the population's global historical best position. Extracting rotation vectors Translation vector ,in These represent the rotation angles of the iOS tooth point cloud around the coordinate axes x, y, and z, respectively. These represent the translation distances along the x, y, and z axes, respectively; the transformation matrix, composed of the rotation and translation vectors, is applied to the iOS tooth point cloud. Complete the initial global registration.
[0035] Furthermore, in step S52, the fitness value is calculated by using KDTree to accelerate the query and calculate the transformed IOS tooth point cloud. Point cloud of each point to CBCT teeth The root mean square error of the Euclidean distance between the nearest points is given by the following formula:
[0036] ,
[0037] In the formula, Represents particles The corresponding rigid transformation matrix, The first point cloud for teeth in iOS One point, The first tooth point cloud in CBCT One point, This represents the total number of points in the iOS tooth point cloud. This represents the squared error of the distance.
[0038] Furthermore, in step S54, the formula used to update the particle's velocity and position is:
[0039] ,
[0040] ,
[0041] In the formula, and They represent the first In the nth iteration The velocity vector and position vector of each particle For inertial weights, and As a learning factor, and for Random numbers between For the first The individual best position in the history of each particle. This represents the global historical best position of the population.
[0042] Furthermore, in step S6, the specific steps of single-tooth local iterative alignment include:
[0043] S61, based on the labeling results generated by the CBCT tooth segmentation model and the IOS tooth segmentation model, extract single tooth data with the same number from the global point cloud; use the IOS local point cloud of this single tooth as the source point cloud. The corresponding CBCT local point cloud is used as the target point cloud. ;
[0044] S62, for target point cloud Construct a KDTree spatial index structure; for each point in the source point cloud... Find the nearest point in the target point cloud using nearest neighbor search. As the initial corresponding point, outlier point pairs with excessive deviations are removed based on the Euclidean distance threshold to improve registration accuracy;
[0045] S63 employs a point-to-point error model and minimizes the objective function between corresponding point pairs using singular value decomposition. The optimal incremental transformation matrix for the current iteration step is calculated. In the formula, for Rotation matrix, for Translation vector, For local point i in iOS, Let j be a local point in CBCT. Indicates the squared error of the distance;
[0046] S64, the calculated Acting on the current source cloud Update its spatial position; repeat steps S62 to S63 to re-establish the correspondence and solve for the new transformation matrix;
[0047] S65, calculate the root mean square error between the IOS local point cloud and the CBCT local point cloud after each iteration; when the error change between two consecutive iterations is less than a set threshold... The iteration stops when the maximum number of iterations reaches 4000; the final fine transformation matrix of the single tooth relative to the global initial registration position is output to achieve accurate anatomical alignment of the single tooth.
[0048] Furthermore, in step S7, the specific steps for full-port global registration optimization include:
[0049] S71, summarize the corresponding point pairs of all individual teeth in the local fine registration state in step S6; take the CBCT nearest neighbor point found after local fine adjustment of each tooth as the target anchor point of that tooth in the whole mouth scale, thus forming a high-precision global corresponding point set covering the entire dental arch. ,in Let represent the i-th pair of paired points, p represent the points in the IOS point cloud, and q represent the points in the CBCT point cloud;
[0050] S72, based on the least squares principle, solves the reconstructed full-mouth corresponding point set through the singular value decomposition algorithm; instead of transforming individual teeth independently, it solves a unique, globally optimal six-degree-of-freedom transformation matrix that acts on the entire IOS dental arch point cloud, thereby minimizing the average registration error across the entire mouth.
[0051] The beneficial effects of this invention are as follows: This invention proposes a registration method for IOS and CBCT images based on particle swarm optimization and single-tooth optimization, which effectively solves the problems of low registration accuracy and poor robustness caused by resolution differences and pose inconsistencies between IOS and CBCT image data in the prior art. Specifically, this method introduces a particle swarm optimization algorithm through a global optimization registration module to search and optimize the six-degree-of-freedom rigid transformation parameters, achieving global initial registration at the dental arch scale and effectively handling pose inconsistencies between different modal data. Furthermore, a local fine optimization module divides the registration object into single-tooth local units based on semantic information, and uses an iterative nearest-point algorithm to perform local iterative alignment of each unit. By minimizing local distance errors, a high-precision registration model is finally output. In summary, this invention achieves the fusion of CBCT and IOS images at different resolutions, improving the accuracy and stability of multimodal 3D model registration tasks in the oral cavity, and providing reliable automated registration support for clinical scenarios such as orthodontics and implantology. Attached Figure Description
[0052] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0053] Figure 1 This is a registration flowchart of the present invention;
[0054] Figure 2 This is a schematic diagram of the tooth contours before and after local correction according to the present invention;
[0055] Figure 3 This is a comparison chart of the registration performance of the method of the present invention with other traditional registration algorithms on real clinical data. Detailed Implementation
[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.
[0057] like Figure 1 As shown, the present invention discloses a method for registering IOS images and CBCT images based on particle swarm optimization and single-tooth optimization, comprising the following steps:
[0058] S1. First, acquire the patient's IOS image data and CBCT image data. For the CBCT image data, read the DICOM voxel information and perform threshold segmentation to extract the hard tissue structure of the teeth and perform semantic segmentation annotation on the segmented 3D model. For the IOS image data, perform fine semantic annotation on different tooth regions to obtain segmentation labels. Subsequently, perform normalization and decentralization preprocessing operations on the IOS image data and CBCT image data to finally construct a CBCT segmentation dataset and an IOS segmentation dataset containing semantic information.
[0059] S2, Based on the convolutional neural network architecture of nnU-Net, a CBCT tooth segmentation model is constructed and trained. The CBCT tooth segmentation model takes CBCT image data as input and outputs the three-dimensional voxel semantic segmentation result of the tooth.
[0060] S3. Based on a two-stage deep learning network framework, an IOS tooth segmentation model is constructed and trained. The two-stage deep learning network framework includes a tooth centroid detection module for localization and a semantic segmentation module for fine tooth segmentation. The IOS tooth segmentation model takes IOS image data as input and outputs the semantic segmentation result of the tooth surface mesh.
[0061] S4. Using the trained CBCT tooth segmentation model and IOS tooth segmentation model, inference is performed on the CBCT image data and IOS image data of the target patient respectively to obtain CBCT tooth semantic segmentation images and IOS tooth semantic segmentation images, and their respective geometric information is extracted to generate CBCT tooth point clouds and IOS tooth point clouds.
[0062] S5. Initialize the particle swarm and establish a spatial index structure for the CBCT tooth point cloud; apply the transformation parameters represented by the particles to the IOS tooth point cloud, and calculate the fitness value between the IOS tooth point cloud and the CBCT tooth point cloud through the spatial index structure; update the individual optimal position and global optimal position of the particle swarm based on the fitness value, and update the particle state according to the individual optimal position and global optimal position; perform constraint processing on the updated particle state and iterate until the preset termination condition is met; determine the optimal transformation parameters based on the global optimal position and construct the transformation matrix to complete the initial registration of the IOS tooth point cloud and the CBCT tooth point cloud.
[0063] S6. Based on the global initial registration result obtained in S5, the semantic label information of the CBCT tooth semantic segmentation image and IOS tooth semantic segmentation image obtained in S4 is used to divide the whole dental arch point cloud into multiple independent local units based on a single tooth, and the IOS tooth local point cloud and CBCT tooth local point cloud corresponding to each tooth are extracted.
[0064] S7. Each IOS tooth local point cloud and CBCT tooth local point cloud unit is independently aligned using single-tooth iteration. By minimizing the distance error of the local correspondence, the small misalignment of a single tooth is eliminated and the position of the overall model is updated, finally obtaining a high-precision three-dimensional tooth registration model.
[0065] Furthermore, the CBCT tooth segmentation model described in step S2 is constructed based on the nnU-Net adaptive configuration framework, and the specific steps of its one-time training iteration include:
[0066] S21, The nnU-Net framework automatically configures and generates an adapted 3D U-Net network topology based on the voxel resolution, data scale, and number of categories of the input CBCT image data, serving as the basic architecture of the model.
[0067] S22, the model adopts the standard U-Net encoder-decoder structure, and its convolutional layer introduces the nnU-Net standardized configuration, including instance normalization and Leaky-ReLU activation function; the encoder extracts multi-scale semantic information through 3D convolution and progressive downsampling; the decoder restores the resolution through upsampling, and fuses shallow structural details and deep semantic features through feature skip connections between corresponding layers to improve segmentation accuracy;
[0068] S23: A batch of data-enhanced CBCT image data is read from the CBCT segmentation dataset in S1 as input and forward propagation is performed; the network output predicts the probability of each voxel belonging to a specific category through Softmax; at the same time, the network outputs the prediction results at multiple downsampling levels of the decoder to achieve deep supervision.
[0069] S24, a hybrid loss function, combines weighted cross-entropy loss. and Dice loss During training, the aforementioned hybrid loss is calculated for each output layer of the deep supervision; ultimately, the total loss of the model is calculated by combining the loss values of each layer's segmentation head with... The weighted sum is obtained by weighting the values.
[0070] Furthermore, the specific steps for one iteration of training the IOS tooth segmentation model described in step S3 include:
[0071] In S31, the IOS segmentation dataset from S1 is input into the tooth centroid detection module to predict the offset vector and distance confidence of each facet relative to the tooth center. This module adopts a PointNet++-based architecture, extracts facet features through multi-scale sampling, and outputs the offset vector and distance confidence of each facet. By moving the facets and filtering high-confidence candidate faces, the DBSCAN clustering algorithm is applied to extract the centroid of each cluster as the tooth center point.
[0072] S32, for each center point, a local region containing a fixed number of facets is cropped using the K-nearest neighbor algorithm; the distance distribution function from each facet within the local region to all tooth center points is calculated. The enhanced feature representation is then concatenated with the original coordinate features and input into the semantic segmentation module. The distance distribution function is calculated as follows:
[0073] ,
[0074] in, These are the coordinates of points on a local area; These are the coordinates of the j-th tooth center point in the set of center points; Indicates Euclidean distance;
[0075] In step S33, the IOS segmentation dataset from S1 is input into the tooth semantic segmentation module. The enhancement features of each point are separated into coordinate features and geometric enhancement features, which are then input into the coordinate feature stream and geometric enhancement feature stream, respectively. At the feature input front end, a spatial transformation network is used to learn the affine transformation matrices of the coordinate stream and the normal vector stream, respectively, to perform spatial alignment correction on the original input to eliminate the influence of pose rotation. Subsequently, learnable parameters are introduced into each layer of the encoder. The adaptive KNN strategy extracts semantic features of the current layer through a semantic encoder and dynamically calculates and fuses the similarity matrix by combining Euclidean spatial distance and semantic spatial cosine similarity.
[0076] This captures the geometric edges and semantically consistent neighborhoods, where Represents learnable parameters , Indicates Euclidean spatial distance. The semantic space cosine similarity is represented. Based on this neighborhood, the coordinate feature stream extracts local fine features through the grouped vector attention module, and the geometric enhancement feature stream aggregates neighborhood information through graph convolution and performs max pooling. Finally, the cross attention module dynamically weights and fuses the two streams by calculating the saliency weights of the features to generate cross attention features with multimodal information.
[0077] S34 first generates multi-scale cross-attention features at each stage of the cascaded encoder, and then calculates the global contribution weights of coordinate flow, geometric flow, and cross-flow. The system achieves adaptive weighted integration of multi-source features, and then performs nonlinear enhancement on the fused feature stream through a feature alignment module. At the network output, a 1×1 convolutional layer is used to transform the dimension and extract the classification score. The softmax function is used to transform it into the probability distribution of each facet belonging to a specific tooth category. Finally, the weighted negative log-likelihood loss function is used to calculate the loss value between the predicted class probability and the true segmentation label.
[0078] Furthermore, step S4 also includes the following steps:
[0079] S41, the tooth segmentation contour voxel information output by the CBCT tooth segmentation model is converted into point cloud data to obtain CBCT tooth point cloud. Specifically, the method is as follows: Read the DICOM voxel data corresponding to the CBCT 3D tooth segmentation result, obtain the 3D voxel array (Volume) and its spatial resolution; perform threshold binarization on the voxel values, where the threshold T is the voxel intensity determination threshold. When not explicitly set, if it is CT data, the default tooth threshold of 1500 HU is automatically selected; if it is a binary mask, a threshold of 0.5 is used. The specific conversion rules are as follows:
[0080]
[0081] In the formula, These are the binarized voxel values. The original voxel values are used; isosurfaces are extracted based on the marchingcubes algorithm to generate a set of triangular facets (Faces) and a set of vertices (Verts), where the vertices are the 3D coordinates of tooth surface elements; the vertex coordinates are used as point cloud data to construct a CBCT tooth point cloud set. ;
[0082] S42, convert the iOS tooth semantic segmentation results output by the iOS tooth segmentation model into point cloud data. Specifically, extract the vertices of the tooth surface mesh from the semantic segmentation results as point cloud data to form an iOS tooth point cloud set. .
[0083] Furthermore, step S5 specifically includes:
[0084] S51, Initialize Particle Swarm: Set the particle swarm size to... Each particle represents a set of six-DOF rigid transformation parameters, including three rotation Euler angles and three translation components. The position vector of each particle is randomly initialized. and velocity vector ,in For particle indexing, the initial range of the rotation angle is set to... The initial range of the translation is , where D is the Euclidean distance between the centroids of the IOS point cloud and the CBCT point cloud;
[0085] S52 employs spatial indexing technology to pre-construct a KDTree of the CBCT tooth point cloud to accelerate the search; the transformation parameters represented by each particle are applied to the IOS tooth point cloud. Query the transformed iOS tooth point cloud using KDTree Each point in the CBCT tooth point cloud Calculate the nearest neighbor distance in the transformed IOS tooth point cloud. With CBCT tooth point cloud The matching error between them is used as the fitness value. The smaller the fitness value, the better the registration effect.
[0086] S53, compare the current fitness value with the historical best value, and update the individual historical best position of each particle. and the global historical best position of the population ;
[0087] S54 updates the particle's velocity and position based on individual and global extrema; during the update process, the velocity vector is thresholded, and the rotation speed is limited to a certain value. Within the arc, the translation speed is limited to the translation range. Within this range, the updated position is subjected to boundary constraints, and the next iteration continues until the maximum number of iterations is reached or the fitness value satisfies the convergence condition.
[0088] S55, based on the population's global historical best position. Extracting rotation vectors Translation vector ,in These represent the rotation angles of the iOS tooth point cloud around the coordinate axes x, y, and z, respectively. These represent the translation distances along the x, y, and z axes, respectively; the transformation matrix, composed of the rotation and translation vectors, is applied to the iOS tooth point cloud. Complete the initial global registration.
[0089] Furthermore, in step S52, the fitness value is calculated by using KDTree to accelerate the query and calculate the transformed IOS tooth point cloud. Point cloud of each point to CBCT teeth The root mean square error of the Euclidean distance between the nearest points is given by the following formula:
[0090] ,
[0091] In the formula, Represents particles The corresponding rigid transformation matrix, The first point cloud for teeth in iOS One point, The first tooth point cloud in CBCT One point, This represents the total number of points in the iOS tooth point cloud. This represents the squared error of the distance.
[0092] Furthermore, in step S54, the formula used to update the particle's velocity and position is:
[0093] ,
[0094] ,
[0095] In the formula, and They represent the first In the nth iteration The velocity vector and position vector of each particle For inertial weights, and As a learning factor, and for Random numbers between For the first The individual best position in the history of each particle. This represents the global historical best position of the population.
[0096] Furthermore, in step S6, the specific steps of single-tooth local iterative alignment include:
[0097] S61, based on the labeling results generated by the CBCT tooth segmentation model and the IOS tooth segmentation model, extract single tooth data with the same number from the global point cloud; use the IOS local point cloud of this single tooth as the source point cloud. The corresponding CBCT local point cloud is used as the target point cloud. ;
[0098] S62, for target point cloud Construct a KDTree spatial index structure; for each point in the source point cloud... Find the nearest point in the target point cloud using nearest neighbor search. As the initial corresponding point, outlier point pairs with excessive deviations are removed based on the Euclidean distance threshold to improve registration accuracy;
[0099] S63 employs a point-to-point error model and minimizes the objective function between corresponding point pairs using singular value decomposition. The optimal incremental transformation matrix for the current iteration step is calculated. In the formula, for Rotation matrix, for Translation vector, For local point i in iOS, Let j be a local point in CBCT. Indicates the squared error of the distance;
[0100] S64, the calculated Acting on the current source cloud Update its spatial position; repeat steps S62 to S63 to re-establish the correspondence and solve for the new transformation matrix;
[0101] S65, calculate the root mean square error between the IOS local point cloud and the CBCT local point cloud after each iteration; when the error change between two consecutive iterations is less than a set threshold... The iteration stops when the maximum number of iterations reaches 4000; the final fine transformation matrix of the single tooth relative to the global initial registration position is output to achieve accurate anatomical alignment of the single tooth.
[0102] Furthermore, in step S7, the specific steps for full-port global registration optimization include:
[0103] S71, summarize the corresponding point pairs of all individual teeth in the local fine registration state in step S6; take the CBCT nearest neighbor point found after local fine adjustment of each tooth as the target anchor point of that tooth in the whole mouth scale, thus forming a high-precision global corresponding point set covering the entire dental arch. ,in Let represent the i-th pair of paired points, p represent the points in the IOS point cloud, and q represent the points in the CBCT point cloud;
[0104] S72, based on the least squares principle, solves the reconstructed full-mouth corresponding point set through the singular value decomposition algorithm; instead of transforming individual teeth independently, it solves a unique, globally optimal six-degree-of-freedom transformation matrix that acts on the entire IOS dental arch point cloud, thereby minimizing the average registration error across the entire mouth.
[0105] Figure 2 This is a schematic diagram of the tooth contour before and after local correction according to the present invention. By comparing the perspectives 1 (coronal plane), 2 (cross section), and 3 (sagittal plane) before and after local registration, it can be seen that the present invention achieves alignment in the X, Y, and Z dimensions, effectively correcting the slight deviations in the local anatomical pose of the global initial registration, so that the tooth contours in the IOS image and CBCT image are highly overlapped, effectively solving the problem that global registration is trapped in local optima because it cannot take into account the pose differences of individual teeth.
[0106] Figure 3 This image compares the registration performance of the method of this invention with other traditional registration algorithms on real clinical data. Experimental results with four different registration algorithms demonstrate that when faced with challenging defect data such as missing teeth, partial dentition, or tooth loss, existing registration algorithms often fail to establish correct geometric correspondences in the defective areas, leading to severe displacement deviations and registration failures. In contrast, this invention, with its "global-local" optimization strategy, effectively solves the complex deformation registration problem between IOS and CBCT images across modal data, exhibiting higher registration accuracy and robustness.
[0107] Example:
[0108] The training dataset was constructed following step S1. The data originated from clinically collected patient oral data, including IOS image data and corresponding CBCT image data. Each sample pair underwent pixel-level / grid-level semantic annotation by a professional dentist to ensure that each tooth had a unique semantic label value.
[0109] The segmentation network is trained according to steps S2 and S3 respectively. For CBCT image data, 5-fold cross-validation training is performed using the nnU-Net framework, and the model weights with the best performance on the validation set are saved. For IOS image data, the centroid detection network is first trained to obtain the region of interest (ROI) of the teeth, and then the ROI regions are cropped and input into the tooth semantic segmentation network for training. The clinical data to be registered is input into the trained model, and point cloud data with semantic information is output according to step S4.
[0110] Perform global registration according to step S5. Set the number of particles N in the particle swarm optimization algorithm to 50 and the maximum number of iterations K to 100. Set the IOS tooth point cloud as the source point cloud and the CBCT tooth point cloud as the target point cloud. During the iteration process, use the root mean square error as the fitness evaluation index. Stop the search when the fitness reaches the threshold or the maximum number of iterations is reached, output the global transformation matrix T, and transform the IOS point cloud to the vicinity of the CBCT coordinate system.
[0111] Perform fine-grained registration according to steps S6 and S7. Using semantic tags, the entire dental arch is automatically segmented into 14-16 single-tooth point cloud pairs. For each single-tooth point cloud pair, local iterative nearest-point registration is performed based on T, and the local transformation matrix is calculated. , where i represents the i-th tooth. Finally, each tooth is reassembled into a complete registered dentition by applying its corresponding optimal transformation matrix.
[0112] Testing: Patient data not involved in training were selected as the test set, and the registration results output by this invention were compared with the labels. Mean surface distance and Hausdorff distance were calculated as evaluation metrics to quantitatively assess the model's registration accuracy.
[0113] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for registering IOS images and CBCT images based on particle swarm optimization and single-tooth optimization, characterized in that, Includes the following steps: S1. First, collect the patient's IOS image data and CBCT image data; For the CBCT image data, read the DICOM voxel information and perform threshold segmentation, extract the hard tissue structure of the teeth, and perform semantic segmentation annotation on the segmented 3D model; For the IOS image data, similarly, perform fine semantic annotation on different tooth regions to obtain segmentation labels. Subsequently, normalization and decentralization preprocessing operations were performed on the IOS image data and CBCT image data to finally construct a CBCT segmentation dataset and an IOS segmentation dataset containing semantic information. S2, Based on the convolutional neural network architecture of nnU-Net, a CBCT tooth segmentation model is constructed and trained. The CBCT tooth segmentation model takes CBCT image data as input and outputs the three-dimensional voxel semantic segmentation result of the tooth. S3. Based on a two-stage deep learning network framework, an IOS tooth segmentation model is constructed and trained. The two-stage deep learning network framework includes a tooth centroid detection module for localization and a semantic segmentation module for fine tooth segmentation. The IOS tooth segmentation model takes IOS image data as input and outputs the semantic segmentation result of the tooth surface mesh. S4. Using the trained CBCT tooth segmentation model and IOS tooth segmentation model, inference is performed on the CBCT image data and IOS image data of the target patient respectively to obtain CBCT tooth semantic segmentation images and IOS tooth semantic segmentation images, and their respective geometric information is extracted to generate CBCT tooth point clouds and IOS tooth point clouds. S5. Initialize the particle swarm and establish the spatial index structure of the CBCT tooth point cloud; apply the transformation parameters represented by the particles to the IOS tooth point cloud, and calculate the fitness value between the IOS tooth point cloud and the CBCT tooth point cloud through the spatial index structure. The individual optimal position and global optimal position of the particle swarm are updated based on the fitness value, and the particle state is updated according to the individual optimal position and global optimal position. The updated particle state is constrained and iterated until the preset termination condition is met. The optimal transformation parameters are determined according to the global optimal position and the transformation matrix is constructed to complete the initial registration of the IOS tooth point cloud and the CBCT tooth point cloud. S6. Based on the global initial registration result obtained in S5, the semantic label information of the CBCT tooth semantic segmentation image and IOS tooth semantic segmentation image obtained in S4 is used to divide the whole dental arch point cloud into multiple independent local units based on a single tooth, and the IOS tooth local point cloud and CBCT tooth local point cloud corresponding to each tooth are extracted. S7. Each IOS tooth local point cloud and CBCT tooth local point cloud unit is independently aligned using single-tooth iteration. By minimizing the distance error of the local correspondence, the small misalignment of a single tooth is eliminated and the position of the overall model is updated, finally obtaining a high-precision three-dimensional tooth registration model.
2. The method for registering IOS and CBCT images based on particle swarm optimization and single-tooth optimization according to claim 1, characterized in that: The CBCT tooth segmentation model described in step S2 is built based on the nnU-Net adaptive configuration framework, and the specific steps of its one-time training iteration include: S21, The nnU-Net framework automatically configures and generates an adapted 3D U-Net network topology based on the voxel resolution, data scale, and number of categories of the input CBCT image data, serving as the basic architecture of the model. S22, the model adopts the standard U-Net encoder-decoder structure, and its convolutional layer introduces the nnU-Net standardized configuration, including instance normalization and Leaky-ReLU activation function; the encoder extracts multi-scale semantic information through 3D convolution and progressive downsampling; the decoder restores the resolution through upsampling, and fuses shallow structural details and deep semantic features through feature skip connections between corresponding layers to improve segmentation accuracy; S23: A batch of data-enhanced CBCT image data is read from the CBCT segmentation dataset in S1 as input and forward propagation is performed; the network output predicts the probability of each voxel belonging to a specific category through Softmax; at the same time, the network outputs the prediction results at multiple downsampling levels of the decoder to achieve deep supervision. S24, a hybrid loss function, combines weighted cross-entropy loss. and Dice loss During training, the aforementioned hybrid loss is calculated for each output layer of the deep supervision; ultimately, the total loss of the model is calculated by combining the loss values of each layer's segmentation head with... The weighted sum is obtained by weighting the values.
3. The method for registering IOS and CBCT images based on particle swarm optimization and single-tooth optimization according to claim 1, characterized in that: The specific steps for one iteration of training the IOS tooth segmentation model described in step S3 include: In S31, the IOS segmentation dataset from S1 is input into the tooth centroid detection module to predict the offset vector and distance confidence of each facet relative to the tooth center. This module adopts a PointNet++-based architecture, extracts facet features through multi-scale sampling, and outputs the offset vector and distance confidence of each facet. By moving the facets and filtering high-confidence candidate faces, the DBSCAN clustering algorithm is applied to extract the centroid of each cluster as the tooth center point. S32, for each center point, a local region containing a fixed number of facets is cropped using the K-nearest neighbor algorithm; the distance distribution function from each facet within the local region to all tooth center points is calculated. The enhanced feature representation is then concatenated with the original coordinate features and input into the semantic segmentation module. The distance distribution function is calculated as follows: ， in, These are the coordinates of points on a local area; These are the coordinates of the j-th tooth center point in the set of center points; Indicates Euclidean distance; In step S33, the IOS segmentation dataset from S1 is input into the tooth semantic segmentation module. The enhancement features of each point are separated into coordinate features and geometric enhancement features, which are then input into the coordinate feature stream and geometric enhancement feature stream, respectively. At the feature input front end, a spatial transformation network is used to learn the affine transformation matrices of the coordinate stream and the normal vector stream, respectively, to perform spatial alignment correction on the original input to eliminate the influence of pose rotation. Subsequently, learnable parameters are introduced into each layer of the encoder. The adaptive KNN strategy extracts semantic features of the current layer through a semantic encoder and dynamically calculates and fuses the similarity matrix by combining Euclidean spatial distance and semantic spatial cosine similarity. This captures the geometric edges and semantically consistent neighborhoods, where Represents learnable parameters , Indicates Euclidean spatial distance. The semantic space cosine similarity is represented. Based on this neighborhood, the coordinate feature stream extracts local fine features through the grouped vector attention module, and the geometric enhancement feature stream aggregates neighborhood information through graph convolution and performs max pooling. Finally, the cross attention module dynamically weights and fuses the two streams by calculating the saliency weights of the features to generate cross attention features with multimodal information. S34 first generates multi-scale cross-attention features at each stage of the cascaded encoder, and then calculates the global contribution weights of coordinate flow, geometric flow, and cross-flow. The system achieves adaptive weighted integration of multi-source features, and then performs nonlinear enhancement on the fused feature stream through a feature alignment module. At the network output, a 1×1 convolutional layer is used to transform the dimension and extract the classification score. The softmax function is used to transform it into the probability distribution of each facet belonging to a specific tooth category. Finally, the weighted negative log-likelihood loss function is used to calculate the loss value between the predicted class probability and the true segmentation label.
4. The method for registering IOS and CBCT images based on particle swarm optimization and single-tooth optimization according to claim 1, characterized in that: Step S4 also includes the following steps: S41, the tooth segmentation contour voxel information output by the CBCT tooth segmentation model is converted into point cloud data to obtain CBCT tooth point cloud. Specifically, the method is as follows: Read the DICOM voxel data corresponding to the CBCT 3D tooth segmentation result, obtain the 3D voxel array (Volume) and its spatial resolution; perform threshold binarization on the voxel values, where the threshold T is the voxel intensity determination threshold. When not explicitly set, if it is CT data, the default tooth threshold of 1500 HU is automatically selected; if it is a binary mask, a threshold of 0.5 is used. The specific conversion rules are as follows: In the formula, These are the binarized voxel values. The original voxel values are used; isosurfaces are extracted based on the marchingcubes algorithm to generate a set of triangular facets (Faces) and a set of vertices (Verts), where the vertices are the 3D coordinates of tooth surface fractals; the vertex coordinates are used as point cloud data to construct a CBCT tooth point cloud set. ; S42, convert the IOS tooth semantic segmentation results output by the IOS tooth segmentation model into point cloud data. Specifically, extract the vertices of the tooth surface mesh from the semantic segmentation results as point cloud data to form an IOS tooth point cloud set. .
5. The method for registering IOS and CBCT images based on particle swarm optimization and single-tooth optimization according to claim 1, characterized in that: Step S5 specifically includes: S51, Initialize Particle Swarm: Set the particle swarm size to... Each particle represents a set of six-DOF rigid transformation parameters, including three rotation Euler angles and three translation components. The position vector of each particle is randomly initialized. and velocity vector ,in For particle indexing, the initial range of the rotation angle is set to... The initial range of the translation is , where D is the Euclidean distance between the centroids of the IOS point cloud and the CBCT point cloud; S52 employs spatial indexing technology to pre-construct a KDTree of the CBCT tooth point cloud to accelerate the search; the transformation parameters represented by each particle are applied to the IOS tooth point cloud. Query the transformed iOS tooth point cloud using KDTree Each point in the CBCT tooth point cloud Calculate the nearest neighbor distance in the transformed IOS tooth point cloud. With CBCT tooth point cloud The matching error between them is used as the fitness value. The smaller the fitness value, the better the registration effect. S53, compare the current fitness value with the historical best value, and update the individual historical best position of each particle. and the global historical best position of the population ; S54 updates the particle's velocity and position based on individual and global extrema; during the update process, the velocity vector is thresholded, and the rotation speed is limited to a certain value. Within the arc, the translation speed is limited to the translation range. Within this range, the updated position is subjected to boundary constraints, and the next iteration continues until the maximum number of iterations is reached or the fitness value satisfies the convergence condition. S55, based on the population's global historical best position. Extracting rotation vectors Translation vector ,in These represent the rotation angles of the iOS tooth point cloud around the coordinate axes x, y, and z, respectively. These represent the translation distances along the x, y, and z axes, respectively; the transformation matrix, composed of the rotation and translation vectors, is applied to the iOS tooth point cloud. Complete the initial global registration.
6. The method for registering IOS and CBCT images based on particle swarm optimization and single-tooth optimization according to claim 5, characterized in that: In step S52, the fitness value is calculated by using KDTree to accelerate the query and calculate the transformed IOS tooth point cloud. Point cloud of each point to CBCT teeth The root mean square error of the Euclidean distance between the nearest points is given by the following formula: ， In the formula, Represents particles The corresponding rigid transformation matrix, The first point cloud for teeth in iOS One point, The first tooth point cloud in CBCT One point, This represents the total number of points in the iOS tooth point cloud. This represents the squared error of the distance.
7. The method for registering IOS and CBCT images based on particle swarm optimization and single-tooth optimization according to claim 5, characterized in that: In step S54, the formula used to update the particle's velocity and position is: ， ， In the formula, and They represent the first In the nth iteration The velocity vector and position vector of each particle For inertial weights, and As a learning factor, and for Random numbers between For the first The individual best position in the history of each particle. This represents the global historical best position of the population.
8. The method for registering IOS and CBCT images based on particle swarm optimization and single-tooth optimization according to claim 1, characterized in that: In step S6, the specific steps for single-tooth local iterative alignment include: S61, based on the labeling results generated by the CBCT tooth segmentation model and the IOS tooth segmentation model, extract single tooth data with the same number from the global point cloud; use the IOS local point cloud of this single tooth as the source point cloud. The corresponding CBCT local point cloud is used as the target point cloud. ; S62, for target point cloud Construct a KDTree spatial index structure; for each point in the source point cloud... Find the nearest point in the target point cloud using nearest neighbor search. As the initial corresponding point, outlier point pairs with excessive deviations are removed based on the Euclidean distance threshold to improve registration accuracy; S63 employs a point-to-point error model and minimizes the objective function between corresponding point pairs using singular value decomposition. The optimal incremental transformation matrix for the current iteration step is calculated. In the formula, for Rotation matrix, for Translation vector, For local point i in iOS, Let j be a local point in CBCT. Indicates the squared error of the distance; S64, the calculated Acting on the current source cloud Update its spatial position; repeat steps S62 to S63 to re-establish the correspondence and solve for the new transformation matrix; S65, calculate the root mean square error between the IOS local point cloud and the CBCT local point cloud after each iteration; when the error change between two consecutive iterations is less than a set threshold... The iteration stops when the maximum number of iterations reaches 4000; the final fine transformation matrix of the single tooth relative to the global initial registration position is output to achieve accurate anatomical alignment of the single tooth.
9. The method for registering IOS and CBCT images based on particle swarm optimization and single-tooth optimization according to claim 1, characterized in that: In step S7, the specific steps for full-port global registration optimization include: S71, summarize the corresponding point pairs of all individual teeth in the local fine registration state in step S6; take the CBCT nearest neighbor point found after local fine adjustment of each tooth as the target anchor point of that tooth in the whole mouth scale, thus forming a high-precision global corresponding point set covering the entire dental arch. ,in Let represent the i-th pair of paired points, p represent the points in the IOS point cloud, and q represent the points in the CBCT point cloud; S72, based on the least squares principle, solves the reconstructed full-mouth corresponding point set through the singular value decomposition algorithm; instead of transforming individual teeth independently, it solves a unique, globally optimal six-degree-of-freedom transformation matrix that acts on the entire IOS dental arch point cloud, thereby minimizing the average registration error across the entire mouth.