CBCT tooth segmentation and modeling algorithm
By combining CBCT and optical scanning data, image registration and deep convolutional neural network technology are used to generate high-quality 3D dental models, solving the problems of long-term data acquisition, low accuracy and low authenticity in traditional technologies, and achieving more efficient and more realistic 3D dental modeling.
Patent Information
- Application Number
- CN202411788545.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-06
- Publication Date
- 2025-03-07
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional CBCT tooth segmentation and modeling technology has long time to collect data, low image registration accuracy, low 3D modeling authenticity, long-term model export and large files, which is not conducive to storage and transmission.
CBCT equipment and optical scanners are used to obtain 3D images and tooth surface data, data are aligned using image registration algorithm, fusion of multiple modal data to generate 3D models, select deep convolutional neural networks to optimize tooth segmentation tasks, and combine physical rendering technology and PBR technology to simulate tooth color and gloss effects.
The accuracy and reliability of data alignment are improved, the generated 3D model has higher quality and realism, the model performs better under different lighting conditions, and its applicability and robustness are improved, and it is suitable for PBR technology applications.
Smart Images
Figure CN119579795A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of CBCT tooth segmentation and modeling, and in particular to a CBCT tooth segmentation and modeling algorithm. Background Art
[0002] CBCT (Cone Beam Computed Tomography) tooth segmentation and modeling is a method that combines CBCT images and optical scanning data to generate high-precision 3D tooth models through image processing and machine learning technology. However, traditional CBCT and optical scanning equipment take a long time in the data acquisition process, affecting the efficiency of the overall process. In addition, the existing image registration method has low accuracy when aligning data of different modalities and is prone to misalignment and distortion. At the same time, the existing 3D modeling method often ignores the optical properties of tooth materials when generating models, resulting in low model authenticity. The model export method takes a long time to generate the final 3D model file, and the file is large, which is not conducive to storage and transmission. Summary of the invention
[0003] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.
[0004] To achieve the above object, the present invention provides the following technical solutions: A CBCT tooth segmentation and modeling algorithm comprises: using a CBCT device to obtain high-resolution 3D image data of a patient's oral cavity, and using an optical scanner to obtain fine structures of tooth surfaces; using an image registration algorithm to align image data from different sources, and using a fusion algorithm to merge data from multiple modalities into a 3D model; selecting a deep convolutional neural network suitable for processing 3D medical images, and optimizing the network structure for tooth segmentation tasks; dividing the preprocessed and annotated data set into a training set, a validation set, and a test set, and establishing a tooth surface reflection and transmission characteristic model based on physical rendering technology and the optical properties of tooth materials; combining the optical model with actual measurement data, and using PBR technology to simulate the real color and gloss effect of teeth; and exporting the optimized 3D tooth model into a universal format file.
[0005] As a further solution of the present invention: the use of a CBCT device to obtain high-resolution 3D image data of the patient's oral cavity and the use of an optical scanner to obtain the fine structure of the tooth surface include: using the CBCT device to scan the patient's oral cavity; using a high-precision optical scanner to scan the patient's tooth surface; using marker points to be placed in the patient's oral cavity, the marker points can be recognized in both CBCT and optical scanning, and by matching the positions of these marker points, accurate alignment of the two modal data is achieved.
[0006] As a further solution of the present invention: the image data from different sources are aligned using an image registration algorithm, and the data of multiple modalities are merged into a 3D model using a fusion algorithm, including preprocessing the CBCT image and optical scanning data, including denoising, grayscale normalization and background removal, denoising using median filtering or Gaussian filtering, grayscale normalization to unify pixel values to the [0, 1] interval, and background removal by setting a threshold to remove the background portion; using the Harris corner detection algorithm based on edge detection to extract landmarks from the CBCT image and optical scanning data respectively; using a rigid registration algorithm for initial registration to find the best rigid transformation between the two groups of point clouds, the expression is: ; in, For the The rotation matrix of the iteration, For the The translation vector of the iteration, The first The coordinates of the landmark points, The optical scanning data The coordinates of the landmark points, is the number of landmark points.
[0007] As a further solution of the present invention: the use of an image registration algorithm to align image data from different sources and the use of a fusion algorithm to merge data from multiple modalities into a 3D model also includes using a non-rigid registration algorithm to improve the registration accuracy, especially at the boundary between teeth and soft tissues, thereby achieving a non-rigid transformation of the entire image; defining a control grid , the grid nodes are ; For each control node Perform displacement , so that the deformed mesh nodes It can better align CBCT images and optical scanning data; through B-spline interpolation, the displacement of the control node is transferred to the entire image to achieve non-rigid transformation, and the expression is: ; in: is a point in the original image, is the point after deformation, is the B-spline basis function, To control the grid nodes, For the The displacement of the nodes, To control the number of nodes in the grid, the registered CBCT images and optical scanning data are fused to generate a comprehensive 3D model, which is expressed as: ; in: CBCT images The pixel value of The midpoint of the optical scan data The pixel value of is the weight of CBCT image, and is the weight of optical scanning data.
[0008] As a further solution of the present invention: the selection of a deep convolutional neural network suitable for processing 3D medical images and the optimization of the network structure for the tooth segmentation task include selecting 3D U-Net as the basic network structure, adding an attention module to the decoder part, introducing a residual connection between the encoder and the decoder, and fusing feature maps from different scales in the decoder part, the expression is:
[0009] in, is the predicted segmentation map The predicted probability of pixels, is the first The label value of each pixel, is the smoothing term, For the The boundary weight of pixels is used to enhance the focus on the boundary. is the total number of pixels.
[0010] As a further solution of the present invention: the pre-processed and annotated data set is divided into a training set, a validation set and a test set, and based on the physical rendering technology, a tooth surface reflection and transmission characteristic model is established according to the optical properties of the tooth material, including: preparing the pre-processed and annotated CBCT image and optical scanning data set; randomly dividing the data set into a training set, a validation set and a test set, with a ratio of 70% for training, 15% for validation, and 15% for testing; using a spectrometer to measure the optical properties of the tooth material, including reflectivity, refractive index and absorption coefficient; based on the measured optical properties, a tooth surface reflection and transmission characteristic model is established, and the Fresnel equation and the Beer-Lambert law are used to describe the reflection characteristics of the tooth surface and the transmission characteristics of the tooth interior, respectively, and the expressions are: ; ; in: is the reflectivity, is the incident angle, is the refractive index of air, is the refractive index of the tooth material, is the transmitted light intensity, is the incident light intensity, is the absorption coefficient, is the distance light travels in tooth material.
[0011] As a further solution of the present invention: the combination of the optical model and the actual measurement data, using the PBR technology to simulate the real color and gloss effect of the teeth includes using the Phong illumination model combined with the PBR technology to calculate the color value of each pixel. The Phong illumination model expression is: ; in: is the final color value, is the diffuse color, is the specular color, is the light source intensity, is the surface normal, is a half-angle vector, is the specular reflection index, and the roughness parameter Related; map the diffuse color, specular color, roughness and metalness parameters to each vertex of the 3D model to generate a PBR material texture; calculate the color value of each pixel based on the PBR material texture and lighting model to generate the final 3D model.
[0012] As a further solution of the present invention: exporting the optimized 3D tooth model into a universal format file includes: using a Laplacian smoothing algorithm to smooth the surface of the 3D model, and using a normal map technique to enhance the details of the model surface; selecting a universal 3D model format, such as STL, OBJ or FBX; setting export parameters to ensure the accuracy and integrity of the model; exporting the optimized 3D tooth model into the selected universal format file and saving it to a specified path.
[0013] The present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of a CBCT tooth segmentation and modeling algorithm as described in the first aspect of the present invention is implemented.
[0014] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of a CBCT tooth segmentation and modeling algorithm as described in the first aspect of the present invention is implemented.
[0015] Compared with the prior art, the present invention has the following beneficial effects: By placing landmarks in the patient's mouth and matching the positions of these landmarks, precise alignment of the two modal data is achieved, ensuring accurate spatial matching of CBCT images and optical scanning data, providing reliable reference points for subsequent image registration and fusion, and improving the accuracy and reliability of data alignment. By fusing the registered CBCT images and optical scanning data, a comprehensive 3D model is generated, combining the advantages of the two modal data and improving the overall quality and realism of the model. By introducing residual connections between the encoder and decoder, the gradient vanishing problem is alleviated, the training stability of the network is improved, and the good performance of the model on complex data is ensured. By combining the optical model with the actual measurement data, the performance of the model under different lighting conditions is ensured, the robustness and applicability of the model are improved, and a solid foundation is provided for the subsequent application of PBR technology. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 A flowchart of a CBCT tooth segmentation and modeling algorithm. DETAILED DESCRIPTION
[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings.
[0018] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0019] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.
[0020] Example See also Figure 1 , is an embodiment of the present invention, which provides a CBCT tooth segmentation and modeling algorithm, specifically an application of the CBCT tooth segmentation and modeling algorithm, including: Using a CBCT device (model: CBCT-3000) and a high-precision optical scanner (model: OptiScan-200), 10 volunteers aged between 20 and 45 years old were selected with good dental health and no obvious caries or defects. Five landmark points were placed in the mouth of each volunteer, which can be identified in both CBCT and optical scans.
[0021] The oral cavity of each volunteer was scanned using a CBCT device to obtain high-resolution 3D image data. The scanning parameters were set as follows: voltage 120 kV, current 10 mA, and scanning time of 2 minutes. The tooth surface of each volunteer was scanned using a high-precision optical scanner to obtain the fine structure of the tooth surface. The scanning parameters were set as follows: resolution 0.1 mm, and scanning time of 1 minute. Five landmark points were placed in the oral cavity of each volunteer to ensure that the landmark points could be recognized in both CBCT and optical scanning.
[0022] The CBCT images and optical scanning data were denoised using the median filter method with a filter window size of 3x3. The pixel values were unified to the [0, 1] interval to ensure data consistency. The background was removed by setting a threshold to retain useful information about the teeth and surrounding tissues.
[0023] The Harris corner detection algorithm based on edge detection is used to extract landmark points in CBCT images and optical scanning data respectively. The rigid registration algorithm is used for initial registration to find the optimal rigid transformation between the two sets of point clouds. The non-rigid registration algorithm is used to improve the registration accuracy, especially at the boundary between teeth and soft tissues. A control grid is defined, and each control node is displaced. The displacement of the control node is transferred to the entire image through B-spline interpolation to achieve non-rigid transformation.
[0024] The registered CBCT images and optical scan data are fused to generate a comprehensive 3D model.
[0025] We select 3D U-Net as the basic network structure, add an attention module to the decoder, introduce a residual connection between the encoder and decoder, and fuse feature maps from different scales in the decoder.
[0026] A spectrometer is used to measure the optical properties of tooth materials, including reflectivity, refractive index and absorption coefficient. Based on the measured optical properties, a reflection and transmission characteristic model of the tooth surface is established. The Fresnel equation and Beer-Lambert law are used to describe the reflection characteristics of the tooth surface and the transmission characteristics of the tooth interior, respectively.
[0027] Use the Phong lighting model combined with PBR technology to calculate the color value of each pixel, map the diffuse color, specular color, roughness and metalness parameters to each vertex of the 3D model, generate PBR material texture, and calculate the color value of each pixel based on the PBR material texture and lighting model to generate the final 3D model.
[0028] Use the Laplacian smoothing algorithm to smooth the surface of the 3D model, and use the normal mapping technology to enhance the details of the model surface. Select a common 3D model format, such as STL, OBJ, or FBX, set the export parameters to ensure the accuracy and integrity of the model, export the optimized 3D tooth model to the selected common format file, and save it to the specified path.
[0029] This embodiment also provides a computer device, which is suitable for the case of CBCT tooth segmentation and modeling algorithm, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to implement the CBCT tooth segmentation and modeling algorithm proposed in the above embodiment.
[0030] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.
[0031] The present embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the CBCT tooth segmentation and modeling algorithm proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, magnetic disk or optical disk.
[0032] Experimental data table: Table 1: Comparison of experimental data of CBCT tooth segmentation and modeling algorithms Parameter name unit Prior art Method of the present invention Percentage increase Scan time minute 3 2 33.33% Image registration accuracy mm 0.5 0.1 80.00% Tooth segmentation accuracy % 80 95 18.75% 3D model realism Fraction 70 90 28.57% Model export time Second 60 30 50.00% Model file size MB 100 50 50.00% Model surface smoothness Fraction 75 95 26.67% Model detail richness Fraction 60 85 41.67% Model Color Fidelity Fraction 70 90 28.57% Model glossiness realism Fraction 65 85 30.77% Data Analysis: The 3D model authenticity score of the prior art is 70 points, while the 3D model authenticity score of the method of the present invention is 90 points, an improvement of 28.57%, indicating that the 3D model generated by the method of the present invention has significant advantages in authenticity and can more realistically reflect the morphology and structure of the teeth.
[0033] The model surface smoothness score of the prior art is 75 points, while the model surface smoothness score of the method of the present invention is 95 points, which is improved by 26.67%, indicating that the method of the present invention has higher smoothness in model surface treatment and can generate a smoother model surface.
[0034] The model detail richness score of the prior art is 60 points, while the model detail richness score of the method of the present invention is 85 points, which is improved by 41.67%, indicating that the method of the present invention has higher richness in model detail processing and can generate a 3D model with more details.
[0035] The model color authenticity score of the prior art is 70 points, while the model color authenticity score of the method of the present invention is 90 points, which is improved by 28.57%, indicating that the method of the present invention has higher authenticity in model color processing and can generate a 3D model with more realistic colors.
[0036] The model gloss authenticity score of the prior art is 65 points, while the model gloss authenticity score of the method of the present invention is 85 points, which is improved by 30.77%, indicating that the method of the present invention has higher authenticity in model gloss processing and can generate a 3D model with more realistic gloss.
[0037] In summary, by placing landmarks in the patient's mouth and matching the positions of these landmarks, accurate alignment of the two modal data is achieved, ensuring accurate spatial matching of CBCT images and optical scanning data, providing reliable reference points for subsequent image registration and fusion, and improving the accuracy and reliability of data alignment. By fusing the registered CBCT images and optical scanning data, a comprehensive 3D model is generated, combining the advantages of the two modal data and improving the overall quality and realism of the model. By introducing residual connections between the encoder and decoder, the gradient vanishing problem is alleviated, the training stability of the network is improved, and the good performance of the model on complex data is ensured. By combining the optical model with the actual measurement data, the performance of the model under different lighting conditions is ensured, the robustness and applicability of the model are improved, and a solid foundation is provided for the subsequent application of PBR technology.
[0038] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A CBCT tooth segmentation and modeling algorithm, characterized by: include, Use CBCT equipment to obtain high-resolution 3D image data of the patient's oral cavity, and use optical scanners to obtain the fine structure of the tooth surface; Use image registration algorithms to align image data from different sources, and use fusion algorithms to merge data from multiple modalities into a 3D model; Select a deep convolutional neural network suitable for processing 3D medical images and optimize the network structure for the tooth segmentation task; The preprocessed and annotated data set is divided into a training set, a validation set, and a test set. Based on the physical rendering technology and the optical properties of tooth materials, a model of tooth surface reflection and transmission characteristics is established. Combining the optical model with actual measurement data, PBR technology is used to simulate the real color and gloss of teeth; Export the optimized 3D tooth model to a common format file.
2. A CBCT tooth segmentation and modeling algorithm according to claim 1, characterized in that: The method of using a CBCT device to obtain high-resolution 3D image data of the patient's oral cavity and using an optical scanner to obtain the fine structure of the tooth surface includes: Scan the patient's oral cavity using a CBCT device; Scan the patient's tooth surface using a high-precision optical scanner; Using landmarks placed in the patient's mouth, the landmarks are identifiable in both CBCT and optical scans, and by matching the positions of these landmarks, precise alignment of the two modal data is achieved.
3. A CBCT tooth segmentation and modeling algorithm according to claim 1, characterized in that: The image calibration algorithm is used to align the image data from different sources, and the fusion algorithm is used to merge the data of multiple modalities into a 3D model, including: Preprocessing of CBCT images and optical scanning data, including denoising, grayscale normalization and background removal. Denoising uses median filtering or Gaussian filtering, grayscale normalization unifies pixel values to the [0, 1] interval, and background removal removes the background by setting a threshold. Harris corner detection algorithm based on edge detection is used to extract landmark points in CBCT images and optical scanning data respectively; Use the rigid registration algorithm for initial registration to find the optimal rigid transformation between the two sets of point clouds. The expression is: ; in, For the The rotation matrix of the iteration, For the The translation vector of the iteration, The first The coordinates of the landmark points, The optical scanning data The coordinates of the landmark points, is the number of landmark points.
4. A CBCT tooth segmentation and modeling algorithm according to claim 1, characterized in that: The method of using an image calibration algorithm to align image data from different sources and using a fusion algorithm to merge data from multiple modalities into a 3D model also includes: Use non-rigid registration algorithms to improve registration accuracy, especially at the boundaries between teeth and soft tissues, thereby achieving a non-rigid transformation of the entire image; Defining a control grid , the grid nodes are ; For each control node Perform displacement , so that the deformed mesh nodes Enables better alignment of CBCT images and optical scan data; Through B-spline interpolation, the displacement of the control node is transferred to the entire image to achieve non-rigid transformation. The expression is: ; in: is a point in the original image, is the point after deformation, is the B-spline basis function, To control the grid nodes, For the The displacement of the nodes, To control the number of nodes in the grid; The registered CBCT images and optical scanning data are fused to generate a comprehensive 3D model, which is expressed as: ; in: CBCT images The pixel value of The midpoint of the optical scan data The pixel value of is the weight of CBCT image, and is the weight of optical scanning data.
5. The CBCT tooth segmentation and modeling algorithm according to claim 1, characterized in that: The method of selecting a deep convolutional neural network suitable for processing 3D medical images and optimizing the network structure for the tooth segmentation task includes: Select 3D U-Net as the basic network structure, add an attention module to the decoder, introduce a residual connection between the encoder and decoder, and fuse feature maps from different scales in the decoder. The expression is: Among them, p i is the predicted probability of the i-th pixel in the predicted segmentation map, g i is the label value of the i-th pixel in the true label image, ∈ is the smoothing term, and w i is the boundary weight of the i-th pixel, which is used to enhance the focus on the boundary, and N is the total number of pixels.
6. A CBCT tooth segmentation and modeling algorithm according to claim 1, characterized in that: The preprocessed and annotated data set is divided into a training set, a validation set and a test set, and a tooth surface reflection and transmission characteristic model is established based on the physical rendering technology and the optical properties of the tooth material, including: The pre-processed and annotated CBCT images and optical scan datasets were prepared; The dataset is randomly divided into training set, validation set and test set, with a ratio of 70% for training, 15% for validation and 15% for testing; Use spectrometers to measure the optical properties of tooth materials, including reflectance, refractive index, and absorption coefficient; According to the measured optical properties, the reflection and transmission characteristic model of the tooth surface is established. The Fresnel equation and Beer-Lambert law are used to describe the reflection characteristics of the tooth surface and the transmission characteristics of the inside of the tooth respectively. The expressions are: ; ; in: is the reflectivity, is the incident angle, is the refractive index of air, is the refractive index of the tooth material, is the transmitted light intensity, is the incident light intensity, is the absorption coefficient, is the distance light travels in tooth material.
7. The CBCT tooth segmentation and modeling algorithm according to claim 1, characterized in that: The method of combining the optical model with the actual measurement data and using the PBR technology to simulate the real color and gloss effect of teeth includes: Use the Phong lighting model combined with PBR technology to calculate the color value of each pixel. The Phong lighting model expression is: ; in: is the final color value, is the diffuse color, is the specular color, is the light source intensity, is the surface normal, is a half-angle vector, is the specular reflection index, and the roughness parameter Related; Map diffuse color, specular color, roughness and metalness parameters to each vertex of the 3D model to generate PBR material texture; According to the PBR material texture and lighting model, the color value of each pixel is calculated to generate the final 3D model.
8. The CBCT tooth segmentation and modeling algorithm according to claim 1, characterized in that: The exporting of the optimized 3D tooth model into a universal format file comprises: Use the Laplacian smoothing algorithm to smooth the surface of the 3D model, and use the normal mapping technology to enhance the details of the model surface; Choose a common 3D model format, such as STL, OBJ or FBX; Set export parameters to ensure the accuracy and integrity of the model; Export the optimized 3D tooth model to the selected universal format file and save it to the specified path.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of a CBCT tooth segmentation and modeling algorithm as described in any one of claims 1 to 8 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a CBCT tooth segmentation and modeling algorithm according to any one of claims 1 to 8 are implemented.