Method, device and medium for identifying and reconstructing jaw and soft tissue based on CBCT images

By constructing a jaw segmentation network model and using Gaussian filters and Marching Cubes algorithms, the problems of oversegment and undersegment in automatic segmentation of CBCT jaws are solved, the accuracy and efficiency of three-dimensional reconstruction of jaws are improved, and the existing technology is solved The problem of low diagnostic error and reconstruction accuracy.

CN114638852BActive Publication Date: 2024-11-22HANSF (HANGZHOU) MEDICAL TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202210177560.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-25
Publication Date
2024-11-22
Estimated Expiration
2042-02-25

AI Technical Summary

Technical Problem

The existing CBCT jaw automatic segmentation algorithm has problems with oversegment and undersegment, resulting in low diagnostic error and reconstruction accuracy. Especially, algorithms based on deep learning models are difficult to accurately identify jaws when category imbalance and low spatial information utilization rate. Bone area.

Method used

By customizing layer by layer annotation of CBCT data, a jaw segmentation network model was constructed, multi-scale features were extracted using VGG-19 backbone network and FPN neck network, jaw recognition was performed by combining Softmax pixel classifier, and Gaussian filters and Marching Cubes The algorithm performs three-dimensional reconstruction, solving the problems of oversegment and undersegment.

Benefits of technology

It improves the accuracy and efficiency of automatic segmentation and reconstruction of CBCT jaw bones, reduces diagnostic errors, and improves the accuracy and computing efficiency of three-dimensional reconstruction of jaw bones.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114638852B_ABST
    Figure CN114638852B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of image processing technology, specifically, a method, device and medium for identifying and reconstructing jaw and soft tissue based on CBCT images. The jaw part in the CT image is manually marked by an oral expert, and then a CBCT jaw segmentation network model is designed and trained based on an open source deep learning platform TensorFlow and a fully convolutional neural network model; the probability value of each voxel is obtained by outputting the trained model, and the voxels of two connected areas above a set threshold are retained to reconstruct the upper and lower alveolar bones, and finally the maxillofacial soft tissue part is segmented by the threshold of the soft tissue to reconstruct a three-dimensional face model. The present invention can use a deep learning method to quickly and automatically reconstruct a three-dimensional model of the jaw and face in batches, solve the problems of over-segmentation and under-segmentation in CBCT jaw segmentation, and improve the segmentation accuracy while improving the reconstruction efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and specifically relates to a method, a device and a storage medium for identifying and reconstructing jaws and soft tissues based on CBCT images. Background Art

[0002] According to data from the Ministry of Health, the number of oral patients in my country has reached 700 million in 2019, and oral health problems have become an important factor affecting the quality of daily life of residents. With the upgrading of residents' consumption and the increase in oral health awareness, the number of people receiving oral treatment in my country has increased significantly. In particular, medical imaging plays a vital reference role in the diagnosis, treatment and prognosis evaluation of oral and maxillofacial diseases. Compared with traditional two-dimensional radiographic imaging technology, cone beam CT (CBCT) has the advantages of low radiation, high precision, and three-dimensional stereoscopic imaging. Therefore, it is widely used in the examination and diagnosis of oral and maxillofacial diseases such as infections, tumors and cystic diseases in the jaw, and maxillofacial fractures. In addition, doctors can also observe the morphology of the patient's maxillofacial soft tissue through CBCT images. However, dentists can only use CBCT to observe the two-dimensional shape characteristics of the patient's jaw from a cross-section. Due to the lack of three-dimensional structural information of the jaw, diagnostic errors are easily caused. Therefore, dentists urgently need an efficient and accurate CBCT jaw automatic segmentation and reconstruction algorithm.

[0003] In response to this clinical need, some researchers have proposed some effective CBCT jaw automatic segmentation algorithms based on traditional image segmentation models and deep learning, as shown below:

[0004] 1) CBCT jaw segmentation algorithm based on traditional image segmentation model: This type of algorithm uses traditional segmentation algorithms such as threshold method, clustering method, active contour method and graph cut method to segment the jaw area of ​​CBCT section layer by layer, and then stacks the binary segmentation results of each layer of jaw into a three-dimensional segmentation result. Finally, the three-dimensional segmentation result is converted into point cloud data with the help of a three-dimensional surface mesh extraction algorithm. However, this type of algorithm is mainly based on the pixel and texture features of the image, while CBCT images have strong noise interference and the shape and pixel intensity of the jaw are complex. Therefore, the robustness and accuracy of the CBCT jaw segmentation algorithm based on the traditional image segmentation model are low. In addition, the traditional segmentation algorithm usually requires the user to provide prior information to the model in advance, so the operation steps are relatively complicated, which affects the computational efficiency of the algorithm and is not suitable for large-scale processing of CBCT images.

[0005] 2) CBCT jaw segmentation algorithm based on deep learning: This type of algorithm uses a deep network model to extract hierarchical features from CBCT images, including shallow features (pixels, textures, shapes, etc.) and deep features (abstract features, semantic features, etc.) of the image. Then, based on the pixel-level classifier and image pyramid features, the image voxels are classified into jaw and non-jaw categories, thereby obtaining the three-dimensional segmentation results of the jaw. Finally, the three-dimensional segmentation results are converted into point cloud data based on the three-dimensional surface mesh extraction algorithm. Since the recognition ability and robustness of deep learning features are much better than those of manually designed image features, and the deep learning model has an end-to-end structural advantage, it can greatly improve the performance and efficiency of CBCT image automatic segmentation algorithms, and has become the mainstream algorithm in this field. Due to the imbalance of categories in pixel classifiers, this type of algorithm is prone to misclassifying some jaw areas as non-jaw areas, resulting in under-segmentation problems. In addition, some segmentation models based on deep learning have low utilization of image spatial information, resulting in rough boundaries of segmentation results, serious over-segmentation problems, and reduced jaw reconstruction accuracy.

[0006] Therefore, how to improve the accuracy of CBCT jaw automatic segmentation and reconstruction algorithm based on deep learning model has become an urgent problem to be solved. Summary of the invention

[0007] The purpose of this application is to solve and overcome the deficiencies of existing technologies and applications, and to provide a method, device and medium for identifying and reconstructing jaws and soft tissues based on CBCT images, thereby solving the problems of over-segmentation and under-segmentation in CBCT jaw segmentation.

[0008] The purpose of this application is to achieve the following technical solution: a method for identifying and reconstructing jaws and soft tissues based on CBCT images, characterized in that it includes the following steps:

[0009] A. Customize layer-by-layer annotation of CBCT data;

[0010] B. Divide the labeled data set into training set, validation set and test set, and perform data processing;

[0011] C. Construct a jaw segmentation network model and train the model using training set data;

[0012] D. Identify and 3D reconstruct the maxillary and mandibular bones; use the trained jaw segmentation network model to process the CBCT data of the test set to obtain the jaw 3D segmentation result, use the domain search method to mark the 3D connected area in the 3D segmentation result, and identify the maxillary and mandibular bones as two connected areas based on the spatial information of the jaw. Use a Gaussian filter to smooth the surface of the jaw segmentation result, and use the Marching Cubes algorithm to extract the mesh information of the maxillary and mandibular segmentation result surface;

[0013] E. Segment the soft tissue and reconstruct it in three dimensions; statistically analyze the pixel intensity of the non-maxillofacial soft tissue area of ​​all CBCT images in the training set data to obtain the pixel average μ and standard deviation σ of the non-maxillofacial soft tissue, use the threshold method to obtain the segmentation result of the maxillofacial soft tissue, and use the Gaussian filter to smooth the 3D segmentation result of the maxillofacial soft tissue.

[0014] Preferably, the step B specifically comprises:

[0015] B-1) Divide the dataset into three parts through cross-validation: training set, validation set and test set;

[0016] B-2) The CBCT data of the training set were processed using the de-averaging, maximum-minimum normalization method and data augmentation method.

[0017] Preferably, the jaw segmentation network model includes a VGG-19 backbone network, an FPN neck network and a Softmax pixel classifier connected in sequence.

[0018] Preferably, the VGG-19 backbone network structure consists of 16 convolutional layers, 4 maximum pooling layers, 5 batch normalization layers and 3 dropout layers, and is used to extract hierarchical features from CBCT images, wherein the resolution of each layer of features is different, and as the resolution decreases, the abstractness and translation invariance of the features increase;

[0019] The FPN neck network structure consists of 5 upsampling layers, 5 connection layers, 12 convolutional layers, 4 dropout layers and 3 channel and spatial attention modules, which are responsible for fusing the multi-scale features extracted by the VGG-19 network into an image feature pyramid with local and global features, and the resolution of the fused features is consistent with the original image;

[0020] The Softmax pixel classifier consists of a convolution layer with a convolution kernel size of 1x1 and a Softmax function, which converts the output value of the deep convolutional neural network into the category probability value of the pixel.

[0021] Preferably, the step D specifically comprises:

[0022] D-1) Use the trained jaw segmentation network to process the CBCT data of the test set and output the probability value P of each voxel in the CBCT data belonging to the jaw a (x), assuming that the predetermined threshold is t a , then the jaw 3D segmentation result of each subject is: S a (x) = P a (x)>t a ;

[0023] D-2) The 3D connected regions in the 3D segmentation results are marked using the domain search method, and the connected regions are sorted from large to small according to the number of voxels in the connected regions. The algorithm only retains the two largest connected regions and identifies them as two connected regions of the upper and lower jaws based on the spatial information of the jaws;

[0024] D-3) Use a Gaussian filter to smooth the surface of the jaw segmentation result. The degree of smoothing can be changed by setting the value of the smoothing factor δ;

[0025] D-4) Use the Marching Cubes algorithm to extract the mesh information of the maxillary and mandibular segmentation result surface and save it in ".stl" format.

[0026] Preferably, the step E is characterized by:

[0027] E-1) Count the pixel intensities of the non-maxillofacial soft tissue areas of all CBCT images in the training data set to obtain the pixel average μ and standard deviation σ of the non-maxillofacial soft tissue, and then set the segmentation threshold as t b =μ+2σ;

[0028] E-2) Then the threshold method is used to obtain the segmentation result of maxillofacial soft tissue: S t (x)=I(x)>t b ,According to the 3D segmentation results of the upper and lower jaws, the spatial information of the upper and lower boundaries of the jaws in the CBCT image can be calculated, and the soft tissue segmentation results below the lower boundary of the jaw and above the upper boundary of the jaw can be removed;

[0029] E-3) Use Gaussian filter to smooth the 3D segmentation results of maxillofacial soft tissue, and use Marching Cubes algorithm to calculate the mesh information of the surface and save it in ".stl" format.

[0030] The present invention also provides a device for identifying and reconstructing jaws and soft tissues based on CBCT images, comprising a scanning device, which implements any one of the above-mentioned methods for identifying and reconstructing jaws and soft tissues based on CBCT images when executed.

[0031] The present invention also provides a computer-readable storage medium storing a computer program, which can be executed by a computer processor to implement any one of the above-mentioned methods for identifying and reconstructing jaws and soft tissues based on CBCT images.

[0032] Compared with the prior art, this application has at least the following obvious advantages and effects:

[0033] In the present invention, a CBCT jaw and soft tissue segmentation and reconstruction method is provided to solve the over-segmentation and under-segmentation problems existing in CBCT jaw segmentation, thereby improving the reconstruction efficiency and the segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The specific implementation modes of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0035] Figure 1 It is a schematic diagram of the process of the present invention.

[0036] Figure 2 This is a schematic diagram of the segmentation network structure of the present invention.

[0037] Figure 3 Schematic diagram of the structure of the channel and spatial attention module (CSA) of the present invention.

[0038] Figure 4 This is a rendering of the jaw reconstruction result of the present invention.

[0039] Figure 5 This is a rendering of the maxillofacial soft tissue reconstruction results of the present invention. DETAILED DESCRIPTION

[0040] Specific embodiments of the present application are described in conjunction with the accompanying drawings and the following description to teach those skilled in the art how to make and use the best mode of the present application. In order to teach the principles of the application, some conventional aspects have been simplified or omitted. Those skilled in the art should understand that variations from these embodiments fall within the scope of the present application. Those skilled in the art should understand that the following features can be combined in various ways to form multiple variations of the present application. Terms such as "upper", "lower", "left", "right", "middle" and "one" cited in this application are only for the convenience of description, and are not intended to limit the scope of the present invention. Changes or adjustments in their relative relationships should also be regarded as the scope of the present invention without substantial changes in the technical content. Therefore, the present application is not limited to the following specific embodiments, but is only limited by the claims and their equivalents.

[0041] The present invention discloses a method, device and storage medium for segmenting and three-dimensional reconstruction of jaw and soft tissue based on CBCT images. According to the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without creative work belong to the protection scope of the present invention. In order to make the purpose, technical solution and advantages of the present invention clearer, the embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0042] like Figure 1The figure shows a flow chart of the present invention, including data acquisition and annotation: collecting volunteer CBCT data, and hiring professional stomatology experts to annotate the jaw area of ​​the CBCT data layer by layer, where the digital labels of the maxillary and mandibular bones are "1" and "2" respectively, and the rest of the background area is annotated as "0"; model construction and training: the data set is divided into three parts by cross-validation method: training set, validation set and test set, and the CBCT data of the training set is processed by removing the average value, maximum and minimum normalization method and data augmentation method (random shearing, image rotation, image contrast change, etc.) to obtain the database required for network model training; maxillary and mandibular segmentation and reconstruction: based on the open source deep learning platform TensorFlow and the fully convolutional neural network model, the CBCT jaw segmentation network model is designed and trained. The probability value of each voxel is obtained by the output of the trained model, and the voxels of the two connected areas above the set threshold are retained to reconstruct the upper and lower alveolar bones. Jaw soft tissue segmentation and reconstruction: finally, the maxillofacial soft tissue part is segmented by the soft tissue threshold, and the three-dimensional facial model is reconstructed. The present invention can use deep learning methods to quickly and automatically reconstruct jaw and facial three-dimensional models in batches, solve the problems of over-segmentation and under-segmentation in CBCT jaw segmentation, and improve segmentation accuracy while improving reconstruction efficiency.

[0043] In this embodiment Figure 2Figure 1 is a schematic diagram of the segmentation network structure, where Conv represents the convolution layer, BatchNorm represents the batch normalization layer, Concat represents the connection layer, MaxPool represents the maximum pooling layer, Upsampl represents the upsampling layer, Softmax represents the classifier, Dropout represents the discard layer, and CSA represents the channel and spatial attention module. Based on the open source deep learning platform TensorFlow and the fully convolutional neural network model, a CBCT jaw segmentation network model is designed; the model consists of a VGG-19 backbone network, a FPN (Feature Pyramid Network) neck network and a Softmax pixel classifier connected in sequence. The VGG-19 network structure consists of 16 convolutional layers, 4 maximum pooling layers, 5 batch normalization layers and 3 dropout layers, which are used to extract hierarchical features from CBCT images. The resolution of each layer of features is different, and the abstractness and translation invariance of the features increase with the decrease in resolution. The FPN network structure consists of 5 upsampling layers, 5 connection layers, 12 convolutional layers, 4 dropout layers and 3 channel and spatial attention modules, which are responsible for fusing the multi-scale features extracted by the VGG-19 network into an image feature pyramid with local and global features, and the resolution of the fused features is consistent with the original image. The Softmax pixel classifier consists of a convolutional layer with a convolution kernel size of 1x1 and a Softmax function, which can convert the output value of the deep convolutional neural network into the category probability value of the pixel.

[0044] In this embodiment Figure 3 The schematic diagram of the channel and spatial attention module (CSA) is shown in Figure 2. The module consists of two vertically symmetrical information transmission pipelines. The calculation steps of this module are as follows: 1) The spatial weighted pipeline consists of a maximum pooling layer, a convolution layer, a deconvolution layer, and a Sigmoid function in sequence. The input feature with a dimension of H×W×1 is calculated by the maximum pooling layer and the convolution layer to obtain a dimension of The compressed spatial weight tensor whose spatial resolution is compressed to the original The number of channels is compressed to 1, and then the spatial resolution is restored through the calculation of the deconvolution layer, and the spatial weight value of the feature is calculated by the Sigmoid function. The spatial weight value is then multiplied with the original input feature to obtain the spatially weighted feature map; 2) The channel weighted pipeline consists of a global pooling layer, two fully connected layers, and a Sigmoid function. The input feature with a dimension of H×W×1 is converted into a feature with a dimension of The compressed channel weight tensor is then processed by the fully connected layer and the Sigmoid function to obtain an expanded channel weight tensor of dimension 1×1×C, which is then multiplied with the original input feature to obtain a channel-weighted feature map; 3) Channel and spatial weighted feature fusion: Finally, the channel-weighted and spatial-weighted feature maps are fused using pixel-by-pixel addition to output the optimized image features. The advantage of this module is that it enables the segmentation model to automatically analyze the correlation between each channel and spatial position in the feature map and the segmentation task, enhance the importance of the channel and spatial position information with strong correlation, and reduce the redundancy of the features, thereby improving the accuracy of pixel classification and the accuracy of target segmentation.

[0045] After constructing the network structure, the CBCT image and the corresponding alveolar bone markers are input, and the loss function is minimized using the Adam optimization algorithm built into TensorFlow to obtain the optimal model parameters. The loss function of the network is as follows:

[0046]

[0047] Where S(I(x); W) is the probability predicted by the network model that pixel x belongs to the jawbone, G(x) is the true label of pixel x (value is 0 or 1), ε is a bias term to avoid the denominator being 0, I is each slice in the training data CBCT image, W is the parameter of the network model, Ω is the spatial domain of the image, the learning rate is set to 0.0001, and the number of iterations is 1000;

[0048] The trained jaw segmentation network is used to process the CBCT data of the test set, and the probability value P of each voxel in the CBCT data belonging to the jaw is output. a (x), assuming that the predetermined threshold is t a , then the jaw 3D segmentation result of each subject is: S a (x) = P a (x)>t a ; Then, the domain search method is used to mark the 3D connected regions in the 3D segmentation results, and the connected regions are sorted from large to small according to the number of voxels in the connected regions. The algorithm only retains the two largest connected regions and identifies them as the upper and lower mandibular connected regions based on the spatial information of the jaw; then, a Gaussian filter is used to smooth the surface of the jaw segmentation result. The degree of smoothing can be changed by setting the value of the smoothing factor δ; finally, the Marching Cubes algorithm is used to extract the mesh information of the upper and lower mandibular segmentation result surface and save it in ".stl" format, as shown in Figure 4 Shown is the rendering of the jaw reconstruction result of this embodiment.

[0049] Finally, the pixel intensity of the non-maxillofacial soft tissue area of ​​all CBCT images in the training data set is counted in advance to obtain the pixel average μ and standard deviation σ of the non-maxillofacial soft tissue, and then the segmentation threshold can be set as t b =μ+2σ; then the threshold method is used to obtain the segmentation result of maxillofacial soft tissue: S t (x)=I(x)>t b According to the 3D segmentation results of the upper and lower jaws, the spatial information of the upper and lower boundaries of the jaw in the CBCT image can be calculated, and the soft tissue segmentation results below the lower boundary of the jaw and above the upper boundary of the jaw can be removed; finally, the 3D segmentation results of the maxillofacial soft tissue are smoothed using a Gaussian filter, and the mesh information of the surface is calculated using the MarchingCubes algorithm and saved in the ".stl" format, such as Figure 5 Shown is the rendering of the maxillofacial soft tissue reconstruction results of this embodiment.

[0050] It should be noted that the present invention also provides an apparatus for identifying and reconstructing jaws and soft tissues based on CBCT images, including a scanning device, wherein the scanning device, when executed, implements the method for identifying and reconstructing jaws and soft tissues based on CBCT images of the above-mentioned embodiment. The scanning device is preferably an oral CBCT scanning device in this embodiment.

[0051] It should be further explained that the present invention also provides a computer-readable storage medium, which stores a computer program, and the computer program can be executed by a computer processor to implement the above-mentioned embodiment of the method for identifying and reconstructing the jaw and soft tissue based on CBCT images. The computer program can use any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0052] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0053] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0054] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0055] The principle of the present invention is: first, the part of the jaw in the CT image is manually marked by a dental expert, and then based on the open source deep learning platform TensorFlow and the fully convolutional neural network model, a CBCT jaw segmentation network model is designed and trained; the probability value of each voxel is obtained by the trained model output, and the voxels of the two connected areas above the set threshold are retained to reconstruct the upper and lower alveolar bones, and finally the maxillofacial soft tissue part is segmented by the soft tissue threshold to reconstruct the facial three-dimensional model. The present invention can use the deep learning method to quickly and automatically reconstruct the jaw and facial three-dimensional model in batches, solve the over-segmentation and under-segmentation problems in CBCT jaw segmentation, and improve the segmentation accuracy while improving the reconstruction efficiency.

[0056] Since it is easy for those skilled in the art to think of it, any modification, equivalent substitution, improvement, etc. made within the concept and principle of the application should be included in the scope of the claims of this application.

Claims

1. A method for identifying and reconstructing jaw bones and soft tissues based on CBCT images, characterized in that: The following steps are involved: A. Customize layer-by-layer annotation of CBCT data; B. Divide the labeled data set into training set, validation set and test set, and perform data processing; C. Construct a jaw segmentation network model and train the model using training set data; D. Identify and 3D reconstruct the maxillary and mandibular bones; use the trained jaw segmentation network model to process the CBCT data of the test set to obtain the jaw 3D segmentation result, use the domain search method to mark the 3D connected area in the 3D segmentation result, and identify the maxillary and mandibular bones as two connected areas based on the spatial information of the jaw. Use a Gaussian filter to smooth the surface of the jaw segmentation result, and use the Marching Cubes algorithm to extract the mesh information of the maxillary and mandibular segmentation result surface; E. Segment the soft tissue and reconstruct it in 3D; statistically analyze the pixel intensity of the non-maxillofacial soft tissue area of ​​all CBCT images in the training set data to obtain the pixel average μ and standard deviation σ of the non-maxillofacial soft tissue, use the threshold method to obtain the segmentation result of the maxillofacial soft tissue, and use the Gaussian filter to smooth the 3D segmentation result of the maxillofacial soft tissue; The jaw segmentation network model in step C includes a VGG-19 backbone network, an FPN neck network and a Softmax pixel classifier connected in sequence; The VGG-19 backbone network structure consists of 16 convolutional layers, 4 maximum pooling layers, 5 batch normalization layers and 3 dropout layers, which are used to extract hierarchical features from CBCT images. The resolution of each layer of features is different, and the abstractness and translation invariance of the features are enhanced as the resolution decreases. The FPN neck network structure consists of 5 upsampling layers, 5 connection layers, 12 convolutional layers, 4 dropout layers and 3 channel and spatial attention modules, which are responsible for fusing the multi-scale features extracted by the VGG-19 network into an image feature pyramid with local and global features, and the resolution of the fused features is consistent with the original image; The Softmax pixel classifier consists of a convolution layer with a convolution kernel size of 1x1 and a Softmax function, which converts the output value of the deep convolutional neural network into the category probability value of the pixel.

2. The method for identifying and reconstructing jaw bones and soft tissues based on CBCT images according to claim 1, characterized in that: The step B specifically comprises: B-1) Divide the data set into training set, validation set and test set through cross-validation method; B-2) The CBCT data of the training set were processed using the de-averaging, maximum and minimum normalization method, and data augmentation method.

3. The method for identifying and reconstructing jaw bones and soft tissues based on CBCT images according to claim 1, characterized in that: The step D specifically comprises: D-1) Use the trained jaw segmentation network to process the CBCT data of the test set and output the probability value of each voxel in the CBCT data belonging to the jaw , assuming that the predetermined threshold is , then the jaw 3D segmentation result of each subject is: ; D-2) The 3D connected regions in the 3D segmentation results are marked using the domain search method, and the connected regions are sorted from large to small according to the number of voxels in the connected regions. The algorithm only retains the two largest connected regions and identifies them as two connected regions of the upper and lower jaws based on the spatial information of the jaws; D-3) Use Gaussian filter to smooth the surface of jaw segmentation result by setting the smoothing factor The value of changes the degree of smoothing; D-4) Use the Marching Cubes algorithm to extract the mesh information of the maxillary and mandibular segmentation result surface and save it in ".stl" format.

4. The method for identifying and reconstructing jaw bones and soft tissues based on CBCT images according to claim 1, characterized in that: The step E specifically comprises: E-1) Count the pixel intensities of the non-maxillofacial soft tissue areas of all CBCT images in the training dataset to obtain the pixel average value of the non-maxillofacial soft tissue and standard deviation , set the segmentation threshold to ; E-2) Use the threshold method to obtain the segmentation results of maxillofacial soft tissue: ,in is the original voxel value of pixel x in the CBCT image slice of the training data. According to the 3D segmentation results of the upper and lower jaws, the spatial information of the upper and lower boundaries of the jaw in the CBCT image is calculated, and the soft tissue segmentation results below the lower boundary of the jaw and above the upper boundary of the jaw are removed; E-3) Use Gaussian filter to smooth the 3D segmentation results of maxillofacial soft tissue, and use Marching Cubes algorithm to calculate the mesh information of the surface and save it in ".stl" format.

5. A device for identifying and reconstructing jaw bones and soft tissues based on CBCT images, characterized in that: The invention comprises a scanning device, wherein the scanning device implements the method according to any one of claims 1 to 4 when executed.

6. A computer-readable storage medium storing a computer program, characterized in that: The computer program may be executed by a computer processor as computer readable instructions to implement the method as claimed in any one of claims 1 to 4.

Citation Information

Patent Citations

  • Three-dimensional jawbone image segmentation method and device based on CBCT and terminal equipment

    CN112150472A

  • Dental image segmentation and registration with machine learning

    US20200175678A1

  • Classification and 3D modelling of 3D dento-maxillofacial structures using deep learning methods

    US20210150702A1

  • Methods and system for autonomous volumetric dental image segmentation

    US20220012888A1