A CBCT tooth segmentation method based on two-dimensional-three-dimensional hybrid feature fusion and double supervision

Abstract

The present application relates to a kind of CBCT tooth segmentation methods based on two-dimensional-three-dimensional hybrid feature fusion and double supervision, belong to artificial intelligence and digital medical technology field.This method includes obtaining CBCT body data, carries out rough segmentation to dental arch region to extract region of interest, and is cut out to obtain the three-dimensional block of tooth to be segmented;For the three-dimensional block of tooth, generate corresponding two-dimensional auxiliary image;Three-dimensional block is input three-dimensional coding branch, two-dimensional auxiliary image is input two-dimensional coding branch, and the fusion of two-dimensional global structure information and three-dimensional local voxel information is realized by self-attention and bidirectional cross attention;Two-dimensional feature and three-dimensional feature are decoded respectively, and are optimized by the double supervision mechanism of two-dimensional segmentation supervision and three-dimensional segmentation supervision;Multi-class classification strategy and multi-scale supervision strategy are simultaneously used;The three-dimensional segmentation result of target tooth is finally output.The present application improves the accuracy, robustness and boundary fidelity of CBCT tooth segmentation while controlling the consumption of display memory.

Description

Technical Field

[0001] This invention relates to the field of digital medical technology, specifically to a CBCT tooth segmentation method based on two-dimensional-three-dimensional hybrid feature fusion and dual supervision. Background Technology

[0002] Orthodontics is a discipline that corrects dental and maxillofacial deformities through tooth movement and jawbone remodeling. Current digital orthodontics requires digital modeling of teeth for planning and orthodontic purposes. CBCT (cone-beam computed tomography) provides high-resolution 3D visualization of teeth, alveolar bone, and related tissues, serving as a crucial imaging foundation for oral diagnosis, orthodontic planning, computer-aided diagnosis, and root-crown fusion analysis. However, manual tooth segmentation is time-consuming and subject to significant subjective variations, making it difficult to meet the demands of automated and large-scale applications.

[0003] Existing tooth segmentation methods mostly employ local 3D blocks for segmentation. While this can extract local voxel features, it often neglects the global anatomical context, particularly in distinguishing adjacent teeth, recognizing the overall positional relationships of different tooth types, and understanding morphological priors. Directly processing full-resolution 3D volumetric data incurs significant memory and computational overhead. In dentistry, panoramic panoramic radiographs of the entire jaw can effectively represent the overall arrangement, morphology, and contour information of the dentition, but simple 2D projection loses some volumetric details. Furthermore, current techniques typically do not directly integrate such global priors into the core segmentation network in a learnable manner, making it difficult to simultaneously consider both global context and local details under memory constraints.

[0004] Therefore, it is necessary to propose a new CBCT tooth segmentation method that combines global anatomical priors from two-dimensional auxiliary images with local voxel information from three-dimensional blocks to improve the accuracy, boundary quality, and morphological integrity of tooth segmentation. Summary of the Invention

[0005] The purpose of this invention is to address the problem mentioned in the background art that existing CBCT tooth segmentation methods suffer from information loss in three-dimensional local segmentation or two-dimensional projection, while directly processing full three-dimensional data involves large computational loads. This invention provides a CBCT tooth segmentation method based on two-dimensional-three-dimensional hybrid feature fusion and dual supervision. It utilizes panoramic images reconstructed from CBCT data to provide global information on the dental arch, and then supplements this with axial, coronal, and sagittal slices of local three-dimensional tooth blocks. After encoding these two-dimensional information, they are fused with the encoded information of the local three-dimensional tooth data to obtain accurate and stable tooth segmentation results.

[0006] The technical solution adopted by this invention to solve its technical problem is: a CBCT tooth segmentation method based on 2D-3D hybrid feature fusion and dual supervision, using a trained 2D-3D dual-branch segmentation network, which includes a 3D encoding branch, a 2D encoding branch, a cross-modal fusion module, and a decoding branch; including the following steps: S1.1, Obtain the CBCT volume data to be segmented; S1.2, Perform coarse segmentation of the dental arch region on the CBCT volume data, extract the region of interest, and crop to obtain the three-dimensional tooth block to be segmented; S1.3, For the three-dimensional tooth block, a two-dimensional auxiliary image is generated from the CBCT volume data. The two-dimensional auxiliary image includes a panoramic view and three orthogonal slice images in the axial, coronal and sagittal directions. S1.4, The three-dimensional tooth block is input into the three-dimensional encoding branch of the trained two-dimensional-three-dimensional bi-branch segmentation network, and the two-dimensional auxiliary image is input into the two-dimensional encoding branch of the two-dimensional-three-dimensional bi-branch segmentation network; S1.5, at the network bottleneck layer, a bidirectional fusion mechanism is executed through a cross-modal fusion module. The bidirectional fusion mechanism includes two-dimensional self-attention, two-dimensional to three-dimensional cross-attention, and three-dimensional to two-dimensional cross-attention. S1.6 uses a decoding branch to decode the fused two-dimensional and three-dimensional features separately, outputs and fuses them to obtain the final three-dimensional segmentation result.

[0007] Preferably, in S1.3, the panoramic image is generated by maximum intensity projection; the three orthogonal slice images are selected from the slices with the largest average gray value in the corresponding direction.

[0008] Preferably, in S1.5, the cross-modal fusion module specifically performs: self-attention enhancement on the two-dimensional feature map; using the enhanced two-dimensional feature map as a query and the three-dimensional feature map as a key and value, performing two-dimensional to three-dimensional cross-attention; using the three-dimensional feature map as a query and the enhanced two-dimensional feature map as a key and value, performing three-dimensional to two-dimensional cross-attention.

[0009] Preferably, the training of the two-dimensional-three-dimensional dual-branch segmentation network includes the following steps: S2.1, Obtain multiple sets of labeled CBCT body data samples; S2.2, Perform coarse segmentation of the dental arch region for each set of CBCT volume data samples, extract the region of interest, and crop to obtain three-dimensional tooth block samples; S2.3 For each three-dimensional tooth block sample, generate a corresponding two-dimensional auxiliary image sample, which includes a panoramic image sample and three orthogonal slice image samples in the axial, coronal and sagittal directions; S2.4, Construct a 2D-3D bi-branch segmentation network, where: the 3D encoding branch is used to extract local voxel features of 3D tooth block samples; the 2D encoding branch is used to extract global structural features contained in 2D auxiliary image samples; S2.5, a cross-modal fusion module is set up in the network bottleneck layer, and a bidirectional fusion mechanism of "two-dimensional self-attention + two-dimensional to three-dimensional cross attention + three-dimensional to two-dimensional cross attention" is adopted to realize the mutual enhancement of two-dimensional global prior and three-dimensional local details; S2.6 Decodes the fused 2D and 3D features separately, and outputs 2D and 3D segmentation results; uses 2D and 3D labels to perform dual-supervised training on the two branches; S2.7, A three-class segmentation strategy is adopted for the two-dimensional and three-dimensional segmentation results, setting the voxel categories as background, target teeth and other teeth to enhance the ability to distinguish the boundaries of adjacent teeth; at the same time, multi-scale supervision is introduced in the three-dimensional decoding stage to improve the ability to preserve the shape and refine the boundary at different resolutions. S2.8, Design the loss function, and jointly optimize the two-dimensional output, three-dimensional output and boundary-dependent output by combining cross-entropy loss and Dice loss.

[0010] Preferably, the training of the two-dimensional-three-dimensional bi-branch segmentation network is repeated in multiple rounds. In each round, multiple sets of labeled CBCT volume data samples are randomly sorted, and the sorted CBCT volume data samples are used to repeat the cycle of S2.2-S2.8 to train and iterate the two-dimensional-three-dimensional bi-branch segmentation network.

[0011] Preferably, in S2.6, the dual supervision mechanism includes: using two-dimensional labels generated from the labeled CBCT volume data samples to supervise the output of the two-dimensional decoding branch, and using three-dimensional labels to supervise the output of the three-dimensional decoding branch.

[0012] As a preferred embodiment, in S2.7, intermediate segmentation results are output at multiple scales of the 3D decoding branch and compared with the corresponding downsampled ground truth labels to achieve multi-scale supervision.

[0013] Preferably, in S2.3, the panoramic image sample is generated by maximum intensity projection; the three orthogonal slice image samples are selected from the slices with the largest average gray value in the corresponding direction.

[0014] This invention introduces panoramic images and triorthogonal slices as 2D auxiliary inputs, enabling the model to utilize not only local voxel details but also global contextual information such as the overall dental arch arrangement, morphology, and relative positions during 3D tooth segmentation. This improves upon the shortcomings of simple local 3D segmentation in separating adjacent teeth. Furthermore, this invention employs a 2D / 3D bi-branch structure, bidirectional cross-attention fusion, and a dual-supervision mechanism to mutually enhance 2D global priors and 3D local details in the feature space, further improving segmentation robustness and result consistency. Therefore, this invention can effectively improve the accuracy, boundary fidelity, and anatomical morphology rationality of CBCT tooth segmentation with relatively low memory overhead. Attached Figure Description

[0015] The invention will now be further described with reference to the accompanying drawings.

[0016] Figure 1 This is a schematic diagram comparing the results of manual annotation and the segmentation method of this invention. The first row is the gold standard of manual annotation, and the second row is the result of the segmentation method of this invention. Detailed Implementation

[0017] The present invention will be further described below with reference to specific embodiments and accompanying drawings.

[0018] Example 1: A CBCT tooth segmentation method based on 2D-3D hybrid feature fusion and dual supervision, comprising the following steps: S1.1, Obtain the CBCT volume data to be segmented; S1.2, Perform coarse segmentation of the dental arch region on the CBCT volume data, extract the region of interest, and crop to obtain the three-dimensional tooth block to be segmented; S1.3, For the three-dimensional tooth block, a two-dimensional auxiliary image is generated from the CBCT volume data. The two-dimensional auxiliary image includes a panoramic image and three orthogonal slice images in the axial, coronal, and sagittal directions. The panoramic image is generated by maximum intensity projection. The three orthogonal slice images are selected from the slices with the largest average gray value in the corresponding direction. S1.4, The three-dimensional tooth block is input into the three-dimensional encoding branch of the trained two-dimensional-three-dimensional bi-branch segmentation network, and the two-dimensional auxiliary image is input into the two-dimensional encoding branch of the two-dimensional-three-dimensional bi-branch segmentation network; S1.5, at the network bottleneck layer, a bidirectional fusion mechanism is executed through a cross-modal fusion module. The bidirectional fusion mechanism includes two-dimensional self-attention, two-dimensional to three-dimensional cross-attention, and three-dimensional to two-dimensional cross-attention. S1.6 uses a decoding branch to decode the fused two-dimensional and three-dimensional features separately, outputs and fuses them to obtain the final three-dimensional segmentation result.

[0019] The above method uses a pre-trained two-dimensional-three-dimensional bi-branch segmentation network (see Example 2). The two-dimensional-three-dimensional bi-branch segmentation network includes a three-dimensional encoding branch, a two-dimensional encoding branch, a cross-modal fusion module, and a decoding branch.

[0020] Figure 1 A comparison chart showing the segmentation results of this method (second row) and the manually labeled gold standard (first row).

[0021] Example 2: A training method for a two-dimensional-three-dimensional dual-branch segmentation network, comprising the following steps: S2.1, Obtain multiple sets of labeled CBCT volume data samples; in this example, more than 100 sets of clinical CBCT volume data were collected, and manual labeling was used to obtain the manual segmentation gold standard.

[0022] S2.2, Perform coarse segmentation of the dental arch region for each set of CBCT volume data samples, extract the region of interest, and cut out three-dimensional tooth block samples; Perform coarse segmentation of the dental arch region for the CBCT volume data described in step 1), extract the region of interest, and cut out the three-dimensional tooth blocks to be segmented based on the tooth positioning results.

[0023] CBCT volume data can be represented as a four-dimensional tensor. Where R represents the real number field, , , These represent the depth, height, and width of the volume data, respectively. Indicates the number of channels. This represents four-dimensional data in the real number field; for single-channel CBCT grayscale data, it typically has... This is such a case in this example. For any voxel in single-channel volume data, its spatial coordinates can be represented as... The goal of the tooth segmentation task is to assign a class label to each voxel, and the set of classes can be represented as: For each voxel, the category label is as follows: Indicates the number of categories; in this example, These correspond to the background, the target tooth, and other teeth, respectively. Given the input volume data... Tooth segmentation model Output prediction results Its form can be expressed as: Where Y is the set of category labels for all voxels. This represents the model parameters. In the specific preprocessing, the original CBCT volume data is first resampled to unify the spatial resolution across different cases; in this example, each voxel is resampled to... Size is adjusted to achieve uniform resolution. Then, grayscale values ​​are truncated and normalized, with the intensity truncated to... Within the HU range, zero-mean and unit variance normalization is performed to reduce the intensity distribution differences caused by different scanning conditions.

[0024] Subsequently, coarse segmentation of the dental region is performed on the preprocessed CBCT volume data to extract the region of interest (ROI) of the dental arch. Based on the tooth center point or tooth positioning results, a 3D block corresponding to a single tooth is cropped to reduce the memory overhead of the subsequent 3D network and improve the localization targeting.

[0025] S2.3 For each three-dimensional tooth block sample, generate a corresponding two-dimensional auxiliary image sample, which includes a panoramic image sample and three orthogonal slice image samples in the axial, coronal and sagittal directions.

[0026] In this example, the panoramic image is generated using the maximum intensity projection method to reflect the overall dentition alignment, tooth contours, and relative positional relationships. Specifically, for each angular direction... Using the data axis center as a reference, maximum intensity projection is performed along the radial path to obtain the grayscale value at depth d, where the angular direction is taken as... , The number of sampling points on the path, compared with the input data. Correspondingly, this image can provide the network with relative orientation information between the target tooth and adjacent teeth, as well as global structural priors for different tooth types within the dentition.

[0027] Since panoramic images are two-dimensional projections, they suffer from the loss of some volumetric information. Therefore, this embodiment further generates three orthogonal slices in the axial, coronal, and sagittal directions to supplement multiplanar anatomical information. For spatial ranges of... , , The three-dimensional block can be selected by choosing a representative slice in each orthogonal direction. Preferably, the slice with the largest average gray value in the corresponding direction is selected as the auxiliary image for that direction. This slice selection strategy can better preserve the main tooth region and its boundary information, thereby providing more effective two-dimensional supplementary features for subsequent segmentation.

[0028] S2.4 Construct a two-dimensional-three-dimensional bi-branch segmentation network, where: the three-dimensional encoding branch is used to extract local voxel features of three-dimensional tooth block samples; the two-dimensional encoding branch is used to extract global structural features contained in two-dimensional auxiliary image samples.

[0029] This example employs a dual-branch coding structure, with each branch using a U-Net-style encoder. In the i-th level coding stage, the three-dimensional feature map is denoted as: , For the i-th level three-dimensional feature map, These represent the depth, height, width, and number of channels of the encoded 3D feature map, respectively. The feature update after three-dimensional convolutional units can be represented as: Then downsampling is performed to obtain the i-th level output feature: in, for convolution, Use a step size of 2 convolution, For batch normalization, This is the ReLU activation function.

[0030] The two-dimensional branch uses a similar feature extraction method, as shown below: in, for convolution, use Max pooling, For batch normalization, This is the ReLU activation function.

[0031] S2.5 sets up a cross-modal fusion module at the network bottleneck layer, and adopts a two-way fusion mechanism of "two-dimensional self-attention + two-dimensional to three-dimensional cross-attention + three-dimensional to two-dimensional cross-attention" to achieve mutual enhancement of two-dimensional global prior and three-dimensional local details.

[0032] The fusion module includes two-dimensional self-attention and bidirectional cross-attention. First, self-attention enhancement is performed on the two-dimensional features; for the input features... ,pass convolution Generate queries, keys, and values: Where R represents the real number field, , , These represent the depth, height, and width of the volume data, respectively. Indicates the number of channels. QKV represents the processing result, obtained by rearranging and splitting it. matrix: ,in The two-dimensional self-attention output can be represented as: in, for The matrix transpose. The softmax function normalizes the result, and the reshape function rearranges the result back into a two-dimensional input. The shape. This step is used to enhance the global structural pattern in the 2D input, especially the continuous shape of the dental arch and the relative layout pattern of the teeth.

[0033] Subsequently, cross-attention fusion is performed on the two-dimensional and three-dimensional features. Let the two-dimensional features provide the query vector. Three-dimensional features provide key vectors Sum value vector Then we have: , , This corresponds to cross attention: in, , , The learnable parameter matrix is ​​initially initialized with random numbers and then modified and adjusted during training. The dimension of the key vector.

[0034] S2.6 Decodes the fused 2D and 3D features separately, and outputs 2D and 3D segmentation results; uses 2D and 3D labels to perform dual-supervised training on the two branches; During the training phase, the 2D and 3D branches are jointly optimized, forming dual supervision. The 3D branch provides more accurate local spatial information of the target tooth, helping the 2D branch distinguish the target tooth from adjacent teeth; the 2D branch provides prior information on the global shape, boundaries, and relative positions of the target tooth, helping the 3D branch produce segmentation results that better conform to the anatomical structure. The panoramic view mainly provides the overall arrangement of the dental arch and the spatial relationship between adjacent teeth, while the triorthogonal slices compensate for the lack of volumetric information in the panoramic view.

[0035] S2.7, A three-class segmentation strategy is adopted for the two-dimensional and three-dimensional segmentation results, and the voxel categories are set as background, target teeth and other teeth; at the same time, multi-scale supervision is introduced in the three-dimensional decoding stage; In this embodiment, to enhance training stability and output consistency, multi-scale supervision is applied to the outputs at multiple scales of the 3D decoding branch. That is, intermediate segmentation results are output at each scale and compared with ground truth labels sampled at the same scale, thereby achieving progressive boundary refinement from coarse to fine. The form is as follows: in, Indicates the first The predicted output at each scale, This represents the scale label obtained after downsampling the ground truth label using methods such as nearest neighbor interpolation. and These are cross-entropy loss and Dice loss, respectively. This represents the loss function calculated at a certain scale across multiple scales in the 3D decoding branch. Indicates the total number of pixels or voxels. Indicates the number of categories. To prevent smooth terms with a denominator of zero.

[0036] S2.8, Design the loss function, and jointly optimize the two-dimensional output, three-dimensional output and boundary-dependent output by combining cross-entropy loss and Dice loss.

[0037] The total loss function can be expressed as the sum of the 3D segmentation result, the 3D boundary, and the 2D segmentation result: in, The loss for 3D segmentation can be further written as a multi-scale sum. , This indicates the number of scales involved in the supervision. The total loss constrains not only the 3D principal segmentation results but also the 2D auxiliary results and boundary-related results, thus taking into account region overlap, boundary quality, and morphological consistency during the optimization process.

[0038] In this example, the training of the 2D-3D bi-branch segmentation network is repeated more than 3000 times. In each round, multiple sets of labeled CBCT volume data samples are randomly sorted, and the sorted CBCT volume data samples are used to repeat the S2.2-S2.8 loop to train and iterate the 2D-3D bi-branch segmentation network.

[0039] The processing steps S1.2-1.6 of the CBCT volume data to be segmented in Example 1 are the same as the processing steps S2.2-2.6 of the CBCT volume data sample modeling in Example 2, only the objects are different.

Claims

1. A CBCT tooth segmentation method based on 2D-3D hybrid feature fusion and dual supervision, characterized in that, Using a pre-trained 2D-3D dual-branch segmentation network, which includes a 3D encoding branch, a 2D encoding branch, a cross-modal fusion module, and a decoding branch, the following steps are included: S1.1, Obtain the CBCT volume data to be segmented; S1.2, Perform coarse segmentation of the dental arch region on the CBCT volume data, extract the region of interest, and crop to obtain the three-dimensional tooth block to be segmented; S1.3, For the three-dimensional tooth block, a two-dimensional auxiliary image is generated from the CBCT volume data. The two-dimensional auxiliary image includes a panoramic view and three orthogonal slice images in the axial, coronal and sagittal directions. S1.4, The three-dimensional tooth block is input into the three-dimensional encoding branch of the trained two-dimensional-three-dimensional bi-branch segmentation network, and the two-dimensional auxiliary image is input into the two-dimensional encoding branch of the two-dimensional-three-dimensional bi-branch segmentation network; S1.5, at the network bottleneck layer, a bidirectional fusion mechanism is executed through a cross-modal fusion module. The bidirectional fusion mechanism includes two-dimensional self-attention, two-dimensional to three-dimensional cross-attention, and three-dimensional to two-dimensional cross-attention. S1.6 uses a decoding branch to decode the fused two-dimensional and three-dimensional features separately, outputs and fuses them to obtain the final three-dimensional segmentation result.

2. The CBCT tooth segmentation method based on two-dimensional-three-dimensional hybrid feature fusion and dual supervision according to claim 1, characterized in that, In S1.3, the panoramic image is generated by maximum intensity projection; the three orthogonal slice images are selected from the slices with the largest average gray value in the corresponding direction.

3. The CBCT tooth segmentation method based on two-dimensional-three-dimensional hybrid feature fusion and dual supervision according to claim 1, characterized in that, In S1.5, the cross-modal fusion module specifically performs the following: self-attention enhancement on the two-dimensional feature map; and cross-attention from two-dimensional to three-dimensional is performed using the enhanced two-dimensional feature map as a query and the three-dimensional feature map as a key and value. The 3D feature map is used as the query, and the enhanced 2D feature map is used as the key and value, to perform 3D-to-2D cross-attention.

4. The CBCT tooth segmentation method based on two-dimensional-three-dimensional hybrid feature fusion and dual supervision according to claim 1, characterized in that, The training of the two-dimensional-three-dimensional dual-branch segmentation network includes the following steps: S2.1, Obtain multiple sets of labeled CBCT body data samples; S2.2, Perform coarse segmentation of the dental arch region for each set of CBCT volume data samples, extract the region of interest, and crop to obtain three-dimensional tooth block samples; S2.3 For each three-dimensional tooth block sample, generate a corresponding two-dimensional auxiliary image sample, which includes a panoramic image sample and three orthogonal slice image samples in the axial, coronal and sagittal directions; S2.4, Construct a 2D-3D bi-branch segmentation network, where: the 3D encoding branch is used to extract local voxel features of 3D tooth block samples; the 2D encoding branch is used to extract global structural features contained in 2D auxiliary image samples; S2.5, a cross-modal fusion module is set up in the network bottleneck layer, and a bidirectional fusion mechanism of "two-dimensional self-attention + two-dimensional to three-dimensional cross attention + three-dimensional to two-dimensional cross attention" is adopted to realize the mutual enhancement of two-dimensional global prior and three-dimensional local details; S2.6 Decodes the fused 2D and 3D features separately, and outputs 2D and 3D segmentation results; uses 2D and 3D labels to perform dual-supervised training on the two branches; S2.7, a three-class segmentation strategy is adopted for the two-dimensional and three-dimensional segmentation results, and the voxel categories are set as background, target teeth and other teeth; at the same time, multi-scale supervision is introduced in the three-dimensional decoding stage. S2.8, Design the loss function, and jointly optimize the two-dimensional output, three-dimensional output and boundary-dependent output by combining cross-entropy loss and Dice loss.

5. The CBCT tooth segmentation method based on two-dimensional-three-dimensional hybrid feature fusion and dual supervision according to claim 4, characterized in that, The training of the two-dimensional-three-dimensional bi-branch segmentation network is repeated multiple times. In each round, multiple sets of labeled CBCT volume data samples are randomly sorted, and the sorted CBCT volume data samples are used to repeat the S2.2-S2.8 loop to train and iterate the two-dimensional-three-dimensional bi-branch segmentation network.

6. The CBCT tooth segmentation method based on two-dimensional-three-dimensional hybrid feature fusion and dual supervision according to claim 4, characterized in that, In S2.6, the dual supervision mechanism includes: using two-dimensional labels generated from the labeled CBCT volume data samples to supervise the output of the two-dimensional decoding branch, and using three-dimensional labels to supervise the output of the three-dimensional decoding branch.

7. The CBCT tooth segmentation method based on two-dimensional-three-dimensional hybrid feature fusion and dual supervision according to claim 4, characterized in that, In S2.7, intermediate segmentation results are output at multiple scales in the 3D decoding branch and compared with the corresponding downsampled ground truth labels to achieve multi-scale supervision.

8. The CBCT tooth segmentation method based on two-dimensional-three-dimensional hybrid feature fusion and dual supervision according to claim 4, characterized in that, In S2.3, the panoramic image sample is generated by maximum intensity projection; the three orthogonal slice image samples are selected from the slices with the largest average gray value in the corresponding direction.

Images