A CBCT tooth instance segmentation method combining global attention and scale awareness

Abstract

This invention discloses a CBCT tooth instance segmentation method combining global attention and scale awareness. After acquiring CBCT images, the original CBCT images are preprocessed and used as input images. These preprocessed images are then fed into a segmentation network combining global attention and scale awareness to obtain the final 3D tooth instance segmentation result. This invention significantly enhances the network's ability to extract detailed features, key features, and multi-scale contextual features, enabling the model to achieve good accuracy and generalization ability in CBCT 3D tooth instance segmentation.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and in particular to a CBCT tooth instance segmentation method that combines global attention and scale awareness. Background Technology

[0002] Cone-beam computed tomography (CBCT) imaging captures 2D images using a flat panel sensor and projection beam, then mathematically converts them into 3D images. Due to its high resolution and ability to create 3D models of the patient's skull, CBCT imaging is extremely useful in orthodontic treatment. It is commonly used to plan the placement of orthodontic implants and micro-implants, determine the location of impacted, supernumerary, and additional teeth, assess root resorption, and evaluate the thickness of the alveolar bone in the maxilla and mandible.

[0003] Computer-aided design (CAD) has been widely applied in digital dentistry for diagnosis, restoration, and orthodontic treatment planning. In these processes, three-dimensional tooth models are typically segmented from cone-beam computed tomography (CBCT) scans. Previously, in clinical practice, dentists had to manually mark each tooth slice by slice from CBCT images, a time-consuming, labor-intensive process highly dependent on the operator's experience. Therefore, accurately and automatically segmenting individual teeth from CBCT images has become a crucial requirement. However, automated tooth segmentation remains a challenging task. The first challenge is the varying tooth alignment among patients, and the second is the diverse morphologies among different tooth categories.

[0004] The patent title is "A CBCT Tooth Image Segmentation Method Based on Center Point Detection," publication number CN110889850B, authorization date July 22, 2022. The patent describes a CBCT tooth image segmentation method based on center point detection. This method first uses the detected center point information to perform initial coarse segmentation of the tooth CBCT image, and then uses the coarse segmentation results to replace manual initialization, achieving a fully automatic "expert"-guided image segmentation. Furthermore, a dual-level set segmentation algorithm and threshold optimization constraint processing are introduced during the segmentation process.

[0005] The patent title is "An End-to-End Interactive 3D CBCT Tooth Image Segmentation Algorithm," application number CN112614127A, publication date 2021-04-06. The patent describes an end-to-end interactive 3D CBCT tooth image segmentation algorithm. The method first inputs a 3D CBCT tooth image, where the user marks the teeth to be segmented within the 3D CBCT image. Next, a 3D ROI image of the tooth is obtained. Then, an end-to-end model for 3D CBCT tooth image segmentation is established. This model utilizes a deep learning neural network (CNN) to extract deep-level features from the acquired 3D ROI image, and then uses a 3D Random Walk algorithm to solve the problem. Prior knowledge of tooth characteristics is incorporated into the loss function to optimize the CNN, resulting in an optimized end-to-end model. Finally, the 3D CBCT tooth image is input into the end-to-end model to obtain the tooth image segmentation result.

[0006] The patent title is "A Deep Learning-Based CBCT Tooth Segmentation Method," authorization number CN112785609B, authorization date June 3, 2022. The patent describes a deep learning-based CBCT tooth segmentation method. This method first preprocesses the cone-beam computed tomography (CBCT) image using prior knowledge of the tooth image to extract the tooth portion, obtaining the region of interest (ROI). Then, a ResNet-FPN network is used to extract features from the image, obtaining a feature map. Next, the CBAM model is used to compress the feature map in both spatial and channel dimensions, thereby encoding the importance of the feature map. Then, an RPN network is used to extract candidate regions from the feature map. Afterwards, ROI Align is used to pool the corresponding regions in the feature map into fixed-size feature maps based on the position coordinates of the pre-selected boxes. Finally, the candidate regions are classified, segmented, bounding box regressed, and segmentation scored.

[0007] Existing technologies for CBCT 3D tooth segmentation using models do not take into account that CBCT images often have abnormal features in real-world applications, such as abnormal tooth alignment, abnormal tooth number, abnormal tooth morphology, and other dental diseases. As a result, existing technologies cannot obtain segmentation results that meet practical needs when dealing with CBCT images containing diseases. Summary of the Invention

[0008] The purpose of this application is to propose a CBCT tooth instance segmentation method that combines global attention and scale awareness, thereby improving the problem of insufficient segmentation accuracy in the existing technology when performing three-dimensional tooth instance segmentation on abnormal CBCT images.

[0009] To achieve the above objectives, the technical solution of this application is as follows:

[0010] A CBCT tooth instance segmentation method combining global attention and scale awareness includes:

[0011] Acquire the raw CBCT images, and use the preprocessed raw CBCT images as input image I;

[0012] The CBCT image I is input into a segmentation network that combines global attention and scale awareness to obtain the segmentation result F of the CBCT image. out .

[0013] Furthermore, the acquisition of raw CBCT images and the application of preprocessing operations to the raw CBCT images include:

[0014] Input a CBCT image, crop the CBCT image to a ROI region containing only teeth, and shrink it to a preset size.

[0015] Furthermore, the CBCT image I is input into a segmentation network that combines global attention and scale awareness to obtain the segmentation result F of the CBCT image. out ,include:

[0016] Step 1.1: Input the CBCT image I into a global attention-based convolutional block to obtain the output feature map F1, which has dimensions C1XD1XH1XW1.

[0017] Step 1.2: Max pool the feature map F1 and then input it into a global attention-based convolutional block to obtain the output feature map F2, which has a dimension of C2XD2XH2XW2.

[0018] Step 1.3: Max pool the feature map F2 and then input it into a global attention-based convolutional block to obtain the output feature map F3, which has dimensions of C3XD3XH3XW3.

[0019] Step 1.4: Perform max pooling on feature map F3, then input it into a scale-aware convolutional block to obtain the output feature map F. deep Its dimensions are C4XD4XH4XW4;

[0020] Step 1.5: Generate the feature map F deep Upsampling is performed, followed by concatenation with feature map F3 channel by channel to obtain a feature map with C4+C3 channels. This is then input into a global attention-based convolutional block to obtain feature map F. up1 Its dimensions are C3XD3XH3XW3;

[0021] Step 1.6: Transfer the feature map F up1Upsampling is performed, followed by concatenation with feature map F2 along channels to obtain a feature map with C3+C2 channels. This feature map is then input into a global attention-based convolutional block to obtain feature map F. up2 Its dimensions are C2XD2XH2XW2;

[0022] Step 1.7: Transfer the feature map F up2 Upsampling is performed, followed by concatenation with feature map F1 channel by channel to obtain a feature map with C2+C1 channels. This feature map is then input into a global attention-based convolutional block to obtain feature map F. up3 Its dimensions are C1XD1XH1XW1;

[0023] Step 1.8: Transfer the feature map F up3 The input is fed into a convolution block with a kernel size of 1x1x1 to obtain the segmentation result F of the CBCT image. out Its dimensions are C5XD1XH1XW1, where C5 represents the number of different categories of C5 teeth.

[0024] Furthermore, the global attention-based convolutional block includes:

[0025] A set of cascaded convolutional layers with a kernel size of 3x3x3, batch normalization layers, activation layers, and a set of global attention modules.

[0026] Furthermore, the global attention module includes:

[0027] The input feature map is processed through a series of operations: a 1x1x1 convolutional layer, an instance normalization layer, and an activation layer; a 3x3x3 convolutional layer, an instance normalization layer, and an activation layer; and a 5x5x5 convolutional layer, an instance normalization layer, and an activation layer. The output feature maps from these three operations are then concatenated by channel to obtain an intermediate feature map. This intermediate feature map is then subjected to one global pooling operation and two 1x1x1 convolutional layers before activation. This activation is then multiplied by the intermediate feature map to obtain a weighted feature map. Finally, the weighted feature map and the intermediate feature map are added together to obtain the final output feature map.

[0028] Furthermore, the scale-aware convolutional block includes:

[0029] A set of cascaded convolutional layers with a kernel size of 3x3x3, batch normalization layers, activation layers, and a set of scale-aware modules.

[0030] Furthermore, the scale-aware module includes:

[0031] The input feature maps are fed into dilated convolutional layers with dilation rates of 1 and 2, respectively. The two feature maps are concatenated and then fed into a set of convolutional blocks with kernel sizes of 1x1x1 and 3x3x3 to obtain the first intermediate feature map. The input feature map is then fed into a dilated convolutional layer with dilation rate of 4. The resulting feature map and the first intermediate feature map are concatenated and then fed into a set of convolutional blocks with kernel sizes of 1x1x1 and 3x3x3 to obtain the second intermediate feature map. The input feature map and the second intermediate feature map are then weighted and added according to a learnable parameter α to obtain the final output feature map.

[0032] This application proposes a CBCT tooth instance segmentation method that combines global attention and scale awareness. On one hand, by incorporating a global attention module, the model can extract features from different receptive fields. Furthermore, a channel attention mechanism is introduced within this module to focus on key feature information, significantly improving the segmentation accuracy of detailed parts and enabling the extraction of valuable feature information even when CBCT morphology varies considerably. On the other hand, by adding a scale awareness module, the model can fuse multi-scale contextual features from high-level feature maps. This allows the model to consistently segment tooth instances of different sizes, positions, and shapes, enhancing its generalization ability and ultimately resulting in more accurate 3D tooth instance segmentation. Attached Figure Description

[0033] Figure 1 This is a flowchart of the CBCT tooth instance segmentation method that combines global attention and scale perception, as described in this application.

[0034] Figure 2 This is a schematic diagram of the network structure in an embodiment of this application;

[0035] Figure 3 This is a schematic diagram of a global attention-based convolutional block in this application;

[0036] Figure 4 This is a schematic diagram of the global attention module in this application;

[0037] Figure 5 This is a schematic diagram of a scale-aware convolutional block according to this application;

[0038] Figure 6 This is a schematic diagram of the scale sensing module of this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0040] In one embodiment, such as Figure 1 As shown, a CBCT 3D tooth instance segmentation method combining global attention and scale awareness is provided, including:

[0041] Step S1: Obtain the original CBCT image and use the preprocessed original CBCT image as input image I.

[0042] This application performs necessary preprocessing on the acquired raw CBCT images. The acquisition of raw CBCT images, and the preprocessing of the raw CBCT images before use as input image I, includes:

[0043] Input the original CBCT image, crop the original CBCT image into a ROI region that only contains teeth, and shrink it to a preset size to obtain input image I.

[0044] It should be noted that this application can also be processed without cropping; the image obtained by shrinking can be used for subsequent processing.

[0045] Step S2: Input the CBCT image I into a segmentation network that combines global attention and scale awareness to obtain the segmentation result F of the CBCT image. out .

[0046] like Figure 2 As shown in the figure, the CBCT image I is input into a segmentation network that combines global attention and scale awareness to obtain the segmentation result F of the CBCT image. out ,include:

[0047] Step 2.1: Input the CBCT image I into a global attention-based convolutional block to obtain the output feature map F1, which has dimensions C1XD1XH1XW1.

[0048] Step 2.2: Max pool the feature map F1 and then input it into a global attention-based convolutional block to obtain the output feature map F2, which has a dimension of C2XD2XH2XW2.

[0049] Step 2.3: Max pool the feature map F2 and then input it into a global attention-based convolutional block to obtain the output feature map F3, which has dimensions of C3XD3XH3XW3.

[0050] Step 2.4: Perform max pooling on feature map F3, then input it into a scale-aware convolutional block to obtain the output feature map F. deep Its dimensions are C4XD4XH4XW4;

[0051] Step 2.5: Generate the feature map F deepUpsampling is performed, followed by concatenation with feature map F3 channel by channel to obtain a feature map with C4+C3 channels. This is then input into a global attention-based convolutional block to obtain feature map F. up1 Its dimensions are C3XD3XH3XW3;

[0052] Step 2.6: Transfer the feature map F up1 Upsampling is performed, followed by concatenation with feature map F2 along channels to obtain a feature map with C3+C2 channels. This feature map is then input into a global attention-based convolutional block to obtain feature map F. up2 Its dimensions are C2XD2XH2XW2;

[0053] Step 2.7: Transfer the feature map F up2 Upsampling is performed, followed by concatenation with feature map F1 channel by channel to obtain a feature map with C2+C1 channels. This feature map is then input into a global attention-based convolutional block to obtain feature map F. up3 Its dimensions are C1XD1XH1XW1;

[0054] Step 2.8: Transfer the feature map F up3 The input is fed into a convolution block with a kernel size of 1x1x1 to obtain the segmentation result F of the CBCT image. out Its dimensions are C5XD1XH1XW1, where C5 represents the number of different categories of C5 teeth.

[0055] The method and technical framework described in this application, such as Figure 2 As shown, in other embodiments, the number of segmentation network layers can be modified.

[0056] The global attention-based convolutional block, such as Figure 3 As shown, it includes a set of cascaded convolutional layers with a kernel size of 3x3x3, batch normalization layers and activation layers, as well as a set of global attention modules.

[0057] The global attention module, such as Figure 4As shown, the process includes performing operations on the input feature map, including a set of convolutional layers with a kernel size of 1X1X1, an instance normalization layer, and an activation layer; a set of convolutional layers with a kernel size of 1X1X1, an instance normalization layer, and an activation layer; a set of convolutional layers with a kernel size of 3X3X3, an instance normalization layer, and an activation layer; and a set of convolutional layers with a kernel size of 1X1X1, an instance normalization layer, and an activation layer; and a set of convolutional layers with a kernel size of 5X5X5, an instance normalization layer, and an activation layer. The feature maps output from these three sets of operations are then concatenated by channel to obtain an intermediate feature map. This intermediate feature map is then subjected to one global pooling operation and two 1X1X1 convolutional layers, activated according to the following formula, and multiplied by the intermediate feature map to obtain a weighted feature map. Finally, the weighted feature map and the intermediate feature map are added together to obtain the final output feature map.

[0058]

[0059] The scale-aware convolutional block, such as Figure 5 As shown, it includes a set of cascaded convolutional layers with a kernel size of 3x3x3, batch normalization layers and activation layers, as well as a set of scale-aware modules.

[0060] The scale-aware module, such as Figure 6 As shown, the process involves inputting the input feature map into dilated convolutional layers with dilation rates of 1 and 2, respectively. The resulting two feature maps are then concatenated and fed into a set of convolutional blocks with kernel sizes of 1x1x1 and 3x3x3 to obtain the first intermediate feature map. The input feature map is then input into a dilated convolutional layer with dilation rate of 4. The resulting feature map and the first intermediate feature map are then concatenated and fed into a set of convolutional blocks with kernel sizes of 1x1x1 and 3x3x3 to obtain the second intermediate feature map. Finally, the input feature map and the second intermediate feature map are weighted and added according to a learnable parameter α to obtain the final output feature map.

[0061] It should be noted that in this application, C represents the number of channels, D represents the depth of the image, H represents the height of the image, W represents the width of the image, and the subscripts of the letters represent the serial numbers used to distinguish the dimensions of different feature maps.

[0062] This application utilizes a global attention module to extract features from different receptive fields and introduces a channel attention mechanism within the module to focus on key feature information, thereby improving the model's ability to segment detailed parts and thus enhancing the segmentation accuracy of teeth. Furthermore, by incorporating a scale-aware module, the model can fuse multi-scale contextual features from high-level feature maps, enabling it to consistently produce correct segmentation results when faced with tooth instances of different sizes, positions, and shapes, thus enhancing the model's generalization ability.

[0063] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A CBCT tooth instance segmentation method combining global attention and scale perception, characterized in that, The CBCT tooth instance segmentation method combining global attention and scale perception comprises: An original CBCT image is acquired, and the original CBCT image is used as an input image I after a pretreatment operation; Input the CBCT image I into the segmentation network combined with global attention and scale perception to obtain the segmentation result of the CBCT image ; The method comprises the following steps: inputting a CBCT image I into a segmentation network combined with global attention and scale perception to obtain a segmentation result of the CBCT image , comprising: Step 1.1, input the CBCT image I into the global attention based convolution block to obtain the output feature map , the dimension size of which is ; Step 1.2, the feature map Max-pooling is performed, and then the output feature map is obtained by inputting to a convolution block based on global attention , and the dimension size is ; Step 1.3, the feature map Max-pooling is performed, and then the output feature map is obtained by inputting the global attention-based convolution block , and the dimension size is ; Step 1.4, the feature map Max-pooling is performed, and then the output feature map is obtained by inputting to a scale-aware convolution block , and the dimension size is ; Step 1.5, the generated feature map is up-sampled, and then concatenated with the feature map ; the channel number of the obtained feature map is ; the feature map is input into a convolution block based on global attention to obtain a feature map with a dimension size of ; Step 1.6, the feature map is up-sampled, and then concatenated with the feature map to obtain a feature map with a channel number of , which is input into a convolution block based on global attention to obtain a feature map with a dimension size of ; Step 1.7, the feature map is up-sampled, and then concatenated with the feature map to obtain a feature map with a channel number of , which is input into a convolution block based on global attention to obtain a feature map with a dimension size of ; Step 1.8, the feature map is input to a convolution block with a kernel size of to obtain a segmentation result of the CBCT image with a dimension size of , where represents the number of categories of different teeth; The scale perception-based convolution block comprises a group of cascaded convolution layers with a kernel size of , a batch normalization layer and an activation layer, and a group of scale perception modules. The scale perception module comprises: The input feature map is respectively input into the dilated convolution layer with the dilated rate of 1 and 2, two obtained feature maps are input into a group of convolution blocks with the convolution kernel size of and after channel splicing, a first intermediate feature map is obtained, the input feature map is input into the dilated convolution layer with the dilated rate of 4, the obtained feature map and the first intermediate feature map are input into a group of convolution blocks with the convolution kernel size of and after channel splicing, a second intermediate feature map is obtained, the input feature map and the second intermediate feature map are weighted and added according to the proportion of a learnable parameter alpha, and a final output feature map is obtained.

2. The CBCT tooth instance segmentation method combining global attention and scale perception of claim 1, wherein, The original CBCT image is acquired, and the original CBCT image is used as an input image I after a pretreatment operation; The scale perception module comprises:

3. The CBCT tooth instance segmentation method combining global attention and scale perception of claim 1, wherein, The global attention-based convolution block includes a set of cascaded convolution layers with a kernel size of , batch normalization layers, and activation layers, and a set of global attention modules.

4. The CBCT tooth instance segmentation method combining global attention and scale perception according to claim 3, characterized in that, The original CBCT image is acquired, and the original CBCT image is used as an input image I after a pretreatment operation; The CBCT image is acquired, the CBCT image is cropped into a ROI region containing only teeth, and is reduced to a preset size. The global attention module comprises: The input feature map is subjected to a set of convolution layers with a convolution kernel size of , an instance normalization layer, and an activation layer, a set of convolution layers with a convolution kernel size of , an instance normalization layer, an activation layer, a convolution layer with a convolution kernel size of , an instance normalization layer, and an activation layer, and a set of convolution layers with a convolution kernel size of , an instance normalization layer, an activation layer, a convolution layer with a convolution kernel size of , an instance normalization layer, and an activation layer, and then the feature maps output by the three sets of operations are spliced by channels to obtain an intermediate feature map, the intermediate feature map is subjected to a global pooling operation once and a convolution layer with a convolution kernel size of twice, and then activated, multiplied by the intermediate feature map to obtain a weight feature map, and the weight feature map and the intermediate feature map are added to obtain a final output feature map.

Images