Oral cavity CBCT image classification method and device, electronic equipment and storage medium

Abstract

The invention discloses an oral cavity CBCT image classification method and device, electronic equipment and a storage medium, and the method comprises the steps: inputting a to-be-classified oral cavity CBCT image into an oral cavity CBCT image processing model, so as to enable the oral cavity CBCT image processing model to generate segmentation information in a segmentation sub-model, and determining a classification image corresponding to the to-be-classified oral cavity CBCT image in the classification sub-model according to the segmentation information. Wherein the segmentation sub-model improves a SwinUnet network model, and adds a residual module to improve the learning ability of the network model; the classification sub-model improves an original 3D-Unet network model, and an attention mechanism module is added to enable the network to selectively learn features of a key area. By implementing the method, the specific structure category of the oral cavity CBCT image can be clearly divided, and the accuracy of a classification result is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an oral CBCT image classification method, device, electronic equipment and storage medium. Background Art

[0002] Cone beam computed tomography (CBCT) is an advanced three-dimensional imaging technology that uses special imaging equipment to obtain detailed three-dimensional images of the patient's oral area. Compared with traditional two-dimensional imaging technology, CBCT can provide higher spatial resolution and richer anatomical details. CBCT uses a rotating X-ray beam and detector system to scan the oral area layer by layer to generate accurate three-dimensional images. This technology can not only clearly display teeth and jaws, but also show alveolar canals, maxillary sinuses, soft tissues and other internal structures of the oral cavity. Due to its superior image quality and three-dimensional visualization capabilities, CBCT has become an indispensable imaging tool in oral implants, orthodontics, maxillofacial surgery and other dental fields.

[0003] In the clinical diagnosis of oral CBCT images, traditional methods mainly rely on manual image analysis and interpretation. Due to the complex structure of the oral region, different anatomical parts in the image often overlap with each other, resulting in a cumbersome and error-prone manual classification process, making it difficult to ensure the accuracy of the classification results. With the continuous development and maturity of deep learning technology, compared with traditional oral CBCT image classification methods, image classification models based on deep learning have higher efficiency, accuracy and robustness. The prior art publicly uses the original 3D-Unet network model to automatically segment teeth in CBCT images. However, the jump connection structure of the original 3D-Unet network model has certain defects. In the direct connection between the encoder and the decoder, when the low-level features output by the encoder are fused with the high-level features of the decoder, there is a semantic gap, which may introduce irrelevant information and reduce the accuracy of the oral CBCT image classification results. Summary of the invention

[0004] The embodiment of the present invention provides an oral CBCT image classification method, device, electronic device and storage medium. By implementing the present invention, various types of teeth in CBCT images can be automatically identified and classified, significantly improving the classification accuracy of oral CBCT images.

[0005] An embodiment of the present invention provides an oral CBCT image classification method, comprising: Acquire oral CBCT images to be classified; Inputting the oral CBCT image into an oral CBCT image processing model, so that the image processing model generates segmentation information in a built-in segmentation sub-model, and determines a classification image corresponding to the oral CBCT image according to the segmentation information in a built-in classification sub-model; The classification sub-model includes several layers, and each layer except the first and last layers is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification decoder. The first layer is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification output layer, and the last layer is provided with a classification decoder; The classification encoder at the first layer is used to extract the features of the segmentation information by downsampling to generate the downsampled features of the second layer; the classification encoders at the remaining layers are used to downsample the downsampled features of the previous layer to generate the downsampled features of the next layer; wherein the downsampled features of the first layer are the segmentation information; The attention mechanism module is used to perform feature extraction, attention generation and attention weighting on the downsampled features of this layer and the convolutional features of the next layer to generate weighted features of this layer; The feature splicing module is used to perform feature splicing and convolution processing according to the weighted features of this layer and the upsampled features of this layer to generate fusion features of this layer; The classification decoder at the last layer is used to perform convolution processing on the downsampled features of the last layer to generate the convolution features of this layer; upsample the convolution features to generate the upsampled features of the previous layer; the classification decoders at the remaining layers are used to perform convolution processing on the fusion features of this layer to generate the convolution features of this layer; upsample the convolution features of this layer to generate the upsampled features of the previous layer; The classification output layer is used to generate a classification image corresponding to the oral CBCT image according to the convolutional features of the first layer.

[0006] Furthermore, the segmentation sub-model includes several layers, and each layer except the first layer and the last layer is provided with a feature extraction module, a first residual module, a second residual module, a feature merging module and a segmentation decoder, the first layer is provided with a first residual module, a feature concatenation module, a second residual module and a segmentation output layer, the last layer is provided with a feature extraction module and a second residual module and a segmentation decoder, and the second layer is also provided with an image processing module; The image processing module is used to perform image segmentation and linear encoding operations according to the oral CBCT images to generate initial features; The feature extraction module located in the second layer is used to perform feature extraction operations and patch merging operations based on the initial features to generate integrated features of the second layer; the feature extraction modules in the remaining layers are used to perform feature extraction operations and patch merging operations based on the integrated features of the previous layer to generate integrated features of the current layer; The first residual module at the first layer is used to generate the first residual features of the first layer according to the oral CBCT image; the first residual modules at other levels are used to generate the first residual features of the current layer according to the integrated features of the current layer; The feature merging module is used to generate a merged feature of this layer according to the first residual feature of this layer and the upsampled feature of this layer; The second residual module located in the last layer is used to perform a convolution operation based on the integrated features of the last layer to generate the second residual features of the last layer; the second residual modules of the remaining layers are used to perform a convolution operation based on the merged features of this layer to generate the second residual features of this layer; The segmentation decoder is used to generate upsampled features of the previous layer according to the second residual features of the current layer; The segmentation output layer is used to generate segmentation information corresponding to the oral CBCT image according to the second residual feature of the first layer; Among them, the structure of the segmentation sub-model is the SwinUnet network structure containing the SwinTransformer module.

[0007] Furthermore, the training of the oral CBCT image processing model includes: Acquire multiple original oral CBCT images for making a data set; Annotating anatomical structures and tooth positions in a plurality of original oral CBCT images to obtain corresponding annotated oral CBCT images; wherein the anatomical structures include: mandible, maxillary bone, mandibular nerve canal, dental bridge, dental crown, implant, etc.; Generate an oral CBCT image dataset according to a plurality of original oral CBCT images and their corresponding annotated oral CBCT images; The oral CBCT image processing model is obtained by training according to the oral CBCT image dataset.

[0008] Furthermore, the attention mechanism module includes: a convolution layer, a normalization layer and a ReLU activation function layer; The attention mechanism module is used to perform feature extraction operation on the downsampled features of the current layer and the convolutional features of the next layer, including: The down-sampled features of the current layer are extracted according to the convolution layer and the normalization layer to generate the first feature; Extract the convolutional features of the next layer according to the convolutional layer and the normalization layer to generate the second feature; The first feature and the second feature are fused to generate a third feature; The third feature is input into the ReLU activation function layer for processing to generate the first activation feature.

[0009] Furthermore, the attention mechanism module further includes: a Sigmoid activation function layer; The attention mechanism module generates attention in the following way: Performing feature extraction on the first activation feature according to the convolution layer and the normalization layer to generate a second activation feature; The second activation feature is input into the Sigmoid activation function layer for processing to generate attention weights.

[0010] Furthermore, the attention mechanism module performs attention weighting in the following manner: The attention weights are applied to the downsampled features of this layer to generate weighted features of this layer.

[0011] Based on the above method embodiment, the present invention provides a corresponding device embodiment.

[0012] An embodiment of the present invention provides an oral CBCT image classification device, comprising: an image acquisition module and an image classification module; The image acquisition module is used to acquire the oral CBCT image to be classified; The image classification module is used to input the oral CBCT image into the oral CBCT image processing model, so that the image processing model generates segmentation information in a built-in segmentation sub-model, and determines a classification image corresponding to the oral CBCT image according to the segmentation information in the built-in classification sub-model; Wherein, the oral CBCT image processing model includes a segmentation sub-model for extracting segmentation information and a classification sub-model for further determining the oral structure category; The classification sub-model includes several layers, and each layer except the first and last layers is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification decoder. The first layer is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification output layer, and the last layer is provided with a classification decoder; The classification encoder at the first layer is used to extract the features of the segmentation information by downsampling to generate the downsampled features of the second layer; the classification encoders at the remaining layers are used to downsample the downsampled features of the previous layer to generate the downsampled features of the next layer; wherein the downsampled features of the first layer are the segmentation information; The attention mechanism module is used to extract features, generate attention and weight attention on the downsampled features of this layer and the convolutional features of the next layer to generate weighted features of this layer; The feature splicing module is used to perform feature splicing and convolution processing according to the weighted features of this layer and the upsampled features of this layer to generate fusion features of this layer; The classification decoder at the last layer is used to perform convolution processing on the downsampled features of the last layer to generate the convolution features of this layer; upsample the convolution features to generate the upsampled features of the previous layer; the classification decoders at the remaining layers are used to perform convolution processing on the fusion features of this layer to generate the convolution features of this layer; upsample the convolution features of this layer to generate the upsampled features of the previous layer; The classification output layer is used to generate a classification image corresponding to the oral CBCT image according to the convolutional features of the first layer.

[0013] Furthermore, the oral CBCT image classification device further includes: a model training module; The model training module is used to obtain a plurality of original oral CBCT images for preparing a data set; Annotating anatomical structures and tooth positions in a plurality of original oral CBCT images to obtain corresponding annotated oral CBCT images; wherein the anatomical structures include: mandible, maxillary bone, mandibular nerve canal, dental bridge, dental crown, implant, etc.; Generate an oral CBCT image dataset according to a plurality of original oral CBCT images and their corresponding annotated oral CBCT images; The oral CBCT image processing model is obtained by training according to the oral CBCT image dataset.

[0014] Based on the above method embodiment, the present invention provides a corresponding electronic device embodiment.

[0015] An embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the oral CBCT image classification method described in any one of the above method embodiments can be implemented.

[0016] Based on the above method item embodiments, the present invention provides a corresponding storage medium item embodiment.

[0017] An embodiment of the present invention provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the oral CBCT image classification method in the above method embodiment can be implemented.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The present invention inputs the oral CBCT image to be classified into the oral CBCT image segmentation model, so that the oral CBCT image segmentation model segments the oral CBCT image to be classified and obtains segmentation information; the segmentation information at this time can clearly identify all target anatomical regions, unify the target anatomical structure into one category, and remove the background and all the voxels of irrelevant regions; and then inputs the segmentation information into the oral CBCT image processing model, so that the oral CBCT image processing model determines the classification image of the oral CBCT image corresponding to the oral CBCT image to be classified according to the segmentation information; wherein the oral CBCT image processing model includes a segmentation sub-model for extracting segmentation information and a classification sub-model for further determining the category of the oral structure; the classification sub-model transforms the original 3D-Unet network model, adds an attention mechanism module, including an encoder, an attention mechanism module, a feature splicing module, a decoder and an output layer; after adding the attention mechanism module, the classification sub-model can weight the features, so that the network can more selectively learn the features of the key areas. wherein the attention generation operation can calculate and generate attention weights, and these weights are used to represent the importance of different regions. The attention weighted operation can adjust the weight of feature information in task-critical areas, while filtering or weakening feature information in irrelevant areas. By increasing the weight of feature information in task-critical areas and suppressing the weight of classification-irrelevant areas, the attention mechanism module can guide the network to better learn task-related features, thereby clearly dividing the specific structural categories of oral CBCT images and ensuring the accuracy of classification results. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flowchart of an oral CBCT image classification method provided by an embodiment of the present invention.

[0020] Figure 2 It is a structural diagram of an oral CBCT image processing model provided by an embodiment of the present invention.

[0021] Figure 3 It is a structural diagram of a segmentation sub-model in an oral CBCT image processing model provided by an embodiment of the present invention.

[0022] Figure 4 It is a structural diagram of the feature extraction layer in the segmentation sub-model provided by an embodiment of the present invention.

[0023] Figure 5 It is a structural diagram of the residual module in the segmentation sub-model provided by an embodiment of the present invention.

[0024] Figure 6 It is a structural diagram of a classification sub-model in an oral CBCT image processing model provided by an embodiment of the present invention.

[0025] Figure 7 It is a structural diagram of the attention mechanism module of the classification sub-model provided in one embodiment of the invention.

[0026] Figure 8 It is a structural schematic diagram of an oral CBCT image classification device provided by an embodiment of the invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] like Figure 1 As shown, an embodiment of the present invention provides an oral CBCT image classification method, which at least includes the following steps: Step S1, obtaining an oral CBCT image to be classified.

[0029] The oral CBCT images to be classified refer to image files acquired from oral CBCT images and having uniform sizes.

[0030] Step S2, inputting the oral CBCT image to be classified into the oral CBCT image processing model, so that the oral CBCT image processing model generates segmentation information in a built-in segmentation sub-model, and determines a classification image corresponding to the oral CBCT image to be classified according to the segmentation information in the built-in classification sub-model; It can be understood that the oral CBCT image processing model includes a segmentation sub-model for extracting segmentation information and a classification sub-model for further determining the category of oral structure; the corresponding classification image contains different anatomical structures and teeth with tooth position information; wherein the anatomical structures include: mandible, maxilla, mandibular nerve canal, dental bridge, crown, implant, etc.; teeth with tooth position information include: incisors, canines, premolars, and molars, wherein the tooth position information refers to the quadrant and tooth sequence number of the tooth.

[0031] It should be noted that the teeth are located by four quadrants and numbers. Each quadrant has 8 teeth, and the numbers are arranged from the midline to the back. The upper right quadrant is numbered 11 to 18, the upper left is 21 to 28, the lower left is 31 to 38, and the lower right is 41 to 48. The teeth in each quadrant are the same type. The first two are incisors, the third is canine, the fourth and fifth are premolars, and the last three are molars. Among them, the third molar (number 8) is a wisdom tooth.

[0032] In an optional embodiment, the structure of the oral CBCT image segmentation model is as follows: Figure 2 The oral CBCT image processing model includes a segmentation sub-model for extracting segmentation information and a classification sub-model for further determining the oral structure category; Specifically, in a preferred embodiment, the structure of the segmentation sub-model is as follows: Figure 3 shown.

[0033] The segmentation sub-model includes several layers, and each layer except the first layer and the last layer is provided with a feature extraction module, a first residual module, a second residual module, a feature merging module and a segmentation decoder. The first layer is provided with a first residual module, a feature splicing module, a second residual module and a segmentation output layer, and the last layer is provided with a feature extraction module, a second residual module and a segmentation decoder, and the second layer is also provided with an image processing module; The image processing module is used to perform image segmentation and linear encoding operations according to the oral CBCT images to generate initial features; The feature extraction module located in the second layer is used to perform feature extraction operations and patch merging operations based on the initial features to generate integrated features of the second layer; the feature extraction modules in the remaining layers are used to perform feature extraction operations and patch merging operations based on the integrated features of the previous layer to generate integrated features of the current layer; The first residual module at the first layer is used to generate the first residual features of the first layer according to the oral CBCT image; the first residual modules at other levels are used to generate the first residual features of the current layer according to the integrated features of the current layer; The feature merging module is used to generate a merged feature of this layer according to the first residual feature of this layer and the upsampled feature of this layer; The second residual module located in the last layer is used to perform a convolution operation based on the integrated features of the last layer to generate the second residual features of the last layer; the second residual modules of the remaining layers are used to perform a convolution operation based on the merged features of this layer to generate the second residual features of this layer; The segmentation decoder is used to generate upsampled features of the previous layer according to the second residual features of the current layer; The segmentation output layer is used to generate segmentation information corresponding to the oral CBCT image according to the second residual feature of the first layer; Among them, the structure of the segmentation sub-model is the SwinUnet network structure containing the SwinTransformer module.

[0034] It should be noted that the image blocking operation of the image processing module divides the oral CBCT image into several local blocks of preset sizes, and the linear coding operation flattens each local block into a one-dimensional feature vector through a linear mapping layer; through image blocking and linear coding operations, the oral CBCT image segmentation model not only uses these one-dimensional feature vectors as initial feature representations, but also reduces the computational burden, provides rich input features for subsequent complex feature extraction, helps to capture more detailed local features, and avoids the high computational cost required for full image processing.

[0035] The structure of the feature extraction layer of the segmentation sub-model is as follows Figure 4 As shown, it includes: a fixed window multi-head attention layer and a sliding multi-head attention layer; Among them, the fixed window multi-head attention layer operates based on the fixed window self-attention mechanism, which divides the features into several fixed-size windows and performs self-attention operations in each window. This mechanism effectively captures local feature information while greatly reducing the computational complexity of global self-attention. By performing calculations only in local windows, the fixed window self-attention mechanism reduces computational overhead and improves the model's ability to process large-scale image data. Fine-grained local feature information can provide an accurate basis for the subsequent feature fusion process.

[0036] The sliding window multi-head attention layer operates based on the sliding window self-attention mechanism, and achieves cross-window feature fusion by translating the window between blocks. By exchanging information between adjacent windows, the sliding window multi-head attention layer can capture global context information across windows and integrate features of different scales. This mechanism enhances the model's ability to capture global features, making the feature representation more comprehensive and accurate, and further improving the model's performance in complex medical image processing.

[0037] The patch merging layer of the segmentation sub-model gradually merges multiple patches divided by the feature map to integrate more contextual information and extract higher-level feature representations. By reducing the spatial resolution of the feature map, patch merging enables the features at each position to cover a larger image area, which not only reduces the amount of computation but also captures a wider range of contextual information. In addition, by integrating features from different positions, patch merging improves the segmentation sub-model's ability to fuse feature information from different regions, ensuring high-quality feature expression.

[0038] The structure of the residual module of the segmentation sub-model is as follows Figure 5 As shown in the figure, by adding the input features to the output features, direct information propagation is achieved, thereby improving the stability of feature transfer and the learning efficiency. The low-level local feature information and the high-level global feature information are effectively integrated to ensure that the segmentation effect of the oral CBCT image segmentation model is accurate and complete.

[0039] The segmentation output layer of the segmentation sub-model is used to generate the final segmentation information. The segmentation output layer uses a single-channel convolutional layer to output a three-dimensional feature map of the same size as the input image. Each voxel value in the feature map is used to indicate whether the location belongs to the target anatomical region. Specifically, the oral CBCT image segmentation model combines all anatomical structures of interest, such as teeth and mandibles, maxilla, mandibular nerve canals, dental bridges, crowns, and implants, into one category, and distinguishes the target area from the background area by binarization. The segmentation information generated by the output layer can clearly identify all target anatomical regions while eliminating voxels in the background and all irrelevant areas.

[0040] Specifically, in a preferred embodiment, the structure of the classification sub-model is as follows: Figure 6 As shown, it includes several layers, including several layers, each layer except the first layer and the last layer is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification decoder, the first layer is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification output layer, and the last layer is provided with a classification decoder; The classification encoder at the first layer is used to extract the features of the segmentation information through downsampling operations to generate the downsampled features of the second layer; the classification encoders at the remaining layers are used to perform downsampling operations on the downsampled features of the previous layer again to generate the downsampled features of the next layer; wherein the downsampled features of the first layer are the segmentation information; The attention mechanism module is used to perform feature extraction operation, attention generation operation and attention weighting operation on the downsampled features of this layer and the convolutional features of the next layer to generate weighted features of this layer; The feature splicing module is used to perform feature splicing and convolution operations according to the weighted features of this layer and the up-sampled features output by the classification decoder of the next layer, so as to generate fusion features of this layer; The classification decoder at the last layer is used to perform convolution processing on the downsampled features of the last layer to generate the convolution features of the last layer; upsample the convolution features to generate the upsampled features of the previous layer; the classification decoders at the remaining levels are used to perform convolution processing on the fusion features of this layer to generate the convolution features of this layer; upsample the convolution features of this layer to generate the upsampled features of the previous layer; The classification output layer is used to generate a classification image corresponding to the oral CBCT image according to the convolutional features of the first layer.

[0041] In actual operation, the classification output layer generates a feature map of the same size as the input image through multiple convolutional layers and Softmax layers, accurately annotating the main anatomical structures such as the mandible, maxilla, mandibular nerve canal, dental bridge, crown, and implant, thus achieving fine classification of oral CBCT images. In addition, the output layer also specifically annotates the teeth in different quadrants, such as incisors, canines, premolars, and molars. The tooth position information refers to the quadrant and tooth number of the tooth, thus achieving high-precision anatomical structure and tooth position classification.

[0042] In an optional embodiment, the training of the oral CBCT image processing model includes: Acquire multiple original oral CBCT images for making a data set; Annotating the anatomical structures and tooth positions in a plurality of original oral CBCT images to obtain corresponding annotated oral CBCT images; wherein the anatomical structures include: mandible, maxillary bone, dental bridge, dental crown, implant, mandibular nerve canal, etc.; Generate an oral CBCT image dataset according to a plurality of original oral CBCT images and their corresponding annotated oral CBCT images; The oral CBCT image processing model is obtained by training according to the oral CBCT image dataset.

[0043] In an optional embodiment, the anatomical structures and tooth positions in a plurality of original oral CBCT images are annotated to obtain corresponding annotated oral CBCT images, including: Use professional medical image annotation software, such as ITK-SNAP, 3D Slicer, and Amira, to annotate the anatomical structures and tooth positions of multiple original oral CBCT images; the anatomical structures include: mandible, maxilla, mandibular nerve canal, dental bridge, crown, implant, etc.; the teeth include: incisors, canines, premolars, and molars, where tooth position information refers to the quadrant and tooth sequence number of the teeth; generate unique label codes for annotated areas such as anatomical structures and tooth positions, and use data processing tools, such as Python's Nibabel or SimpleITK library, to automatically generate label files, and record these label codes in detail in the label files; use the label files as the corresponding annotated oral CBCT images; Specifically, an oral CBCT image dataset is generated according to a plurality of original oral CBCT images and their corresponding annotated oral CBCT images, including: Use Python to unify the size of multiple original oral CBCT images. First, use the ReadImage function of the SimpleITK library to read the original oral CBCT images; use the GetSize function of the SimpleITK library to obtain the original size of the original oral CBCT images, and use the GetSpacing function to obtain the original spacing; set the target size of the original oral CBCT images, where the target size is the number of slices of the Axial axis set to 128, and the Coronal axis and Sagittal axis set to 192; use the ResampleImageFilter function of the SimpleITK library to resample, adjust the size of the original oral CBCT images to the target size, and calculate the new spacing; The corresponding annotated oral CBCT images are resized using Python. First, the ReadImage function of the SimpleITK library is used to read the corresponding label data; the ResampleImageFilter function of the SimpleITK library is used to resample the label data, which is the same as the processing steps for the image data.

[0044] Use the WriteImage function of the SimpleITK library to save the resampled original oral CBCT images and the annotated oral CBCT images; The resampled original oral CBCT images and the resampled annotated oral CBCT images are used as an oral CBCT image dataset; In actual operation, when training the oral CBCT image processing model, the oral CBCT image dataset is randomly divided into a training set and a validation set according to a certain ratio to obtain an oral CBCT image classification dataset; Firstly, the oral CBCT image classification dataset is binary processed separately to obtain the oral CBCT image segmentation dataset of the segmented target area.

[0045] The segmentation sub-model is trained separately based on the oral CBCT image segmentation data set to obtain the weight of the trained segmentation sub-model.

[0046] Based on the transfer learning technology, the weights of all parameters of the segmentation sub-model in the oral CBCT image processing model are frozen and changed to the weights of the trained segmentation sub-model.

[0047] Secondly, a series of specific experimental parameters are set to optimize the model performance. The batch size is set to 32, the initial learning rate is 0.0001, the minimum learning rate is 0.000001, and the cosine annealing strategy is used to dynamically adjust the learning rate so that the learning rate can be adaptively adjusted at different stages of training to improve the convergence speed and effect of the model. The Adam optimizer is selected as the optimizer because of its stability and efficiency in processing large-scale data sets and complex models. The loss function uses the Dice coefficient loss, which can effectively measure the overlap between the predicted results and the true labels and optimize the accuracy of the model in the segmentation task. Secondly, a batch of resampled original oral CBCT images is randomly captured from the training set, and the resampled original oral CBCT images are input into the oral CBCT image processing model to be trained to obtain the classification results; the classification results and the corresponding resampled labeled oral CBCT images are used to calculate the loss function to generate the loss function value; the oral CBCT image processing model to be trained is optimized by the Adam optimizer according to the loss function value and the preset learning rate to complete a batch of model training; Repeat a batch of model training steps until the resampled original oral CBCT images in the training set are trained, and a round of model training is completed; The entire training process was conducted for a total of 300 rounds, and after each round of training, the model performance was evaluated based on the validation set, and the model with the best performance was retained as the final oral CBCT image processing model. This method ensured the high accuracy and reliability of the final model in the classification task.

[0048] In a preferred embodiment, the structure of the attention mechanism module of the classification sub-model is as follows: Figure 7 As shown, it includes: convolution layer, normalization layer and ReLU activation function layer; The attention mechanism module is used to perform feature extraction operation on the downsampled features of the current layer and the convolutional features of the next layer, including: The down-sampled features of the current layer are extracted according to the convolution layer and the normalization layer to generate the first feature; Extract the convolutional features of the next layer according to the convolutional layer and the normalization layer to generate the second feature; The first feature and the second feature are fused to generate a third feature; The third feature is input into the ReLU activation function layer for processing to generate the first activation feature.

[0049] The attention mechanism module further includes: a Sigmoid activation function layer; The attention mechanism module performs attention generation operation in the following way: Performing feature extraction on the first activation feature according to the convolution layer and the normalization layer to generate a second activation feature; The second activation feature is input into the Sigmoid activation function layer for processing to generate attention weights.

[0050] The attention mechanism module performs attention weighting operation in the following way: The attention weights are applied to the downsampled features of this layer to generate weighted features of this layer.

[0051] It can be understood that by adjusting the regional information of the input features according to the attention weight, the feature information weight of the task-critical area is enhanced, while the feature information of the irrelevant area is filtered or weakened. This process makes the oral CBCT image processing model more focused on the task-related area, thereby improving the classification performance.

[0052] In actual operation, this application uses the Python 3.8 programming language and the PyTorch deep learning framework to train the oral CBCT image processing model. During the training process, the NVIDIA GeForce RTX 4090 graphics card was used to improve computing efficiency and speed up model training. In order to effectively utilize the video memory and prevent memory overflow, the size of the input data is dynamically adjusted and compressed according to the video memory occupancy. This strategy ensures that hardware resources are fully utilized to process large-scale three-dimensional medical imaging data without reducing the efficiency of model training. In addition, after the training of the oral CBCT image processing model is completed, the entire oral CBCT image processing model is exported to the ONNX format to facilitate cross-platform compatibility and efficient reasoning in different reasoning environments. The export in the ONNX format enables the model to be loaded into a variety of deep learning frameworks and hardware accelerators, thereby achieving flexible deployment and efficient reasoning performance.

[0053] Based on the above method embodiment, the present invention provides a corresponding device embodiment.

[0054] like Figure 8 As shown, an embodiment of the present invention provides an oral CBCT image classification device, including: an image acquisition module 101 and an image classification module 102 .

[0055] The image acquisition module 101 is used to acquire oral CBCT images to be classified; The image classification module 102 is used to input the oral CBCT image into the oral CBCT image processing model, so that the image processing model generates segmentation information in a built-in segmentation sub-model, and determines a classification image corresponding to the oral CBCT image according to the segmentation information in the built-in classification sub-model; The classification sub-model includes several layers, and each layer except the first and last layers is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification decoder. The first layer is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification output layer, and the last layer is provided with a classification decoder; The classification encoder at the first layer is used to extract the features of the segmentation information by downsampling to generate the downsampled features of the second layer; the classification encoders at the remaining layers are used to downsample the downsampled features of the previous layer to generate the downsampled features of the next layer; wherein the downsampled features of the first layer are the segmentation information; The attention mechanism module is used to perform feature extraction, attention generation and attention weighting on the downsampled features of this layer and the convolutional features of the next layer to generate weighted features of this layer; The feature splicing module is used to perform feature splicing and convolution processing according to the weighted features of this layer and the upsampled features of this layer to generate fusion features of this layer; The classification decoder at the last layer is used to perform convolution processing on the downsampled features of the last layer to generate the convolution features of this layer; upsample the convolution features to generate the upsampled features of the previous layer; the classification decoders at the remaining layers are used to perform convolution processing on the fusion features of this layer to generate the convolution features of this layer; upsample the convolution features of this layer to generate the upsampled features of the previous layer; The classification output layer is used to generate a classification image corresponding to the oral CBCT image according to the convolutional features of the first layer.

[0056] In an optional embodiment, the oral CBCT image classification device further includes: a model training module; The model training module is used to obtain a plurality of original oral CBCT images for preparing a data set; Annotating anatomical structures and tooth positions in a plurality of original oral CBCT images to obtain corresponding annotated oral CBCT images; wherein the anatomical structures include: mandible, maxillary bone, mandibular nerve canal, dental bridge, dental crown, implant, etc.; Generate an oral CBCT image dataset according to a plurality of original oral CBCT images and their corresponding annotated oral CBCT images; The oral CBCT image processing model is obtained by training according to the oral CBCT image dataset.

[0057] It should be noted that the embodiments of the device described above correspond to the above embodiments of the present invention, and can implement any of the methods described above in the present invention. In addition, the embodiments of the above-mentioned device are merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0058] Based on the above method embodiment of the present invention, a corresponding electronic device embodiment is provided.

[0059] An embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the oral CBCT image classification method described in any one of the present invention is implemented, or when the processor executes the computer program, the functions of the modules in the above-mentioned device embodiments are implemented.

[0060] Exemplarily, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the terminal device.

[0061] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0062] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0063] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the terminal device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device or other volatile solid-state storage device.

[0064] Based on the above method embodiment, the present invention provides a storage medium embodiment; Another embodiment of the present invention provides a storage medium, wherein the storage medium includes a stored computer program, wherein when the computer program is executed, the device where the storage medium is located is controlled to execute any one of the above-mentioned oral CBCT image classification methods of the present invention.

[0065] The storage medium is a computer-readable storage medium, and the computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0066] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0067] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for classifying oral CBCT images, characterized in that: include: Acquire oral CBCT images to be classified; Inputting the oral CBCT image into an oral CBCT image processing model, so that the image processing model generates segmentation information in a built-in segmentation sub-model, and determines a classification image corresponding to the oral CBCT image according to the segmentation information in a built-in classification sub-model; The classification sub-model includes several layers, and each layer except the first and last layers is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification decoder. The first layer is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification output layer, and the last layer is provided with a classification decoder; The classification encoder at the first layer is used to extract the features of the segmentation information by downsampling to generate the downsampled features of the second layer; the classification encoders at the remaining layers are used to downsample the downsampled features of the previous layer to generate the downsampled features of the next layer; wherein the downsampled features of the first layer are the segmentation information; The attention mechanism module is used to perform feature extraction, attention generation and attention weighting on the downsampled features of this layer and the convolutional features of the next layer to generate weighted features of this layer; The feature splicing module is used to perform feature splicing and convolution processing according to the weighted features of this layer and the upsampled features of this layer to generate fusion features of this layer; The classification decoder at the last layer is used to perform convolution processing on the downsampled features of the last layer to generate the convolution features of this layer; upsample the convolution features to generate the upsampled features of the previous layer; the classification decoders at the remaining layers are used to perform convolution processing on the fusion features of this layer to generate the convolution features of this layer; upsample the convolution features of this layer to generate the upsampled features of the previous layer; The classification output layer is used to generate a classification image corresponding to the oral CBCT image according to the convolutional features of the first layer.

2. The oral CBCT image classification method according to claim 1, characterized in that: The segmentation sub-model includes several layers, and each layer except the first layer and the last layer is provided with a feature extraction module, a first residual module, a second residual module, a feature merging module and a segmentation decoder. The first layer is provided with a first residual module, a feature splicing module, a second residual module and a segmentation output layer, and the last layer is provided with a feature extraction module, a second residual module and a segmentation decoder, and the second layer is also provided with an image processing module; The image processing module is used to perform image segmentation and linear encoding operations according to the oral CBCT images to generate initial features; The feature extraction module located in the second layer is used to perform feature extraction operations and patch merging operations based on the initial features to generate integrated features of the second layer; The feature extraction modules of the remaining layers are used to perform feature extraction operations and patch merging operations based on the integrated features of the previous layer to generate the integrated features of this layer; The first residual module located at the first layer is used to generate the first residual features of the first layer according to the oral CBCT image; The first residual modules of the remaining layers are used to generate the first residual features of the layer according to the integrated features of the layer; The feature merging module is used to generate a merged feature of this layer according to the first residual feature of this layer and the upsampled feature of this layer; The second residual module located in the last layer is used to perform a convolution operation according to the integrated features of the last layer to generate the second residual features of the last layer; The second residual modules of the remaining layers are used to perform convolution operations based on the merged features of this layer to generate the second residual features of this layer; The segmentation decoder is used to generate upsampled features of the previous layer according to the second residual features of the current layer; The segmentation output layer is used to generate segmentation information corresponding to the oral CBCT image according to the second residual feature of the first layer; Among them, the structure of the segmentation sub-model is the SwinUnet network structure containing the SwinTransformer module.

3. The oral CBCT image classification method according to claim 1, characterized in that: The training of the oral CBCT image processing model includes: Acquire multiple original oral CBCT images for making a data set; Annotating anatomical structures and tooth positions in a plurality of original oral CBCT images to obtain corresponding annotated oral CBCT images; wherein the anatomical structures include: mandible, maxillary bone, mandibular nerve canal, dental bridge, dental crown, implant, etc.; Generate an oral CBCT image dataset according to a plurality of original oral CBCT images and their corresponding annotated oral CBCT images; The oral CBCT image processing model is obtained by training according to the oral CBCT image dataset.

4. The oral CBCT image classification method according to claim 1, characterized in that: The attention mechanism module includes: a convolution layer, a normalization layer and a ReLU activation function layer; The attention mechanism module is used to perform feature extraction operation on the downsampled features of the current layer and the convolutional features of the next layer, including: The down-sampled features of the current layer are extracted according to the convolution layer and the normalization layer to generate the first feature; Extract the convolutional features of the next layer according to the convolutional layer and the normalization layer to generate the second feature; The first feature and the second feature are fused to generate a third feature; The third feature is input into the ReLU activation function layer for processing to generate the first activation feature.

5. The oral CBCT image classification method according to claim 4, characterized in that: The attention mechanism module further includes: a Sigmoid activation function layer; The attention mechanism module generates attention in the following way: Performing feature extraction on the first activation feature according to the convolution layer and the normalization layer to generate a second activation feature; The second activation feature is input into the Sigmoid activation function layer for processing to generate attention weights.

6. The oral CBCT image classification method according to claim 5, characterized in that: The attention mechanism module performs attention weighting in the following way: The attention weights are applied to the downsampled features of this layer to generate weighted features of this layer.

7. An oral CBCT image classification device, characterized in that: include: Image acquisition module and image classification module; The image acquisition module is used to acquire the oral CBCT image to be classified; The image classification module is used to input the oral CBCT image into the oral CBCT image processing model, so that the image processing model generates segmentation information in a built-in segmentation sub-model, and determines a classification image corresponding to the oral CBCT image according to the segmentation information in the built-in classification sub-model; Wherein, the oral CBCT image processing model includes a segmentation sub-model for extracting segmentation information and a classification sub-model for further determining the oral structure category; The classification sub-model includes several layers, and each layer except the first and last layers is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification decoder. The first layer is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification output layer, and the last layer is provided with a classification decoder; The classification encoder at the first layer is used to extract the features of the segmentation information by downsampling to generate the downsampled features of the second layer; the classification encoders at the remaining layers are used to downsample the downsampled features of the previous layer to generate the downsampled features of the next layer; wherein the downsampled features of the first layer are the segmentation information; The attention mechanism module is used to extract features, generate attention and weight attention on the downsampled features of this layer and the convolutional features of the next layer to generate weighted features of this layer; The feature splicing module is used to perform feature splicing and convolution processing according to the weighted features of this layer and the upsampled features of this layer to generate fusion features of this layer; The classification decoder at the last layer is used to perform convolution processing on the downsampled features of the last layer to generate the convolution features of this layer; upsample the convolution features to generate the upsampled features of the previous layer; the classification decoders at the remaining layers are used to perform convolution processing on the fusion features of this layer to generate the convolution features of this layer; upsample the convolution features of this layer to generate the upsampled features of the previous layer; The classification output layer is used to generate a classification image corresponding to the oral CBCT image according to the convolutional features of the first layer.

8. The oral CBCT image classification device according to claim 7, characterized in that: Also includes: Model training module; The model training module is used to obtain a plurality of original oral CBCT images for preparing a data set; Annotating anatomical structures and tooth positions in a plurality of original oral CBCT images to obtain corresponding annotated oral CBCT images; wherein the anatomical structures include: mandible, maxillary bone, mandibular nerve canal, dental bridge, dental crown, implant, etc.; Generate an oral CBCT image dataset according to a plurality of original oral CBCT images and their corresponding annotated oral CBCT images; The oral CBCT image processing model is obtained by training according to the oral CBCT image dataset.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the oral CBCT image classification method according to any one of claims 1 to 6 can be implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it can implement the oral CBCT image classification method described in any one of claims 1 to 6.

Images