Oral CBCT image classification method, device, electronic device and storage medium

Abstract

The present invention discloses a method, device, electronic device and storage medium for classifying oral CBCT images, comprising: inputting an oral CBCT image to be classified into an oral CBCT image processing model, so that the oral CBCT image processing model generates segmentation information in a segmentation sub-model, and determining a classification image corresponding to the oral CBCT image to be classified based on the segmentation information in the classification sub-model. The segmentation sub-model improves the SwinUnet network model by adding a residual module to improve the learning ability of the network model; the classification sub-model improves the original 3D-Unet network model by adding an attention mechanism module to enable the network to selectively learn the features of key areas. By implementing the present invention, the specific structural categories of oral CBCT images can be clearly divided, ensuring the accuracy of the classification results.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an oral CBCT image classification method, device, electronic equipment and storage medium. Background Art

[0002] Cone-beam computed tomography (CBCT) is an advanced three-dimensional imaging technique that uses specialized imaging equipment to obtain detailed three-dimensional images of a patient's oral cavity. Compared to traditional two-dimensional imaging techniques, CBCT provides higher spatial resolution and greater anatomical detail. Utilizing a rotating X-ray beam and detector system, CBCT scans the oral cavity layer by layer, generating precise three-dimensional images. This technology not only clearly displays the teeth and jaws, but also reveals the alveolar canals, sinuses, soft tissues, and other internal oral structures. Due to its superior image quality and three-dimensional visualization capabilities, CBCT has become an indispensable imaging tool in dental implantology, orthodontics, maxillofacial surgery, and other dental fields.

[0003] In the clinical diagnosis of oral CBCT images, traditional methods mainly rely on manual image analysis and interpretation. Due to the complex structure of the oral region, different anatomical parts in the image often overlap with each other, resulting in a cumbersome and error-prone manual classification process, making it difficult to ensure the accuracy of the classification results. With the continuous development and maturity of deep learning technology, compared with traditional oral CBCT image classification methods, image classification models based on deep learning have higher efficiency, accuracy and robustness. The existing technology discloses the use of the original 3D-Unet network model to fully automatically segment teeth in CBCT images. However, the jump connection structure of the original 3D-Unet network model has certain defects. In the direct connection between the encoder and the decoder, there is a semantic gap when the low-level features output by the encoder are fused with the high-level features of the decoder, which may introduce irrelevant information and reduce the accuracy of the oral CBCT image classification results. Summary of the Invention

[0004] The present invention provides an oral CBCT image classification method, device, electronic device, and storage medium. The present invention can automatically identify and classify various types of teeth in CBCT images, significantly improving the classification accuracy of oral CBCT images.

[0005] An embodiment of the present invention provides an oral CBCT image classification method, comprising:

[0006] Obtaining oral CBCT images to be classified;

[0007] Inputting the oral CBCT image into an oral CBCT image processing model, so that the image processing model generates segmentation information in a built-in segmentation sub-model, and determines a classification image corresponding to the oral CBCT image according to the segmentation information in a built-in classification sub-model;

[0008] The classification sub-model includes several layers, and each layer except the first and last layers is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification decoder. The first layer is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification output layer, and the last layer is provided with a classification decoder.

[0009] The classification encoder in the first layer is used to extract the features of the segmentation information through downsampling to generate the downsampled features of the second layer; the classification encoders in the remaining layers are used to downsample the downsampled features of the previous layer to generate the downsampled features of the next layer; the downsampled features of the first layer are the segmentation information;

[0010] The attention mechanism module is used to perform feature extraction, attention generation and attention weighting on the downsampled features of this layer and the convolutional features of the next layer to generate weighted features of this layer;

[0011] The feature splicing module is used to perform feature splicing and convolution processing based on the weighted features of this layer and the upsampled features of this layer to generate the fused features of this layer;

[0012] The classification decoder at the last layer is used to perform convolution processing on the downsampled features of the last layer to generate the convolution features of this layer; upsample the convolution features to generate the upsampled features of the previous layer; the classification decoders at the remaining layers are used to perform convolution processing on the fused features of this layer to generate the convolution features of this layer; upsample the convolution features of this layer to generate the upsampled features of the previous layer;

[0013] The classification output layer is used to generate a classification image corresponding to the oral CBCT image based on the convolution features of the first layer.

[0014] Furthermore, the segmentation sub-model includes several layers, and each layer except the first and last layers is provided with a feature extraction module, a first residual module, a second residual module, a feature merging module and a segmentation decoder. The first layer is provided with a first residual module, a feature splicing module, a second residual module and a segmentation output layer, the last layer is provided with a feature extraction module, a second residual module and a segmentation decoder, and the second layer is also provided with an image processing module;

[0015] The image processing module is used to perform image segmentation and linear encoding operations based on the oral CBCT image to generate initial features;

[0016] The feature extraction module in the second layer is used to perform feature extraction and patch merging operations based on the initial features to generate the integrated features of the second layer. The feature extraction modules in the remaining layers are used to perform feature extraction and patch merging operations based on the integrated features of the previous layer to generate the integrated features of the current layer.

[0017] The first residual module at the first layer is used to generate the first residual features of the first layer based on the oral CBCT image; the first residual modules at other levels are used to generate the first residual features of the current layer based on the integrated features of the current layer;

[0018] The feature merging module is used to generate a merged feature of this layer based on the first residual feature of this layer and the upsampled feature of this layer;

[0019] The second residual module in the last layer is used to perform a convolution operation based on the integrated features of the last layer to generate the second residual features of the last layer; the second residual modules in the remaining layers are used to perform a convolution operation based on the merged features of this layer to generate the second residual features of this layer;

[0020] The segmentation decoder is used to generate upsampled features of the previous layer based on the second residual features of the current layer;

[0021] The segmentation output layer is used to generate segmentation information corresponding to the oral CBCT image based on the second residual feature of the first layer;

[0022] Among them, the structure of the segmentation sub-model is the SwinUnet network structure containing the SwinTransformer module.

[0023] Furthermore, the training of the oral CBCT image processing model includes:

[0024] Acquire multiple original oral CBCT images for preparing a data set;

[0025] annotating anatomical structures and tooth positions in multiple original oral CBCT images to obtain corresponding annotated oral CBCT images; wherein the anatomical structures include: mandible, maxillary bone, mandibular nerve canal, dental bridge, dental crown, implant, etc.;

[0026] Generate an oral CBCT image dataset based on multiple original oral CBCT images and their corresponding annotated oral CBCT images;

[0027] The oral CBCT image processing model is obtained by training according to the oral CBCT image dataset.

[0028] Furthermore, the attention mechanism module includes: a convolution layer, a normalization layer and a ReLU activation function layer;

[0029] The attention mechanism module is used to perform feature extraction operations on the downsampled features of the current layer and the convolutional features of the next layer, including:

[0030] The downsampled features of this layer are extracted according to the convolution layer and the normalization layer to generate the first feature;

[0031] The convolutional features of the next layer are extracted according to the convolutional layer and the normalization layer to generate the second feature;

[0032] Fusing the first feature and the second feature to generate a third feature;

[0033] The third feature is input into the ReLU activation function layer for processing to generate the first activation feature.

[0034] Furthermore, the attention mechanism module further includes: a Sigmoid activation function layer;

[0035] The attention mechanism module generates attention in the following way:

[0036] Performing feature extraction on the first activation feature according to the convolution layer and the normalization layer to generate a second activation feature;

[0037] The second activation feature is input into the Sigmoid activation function layer for processing to generate attention weights.

[0038] Furthermore, the attention mechanism module performs attention weighting in the following way:

[0039] The attention weight is applied to the downsampled features of this layer to generate weighted features of this layer.

[0040] Based on the above method embodiments, the present invention provides corresponding device embodiments.

[0041] An embodiment of the present invention provides an oral CBCT image classification device, comprising: an image acquisition module and an image classification module;

[0042] The image acquisition module is used to acquire oral CBCT images to be classified;

[0043] The image classification module is configured to input the oral CBCT image into the oral CBCT image processing model, so that the image processing model generates segmentation information in a built-in segmentation sub-model, and determines a classification image corresponding to the oral CBCT image based on the segmentation information in the built-in classification sub-model;

[0044] The oral CBCT image processing model includes a segmentation sub-model for extracting segmentation information and a classification sub-model for further determining the oral structure category;

[0045] The classification sub-model includes several layers, and each layer except the first and last layers is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification decoder. The first layer is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification output layer, and the last layer is provided with a classification decoder.

[0046] The classification encoder in the first layer is used to extract the features of the segmentation information through downsampling to generate the downsampled features of the second layer; the classification encoders in the remaining layers are used to downsample the downsampled features of the previous layer to generate the downsampled features of the next layer; the downsampled features of the first layer are the segmentation information;

[0047] The attention mechanism module is used to extract features, generate attention, and weight attention by combining the downsampled features of this layer with the convolutional features of the next layer to generate weighted features of this layer;

[0048] The feature splicing module is used to perform feature splicing and convolution processing based on the weighted features of this layer and the upsampled features of this layer to generate the fused features of this layer;

[0049] The classification decoder at the last layer is used to perform convolution processing on the downsampled features of the last layer to generate the convolution features of this layer; upsample the convolution features to generate the upsampled features of the previous layer; the classification decoders at the remaining layers are used to perform convolution processing on the fused features of this layer to generate the convolution features of this layer; upsample the convolution features of this layer to generate the upsampled features of the previous layer;

[0050] The classification output layer is used to generate a classification image corresponding to the oral CBCT image based on the convolution features of the first layer.

[0051] Furthermore, the oral CBCT image classification device further includes: a model training module;

[0052] The model training module is used to obtain a plurality of original oral CBCT images for preparing a data set;

[0053] annotating anatomical structures and tooth positions in multiple original oral CBCT images to obtain corresponding annotated oral CBCT images; wherein the anatomical structures include: mandible, maxillary bone, mandibular nerve canal, dental bridge, dental crown, implant, etc.;

[0054] Generate an oral CBCT image dataset based on multiple original oral CBCT images and their corresponding annotated oral CBCT images;

[0055] The oral CBCT image processing model is obtained by training according to the oral CBCT image dataset.

[0056] Based on the above method embodiment, the present invention provides a corresponding electronic device embodiment.

[0057] An embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the oral CBCT image classification method described in any one of the above-mentioned method embodiments can be implemented.

[0058] Based on the above method embodiment, the present invention provides a corresponding storage medium embodiment.

[0059] An embodiment of the present invention provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the oral CBCT image classification method in the above method embodiment can be implemented.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] The present invention inputs an oral CBCT image to be classified into an oral CBCT image segmentation model, so that the oral CBCT image segmentation model segments the oral CBCT image to be classified and obtains segmentation information. The segmentation information at this time can clearly identify all target anatomical regions, unify the target anatomical structures into one category, and eliminate voxels of background and all irrelevant regions. The segmentation information is then input into an oral CBCT image processing model, so that the oral CBCT image processing model determines a classification image of the oral CBCT image corresponding to the oral CBCT image to be classified based on the segmentation information. The oral CBCT image processing model includes a segmentation sub-model for extracting the segmentation information and a classification sub-model for further determining the oral structure category. The classification sub-model modifies the original 3D-Unet network model and adds an attention mechanism module, including an encoder, an attention mechanism module, a feature splicing module, a decoder, and an output layer. After adding the attention mechanism module, the classification sub-model can weight features, so that the network can more selectively learn features of key regions. The attention generation operation can calculate and generate attention weights, which are used to represent the importance of different regions. The attention weighting operation can adjust the weight of feature information in task-critical areas while filtering or weakening feature information in irrelevant areas. By increasing the weight of feature information in task-critical areas and suppressing the weight of classification-irrelevant areas, the attention mechanism module can guide the network to better learn task-related features, thereby clearly classifying the specific structural categories of oral CBCT images and ensuring the accuracy of classification results. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1The figure is a flow chart of an oral CBCT image classification method provided by one embodiment of the present invention.

[0063] Figure 2 This is a structural diagram of an oral CBCT image processing model provided by one embodiment of the present invention.

[0064] Figure 3 This is a structural diagram of a segmentation sub-model in an oral CBCT image processing model provided by an embodiment of the present invention.

[0065] Figure 4 4 is a structural diagram of the feature extraction layer in the segmentation sub-model provided by one embodiment of the present invention.

[0066] Figure 5 4 is a structural diagram of the residual module in the segmentation sub-model provided by one embodiment of the present invention.

[0067] Figure 6 This is a structural diagram of a classification sub-model in an oral CBCT image processing model provided by an embodiment of the present invention.

[0068] Figure 7 This is a structural diagram of the attention mechanism module of the classification sub-model provided in one embodiment of the invention.

[0069] Figure 8 It is a structural schematic diagram of an oral CBCT image classification device provided by one embodiment of the invention. DETAILED DESCRIPTION

[0070] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0071] like Figure 1 As shown, an embodiment of the present invention provides an oral CBCT image classification method, which includes at least the following steps:

[0072] Step S1: Acquire oral CBCT images to be classified.

[0073] The oral CBCT images to be classified refer to image files acquired from oral CBCT images and having uniformed sizes.

[0074] Step S2: inputting the oral CBCT image to be classified into the oral CBCT image processing model, so that the oral CBCT image processing model generates segmentation information in a built-in segmentation sub-model, and determines a classification image corresponding to the oral CBCT image to be classified according to the segmentation information in the built-in classification sub-model;

[0075] It can be understood that the oral CBCT image processing model includes a segmentation sub-model for extracting segmentation information and a classification sub-model for further determining the category of oral structure; the corresponding classification image contains different anatomical structures and teeth with tooth position information; wherein, the anatomical structures include: mandible, maxilla, mandibular nerve canal, dental bridge, crown, implant, etc.; teeth with tooth position information include: incisors, canines, premolars, and molars, wherein the tooth position information refers to the quadrant where the tooth is located and the tooth sequence number.

[0076] It's important to note that teeth are numbered and located in four quadrants. Each quadrant contains eight teeth, numbered from the midline backward. The upper right quadrant is numbered 11 to 18, the upper left quadrant is numbered 21 to 28, the lower left quadrant is numbered 31 to 38, and the lower right quadrant is numbered 41 to 48. The teeth in each quadrant are the same: the first two are incisors, the third is a canine, the fourth and fifth are premolars, and the last three are molars. The third molar (number 8) is a wisdom tooth.

[0077] In an optional embodiment, the structure of the oral CBCT image segmentation model is as follows: Figure 2 The oral CBCT image processing model includes a segmentation sub-model for extracting segmentation information and a classification sub-model for further determining the oral structure category;

[0078] Specifically, in a preferred embodiment, the structure of the segmentation sub-model is as follows: Figure 3 shown.

[0079] The segmentation sub-model includes several layers, and each layer except the first and last layers is provided with a feature extraction module, a first residual module, a second residual module, a feature merging module and a segmentation decoder. The first layer is provided with a first residual module, a feature splicing module, a second residual module and a segmentation output layer, and the last layer is provided with a feature extraction module, a second residual module and a segmentation decoder. The second layer is also provided with an image processing module;

[0080] The image processing module is used to perform image segmentation and linear encoding operations based on the oral CBCT image to generate initial features;

[0081] The feature extraction module in the second layer is used to perform feature extraction and patch merging operations based on the initial features to generate the integrated features of the second layer. The feature extraction modules in the remaining layers are used to perform feature extraction and patch merging operations based on the integrated features of the previous layer to generate the integrated features of the current layer.

[0082] The first residual module at the first layer is used to generate the first residual features of the first layer based on the oral CBCT image; the first residual modules at other levels are used to generate the first residual features of the current layer based on the integrated features of the current layer;

[0083] The feature merging module is used to generate a merged feature of this layer based on the first residual feature of this layer and the upsampled feature of this layer;

[0084] The second residual module in the last layer is used to perform a convolution operation based on the integrated features of the last layer to generate the second residual features of the last layer; the second residual modules in the remaining layers are used to perform a convolution operation based on the merged features of this layer to generate the second residual features of this layer;

[0085] The segmentation decoder is used to generate upsampled features of the previous layer based on the second residual features of the current layer;

[0086] The segmentation output layer is used to generate segmentation information corresponding to the oral CBCT image based on the second residual feature of the first layer;

[0087] Among them, the structure of the segmentation sub-model is the SwinUnet network structure containing the SwinTransformer module.

[0088] It should be noted that the image processing module's image segmentation operation divides the oral CBCT image into several local blocks of preset sizes, and the linear encoding operation flattens each local block into a one-dimensional feature vector through a linear mapping layer. Through image segmentation and linear encoding operations, the oral CBCT image segmentation model not only uses these one-dimensional feature vectors as initial feature representations but also reduces the computational burden, providing rich input features for subsequent complex feature extraction, helping to capture more detailed local features while avoiding the high computational cost required for full image processing.

[0089] The structure of the feature extraction layer of the segmentation sub-model is as follows Figure 4 As shown, it includes: fixed window multi-head attention layer and sliding multi-head attention layer;

[0090] The fixed-window multi-head attention layer operates based on the fixed-window self-attention mechanism, partitioning features into several fixed-size windows and performing self-attention within each window. This mechanism effectively captures local feature information while significantly reducing the computational complexity of global self-attention. By performing calculations only within the local window, the fixed-window self-attention mechanism reduces computational overhead and improves the model's ability to process large-scale image data. This fine-grained local feature information provides a precise foundation for subsequent feature fusion.

[0091] The sliding window multi-head attention layer operates based on the sliding window self-attention mechanism, achieving cross-window feature fusion by shifting the window between blocks. By exchanging information between adjacent windows, the sliding window multi-head attention layer is able to capture global contextual information across windows and integrate features at different scales. This mechanism enhances the model's ability to capture global features, making feature representation more comprehensive and accurate, and further improving the model's performance in complex medical image processing.

[0092] The segmentation sub-model's patch merging layer gradually merges multiple patches from the feature map to incorporate more contextual information and extract higher-level feature representations. By reducing the spatial resolution of the feature map, patch merging enables features at each location to cover a larger image area, reducing computational effort while capturing a wider range of contextual information. Furthermore, by integrating features from different locations, patch merging improves the segmentation sub-model's ability to fuse feature information from different regions, ensuring high-quality feature representation.

[0093] The structure of the residual module of the segmentation sub-model is as follows Figure 5 As shown in the figure, by adding the input features to the output features, direct information propagation is achieved, thereby improving the stability of feature transfer and the learning efficiency. Low-level local feature information and high-level global feature information are effectively integrated, ensuring that the segmentation effect of the oral CBCT image segmentation model is accurate and complete.

[0094] The segmentation output layer of the segmentation sub-model is used to generate the final segmentation information. The segmentation output layer uses a single-channel convolutional layer to output a three-dimensional feature map of the same size as the input image. The value of each voxel in the feature map is used to indicate whether the location belongs to the target anatomical region. Specifically, the oral CBCT image segmentation model combines all anatomical structures of interest, such as teeth and mandible, maxillary bone, mandibular nerve canal, dental bridge, crown, implant, etc., into one category, and distinguishes the target area from the background area through binarization. The segmentation information generated by the output layer can clearly identify all target anatomical regions while eliminating voxels in the background and all irrelevant areas.

[0095] Specifically, in a preferred embodiment, the structure of the classification sub-model is as follows Figure 6 As shown, it includes several layers, including several layers. Except for the first and last layers, each layer is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification decoder. The first layer is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification output layer, and the last layer is provided with a classification decoder.

[0096] The classification encoder at the first layer is used to extract the features of the segmentation information through downsampling operations to generate the downsampled features of the second layer. The classification encoders at the remaining layers are used to downsample the downsampled features of the previous layer again to generate the downsampled features of the next layer. Among them, the downsampled features of the first layer are the segmentation information.

[0097] The attention mechanism module is used to perform feature extraction, attention generation and attention weighting operations on the downsampled features of this layer and the convolutional features of the next layer to generate weighted features of this layer;

[0098] The feature splicing module is used to perform feature splicing and convolution operations based on the weighted features of this layer and the up-sampled features output by the classification decoder of the next layer to generate the fused features of this layer;

[0099] The classification decoder at the last layer is used to perform convolution processing on the downsampled features of the last layer to generate the convolution features of the last layer; upsample the convolution features to generate the upsampled features of the previous layer; the classification decoders at the remaining layers are used to perform convolution processing on the fusion features of the current layer to generate the convolution features of the current layer; upsample the convolution features of the current layer to generate the upsampled features of the previous layer;

[0100] The classification output layer is used to generate a classification image corresponding to the oral CBCT image based on the convolutional features of the first layer.

[0101] In practice, the classification output layer generates a feature map of the same size as the input image through multiple convolutional and softmax layers. This accurately labels key anatomical structures, such as the mandible, maxilla, mandibular nerve canal, dental bridges, crowns, and implants, enabling refined classification of oral CBCT images. Furthermore, the output layer specifically annotates teeth in different quadrants, such as incisors, canines, premolars, and molars. Tooth position information refers to the quadrant in which the tooth is located and its ordinal number, enabling highly accurate anatomical structure and tooth position classification.

[0102] In an optional embodiment, the training of the oral CBCT image processing model includes:

[0103] Acquire multiple original oral CBCT images for preparing a data set;

[0104] annotating anatomical structures and tooth positions in multiple original oral CBCT images to obtain corresponding annotated oral CBCT images; wherein the anatomical structures include: mandible, maxilla, dental bridge, dental crown, implant, mandibular nerve canal, etc.;

[0105] Generate an oral CBCT image dataset based on multiple original oral CBCT images and their corresponding annotated oral CBCT images;

[0106] The oral CBCT image processing model is obtained by training according to the oral CBCT image dataset.

[0107] In an optional embodiment, the anatomical structures and tooth positions in a plurality of original oral CBCT images are annotated to obtain corresponding annotated oral CBCT images, including:

[0108] Professional medical image annotation software, such as ITK-SNAP, 3D Slicer, and Amira, is used to annotate the anatomical structures and tooth positions of multiple original oral CBCT images. Anatomical structures include the mandible, maxilla, mandibular nerve canal, bridges, crowns, and implants. Teeth include incisors, canines, premolars, and molars. Tooth position information refers to the quadrant and tooth number of the tooth. Unique label codes are generated for the annotated areas, including anatomical structures and tooth positions. Data processing tools, such as Python's Nibabel or SimpleITK library, are used to automatically generate label files, which contain detailed information about these label codes. The label files are then used as the corresponding annotated oral CBCT images.

[0109] Specifically, an oral CBCT image dataset is generated based on multiple original oral CBCT images and their corresponding annotated oral CBCT images, including:

[0110] Multiple original oral CBCT images were resized using Python. First, the original oral CBCT images were read using the ReadImage function of the SimpleITK library. The original size of the original oral CBCT images was obtained using the GetSize function of the SimpleITK library, and the original spacing was obtained using the GetSpacing function of the SimpleITK library. The target size of the original oral CBCT images was set to 128 slices in the axial axis and 192 slices in the coronal and sagittal axes. The ResampleImageFilter function of the SimpleITK library was used to resample the original oral CBCT images to the target size and calculate the new spacing.

[0111] The corresponding annotated oral CBCT images were resized using Python. First, the corresponding label data was read using the ReadImage function of the SimpleITK library. The label data was resampled using the ResampleImageFilter function of the SimpleITK library, following the same steps as for the image data.

[0112] Use the WriteImage function of the SimpleITK library to save the resampled original oral CBCT image and the annotated oral CBCT image;

[0113] The resampled original oral CBCT images and the resampled annotated oral CBCT images are used as the oral CBCT image dataset;

[0114] In actual operation, when training the oral CBCT image processing model, the oral CBCT image dataset is randomly divided into a training set and a validation set according to a certain ratio to obtain the oral CBCT image classification dataset;

[0115] Firstly, the oral CBCT image classification dataset is binary processed separately to obtain the oral CBCT image segmentation dataset of the segmented target area.

[0116] The segmentation sub-model is trained separately based on the oral CBCT image segmentation dataset to obtain the weight of the trained segmentation sub-model.

[0117] Based on the transfer learning technology, the weights of all parameters of the segmentation sub-model in the oral CBCT image processing model are frozen and replaced with the weights of the trained segmentation sub-model.

[0118] Secondly, a series of specific experimental parameters were set to optimize model performance. The batch size was set to 32, the initial learning rate was 0.0001, the minimum learning rate was 0.000001, and a cosine annealing strategy was used to dynamically adjust the learning rate. This allowed for adaptive adjustment at different stages of training, improving the model's convergence speed and effectiveness. The Adam optimizer was chosen for its stability and efficiency when processing large-scale datasets and complex models. The Dice coefficient loss was used as the loss function, which effectively measures the overlap between the predicted results and the true labels, optimizing the model's accuracy on the segmentation task.

[0119] Next, a batch of resampled original oral CBCT images is randomly captured from the training set and input into the oral CBCT image processing model to be trained to obtain classification results. The classification results are compared with the corresponding resampled annotated oral CBCT images to calculate the loss function and generate a loss function value. The Adam optimizer is used to optimize the oral CBCT image processing model to be trained based on the loss function value and the preset learning rate to complete a batch of model training.

[0120] Repeat the model training steps for a batch until all the resampled original oral CBCT images in the training set are trained, thus completing one round of model training.

[0121] The entire training process was conducted over 300 rounds. After each round, model performance was evaluated based on the validation set. The best performing model was retained as the final oral CBCT image processing model. This approach ensured the high accuracy and reliability of the final model in classification tasks.

[0122] In a preferred embodiment, the structure of the attention mechanism module of the classification sub-model is as follows Figure 7 As shown, it includes: convolution layer, normalization layer and ReLU activation function layer;

[0123] The attention mechanism module is used to perform feature extraction operations on the downsampled features of the current layer and the convolutional features of the next layer, including:

[0124] The downsampled features of this layer are extracted according to the convolution layer and the normalization layer to generate the first feature;

[0125] The convolutional features of the next layer are extracted according to the convolutional layer and the normalization layer to generate the second feature;

[0126] Fusing the first feature and the second feature to generate a third feature;

[0127] The third feature is input into the ReLU activation function layer for processing to generate the first activation feature.

[0128] The attention mechanism module further includes: a Sigmoid activation function layer;

[0129] The attention mechanism module performs attention generation operations in the following ways:

[0130] Performing feature extraction on the first activation feature according to the convolution layer and the normalization layer to generate a second activation feature;

[0131] The second activation feature is input into the Sigmoid activation function layer for processing to generate attention weights.

[0132] The attention mechanism module performs attention weighting operations in the following way:

[0133] The attention weight is applied to the downsampled features of this layer to generate weighted features of this layer.

[0134] It can be understood that by adjusting the regional information of the input features based on the attention weight, the feature information weight of the task-critical areas is enhanced, while the feature information of irrelevant areas is filtered or weakened. This process allows the oral CBCT image processing model to focus more on the areas relevant to the task, thereby improving classification performance.

[0135] In practice, this application uses the Python 3.8 programming language and the PyTorch deep learning framework to train the oral CBCT image processing model. During the training process, the NVIDIA GeForce RTX 4090 graphics card was used to improve computing efficiency and speed up model training. In order to effectively utilize the video memory and prevent memory overflow, the size of the input data is dynamically adjusted and compressed according to the video memory occupancy. This strategy ensures that hardware resources are fully utilized to process large-scale three-dimensional medical imaging data without reducing the efficiency of model training. In addition, after the training of the oral CBCT image processing model is completed, the entire oral CBCT image processing model is exported to the ONNX format to facilitate cross-platform compatibility and efficient reasoning in different inference environments. Exporting in the ONNX format enables the model to be loaded into a variety of deep learning frameworks and hardware accelerators, thereby achieving flexible deployment and efficient reasoning performance.

[0136] Based on the above method embodiments, the present invention provides corresponding device embodiments.

[0137] like Figure 8 As shown, an embodiment of the present invention provides an oral CBCT image classification device, including: an image acquisition module 101 and an image classification module 102.

[0138] The image acquisition module 101 is used to acquire oral CBCT images to be classified;

[0139] The image classification module 102 is configured to input the oral CBCT image into an oral CBCT image processing model, so that the image processing model generates segmentation information in a built-in segmentation sub-model, and determines a classification image corresponding to the oral CBCT image based on the segmentation information in the built-in classification sub-model;

[0140] The classification sub-model includes several layers, and each layer except the first and last layers is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification decoder. The first layer is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification output layer, and the last layer is provided with a classification decoder.

[0141] The classification encoder in the first layer is used to extract the features of the segmentation information through downsampling to generate the downsampled features of the second layer; the classification encoders in the remaining layers are used to downsample the downsampled features of the previous layer to generate the downsampled features of the next layer; the downsampled features of the first layer are the segmentation information;

[0142] The attention mechanism module is used to perform feature extraction, attention generation and attention weighting on the downsampled features of this layer and the convolutional features of the next layer to generate weighted features of this layer;

[0143] The feature splicing module is used to perform feature splicing and convolution processing based on the weighted features of this layer and the upsampled features of this layer to generate the fused features of this layer;

[0144] The classification decoder at the last layer is used to perform convolution processing on the downsampled features of the last layer to generate the convolution features of this layer; upsample the convolution features to generate the upsampled features of the previous layer; the classification decoders at the remaining layers are used to perform convolution processing on the fused features of this layer to generate the convolution features of this layer; upsample the convolution features of this layer to generate the upsampled features of the previous layer;

[0145] The classification output layer is used to generate a classification image corresponding to the oral CBCT image based on the convolution features of the first layer.

[0146] In an optional embodiment, the oral CBCT image classification device further includes: a model training module;

[0147] The model training module is used to obtain a plurality of original oral CBCT images for preparing a data set;

[0148] annotating anatomical structures and tooth positions in multiple original oral CBCT images to obtain corresponding annotated oral CBCT images; wherein the anatomical structures include: mandible, maxillary bone, mandibular nerve canal, dental bridge, dental crown, implant, etc.;

[0149] Generate an oral CBCT image dataset based on multiple original oral CBCT images and their corresponding annotated oral CBCT images;

[0150] The oral CBCT image processing model is obtained by training according to the oral CBCT image dataset.

[0151] It should be noted that the embodiments of the device described above correspond to the above-mentioned embodiments of the present invention, and can implement any of the methods described above in the present invention. In addition, the embodiments of the above-mentioned device are merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the embodiment of the device provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0152] Based on the above method embodiment of the present invention, a corresponding electronic device embodiment is provided.

[0153] An embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the oral CBCT image classification method described in any one of the present invention is implemented, or when the processor executes the computer program, the functions of the modules in the above-mentioned device embodiments are implemented.

[0154] Exemplarily, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device.

[0155] The terminal device may be a computing device such as a desktop computer, a notebook computer, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0156] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor. The processor is the control center of the terminal device and connects various parts of the entire terminal device using various interfaces and lines.

[0157] The memory can be used to store the computer programs and / or modules. The processor implements the various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and accessing the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store an operating system, at least one application required for a function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory and non-volatile memory, such as a hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0158] Based on the above method embodiment, the present invention provides a corresponding storage medium embodiment;

[0159] Another embodiment of the present invention provides a storage medium, wherein the storage medium includes a stored computer program, wherein when the computer program is executed, the device where the storage medium is located is controlled to execute any of the above-mentioned oral CBCT image classification methods of the present invention.

[0160] The aforementioned storage medium is a computer-readable storage medium, and the computer program includes computer program code, which may be in source code form, object code form, an executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a removable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunications signal, and a software distribution medium. It should be noted that the content of the computer-readable medium may be appropriately increased or decreased based on the requirements of legislation and patent practice within a jurisdiction. For example, in some jurisdictions, based on legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunications signals.

[0161] In the description of this specification, the reference terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials, or characteristics described may be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and integrate different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless they are mutually inconsistent.

[0162] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for classifying oral CBCT images, characterized in that: include: Obtaining oral CBCT images to be classified; Inputting the oral CBCT image into an oral CBCT image processing model, so that the image processing model generates segmentation information in a built-in segmentation sub-model, and determines a classification image corresponding to the oral CBCT image according to the segmentation information in a built-in classification sub-model; The classification sub-model includes several layers, and each layer except the first and last layers is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification decoder. The first layer is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification output layer, and the last layer is provided with a classification decoder. The classification encoder in the first layer is used to extract the features of the segmentation information through downsampling to generate the downsampled features of the second layer; the classification encoders in the remaining layers are used to downsample the downsampled features of the previous layer to generate the downsampled features of the next layer; the downsampled features of the first layer are the segmentation information; The attention mechanism module is used to perform feature extraction, attention generation and attention weighting on the downsampled features of this layer and the convolutional features of the next layer to generate weighted features of this layer; The feature splicing module is used to perform feature splicing and convolution processing based on the weighted features of this layer and the upsampled features of this layer to generate the fused features of this layer; The classification decoder at the last layer is used to perform convolution processing on the downsampled features of the last layer to generate the convolution features of this layer; upsample the convolution features to generate the upsampled features of the previous layer; the classification decoders at the remaining layers are used to perform convolution processing on the fused features of this layer to generate the convolution features of this layer; upsample the convolution features of this layer to generate the upsampled features of the previous layer; The classification output layer is used to generate a classification image corresponding to the oral CBCT image based on the convolution features of the first layer; The segmentation sub-model includes a segmentation output layer; The segmentation output layer is used to generate the final segmentation information and output a first three-dimensional feature map with the same size as the input image; each voxel value in the first three-dimensional feature map indicates whether the location belongs to the target anatomical region; The classification output layer generates a second feature map with the same size as the input image through multiple convolutional layers and Softmax layers to mark the anatomical structure.

2. The oral CBCT image classification method according to claim 1, wherein: The segmentation sub-model also includes several layers, and each layer except the first and last layers is provided with a feature extraction module, a first residual module, a second residual module, and a feature merging module. The first layer is provided with a first residual module, a feature splicing module, a second residual module, and a segmentation output layer. The last layer is provided with a feature extraction module, a second residual module, and a segmentation decoder. The second layer is also provided with an image processing module. The image processing module is used to perform image segmentation and linear encoding operations based on the oral CBCT image to generate initial features; The feature extraction module at the second layer is used to perform feature extraction and patch merging operations based on the initial features to generate integrated features at the second layer; The feature extraction modules of the remaining layers are used to perform feature extraction and patch merging operations based on the integrated features of the previous layer to generate the integrated features of this layer; The first residual module located in the first layer is used to generate the first residual features of the first layer according to the oral CBCT image; The first residual modules of the remaining layers are used to generate the first residual features of the layer based on the integrated features of the layer; The feature merging module is used to generate a merged feature of this layer based on the first residual feature of this layer and the upsampled feature of this layer; The second residual module located in the last layer is used to perform a convolution operation based on the integrated features of the last layer to generate the second residual features of the last layer; The second residual modules of the remaining layers are used to perform convolution operations based on the merged features of this layer to generate the second residual features of this layer; The segmentation decoder is used to generate upsampled features of the previous layer based on the second residual features of the current layer; The segmentation output layer is used to generate segmentation information corresponding to the oral CBCT image based on the second residual feature of the first layer; Among them, the structure of the segmentation sub-model is the SwinUnet network structure containing the SwinTransformer module.

3. The oral CBCT image classification method according to claim 1, wherein: The training of the oral CBCT image processing model includes: Acquire multiple original oral CBCT images for preparing a data set; annotating anatomical structures and tooth positions in a plurality of original oral CBCT images to obtain corresponding annotated oral CBCT images; wherein the anatomical structures include: mandible, maxillary bone, mandibular nerve canal, dental bridge, dental crown, and implant; Generate an oral CBCT image dataset based on multiple original oral CBCT images and their corresponding annotated oral CBCT images; The oral CBCT image processing model is obtained by training according to the oral CBCT image dataset.

4. The oral CBCT image classification method according to claim 1, wherein: The attention mechanism module includes: a convolution layer, a normalization layer and a ReLU activation function layer; The attention mechanism module is used to perform feature extraction operations on the downsampled features of the current layer and the convolutional features of the next layer, including: The downsampled features of this layer are extracted according to the convolution layer and the normalization layer to generate the first feature; The convolutional features of the next layer are extracted according to the convolutional layer and the normalization layer to generate the second feature; Fusing the first feature and the second feature to generate a third feature; The third feature is input into the ReLU activation function layer for processing to generate the first activation feature.

5. The oral CBCT image classification method according to claim 4, characterized in that: The attention mechanism module further includes: a Sigmoid activation function layer; The attention mechanism module generates attention in the following way: Performing feature extraction on the first activation feature according to the convolution layer and the normalization layer to generate a second activation feature; The second activation feature is input into the Sigmoid activation function layer for processing to generate attention weights.

6. The oral CBCT image classification method according to claim 5, characterized in that: The attention mechanism module performs attention weighting in the following way: The attention weight is applied to the downsampled features of this layer to generate weighted features of this layer.

7. An oral CBCT image classification device, characterized in that: include: Image acquisition module and image classification module; The image acquisition module is used to acquire oral CBCT images to be classified; The image classification module is configured to input the oral CBCT image into the oral CBCT image processing model, so that the image processing model generates segmentation information in a built-in segmentation sub-model, and determines a classification image corresponding to the oral CBCT image based on the segmentation information in the built-in classification sub-model; The oral CBCT image processing model includes a segmentation sub-model for extracting segmentation information and a classification sub-model for further determining the oral structure category; The classification sub-model includes several layers, and each layer except the first and last layers is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification decoder. The first layer is provided with a classification encoder, an attention mechanism module, a feature splicing module and a classification output layer, and the last layer is provided with a classification decoder. The classification encoder in the first layer is used to extract the features of the segmentation information through downsampling to generate the downsampled features of the second layer; the classification encoders in the remaining layers are used to downsample the downsampled features of the previous layer to generate the downsampled features of the next layer; the downsampled features of the first layer are the segmentation information; The attention mechanism module is used to extract features, generate attention, and weight attention by combining the downsampled features of this layer with the convolutional features of the next layer to generate weighted features of this layer; The feature splicing module is used to perform feature splicing and convolution processing based on the weighted features of this layer and the upsampled features of this layer to generate the fused features of this layer; The classification decoder at the last layer is used to perform convolution processing on the downsampled features of the last layer to generate the convolution features of this layer; upsample the convolution features to generate the upsampled features of the previous layer; the classification decoders at the remaining layers are used to perform convolution processing on the fused features of this layer to generate the convolution features of this layer; upsample the convolution features of this layer to generate the upsampled features of the previous layer; The classification output layer is used to generate a classification image corresponding to the oral CBCT image based on the convolution features of the first layer; The segmentation sub-model includes a segmentation output layer; The segmentation output layer is used to generate the final segmentation information and output a first three-dimensional feature map with the same size as the input image; each voxel value in the first three-dimensional feature map indicates whether the location belongs to the target anatomical region; The classification output layer generates a second feature map with the same size as the input image through multiple convolutional layers and Softmax layers to mark the anatomical structure.

8. The oral CBCT image classification device according to claim 7, characterized in that: Also includes: Model training module; The model training module is used to obtain a plurality of original oral CBCT images for preparing a data set; annotating anatomical structures and tooth positions in a plurality of original oral CBCT images to obtain corresponding annotated oral CBCT images; wherein the anatomical structures include: mandible, maxillary bone, mandibular nerve canal, dental bridge, dental crown, and implant; Generate an oral CBCT image dataset based on multiple original oral CBCT images and their corresponding annotated oral CBCT images; The oral CBCT image processing model is obtained by training according to the oral CBCT image dataset.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it can implement the oral CBCT image classification method according to any one of claims 1 to 6.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it can implement the oral CBCT image classification method according to any one of claims 1 to 6.

Images