A deep learning dental implant classification method based on text prompt training

Abstract

The present invention provides a deep learning dental implant classification method based on text prompt training, which relates to the technical field of dental implant classification, and comprises: fusing knowledge features and image features of CBCT image feature maps in a training set to form a joint feature; embedding a category label feature after splicing the joint feature and the feature map, and obtaining a label-pixel similarity map; processing the label-pixel similarity map to obtain a category prediction result, and minimizing a loss function to update a model parameter; inputting a CBCT image of a to-be-tested dental implant into a trained dental implant classification model to obtain the category of the to-be-tested dental implant; the present invention processes text information related to dental implants as knowledge features, and then fuses the knowledge features and the image features of the CBCT images to form a joint feature, thereby constraining the learning and training of a dental implant classification model through the text information of the dental implant, thereby improving the learning efficiency of the dental implant classification model and the accuracy of predicting the category of the dental implant.

Description

Technical Field

[0001] The present invention relates to the technical field of dental implant classification, and in particular to a deep learning dental implant classification method based on text prompt training. Background Art

[0002] In the field of oral implantology, the classification of oral implants plays a vital role in the subsequent treatment of patients. Due to the problems of missing teeth and loose teeth caused by oral diseases, oral implants are widely used. Currently, dental implants are considered to be the most reliable method for replacing long-term missing teeth. Oral implants usually have some complications during or after treatment, such as biological complications such as peri-implant mucositis and peri-implant inflammation, as well as mechanical complications such as fracture of implants or firmware and loose screws. The cumulative incidence of these complications is as high as 48.03%. Although oral diseases are very common in the population, they may have long-term effects on the overall health of the human body and are closely related to systemic diseases such as stroke, heart disease, diabetes, digestive system problems, and respiratory diseases.

[0003] Once a problem occurs with an implant, or if similar signs of complications are foreseen during a follow-up visit, it must be removed from the mouth using specific tools. The clinician's choice of removal tools is closely related to the shape, surface texture, base connection type, and fixing screw information of the oral implant. This information is very important for the subsequent treatment of the implant, so it is necessary to accurately classify the implants in the mouth. Traditional treatment methods mainly rely on the doctor's professional skills and clinical experience, which to a certain extent limits the stability of the treatment effect.

[0004] There are a series of challenges in manually identifying implants: there are many implant systems on the market, more than a thousand in variety, and with different shapes, even experienced doctors find it difficult to identify them with the naked eye. Traditional classification methods usually use the K nearest neighbor algorithm, which can improve the classification efficiency to a certain extent, but it relies on prior knowledge and requires manual interaction, and cannot be fully automated; in the field of deep learning, pre-trained models are often fine-tuned on large natural image datasets, but these methods have not effectively solved the problem of implant classification, for example:

[0005] 1) Although the overall morphology of implant systems is similar, there are differences in details, such as taper and thread shape, and existing methods cannot effectively analyze these differences;

[0006] 2) Traditional methods and general deep learning methods ignore the anatomical structure description and material information of dental implants, which makes the network unable to fully understand the knowledge text characteristics of implants.

[0007] Therefore, there is an urgent need for a new automated method that can automatically and accurately classify implants in three-dimensional CBCT images. Summary of the invention

[0008] The purpose of the present invention is to provide a deep learning dental implant classification method based on text prompt training to solve the problems raised in the above background technology.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] A deep learning dental implant classification method based on text prompt training includes the following steps:

[0011] S1, obtain CBCT image datasets of various types of dental implants, and randomly divide the CBCT image data into training sets and test sets;

[0012] S2, inputting the CBCT image into the image encoder in the dental implant classification model for feature extraction to obtain a feature map, and inputting the feature map into the fully connected module to obtain image features;

[0013] S3, taking the text information corresponding to the CBCT images of the dental implants in the training set as text labels, and inputting them into the knowledge text encoder of the dental implant classification model to obtain knowledge features;

[0014] S4, concatenating the image features and knowledge features and inputting them into the knowledge attention module of the dental implant classification model to obtain the joint features;

[0015] S5, inputting both the joint feature and the feature map into the upsampling module of the dental implant classification model for decoding and concatenation, obtaining a joint feature map containing the joint feature and embedding the corresponding category label feature thereon, calculating the similarity between the category label feature and the image feature to obtain a label-pixel similarity map;

[0016] S6, processing the label-pixel similarity map through the classifier of the dental implant classification model to obtain the category prediction result, and calculating the classification loss function, and continuously updating the parameters of the model through gradient back propagation. When the loss function converges to the minimum value, the model training is completed to obtain the trained dental implant classification model;

[0017] S7, testing the trained dental implant classification model through the CBCT images in the test set, when the accuracy of the output result of the dental implant classification model exceeds a preset threshold, inputting the CBCT image of the dental implant to be tested into the trained dental implant classification model to obtain the category of the dental implant to be tested, and retraining the dental implant classification model when the accuracy of the output result of the dental implant classification model does not exceed the preset threshold.

[0018] Furthermore, the specific logic of concatenating the image features and the knowledge features and inputting them into the knowledge attention module of the dental implant classification model to obtain the joint features is as follows:

[0019] S41, concatenate the image features and the knowledge features in the channel dimension, so as to splice the image features and the knowledge features to obtain spliced ​​features, which are as follows:

[0020] The image features are labeled and The knowledge feature is labeled as K, and K∈R N*C , mark the splicing feature as T fuse , then T fuse ∈R (M+N)*C , where R represents the number domain, M represents the feature map, C represents the number of channels, and N represents the number of categories of dental implants;

[0021] S42, after normalizing the concatenated features, input them into the knowledge attention module to perform self-attention calculation to obtain self-attention features;

[0022] S43: After adding the concatenated features and the self-attention features, the concatenated features are passed through a multi-head perceptron to obtain a joint feature enhanced with knowledge information and image information.

[0023] Furthermore, the process of calculating the similarity between the category label feature and the image feature is as follows:

[0024] F n (i,j,k)=I(i,j,k)*L(n)∈R L*H*W*N

[0025] Among them, i, j, k represent the coordinate value of a pixel point on the label-pixel similarity map, F n (i,j,k) represents the probability of the pixel point (i,j,k) in the label-pixel similarity map being classified into the nth category label, I(i,j,k) represents the image feature at the pixel point (i,j,k) in the label-pixel similarity map, L(n) represents the category label value of the nth category label feature of the label-pixel similarity map, H, W, L represent the height, width and length of the label-pixel similarity map image respectively.

[0026] Furthermore, the label-pixel similarity map is processed by the classifier of the dental implant classification model to obtain the category prediction result, and the classification loss function is calculated. The parameters of the model are continuously updated through gradient back propagation. When the loss function converges to the minimum value, the model training is completed. The specific logic of the trained dental implant classification model is as follows:

[0027] S61, processing the label-pixel similarity map by a classifier of the dental implant classification model to obtain a classification prediction, as follows:

[0028] S611, performing deviation normalization processing on the label-pixel similarity map, and mapping the result value to between [0-1] by linear transformation of the original data, so as to obtain the foreground of the label-pixel similarity map. The calculation process is as follows:

[0029]

[0030] in, Indicates the foreground value of the pixel (i, j, k) classified into the nth category label, F n (i,j,k) represents the probability of the pixel point (i,j,k) being classified into the nth category label, max(F n (i,j,k)) represents the maximum probability of the pixel point (i,j,k) being classified into each category label, min(F n (i,j,k)) represents the minimum probability of the pixel point (i,j,k) being classified into each category label;

[0031] S612, obtaining the label-pixel similarity map background through the label-pixel similarity map foreground, and the calculation formula is as follows:

[0032]

[0033] Among them, F bg (i,j,k) represents the background value at pixel (i,j,k), It represents the maximum value of the foreground value classified into each category label at the pixel point (i, j, k), α represents the hyperparameter for adjusting the background confidence score, and α≥1;

[0034] S613, fusion of label-pixel similarity map foreground and label-pixel similarity map background, the fusion formula is as follows:

[0035]

[0036] in, represents the fusion value of the pixel point (i, j, k) classified into the nth category label, γ represents the fusion weight, which is used to control the proportion of foreground and background in the fusion process, and γ∈[0,1];

[0037] S614, yes Perform argmax operation, return the category label with the maximum value, and get the final category prediction result;

[0038] S62, a loss function is calculated based on the binary cross entropy loss between the classification label and the classification prediction. The calculation formula of the loss function is as follows:

[0039]

[0040] Where L represents the loss function value, P represents the category prediction output by the dental implant classification model, and P v ∈R 1×N , Y represents the category label, and Y∈R 1×N , σ represents the preset parameters of the dental implant classification model, and 0<σ<1;

[0041] S63, through reverse gradient propagation, the parameters of the image encoder, upsampling module, knowledge attention module and classifier are updated. When the loss function converges to the minimum value, the model training is completed to obtain a trained dental implant classification model. The input of the trained dental implant classification model is the CBCT image, and the output is the dental implant category.

[0042] Furthermore, step S2 includes the following steps:

[0043] S21, performing multi-resolution feature extraction on the CBCT image through an image encoder to obtain feature maps with different resolutions at multiple stages;

[0044] S22, inputting the feature map of the last stage of the image encoder into the fully connected module to obtain a one-dimensional image feature;

[0045] S23, the features of the image encoder except the first stage Figure 1 The densely connected feature enhancement module corresponding to the input is used to enhance the features to obtain the features output by the image encoder in multiple stages. Figure 1 A corresponding stage-by-stage enhanced feature map;

[0046] Among them, in step S5: the joint features and the enhanced feature map of the last stage are input into the upsampling module for decoding and splicing to obtain the joint feature map corresponding to the last stage, and the joint feature map is spliced ​​with the enhanced feature map of the previous stage of the last stage in the upsampling module to obtain the joint feature map corresponding to the previous stage of the last stage, and so on to obtain joint feature maps corresponding to multiple stages one by one, and embed the corresponding category label features in each joint feature map, calculate the similarity between the category label features and the image features to obtain the label-pixel similarity map in each stage except the first stage,

[0047] Among them, in step S614, the After finding the average value, perform the argmax operation;

[0048] In step S62, the loss function values ​​at each stage are accumulated as the total loss function value.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The deep learning dental implant classification method based on text prompt training of the present invention processes the text information related to dental implants as knowledge features, and then fuses the knowledge features and the image features of CBCT images to form joint features. In this way, the learning and training of the dental implant classification model is constrained by the text information of the dental implants, thereby improving the learning efficiency of the dental implant classification model and the accuracy of predicting the dental implant category. The densely connected feature enhancement module adjusts the effective kernel function of the feature point receptive field without reducing the resolution of the feature map, which can effectively restore the global information of the feature map and further improve the classification accuracy of the dental implant classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of the process of the deep learning dental implant classification method based on text prompt training in the present invention;

[0052] Figure 2 It is a framework diagram of the dental implant classification model in the present invention;

[0053] Figure 3 It is a framework diagram of each convolution module in the feature extraction of the present invention;

[0054] Figure 4 It is a framework diagram of the densely connected feature enhancement module in the present invention;

[0055] Figure 5 This is a framework diagram of the knowledge attention module in the present invention. DETAILED DESCRIPTION

[0056] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0057] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0058] See also Figure 1-5 , the present invention provides the following embodiments:

[0059] Embodiment 1: The present invention provides a deep learning dental implant classification method based on text prompt training, comprising:

[0060] S1, obtain CBCT image datasets of various types of dental implants, and randomly divide the CBCT image data into training sets and test sets;

[0061] It should be noted that the CBCT image data of various dental implants can be obtained from a dental implant image database, specifically from a database of a relevant dental hospital or public data on the Internet. The ratio of the training set to the test set can be 70:30 or 80:20, and there is no limitation here.

[0062] Furthermore, since the acquired CBCT image data are from multiple centers, the acquired CBCT image data of the dental implant are preprocessed, and the preprocessing includes standardization and normalization of image intensity values, and of course, may also include one or more preprocessing operations such as resizing, cropping, histogram equalization, and denoising, which are not limited here;

[0063] Among them, the voxel intensity value of the CBCT image data after preprocessing is:

[0064]

[0065] It should be noted that P mean , P sta , P min , P max , P, and P' respectively represent the average value of voxel intensity values, the standard deviation of voxel intensity values, the minimum value of voxel intensity values, the maximum value of voxel intensity values, the initial voxel intensity value, and the voxel intensity value after preprocessing in the CBCT image data.

[0066] S2, inputting the CBCT image into the image encoder in the dental implant classification model for feature extraction to obtain a feature map, and inputting the feature map into the fully connected module to obtain image features;

[0067] It should be noted that the CBCT image is input into the image encoder in the dental implant classification model for downsampling to extract features to obtain a feature map, and then a one-dimensional image feature is obtained from the feature map through a fully connected module. The existing technology can be used and will not be elaborated here.

[0068] S3, the text information corresponding to the CBCT images of the dental implants in the training set is used as a text label and input into the knowledge text encoder of the dental implant classification model to obtain the knowledge feature. The expression of the knowledge feature is as follows:

[0069] K=FC(BERT(W))

[0070] The text information is calibrated as W, and the text information is the text information of the thread shape, implant proportion and other categories of dental implants. The categories of dental implants include but are not limited to Nobel ActiveA, Nobel ReplaceA ConicalConnection, CAMLOGA SCREW-LINE, OSSTEMA ET III, and SKYA. The knowledge features are calibrated as K, BERT(·) represents the knowledge text encoder, and FC(·) represents the full connection, so as to output the knowledge features of a one-dimensional vector. The text information is a key component of knowledge guidance, and its role is to convert the description of subtype performance into knowledge features. The knowledge features guide the image features to focus on the area related to the target tissue, in order to encode the subtype performance description into a more general semantic feature, so as to constrain the dental implant classification model to better learn and train;

[0071] It should be noted that the core of the knowledge text encoder is a learnable model that can understand and process natural language descriptions and convert these descriptions into numerical features that can be understood by computers. In this embodiment, ClinicalBert is used as the knowledge text encoder and is further fine-tuned on the MIMIC-III dataset. MIMIC-III is a large medical database that contains a large amount of clinical data. This enables ClinicalBert to learn rich medical terms and knowledge, and to better understand and process medical image data, which is very suitable for fine-grained image classification of dental implants.

[0072] S4, the image features and knowledge features are concatenated and input into the knowledge attention module of the dental implant classification model to obtain the joint features. The knowledge attention module is as follows Figure 5 As shown, the following steps are included:

[0073] S41, concatenate the image features and the knowledge features in the channel dimension, so as to splice the image features and the knowledge features to obtain spliced ​​features, which are as follows:

[0074] The image features are labeled and Knowledge feature K∈R N*C , mark the splicing feature as T fuse , then T fuse ∈R (M+N)*C , where R represents the number domain, M represents the feature map, C represents the number of channels, and N represents the number of categories of dental implants;

[0075] S42, after normalizing the concatenated features, input them into the knowledge attention module to perform self-attention calculation to obtain self-attention features;

[0076] It should be noted that in the self-attention calculation process, each head learns different attention weights. These attention heads can be calculated in parallel to speed up the training. The knowledge attention module is only added after the last stage of the visual encoder to save computing resources. The self-attention technology can adopt the existing technology and will not be described in detail here.

[0077] S43: After adding the concatenated features and the self-attention features, a multi-head perceptron is used to obtain a joint feature after the knowledge information and the image information are enhanced. The multi-head perceptron is a prior art and will not be described in detail here.

[0078] S5, the joint features and feature maps are input into the upsampling module of the dental implant classification model for decoding and concatenation, a joint feature map containing the joint features is obtained, and the corresponding category label features are embedded thereon, and the similarity between the category label features and the image features is calculated to obtain a label-pixel similarity map. The calculation process is as follows:

[0079] F n (i,j,k)=I(i,j,k)*L(n)∈R L*H*W*N

[0080] Among them, i, j, k represent the coordinate value of a pixel point on the label-pixel similarity map, F n (i,j,k) represents the probability of the pixel point (i,j,k) in the label-pixel similarity map being classified into the nth category label, I(i,j,k) represents the image feature at the pixel point (i,j,k) in the label-pixel similarity map, L(n) represents the category label value of the nth category label feature of the label-pixel similarity map, H, W, L represent the height, width and length of the label-pixel similarity map image respectively.

[0081] S6, processing the label-pixel similarity map through the classifier of the dental implant classification model to obtain the category prediction result, and calculating the classification loss function, continuously updating the parameters of the model through gradient back propagation, and completing the model training when the loss function converges to the minimum value to obtain the trained dental implant classification model, including the following steps:

[0082] S61, processing the label-pixel similarity map by a classifier of the dental implant classification model to obtain a classification prediction, as follows:

[0083] S611, performing deviation normalization processing on the label-pixel similarity map, and mapping the result value to between [0-1] by linear transformation of the original data, so as to obtain the foreground of the label-pixel similarity map. The calculation process is as follows:

[0084]

[0085] It should be noted that Indicates the foreground value of the pixel (i, j, k) classified into the nth category label, F n (i,j,k) represents the probability of the pixel point (i,j,k) being classified into the nth category label, max(F n (i,j,k)) represents the maximum probability of the pixel point (i,j,k) being classified into each category label, min(F n (i,j,k)) represents the minimum probability of the pixel point (i,j,k) being classified into each category label;

[0086] S612, obtaining the label-pixel similarity map background through the label-pixel similarity map foreground, and the calculation formula is as follows:

[0087]

[0088] It should be noted that F bg (i,j,k) represents the background value at pixel (i,j,k), It represents the maximum value of the foreground value classified into each category label at the pixel point (i, j, k), α represents the hyperparameter for adjusting the background confidence score, and α≥1.

[0089] S613, fusion of label-pixel similarity map foreground and label-pixel similarity map background, the fusion formula is as follows:

[0090]

[0091] It should be noted that represents the fusion value of the pixel (i, j, k) classified into the nth category label, γ represents the fusion weight, which is used to control the proportion of foreground and background in the fusion process, and γ∈[0,1], where γ=0 represents the complete background and γ=1 represents the complete foreground;

[0092] S614, yes Perform argmax operation, return the category label with the maximum value, and get the final category prediction result;

[0093] S62, a loss function is calculated based on the binary cross entropy loss between the classification label and the classification prediction. The calculation formula of the loss function is as follows:

[0094]

[0095] It should be noted that L represents the loss function value, P represents the category prediction output by the dental implant classification model, and P v ∈R 1×N , Y represents the category label, and Y∈R 1×N , σ represents the preset parameters of the dental implant classification model, and 0<σ<1;

[0096] S63, through reverse gradient propagation, update the parameters of the image encoder, upsampling module, knowledge attention module and classifier, and complete the model training when the loss function converges to the minimum value, and obtain the trained dental implant classification model. The input of the trained dental implant classification model is the CBCT image, and the output is the dental implant category. The model parameters are updated according to the existing technology, which will not be elaborated here.

[0097] S7, testing the trained dental implant classification model using the CBCT images in the test set, and when the accuracy of the output result of the dental implant classification model exceeds a preset threshold, inputting the CBCT image of the dental implant to be tested into the trained dental implant classification model to obtain the category of the dental implant to be tested, and retraining the dental implant classification model when the accuracy of the output result of the dental implant classification model does not exceed the preset threshold;

[0098] It should be noted that the preset threshold value may be 95%, 98%, etc., which is set by the staff according to the actual situation and is not limited here.

[0099] Embodiment 2: Embodiment 2 is optimized on the basis of Embodiment 1 to improve the classification accuracy of the dental implant classification model, as follows:

[0100] Step S2 includes the following steps:

[0101] S21, performing multi-resolution feature extraction on the CBCT image through an image encoder to obtain feature maps with different resolutions at multiple stages, as follows:

[0102] The CBCT image data is input into the image encoder of the dental implant classification model for multi-resolution feature extraction, and the feature map of the resolution is saved before each downsampling stage, so as to obtain feature maps with different resolutions in multiple stages;

[0103] As an implementation method, feature extraction is as follows Figure 3 As shown in the figure, it includes 1 basic convolution block and 4 residual convolution modules. Each convolution module is followed by a max pooling module for downsampling. Each basic convolution module and residual convolution module includes 2 3×3×3 convolution layers, 2 batch normalization layers, and 2 ReLU activation functions. Each time the data is extracted, the size of the feature map is reduced to half of the original size. The relationship between the feature maps before and after downsampling is as follows:

[0104]

[0105] It should be noted that M v-1 represents the feature map output by the image encoder before the vth downsampling stage of the CBCT image, C represents the number of feature map channels, R represents the number domain, H', W', L' represent the height, width and length of the CBCT image respectively, and M v It represents the feature map output by the image encoder after the vth downsampling stage of the CBCT image, v represents the number of different stages, and v = 2, 3, ..., V, V represents the total number of stages, and V∈N + In this embodiment, the image encoder performs four downsampling stages in total, where V=4;

[0106] S22, inputting the feature map of the last stage of the image encoder into the fully connected module to obtain a one-dimensional image feature;

[0107] S23, the features of the image encoder except the first stage Figure 1 The densely connected feature enhancement module corresponding to the input is used to enhance the features to obtain the features output by the image encoder in multiple stages. Figure 1 A corresponding stage-by-stage enhanced feature map;

[0108] As an implementation method, step S23 includes the following steps:

[0109] S231, the feature map output by the image encoder at the vth stage is recorded as X v , and the feature map X vThe input is sent to the dense connection feature enhancement module and undergoes a dilated convolution with a dilation factor of 1, and the feature map Y1 output by the dense connection feature enhancement module is saved. v ;

[0110] S232: Feature map Y1 v and X v Add and go through a dilated convolution with a dilation factor of 2 to save the feature map Y2 output by the densely connected feature enhancement module v ;

[0111] S233: Feature map Y2 v With X v Add and go through a dilated convolution with a dilation factor of 4 to save the feature map Y3 output by the densely connected feature enhancement module v , X v 、Y1 v 、Y2 v and Y3 v Perform attention-based multi-channel feature fusion to obtain enhanced feature maps

[0112] Among them, record X v =Y0 v , the feature map output by the densely connected feature enhancement module after each dilated convolution is expressed as:

[0113]

[0114] Among them, Y k v Represents the feature map output by the densely connected feature enhancement module after the xth dilated convolution;

[0115] The feature map finally output by the densely connected feature enhancement module is expressed as:

[0116]

[0117] in, Denotes the densely connected feature enhancement module for the feature map M vThe enhanced feature map finally output after processing, Z is the total number of layers of the densely connected feature enhancement module, z represents the number of different layers in the densely connected feature enhancement module, l is the number of dilated convolutions in each layer of the densely connected feature enhancement module, Dilate(A) represents the dilated convolution operation, y is the dilation coefficient, in this embodiment, y={1, 2, 4}, in the process of performing attention-based multi-channel feature fusion, the sum of the four features is passed through two branches to extract the channel attention weight, one branch uses global average pooling to extract the attention of the global feature, and the other branch directly uses the convolution with a convolution kernel size of 1 to extract the channel attention of the local feature, so that the final fused feature map has not only global features but also local features, so that the final result is more accurate. Compared with other feature map fusion methods, the more complex attention-based multi-channel feature fusion method for feature fusion has great potential to produce better results;

[0118] It should be noted that the dense connection feature enhancement module is as follows Figure 4 As shown in the figure, the dilated convolution in the densely connected feature enhancement module is an effective kernel function that adjusts the receptive field of feature points without reducing the resolution of the feature map. By using this feature, the feature map after each dilation is added to the feature map input in the previous stage, and the feature enhancement is performed again, which can show a strong ability to improve the segmentation accuracy.

[0119] Among them, in the step S5 of obtaining the joint feature map: the joint feature and the enhanced feature map of the last stage are input into the upsampling module for decoding and splicing to obtain the joint feature map corresponding to the last stage, and the joint feature map is spliced ​​with the enhanced feature map of the previous stage of the last stage in the upsampling module to obtain the joint feature map corresponding to the previous stage of the last stage, and so on to obtain joint feature maps corresponding to multiple stages one by one, and embed the corresponding category label features in each joint feature map, calculate the similarity of the category label features and the image features to obtain the label-pixel similarity map in each stage except the first stage, and the specific calculation method is the same as the method of step S5 in the above-mentioned embodiment 1, which will not be repeated here;

[0120] Among them, in step S614: After finding the average value, perform the argmax operation;

[0121] Wherein, in step S62: the loss function values ​​at each stage are accumulated as the total loss function value.

[0122] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0123] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0124] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0125] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A deep learning dental implant classification method based on text prompt training, characterized in that: The steps include: S1, obtain CBCT image datasets of various dental implants, and randomly divide the CBCT image data into training sets and test sets; S2, inputting the CBCT image into the image encoder in the dental implant classification model for feature extraction to obtain a feature map, and inputting the feature map into the fully connected module to obtain image features; S3, taking the text information corresponding to the CBCT images of the dental implants in the training set as text labels, and inputting them into the knowledge text encoder of the dental implant classification model to obtain knowledge features; S4, concatenating the image features and knowledge features and inputting them into the knowledge attention module of the dental implant classification model to obtain the joint features; S5, inputting both the joint feature and the feature map into the upsampling module of the dental implant classification model for decoding and concatenation, obtaining a joint feature map containing the joint feature and embedding the corresponding category label feature thereon, calculating the similarity between the category label feature and the image feature to obtain a label-pixel similarity map; S6, processing the label-pixel similarity map through the classifier of the dental implant classification model to obtain the category prediction result, and calculating the classification loss function, and continuously updating the parameters of the model through gradient back propagation. When the loss function converges to the minimum value, the model training is completed to obtain the trained dental implant classification model; S7, testing the trained dental implant classification model using the CBCT images in the test set, and when the accuracy of the output result of the dental implant classification model exceeds a preset threshold, inputting the CBCT image of the dental implant to be tested into the trained dental implant classification model to obtain the category of the dental implant to be tested, and retraining the dental implant classification model when the accuracy of the output result of the dental implant classification model does not exceed the preset threshold; The specific logic of concatenating image features and knowledge features and inputting them into the knowledge attention module of the dental implant classification model to obtain the joint features is as follows: S41, concatenate the image features and the knowledge features in the channel dimension, so as to splice the image features and the knowledge features to obtain spliced ​​features, which are as follows: The image features are labeled and The knowledge feature is marked as K, and K∈R N*C , mark the splicing feature as T fuse , then T fuse ∈R (M+N)*C , where R represents the number domain, M represents the feature map, C represents the number of channels, and N represents the number of categories of dental implants; S42, after normalizing the concatenated features, input them into the knowledge attention module to perform self-attention calculation to obtain self-attention features; S43: After adding the concatenated features and the self-attention features, the concatenated features are passed through a multi-head perceptron to obtain a joint feature enhanced with knowledge information and image information.

2. The deep learning dental implant classification method based on text prompt training according to claim 1, characterized in that: The process of calculating the similarity between the category label feature and the image feature is as follows: F n (i,j,k)=I(i,j,k)*L(n)∈R L*H*W*N Among them, i, j, k represent the coordinate value of a pixel point on the label-pixel similarity map, F n (i, j, k) represents the probability of the pixel point (i, j, k) in the label-pixel similarity map being classified into the nth category label, I(i, j, k) represents the image feature at the pixel point (i, j, k) in the label-pixel similarity map, L(n) represents the category label value of the nth category label feature of the label-pixel similarity map, H, W, L represent the height, width and length of the label-pixel similarity map image respectively.

3. The deep learning dental implant classification method based on text prompt training according to claim 1, characterized in that: The label-pixel similarity map is processed by the classifier of the dental implant classification model to obtain the category prediction result, and the classification loss function is calculated. The parameters of the model are continuously updated through gradient back propagation. When the loss function converges to the minimum value, the model training is completed. The specific logic of the trained dental implant classification model is as follows: S61, processing the label-pixel similarity map by a classifier of the dental implant classification model to obtain a classification prediction, as follows: S611, performing deviation normalization processing on the label-pixel similarity map, and mapping the result value to between [0-1] by linear transformation of the original data, so as to obtain the foreground of the label-pixel similarity map. The calculation process is as follows: in, Indicates the foreground value of the pixel (i, j, k) classified into the nth category label, F n (i, j, k) represents the probability of the pixel point (i, j, k) being classified into the nth category label, max(F n (i, j, k)) represents the maximum probability of the pixel point (i, j, k) being classified into each category label, min(F n (i, j, k)) represents the minimum probability of the pixel point (i, j, k) being classified into each category label; S612, obtaining the label-pixel similarity map background through the label-pixel similarity map foreground, and the calculation formula is as follows: Among them, F bg (i, j, k) represents the background value at pixel point (i, j, k), It represents the maximum value of the foreground value classified into each category label at the pixel point (i, j, k), α represents the hyperparameter for adjusting the background confidence score, and α≥1; S613, fusion of label-pixel similarity map foreground and label-pixel similarity map background, the fusion formula is as follows: in, represents the fusion value of the pixel (i, j, k) classified into the nth category label, γ represents the fusion weight, which is used to control the proportion of foreground and background in the fusion process, and γ∈[0,1]; S614, yes Perform argmax operation, return the category label with the maximum value, and get the final category prediction result; S62, a loss function is calculated based on the binary cross entropy loss between the classification label and the classification prediction. The calculation formula of the loss function is as follows: in, represents the loss function value, P represents the category prediction output by the dental implant classification model, and P v ∈R 1×N , Y represents the category label, and Y∈R 1×N , σ represents the preset parameters of the dental implant classification model, and 0<σ<1; S63, through reverse gradient propagation, the parameters of the image encoder, upsampling module, knowledge attention module and classifier are updated. When the loss function converges to the minimum value, the model training is completed to obtain a trained dental implant classification model. The input of the trained dental implant classification model is the CBCT image, and the output is the dental implant category.

4. The method for dental implant classification based on deep learning and text prompt training according to claim 3, characterized in that: The step S2 comprises the following steps: S21, performing multi-resolution feature extraction on the CBCT image through an image encoder to obtain feature maps with different resolutions at multiple stages; S22, inputting the feature map of the last stage of the image encoder into the fully connected module to obtain a one-dimensional image feature; S23, inputting the feature maps of the image encoder except the first stage into the corresponding densely connected feature enhancement module one by one for feature enhancement, so as to obtain stage-by-stage enhanced feature maps corresponding one by one to the feature maps output by the image encoder in multiple stages; Among them, in step S5: the joint features and the enhanced feature map of the last stage are input into the upsampling module for decoding and splicing to obtain the joint feature map corresponding to the last stage, and the joint feature map is spliced ​​with the enhanced feature map of the previous stage of the last stage in the upsampling module to obtain the joint feature map corresponding to the previous stage of the last stage, and so on to obtain joint feature maps corresponding to multiple stages one by one, and embed the corresponding category label features in each joint feature map, calculate the similarity between the category label features and the image features to obtain the label-pixel similarity map in each stage except the first stage, Among them, in step S614, the After finding the average value, perform the argmax operation; In step S62, the loss function values ​​at each stage are accumulated as the total loss function value.

Images