Auxiliary dental implant generation method based on diffusion model

Abstract

The present invention relates to the technical field of stomatology. Provided is an auxiliary dental implant generation method based on a diffusion model. The method in the present invention comprises: acquiring oral CBCT image data of historical patients, preprocessing the oral CBCT image data of the historical patients to obtain a CBCT image dataset, using the CBCT image dataset to train a multi-task segmentation network, and using the segmentation network to obtain an intraoral tissue segmentation result; using the intraoral tissue segmentation result to train detection networks from the three dimensions of a cross-sectional plane, a coronal plane and a sagittal plane, respectively; using the detection networks to obtain detection results in the three directions of the cross-sectional plane, the coronal plane and the sagittal plane; fusing the detection results in the three directions of the cross-sectional plane, the coronal plane and the sagittal plane, and using a majority voting algorithm to construct a three-dimensional bounding box, so as to acquire an edentulous area; and using the intraoral segmentation result and the edentulous area as prompt information to guide, by means of an iterative process, a network to generate a post-implantation effect. The implantation effect obtained by the present invention is highly accurate, thereby providing a more precise auxiliary tool for stomatology.

Description

An auxiliary dental implant generation method based on a diffusion model TECHNICAL FIELD

[0001] The present application relates to the field of oral medical technology, in particular to an auxiliary dental implant generation method based on a diffusion model. BACKGROUND

[0002] With the continuous development of oral medicine, digital technology is revolutionizing the design and implementation of dental treatment programs, and has been widely used in assisting dentists in designing programs. Among them, dental implant surgery is a key means to compensate for tooth loss, restore masticatory function and aesthetic appearance, and its intelligent and precise development needs are increasingly urgent. In order to meet the needs of modern dentistry, exploring an intelligent, rapid and efficient dental implant generation method to assist doctors in designing surgical programs can not only greatly improve the convenience and accuracy of surgical design, but also provide strong technical support for subsequent diagnosis and treatment tasks, thereby effectively ensuring the efficient execution and success rate of implant surgery.

[0003] The traditional dental implant generation method uses materials such as alginate and silica gel to make precise impressions of teeth and surrounding tissues in the patient's mouth, pours plaster into the impressions to form solid tooth models, and then the doctor designs the expected dental implant position and shape on the model. However, the intraoral impression will cause severe discomfort to the patient, and the design process is heavily dependent on the experience and technical level of the doctor. There are problems of low precision, low efficiency and long time consumption.

[0004] The existing digital implant program design needs multiple steps to cooperate, and the limitations of each step limit the accuracy of the implant position. First, the contrast between different tissues in the CBCT image is low and closely arranged, and the traditional segmentation method cannot distinguish the boundary information between the intraoral tissues, which directly leads to low-quality segmentation images, affecting the subsequent position determination. At the same time, the traditional method is also limited for the determination of the missing tooth position in the three-dimensional image, usually needs to project the three-dimensional image into a two-dimensional depth map for detection, which lacks spatial information for the positioning of the position. For the generation of images, the traditional method often uses a generative adversarial network, the training process is unstable, and mode collapse is prone to occur, and the calculation resources and time consumption for high-resolution images are large.

[0005] The above information disclosed in the BACKGROUND section is only used to enhance the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0006] The purpose of the present application is to provide an auxiliary dental implant generation method based on a diffusion model to solve the problems raised in the above BACKGROUND.

[0007] To achieve the above object, the present application provides the following technical solutions:

[0008] An auxiliary dental implant generation method based on a diffusion model, the specific steps comprising:

[0009] S1: obtaining historical patient oral cavity CBCT image data, preprocessing the historical patient oral cavity CBCT image data to obtain a CBCT image set, and determining a segmentation result set, a cross-sectional detection result set, a coronal plane detection result set and a sagittal plane detection result set based on the historical patient oral cavity CBCT image data, wherein the segmentation result set comprises an edge information set and a mask information set;

[0010] S2: training a multi-task segmentation network with the CBCT image set as input and the segmentation result set as label, obtaining and preprocessing oral cavity CBCT image data of a to-be-predicted object, inputting the obtained CBCT image data into the trained segmentation network, and obtaining an intraoral segmentation result of the intraoral tissue of the to-be-predicted object, wherein the intraoral segmentation result comprises edge information and mask information;

[0011] S3: training corresponding detection networks with the mask information set as input and the cross-sectional detection result set, the coronal plane detection result set and the sagittal plane detection result set as labels, inputting the mask information of the intraoral tissue of the to-be-predicted object into the trained detection networks to obtain detection results, and the detection results comprising cross-sectional direction detection results, coronal plane direction detection results and sagittal plane direction detection results;

[0012] S4: fusing the cross-sectional direction detection results, the coronal plane direction detection results and the sagittal plane direction detection results, constructing a three-dimensional bounding box by using a majority voting algorithm, and obtaining a tooth loss region;

[0013] S5: inputting the CBCT image and a random noise image into the trained diffusion model, continuously diffusing and denoising the noise image through an encoder, an intermediate layer and a decoder, taking the intraoral segmentation result and the tooth loss region as prompt information, and guiding the network to generate an implantation effect through an iterative process.

[0014] Further, the process of network training in the segmentation network comprises:

[0015] S21: performing image enhancement on the preprocessed CBCT image by using a Haar filtering algorithm;

[0016] S22: The pre-processed CBCT image and the enhanced image are spliced to form a feature map, and the feature map is input into an encoder for feature extraction to obtain a first feature map, specifically: the feature map is evenly divided into 4 groups of features along the channel dimension, the 1st group adopts 1*1*1 convolution to maintain the current scale; the 2nd group and the 3rd group adopt 3*3*3 deep convolution with different expansion rates to capture information at different levels, wherein the expansion rate and the padding rate are set to 1 and 2 respectively, and the 4th group adopts image pooling operation to statistically process global spatial information; the features after hierarchical processing are spliced and input into batch normalization integration and ReLU activation function to form output features, and the output features and the feature map are superimposed through residual connection to form the first feature map;

[0017] S23: The first feature map is subjected to feature enhancement through a cross-attention network, specifically: the channel dimension of the first feature map is adjusted by 1*1*1 convolution, and two parallel convolution layers further extract context information in the horizontal and vertical directions; the importance of the features is adaptively enhanced through a multi-head attention module, and the enhanced features are fused through a multi-layer perceptron to integrate information in different channels and spatial positions;

[0018] S24: The decoder gradually restores the spatial resolution of the cross-attention network output features through upsampling and multi-scale convolution, and connects the cross-attention network output features and the first feature map through a skip connection to obtain a second feature map;

[0019] S25: The second feature map is input into an output layer and processed by an activation function to output a final intraoral segmentation result;

[0020] S26: The training process adopts a weighted loss combining Dice and cross-entropy to optimize the boundary morphology.

[0021] Further, the expression of the Haar filter is:

[0022]

[0023] where H N is the Haar filter matrix, h N-1 (N-1 / N) is the N-1 order Haar basis function, and N*N is the order of the Haar filter matrix.

[0024] Further, the output of the multi-scale convolution is represented as:

[0025] F = δ (X (Concat (W1 (S'1), W2 (S'2), W3 (S'3), P (S'4))) ) + S (1)

[0026] Wherein, F is the first feature map, W1, W2, W3, W4 are the weights of the results of the corresponding processing of the four groups of features respectively; Concat represents the splicing operation, P represents the global pooling operation, X represents the normalization operation, δ represents the ReLU activation function operation, S'1, S'2, S'3, S'4 are the results of the corresponding processing of the four groups of features respectively, and S(1) represents the feature map.

[0027] Further, the output calculation method of the segmentation network is:

[0028]

[0029] Wherein, P represents the final segmentation result, Softmax(·) represents the normalized exponential function, M represents the pixel value output by the decoder, and C represents the number of classes; M c represents the prediction result corresponding to the cth class in M, and c is the class index;

[0030] The loss function expression of the segmentation network is:

[0031]

[0032]

[0033] Wherein, represents the Dice loss function, K represents the total number of voxels, i represents the voxel index, represents the cross-entropy loss function, represents the true label of the cth class of the ith voxel, represents the total loss function of the segmentation network; λ1 and λ2 represent the balance loss parameters, and the tooth classes considered by the segmentation result include: central incisor, lateral incisor, canine, first premolar, second premolar, first molar, and second molar.

[0034] Further, the process of training the detection network includes:

[0035] S31: input the intraoral tissue segmentation result set into the backbone network, and use multiple convolutional layers to extract image features;

[0036] S32: input the extracted feature map into the global attention module for feature enhancement;

[0037] S33: the last layer of the backbone network uses spatial pyramid pooling to integrate features of different regions;

[0038] S34: the output of the backbone network enters the fusion network, and the feature information from different scales is fused through the pyramid network;

[0039] S35: input the fused feature map into the detector for regression prediction, output multi-scale prediction results containing boundary box, confidence and class information;

[0040] S36: input the multi-scale prediction results into the loss function and flexible non-maximum suppression algorithm for post-processing, remove the overlapping boundary box through Gaussian confidence decay, and obtain the final detection result;

[0041] S37: the network continuously iterates and adjusts the parameters to ensure that the loss parameter converges to a minimum value when the entire training process is completed, and the optimal weight is obtained; the loss function expression of the detection network is:

[0042] Loss=a*loss obj +b*loss rect +c*loss cls

[0043] Wherein, a, b, c represent weight coefficients, loss obj represents the confidence loss, loss rect represents the rectangular frame loss, loss cls represents the classification loss, and Loss represents the total loss of the detection network.

[0044] Further, the specific steps of inputting the extracted feature map into the global attention module for feature enhancement include:

[0045] S321: input the features extracted by the convolution layer into the channel attention for enhancement; through dimension exchange and multilayer perceptron, the mutual relationship between channels is learned;

[0046] S322: the output of the multilayer perceptron is exchanged in dimension, and the channel attention map is generated through the Sigmoid activation function;

[0047] S323: the output of the channel attention module is spliced with the output features extracted by the convolution layer to obtain the intermediate state, and the intermediate state is sent to the spatial attention module to further extract the spatial features through two 7*7 convolutions;

[0048] S324: the convolution output obtains the channel attention map through the Sigmoid activation function;

[0049] S325: fuse the channel attention weight, spatial attention weight and output features extracted by the convolution layer to make the network focus on the most important features in the image, and output the features F3 for subsequent network layer; the output of the intermediate state and GAM is:

[0050]

[0051] wherein, F1 represents an input feature, F2 represents an intermediate state, F3 represents an output feature, M P is a channel attention map, M S is a spatial attention map, represents a product of pixels; M P (F1) represents applying the channel attention map M P on the input feature F1, and M S (F2) represents applying the spatial attention map M T on the intermediate state F2.

[0052] Further, the specific steps of obtaining the edentulous region are as follows:

[0053] S41: using a two-dimensional convolutional network to detect the edentulous position on the CBCT image in the transverse plane, coronal plane and sagittal plane respectively;

[0054] S42: splicing the candidate boxes output by the detection network in the transverse plane, coronal plane and sagittal plane, and obtaining a three-dimensional bounding box by using a majority voting algorithm to obtain the edentulous region.

[0055] Further, the specific steps of step 5 are as follows:

[0056] S51: inputting a target image x0 into an encoder to map it to a latent space and obtaining a latent variable, and the expression of the latent variable is:

[0057] z0=E(x0)

[0058] wherein, z0 is a latent variable, x0 is an input target image, and E is an encoder encoding function;

[0059] S52: defining a forward diffusion process, adding noise to the latent variable z ~ q(z0) to simulate the data degradation under the real condition, and gradually evolving the target image into a Gaussian noise distribution z T ; the forward diffusion process q(z 1:T |z0) is represented as:

[0060]

[0061] wherein, β t represents a hyperparameter of noise amount from 0 to 1 variance, z1,…, z T is a latent variable with the same dimension as the data z ~ q(z0); q(z 1:T |z0) is a forward diffusion process, t is an index of diffusion steps, and T represents the total number of diffusion steps, represents a Gaussian distribution with a mean of and a variance of β t ; I represents a unit matrix;

[0062] S53: the located toothless position, the segmentation result of the intraoral tissue and the original image are sent into a condition encoder to extract position features, shape features and image features, and the fusion result is obtained by fusing the position features, the shape features and the image features, and the fusion result is represented as:

[0063]

[0064] wherein W is the fusion result, represents the embedded position features, represents the embedded shape features, represents the feature embedding of the current step;

[0065] S54: a reverse diffusion process is defined, and the fusion result is used as the condition information to guide the model to iteratively denoise from a noise image z T , and after T steps, a feature map z0 without noise is obtained;

[0066] Specifically, the reverse diffusion process is represented as:

[0067]

[0068] wherein p θ (z 0:T |W) is the reverse diffusion process, and is a Gaussian distribution with a mean and a variance t ; and θ represents the sampled random noise, and t : = 1-β t ,

[0069] S55: the feature map z0 obtained above is subjected to feature decoding through a convolution layer and an up-sampling layer, the latent space features are mapped back to the explicit space, and a final target image is generated, and the decoding process is represented as:

[0070] x' = D(z0)

[0071] wherein x' is the final target image, and D is a decoder decoding function.

[0072] The improved multi-task three-dimensional convolutional neural network is adopted to segment the intraoral tissue of the CBCT image, the boundary contrast is strengthened by the Haar filter, the perception ability of the network to the edge information is improved, the multi-scale convolution module is used to extract the multi-granularity feature information, the cross attention module is introduced at the connection position of the encoder and the decoder, and the representation ability and the context understanding ability of the network to the image features are improved.

[0073] The application adopts a two-dimensional detection network to detect the tooth loss position from the CBCT image, detects three directions on the cross section, coronal plane and sagittal plane, splices the detection results of the three branches, obtains a three-dimensional bounding box by using a majority voting algorithm, and locates the tooth loss region; compared with projecting the CBCT image into a two-dimensional panoramic film, more spatial information can be retained, and the tooth loss region can be more accurately located;

[0074] The conditional diffusion model is used in the generation network part, images are generated in the gradual denoising process, the generation is more stable compared with the generative adversarial network, and high-resolution images with fine detailed features can be generated. The position information and structure information are used to guide the diffusion process, so that the generated image is more accurate. BRIEF DESCRIPTION OF DRAWINGS

[0075] Fig. 1 is a schematic diagram of the overall method of the application.

[0076] Fig. 2 is a schematic diagram of the segmentation network of the application.

[0077] Fig. 3 is a schematic diagram of the multi-scale convolution module of the application.

[0078] Fig. 4 is a schematic diagram of the tooth loss position detection of the application.

[0079] Fig. 5 is a structural schematic diagram of the diffusion model of the application. DETAILED DESCRIPTION

[0080] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below with specific examples.

[0081] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the application should be understood as the usual meaning understood by those skilled in the art to which the application belongs. The terms "first", "second" and similar words used in the application do not represent any order, quantity or importance, but are only used to distinguish different components. The terms "include" or "contain" and similar words mean that the elements or objects before the word cover the elements or objects listed after the word and their equivalents, and do not exclude other elements or objects. The terms "connect" or "connect" and similar words are not limited to physical or mechanical connection, but can include electrical connection, whether direct or indirect. The terms "up", "down", "left", "right" and the like only represent relative positional relationships, which may change accordingly when the absolute position of the described object changes.

[0082] Embodiment:

[0083] Referring to Fig. 1, the application provides a technical scheme:

[0084] An auxiliary dental implant generation method based on a diffusion model, the specific steps comprising:

[0085] S1: Obtain historical patient oral cavity CBCT image data, pre-process the historical patient oral cavity CBCT image data to obtain a CBCT image set, and determine a segmentation result set, a cross-sectional detection result set, a coronal plane detection result set and a sagittal plane detection result set based on the historical patient oral cavity CBCT image data, wherein the segmentation result set comprises an edge information set and a mask information set;

[0086] Obtain a CBCT data set containing images of edentulous cases, process the data set by using a preprocessing unit, and normalize the pixel value and sampling interval due to the differences in gray information and sampling interval of the same tissue collected by different devices in different hospitals, and divide the data set according to a standard ratio of 7:3. The normalized pixel value is calculated as:

[0087]

[0088] wherein X represents an original pixel value, X mean represents an average value, X sd represents a labeling difference, and X nor represents a normalized pixel value.

[0089] S2: Train a multi-task segmentation network with the CBCT image set as input and the segmentation result set as label, obtain oral cavity CBCT image data of a to-be-predicted object and pre-process the same, input the pre-processed CBCT image to the trained segmentation network, and obtain an intraoral segmentation result of the intraoral tissue of the to-be-predicted object, wherein the intraoral segmentation result comprises edge information and mask information;

[0090] The segmentation result considers tooth categories, specifically including central incisors, lateral incisors, canines, first premolars, second premolars, first molars and second molars.

[0091] Referring to FIG. 2, the segmentation network structure of the dental implant generation model is shown in FIG. 2;

[0092] The process of network training in the segmentation branch of the dental implant generation model comprises:

[0093] S21: Perform image enhancement on the pre-processed CBCT image by using a Haar filtering algorithm;

[0094] The expression of the Haar filtering is:

[0095]

[0096] wherein H N is a Haar filtering matrix, h N-1(N-1 / N) is the N-1 order Haar base function, and N*N is the order of the Haar filter matrix.

[0097] S22: The pre-processed CBCT image and the enhanced image are spliced to form a feature map, and the feature map is input into an encoder for feature extraction to obtain a first feature map. Specifically, the feature map is evenly divided into 4 groups of features along the channel dimension, the first group adopts 1*1*1 convolution to maintain the current scale, the second group and the third group adopt 3*3*3 deep convolution with different expansion rates to capture information at different levels, wherein the expansion rate and the padding rate are set to 1 and 2 respectively, and the fourth group adopts image pooling operation to statistically process global spatial information; the features after hierarchical processing are spliced and input into batch normalization integration and ReLU activation function to form output features, and the output features and the feature map are superimposed through residual connection to form the first feature map.

[0098] The filtered image and the original CBCT image are spliced and input into an encoder for feature extraction. The encoder includes three multi-scale convolution layers, three maximum pooling layers and three down-sampling layers. The multi-scale convolution layer and the pooling layer are sequentially connected, and down-sampling is performed after each pooling layer. The down-sampling ratio is set to 2. The feature map after sampling is represented as:

[0099]

[0100] wherein S' represents the feature map after sampling, S represents the feature map before sampling, J represents the number of channels of the feature map, and S' represents the feature map after sampling. represents the number field of the image, H represents the height of the feature map, and W represents the width of the feature map.

[0101] Referring to FIG. 3, the multi-scale convolution module is shown in FIG. 3, which specifically includes the following processes:

[0102] The input image is evenly divided into 4 groups along the channel dimension, the first group adopts 1*1*1 convolution to maintain the current scale. The second group and the third group adopt 3*3*3 deep convolution with different expansion rates to capture information at different levels, wherein the expansion rate and the padding rate are set to 1 and 2 respectively, and the fourth group adopts image global pooling operation to statistically process global spatial information.

[0103] The data after hierarchical processing is spliced and integrated, and output through batch normalization and ReLU activation function. The residual connection superimposes the processed features and the original input to obtain the final output feature F.

[0104] The output of the multi-scale convolution is represented as:

[0105] F = δ (X (Concat (W1 (s'1), W2 (s'2), W3 (s'3), P (s'4))) ) + s (1)

[0106] Wherein, F is the first feature map, W1, W2, W3, W4 are respectively the weights of the results of the corresponding processing of the four groups of features; Concat represents the splicing operation, P represents the global pooling operation, X represents the normalization operation, δ represents the ReLU activation function operation, S'1, S'2, S'3, S'4 are respectively the results of the corresponding processing of the four groups of features, and S(1) represents the feature map.

[0107] S23: The first feature map is subjected to feature enhancement through the cross attention network, specifically: the channel dimension in the first feature map is adjusted by a 1x1x1 convolution, two parallel convolution layers further extract context information in the horizontal and vertical directions; the importance of the features is adaptively enhanced through a multi-head attention module, and the enhanced features fuse information in different channels and spatial positions through a multilayer perceptron;

[0108] S231: The channel dimension of the encoder output is adjusted by a 1x1x1 convolution;

[0109] S232: Two parallel convolutions respectively extract context information in the horizontal and vertical directions;

[0110] S232: The multi-head attention module adaptively enhances the importance of the related features and reduces the interference of irrelevant pixels;

[0111] S233: The enhanced features fuse information in different channels and spatial positions through a multilayer perceptron, and finally are sent to the decoder through residual connection and the output of the encoder.

[0112] S24: The decoder gradually restores the spatial resolution of the cross attention network output feature through upsampling and multi-scale convolution, and connects the cross attention network output feature and the first feature map in a skip connection manner to obtain a second feature map;

[0113] S25: The second feature map is sent to an output layer and is processed through an activation function to output a final intraoral segmentation result;

[0114] The output calculation method of the segmentation network is:

[0115]

[0116] Wherein, P represents the final segmentation result, Softmax(·) represents a normalized exponential function, M represents the pixel value output by the decoder, and C represents the number of classes; M c represents the prediction result corresponding to the cth class in M, and c is a class index;

[0117] S26: The training process optimizes the boundary morphology by using a weighted loss combining Dice and cross entropy.

[0118] The loss function expression of the segmentation network is:

[0119]

[0120]

[0121] wherein, represents a Dice loss function, K represents the total number of voxels, i represents a voxel index, represents a cross-entropy loss function, represents a true label of the i-th voxel of the c-th class, represents a total loss function of the segmentation network; λ1 and λ2 represent balance loss parameters, and the tooth classes considered by the segmentation result specifically include: central incisor, lateral incisor, canine, first premolar, second premolar, first molar, and second molar. λ1> λ2, because the Dice loss function performs better on an unbalanced data set, especially when the target class occupies a smaller volume, because it pays more attention to the quality of the overlapping area. While the cross-entropy loss function performs well in terms of overall accuracy. In the present fact instance, the target volume is small, and the overlapping area is not important for determining the edentulous region in the subsequent step, so the balance loss parameters are set as λ1> λ2.

[0122] S3: taking the mask information set as input, taking the cross-sectional detection result set, the coronal detection result set and the sagittal detection result set as labels to train the corresponding detection network respectively, and inputting the mask information of the intraoral tissue of the object to be predicted into the trained detection network to obtain a detection result, wherein the detection result includes a cross-sectional direction detection result, a coronal direction detection result and a sagittal direction detection result;

[0123] The mask image of the segmentation network is detected for edentulous from three anatomical directions of cross section, coronal section and sagittal section, and one detection network is designed and trained for each direction.

[0124] The process of training the detection network includes:

[0125] S31: inputting the intraoral tissue segmentation result set into the backbone network, and performing image feature extraction by using a plurality of convolutional layers;

[0126] S32: inputting the extracted feature map into a global attention module for feature enhancement;

[0127] S321: inputting the features extracted by the convolutional layer into a channel attention for enhancement; and learning the mutual relationship between channels through dimension exchange and a multilayer perceptron;

[0128] S322: performing dimension exchange on the output of the multilayer perceptron, and generating a channel attention map through a Sigmoid activation function;

[0129] S323: The output of the channel attention module is spliced with the output feature extracted by the convolution layer to obtain an intermediate state, and the intermediate state is sent to the spatial attention module to further extract spatial features through two 7x7 convolutions;

[0130] S324: The convolution output obtains a channel attention map through a Sigmoid activation function;

[0131] S325: The channel attention weight, the spatial attention weight, and the output feature extracted by the convolution layer are fused to make the network focus on the most important features in the image, and the output feature F3 is used for subsequent network layers; the intermediate state and the output of the GAM are represented as:

[0132]

[0133]

[0134] wherein F1 represents an input feature, F2 represents an intermediate state, F3 represents an output feature, M C is a channel attention map, M S is a spatial attention map, represents the product of pixels; M C (F1) represents applying the channel attention map M C to the input feature F1, and M S represents applying the spatial attention map M

[0135] S33: The last layer of the backbone network uses spatial pyramid pooling to integrate features of different regions;

[0136] S34: The output of the backbone network enters the fusion network, and features from different scales are fused through a pyramid network;

[0137] S35: The fused feature map is input into the detector for regression prediction, and a multi-scale prediction result containing a bounding box, a confidence, and class information is output;

[0138] S36: The multi-scale prediction result is input into a loss function and a flexible non-maximum suppression algorithm for post-processing, and overlapping bounding boxes are removed through Gaussian confidence decay to obtain the final detection result;

[0139] S37: The network continuously iterates and adjusts parameters to ensure that the loss parameter converges to a minimum value when the entire training process is completed, and the optimal weight is obtained; the loss function expression of the detection network is:

[0140] Loss = a x loss obj + b x loss rect + c x losscls

[0141] wherein a, b, c represent weight coefficients, loss obj represents a confidence loss, loss rect represents a bounding box loss, loss cls represents a classification loss, Loss represents the total loss of the detection network. The weight coefficients satisfy a < b < c, a + b + c = 1. The classification loss is usually the most important part in target detection, because it directly affects the ability of the model to identify target categories; therefore, giving a larger weight can improve the classification accuracy; the bounding box loss is responsible for the accuracy of positioning the target, although it is important, but the priority is slightly lower than classification; a moderate weight can help the model learn more accurate positioning; the confidence loss is mainly used to evaluate the model's confidence in the existence of the target; the influence of this part of the loss is relatively small, so the weight is set to the minimum.

[0142] S4: Fuse the cross-sectional direction detection result, the coronal plane direction detection result and the sagittal plane direction detection result, construct a three-dimensional bounding box using a majority voting algorithm, and obtain a missing tooth region;

[0143] Referring to FIG. 4, the missing tooth position detection is as shown in FIG. 4,

[0144] The segmented mask image is input into the trained detection network to predict the missing tooth position from the cross-sectional, coronal and sagittal directions;

[0145] The detection results of the three branches of the cross-sectional, coronal and sagittal directions are combined, and a three-dimensional bounding box of the missing tooth position is obtained using a majority voting algorithm to locate the position of the missing tooth.

[0146] S51: Input a target image x0 mapped to a latent space by an encoder to obtain a latent variable, and the expression of the latent variable is:

[0147] z0 = E(x0)

[0148] wherein z0 is a latent variable, x0 is an input target image, and E is an encoder encoding function;

[0149] S52: Define a forward diffusion process, simulate the data degradation situation under the real condition by adding noise to the latent variable z ~ q(z0), and gradually evolve the target image into a Gaussian noise distribution z T ; The forward diffusion process q(z 1:T |z0) is expressed as:

[0150]

[0151] wherein β t represents a hyperparameter of the noise amount, and z1, …, z Tis a latent variable with the same dimension as data z ~ q(z0);q(z 1:T |z0) is a forward diffusion process, t is the index of diffusion steps, T represents the total number of diffusion steps, represents a Gaussian distribution with mean and variance β t ; I represents an identity matrix;

[0152] The specific method of adding noise is to add V times of noise to the latent variable, The added noise satisfies 0 < σ β < 1, where, is a rounding up operation, σ β is the variance of the added noise.

[0153] S53: The located toothless position, the segmentation result of the intraoral tissue and the original image are sent into the conditional encoder to extract position features, morphological features and image features, and the fusion result is obtained by fusing them, and the fusion result is represented as:

[0154]

[0155] Where W is the fusion result, represents the embedded position feature, represents the embedded morphological feature, represents the feature embedding of the current step;

[0156] S54: Define a reverse diffusion process starting from the noise image z T , and guide the model to iteratively denoise with the fusion result as the conditional information. After T steps, the feature map z0 without noise is obtained;

[0157] Specifically, the reverse diffusion process is represented as:

[0158]

[0159] Where p θ (z 0:T |W) is the reverse diffusion process, and is a Gaussian distribution with mean and variance β t ; ∈ θ represents the sampled random noise, and α t : = 1-β t ,

[0160] S55: The feature map z0 obtained above is subjected to feature decoding through convolution layer and up-sampling layer, and the latent space feature is mapped back to the explicit space to generate the final target image, and the decoding process is represented as:

[0161] x' = D(z0)

[0162] wherein x' is the final target image, and D is a decoder decoding function.

[0163] The above formulas are all dimensionless values calculated, the formula is obtained by collecting a large amount of data to simulate the latest real situation, and the preset parameters in the formula are set by a person skilled in the art according to the actual situation.

[0164] The above embodiments can be realized wholly or partially by software, hardware, firmware or any combination thereof. When realized by software, the above embodiments can be realized wholly or partially in the form of a computer program product. Those skilled in the art can realize that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized by hardware or software methods depends on the specific application and design constraints of the technical solutions.

[0165] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, which can be located in one place or distributed on multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiments according to actual needs.

[0166] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for generating an auxiliary dental implant based on a diffusion model, characterized in that, The specific steps include: S1: obtaining historical patient oral CBCT image data, preprocessing the historical patient oral CBCT image data to obtain a CBCT image set, and determining a segmentation result set, a cross-sectional detection result set, a coronal plane detection result set and a sagittal plane detection result set based on the historical patient oral CBCT image data, wherein the segmentation result set includes an edge information set and a mask information set; S2: training a multi-task segmentation network with the CBCT image set as input and the segmentation result set as label, obtaining and preprocessing oral CBCT image data of a to-be-predicted object, inputting the to-be-measured CBCT image obtained after preprocessing into the trained segmentation network, and obtaining an intraoral segmentation result of intraoral tissue of the to-be-predicted object, wherein the intraoral segmentation result includes edge information and mask information; S3: training corresponding detection networks with the mask information set as input and the cross-sectional detection result set, the coronal plane detection result set and the sagittal plane detection result set as labels, inputting the mask information of the intraoral tissue of the to-be-predicted object into the trained detection network to obtain a detection result, wherein the detection result includes cross-sectional direction detection results, coronal plane direction detection results and sagittal plane direction detection results; S4: fusing the cross-sectional direction detection results, the coronal plane direction detection results and the sagittal plane direction detection results, constructing a three-dimensional bounding box by using a majority voting algorithm, and obtaining a missing tooth region; S5: inputting the to-be-measured CBCT image and a random noise image into the trained diffusion model, continuously diffusing and denoising the noise image through an encoder, an intermediate layer and a decoder, taking the intraoral segmentation result and the missing tooth region as prompt information, and guiding the network to generate an implantation effect through an iterative process.

2. The method of claim 1, wherein the method is based on a diffusion model. The process of network training in the segmentation network includes: S21: performing image enhancement on the preprocessed CBCT image by using a Haar filter algorithm; S22: splicing the preprocessed CBCT image and the enhanced image to form a feature map, inputting the feature map into an encoder for feature extraction to obtain a first feature map, specifically: dividing the feature map into 4 groups of features along the channel dimension, the first group adopts 1×1×1 convolution to maintain the current proportion; the second group and the third group adopt 3×3×3 deep convolution with different expansion rates to capture information at different levels, wherein the expansion rate and the padding rate are set to 1 and 2 respectively, and the fourth group adopts image pooling operation to statistically analyze global spatial information; after hierarchical processing, the features are spliced into batch normalization integration and ReLU activation function to form output features, and the output features and the feature map are superimposed through residual connection to form the first feature map; S23: performing feature enhancement on the first feature map through a cross-attention network, specifically: adjusting the dimension of the channels in the first feature map by 1×1×1 convolution, and further extracting context information in the horizontal and vertical directions through two parallel convolution layers; the importance of the enhanced features is adaptively enhanced through a multi-head attention module, and the enhanced features are fused with information in different channels and spatial positions through a multilayer perceptron. S24: The decoder gradually restores the spatial resolution of the cross-attention network output feature by upsampling and multi-scale convolution, and connects the cross-attention network output feature and the first feature map in a jump connection manner to obtain a second feature map; S25: The second feature map is sent to the output layer and processed by using an activation function to output the final intraoral segmentation result; S26: The training process optimizes the boundary morphology by using a weighted loss combining Dice and cross-entropy.

3. The method of claim 2, wherein the method further comprises: The expression of the Haar filter is: where H N is a Haar filter matrix, h N-1 (N-1 / N) is an N-1 order Haar basis function, and N x N is the order of the Haar filter matrix.

4. The method of claim 3, wherein the method further comprises: The output of the multi-scale convolution is represented as: F = δ(X(Concat(W1(S'1), W2(S'2), W3(S'3), P(S'4)))) + S(1) Wherein, F is the first feature map, W1, W2, W3, W4 are the weights of the results of the four groups of features after corresponding processing; Concat represents the splicing operation, P represents the global pooling operation, X represents the normalization operation, δ represents the ReLU activation function operation, S'1, S'2, S'3, S'4 are the results of the four groups of features after corresponding processing, and S(1) represents the feature map.

5. The method of claim 3, wherein the method further comprises: The output of the segmentation network is calculated in the following manner: wherein P represents the final segmentation result, Softmax(·) represents a normalized exponential function, M represents pixel values output by the decoder, and C represents the number of classes; M c represents a prediction result corresponding to the cth class in M, and c is a class index. The loss function expression of the segmentation network is: wherein denotes the Dice loss function, K denotes the total number of voxels, and i denotes the voxel index, denotes a cross-entropy loss function, a label representing a true class of the i-th voxel, The total loss function of the segmentation network is represented as: L = λ1Ldice + λ2Lcross-entropy 6. The method of claim 1, wherein: The process of training the detection network includes: S31: The intraoral tissue segmentation result set is input into the backbone network, and a plurality of convolutional layers are used for image feature extraction; S32: The extracted feature map is input into the global attention module for feature enhancement; S33: The last layer of the backbone network uses spatial pyramid pooling to integrate features of different regions; S34: The output of the backbone network enters the fusion network, and features from different scales are fused through the pyramid network; S35: The fused feature map is input into the detector for regression prediction, and a multi-scale prediction result containing a bounding box, a confidence and class information is output; S36: The multi-scale prediction result is input into the loss function and the flexible non-maximum suppression algorithm for post-processing, and the overlapping bounding boxes are removed through Gaussian confidence decay to obtain the final detection result; S37: The network continuously iterates and adjusts the parameters to ensure that the loss parameter converges to a minimum value to complete the entire training process and obtain the optimal weight; the loss function expression of the detection network is: Loss = a x loss obj + b x loss rect + c x loss cls wherein a, b, c represent weight coefficients, loss obj represents the confidence loss, loss rect represents the rectangular frame loss, loss cls represents the classification loss, Loss represents the total loss of the detection network.

7. The method of claim 1, wherein: The specific steps of inputting the extracted feature map into the global attention module for feature enhancement include: S321: The features extracted by the convolutional layer are input into the channel attention for enhancement; the mutual relationship between channels is learned through dimension exchange and multilayer perceptron; S322: The output of the multilayer perceptron is dimension-exchanged, and a channel attention map is generated through a Sigmoid activation function; S323: The output of the channel attention module is spliced with the output features extracted by the convolutional layer to obtain an intermediate state, and the intermediate state is input into the spatial attention module to further extract spatial features through two 7x7 convolutions; S324: The convolution output obtains a channel attention map through a Sigmoid activation function; S325: The channel attention weight, the spatial attention weight and the output feature extracted by the convolution layer are fused, so that the network focuses on the most important features in the image, and the output feature F3 is used for subsequent network layers; the output of the intermediate state and the GAM is represented as: wherein, F1 represents input features, F2 represents intermediate states, F3 represents output features, M P is a channel attention map, M S is a spatial attention map, denotes the product of the pixels; M P (F1) denotes applying the channel attention map M P on the input features F1, for applying the spatial attention map M S on the intermediate state F2.

8. The method of claim 1, wherein: The specific steps of obtaining the edentulous region are: S41: The CBCT image is respectively detected on the cross section, coronal section and sagittal section using a two-dimensional convolution network to detect the tooth loss position; S42: The candidate boxes output by the detection network in the cross section, coronal section and sagittal section are spliced, a three-dimensional bounding box is obtained by using a majority voting algorithm, and a tooth loss region is obtained.

9. The method of claim 1, wherein: The specific steps of step 5 are: S51: Input the target image x0 into the encoder to map it to the latent space to obtain the latent variable, and the expression of the latent variable is: z0=E(x0) Wherein, z0 is the latent variable, x0 is the input target image, and E is the encoder coding function; S52: Define a forward diffusion process to simulate the data degradation under realistic conditions by adding noise to the latent variable z ~ q(z0) to gradually evolve the target image into a Gaussian noise distribution z T ; the forward diffusion process q(z1: T |z0) is represented as: where β t is a hyperparameter representing the amount of noise, z1,..., z T are latent variables of the same dimension as the data z ~ q(z 1：T |z0) is a forward diffusion process, t is an index of the diffusion step, and T represents the total number of diffusion steps, representing the mean variance of β t a Gaussian distribution; I denotes the identity matrix; S53: send the located edentulous position, segmentation result of intraoral tissue and original image into the condition encoder to extract position features, morphological features and image features, and fuse them to obtain a fusion result, which is represented as: W is the fusion result, representing an embedded location feature, representing the embedded morphological features, The feature embedding of the current step is represented as: S54: define a reverse diffusion process from the noisy image z T First, the fusion result is used as the condition information to guide the iterative denoising of the model, and after T steps, the feature map z0 without noise is obtained. In particular, the back-diffusion process is represented as: where p θ (z 0：T |W) is the back-diffusion process, with mean variance is β t a Gaussian distribution; ∈ θ represents random noise of the sampling, α t : = 1 - β t , S52: The feature embedding of the current step is represented as: S53: The feature embedding of the current step is represented as: S54: The feature embedding of the current step is represented as: S55: The feature map z0 obtained above is subjected to feature decoding through a convolution layer and an upsampling layer, the latent space feature is mapped back to the explicit space, and the final target image is generated, and the decoding process is represented as: x′=D(z0) Wherein, x′ is the final target image, and D is the decoder decoding function.

Images