Method for fine segmentation of mandibular nerve canal under CBCT based on multi-view feature fusion

Abstract

The application discloses a CBCT mandibular nerve canal fine segmentation method based on multi-view feature fusion. First, the CBCT image and the mandibular nerve canal annotation constitute a data set; the training set is cut into several patches to obtain input data, and the position coordinates of each patch are encoded. Second, each patch and its corresponding position code are sent into a pre-segmentation network model to obtain a pre-segmentation result, and a cubic spline interpolation is used to obtain a mandibular canal center line. Finally, multi-view feature extraction is performed on the CBCT and the mandibular canal center line, feature fusion is performed through 3D Attention, the obtained features are decoded, and finally the segmentation result of the mandibular canal is obtained and optimized. The application effectively fuses the feature information of multiple views, and ensures the accuracy and integrity of the segmentation result.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of computer vision, medical image analysis and numerical analysis, and particularly relates to a CBCT mandibular nerve canal fine segmentation method based on multi-view feature fusion. BACKGROUND

[0002] In the field of computer vision and image processing, cone beam computed tomography (CBCT) technology is widely used to obtain fine three-dimensional image data. CBCT can provide high-resolution three-dimensional reconstruction images and has a wide range of applications in industrial detection, material analysis, and biological engineering. However, due to the complexity and high noise of CBCT image data, accurate identification and segmentation of important structures have always been a technical challenge.

[0003] Traditional image segmentation methods rely on simple image processing techniques or manual annotation, which have low efficiency and poor accuracy. With the rapid development of deep learning technology, automatic segmentation methods based on convolutional neural networks (CNN) have gradually become a research hotspot. These methods can handle large-scale image data and exhibit excellent performance in image segmentation. However, relying solely on CNN for segmentation often fails to balance global information and local details, especially when dealing with high-noise data, which can lead to missegmentation or missed segmentation. SUMMARY

[0004] The present application proposes a CBCT (cone beam computed tomography) mandibular nerve canal fine segmentation method based on multi-view feature fusion to address the shortcomings of existing technology. This method involves multiple steps. First, samples containing CBCT images and corresponding annotations are selected to form a training dataset, and data augmentation is performed to improve the model's generalization ability. Then, the Unet model is used as a pre-segmentation network to perform convolution and pooling operations on the CBCT data to generate feature maps. The feature maps are then concatenated with position encodings and decoded to obtain preliminary segmentation results. Subsequently, denoising is performed on the pre-segmentation results, and the left and right mandibular canal preliminary segmentation results and candidate boxes are obtained by merging the smallest connected subdomains. Then, cubic spline interpolation is performed on the left and right mandibular canal candidate boxes to fit the centerline of the mandibular canal, and the centerline is spliced back into the original image. Based on the fitted results, the mandibular canal centerline and the original CBCT image are input into the second-stage dynamic mandibular canal convolution (DIConv) neural network model as a dual-channel input, and feature fusion is performed through multi-view feature extraction and a three-dimensional attention mechanism (3D attention) module. Finally, the fused feature maps and the feature maps obtained by ordinary convolution are spliced and fed into the encoder for encoding and decoding operations to obtain fine mandibular canal segmentation results. The Jaccard Loss is used as the loss function to optimize the model and improve the segmentation accuracy.

[0005] The method gradually improves the accuracy and robustness of the segmentation result by introducing a multi-stage segmentation strategy, combining pre-segmentation model, position encoding, curve fitting and multi-view feature extraction technology. Specifically, first, the CBCT image is roughly segmented to obtain the preliminary mandibular nerve canal region; then the boundary and morphology of the mandibular nerve canal are further optimized through curve fitting; finally, the multi-view dynamic mandibular canal convolution (DIConv) model is used for feature extraction and fusion to ensure the accuracy of the segmentation result.

[0006] The CBCT mandibular nerve canal fine segmentation method based on multi-view feature fusion has the following specific steps:

[0007] Step (1), select the sample containing cone beam computed tomography (CBCT) image and corresponding label {X, Y} to constitute the data set, and divide it into training set, test set and validation set; wherein X and Y represent the CBCT image and its corresponding mandibular nerve canal label in the data set, respectively.

[0008] Step (2), for the obtained data set, in the training stage, cut the training set into several patch blocks, and record the position coordinates of the patch block, then perform data enhancement operation on the patch block to obtain the input data X0.

[0009] Step (3), based on Unet, a pre-segmentation model is constructed, the position coordinates of each patch block of CBCT are encoded to obtain the position encoding P0, and the data X0 and its corresponding position encoding P0 are input into the pre-segmentation network model to obtain the corresponding pre-segmentation result Y0. The pre-segmentation model will first perform a series of convolution and pooling encoding operations on X0 to obtain the final feature map Feature0.

[0010] Step (4), splice the final encoded Feature0 and the previously calculated position encoding P0 for feature fusion.

[0011] Step (5), decode the feature map Con_Feature0 after encoding and position encoding fusion of the CBCT image, and constantly perform upsampling and pooling operations, and perform jump connection on the corresponding layer output of encoding and decoding.

[0012] Step (6), use the current training set and validation set to train the pre-segmentation model, and save its weight. After obtaining the model and weight, all data sets need to be input into the pre-segmentation network for inference to obtain the pre-segmentation result Y'0 of all data.

[0013] Step (7), based on the pre-segmentation result Y'0 obtained in step (6), denoising is performed: based on the pre-segmentation result Y'0, the number of voxels of each minimum connected domain is counted, and the connected domains smaller than the set threshold are deleted. The N minimum connected subdomains obtained by using the clustering algorithm are respectively initially assigned to N categories. The center distance between all categories is calculated, and the two categories with the closest distance are selected for merging, and the center coordinates of the merged categories are updated. If the number of final categories is equal to 2, the next step is performed, and if it is greater than 2, the merging operation is continued.

[0014] Step (8), by merging in step (7), the mandibular canal is divided into left and right mandibular canals Y left and Y right The starting point of the candidate box and the size of the candidate box of the left and right mandibular canals are obtained at the same time. For each mandibular canal in the obtained candidate box, a cubic spline interpolation is used to fit the center line of the mandibular canal, and the obtained left and right mandibular canal center lines are respectively spliced back to the position of the original image, denoted as X Line .

[0015] Step (9), the spliced mandibular canal center line X Line and X0 are input into the DIConv network model of the second stage as a dual-channel input. The previously generated mandibular canal center line is used as a prompt mask to perform multi-view feature extraction on the mandibular canal. Then, 3D attention is used to fuse the features extracted by multi-view, and a feature map X DIConv is obtained.

[0016] Step (10), the obtained X DIConv is spliced with the feature map obtained by performing ordinary convolution on X0, and then input into the DIConv network model. The same operation as steps (3), (4) and (5) is performed to encode, then embed the position coding P0, and decode to obtain the final mandibular canal segmentation result U1.

[0017] DIConv uses Unet network as the basis, uses dual-channel feature input, and simultaneously embeds the position coding information of the patch block.

[0018] Step (11), the model is optimized by the following loss function, and the performance is tested by the test set. The loss function is as follows:

[0019]

[0020] Wherein, N represents the number of samples, y i is the true label, is the predicted label, and ∈ is a parameter to prevent the denominator from being zero, thereby avoiding numerical instability or calculation error.

[0021] Further, the pre-segmentation model in step 3 is implemented as follows:

[0022] Step 3.1, encode the position coordinates of each patch block of the CBCT to obtain position encoding P0; input data X0 and its corresponding position encoding P0 into the pre-segmentation model, first perform convolution and pooling encoding operation on X0 to obtain feature map Feature0;

[0023] Step 3.2, splice the feature map Feature0 and the position encoding P0 to obtain the feature map Con_Feature0;

[0024] Step 3.3, decode the feature map Con_Feature0, constantly perform upsampling and pooling operations, and perform jump connection on the corresponding layer outputs of encoding and decoding, and obtain the corresponding pre-segmentation result Y0 through the full connection layer.

[0025] Further, the denoising in step 7 is specifically: based on the pre-segmentation result Y'0, the number of voxels in each minimum connected domain is counted, and the connected domain with a voxel number less than a set threshold is deleted; the merging operation is specifically: the N minimum connected domains after denoising are respectively initialized as N classes, the center distance between all classes is calculated, the two classes with the closest center distance are selected through a clustering algorithm, and the center coordinates of the merged class are updated, if the number of classes is equal to 2, the next step is performed, if it is greater than 2, the merging operation is continued.

[0026] Further, the curve fitting of the mandibular nerve canal is implemented as follows in step (8):

[0027] Step 8.1, for the left mandibular canal Y left and the right mandibular canal Y right , according to the size of the candidate frame, divide the candidate frame into eleven equal parts along the y-axis, extract ten xoz sections, for each section, extract the center point coordinates of the pre-segmentation result Y'0 on the section, thereby obtaining ten center line coordinate points P1 to P 10 If there is a point without a value in the ten center line coordinate points (i.e. the xoz section has no predicted mask result point), the center line coordinate point is discarded;

[0028] Step 8.2, construct a geometric model of the mandibular canal, use the center line coordinate points P1 to P 10 Construct a cubic spline interpolation curve C(t); each center line coordinate point P i is assigned a corresponding parameter t i , the formula is as follows:

[0029]

[0030] where N = 10 is the total number of centerline coordinate points; the coordinates of each point C(t) on the curve are defined by independent cubic spline functions S x (t), S y (t), S z (t), with the specific formulas being:

[0031] For x coordinates: S xi (t i ) = a i + b i (t-t i ) + c i (t-t i ) 2 + d i (t-t i ) 3 , t e [t i , t i+1 ]

[0032] For y coordinates: S yi (t i ) = e i + f i (t-t i ) + g i (t-t i ) 2 + h i (t-t i ) 3 , t e [t i , t i+1 ]

[0033] For z coordinates: S zi (t i ) = m i + n i (t-t i ) + p i (t-t i ) 2 + q i (t-t i ) 3 , t e [t i , t i+1 ]

[0034] where the coefficients a i , b i , c i , d i , e i , f i , g i , h i , mi ,n i ,p i ,q i By solving the linear equations, the continuity of the first and second derivatives at the end points of each segment is ensured; in addition, the boundary conditions are that the second derivatives of the curve at the start and end points are zero;

[0035] The three-dimensional curve C(t) is a geometric model describing the center line of the mandibular canal, wherein the coordinates of each point are determined by three independent cubic polynomials defined in each interval [t i ,t i+1 ], which can accurately depict the spatial orientation of the mandibular canal;

[0036] Step 8.3, by fitting the curve constructed in step 8.2, the left and right mandibular canal center lines are obtained, since the voxel proportion of the mandibular canal center line is small, and expansion is used for expansion, finally the obtained mandibular canal is put back into the original image according to the recorded starting position between them, and restored to X Line Mandibular nerve canal center line mask.

[0037] Further, the multi-view mandibular canal convolution DIConv is implemented as follows in step (9):

[0038] 9-1. For the double-channel input of the mandibular canal center line X Line and the original CBCT image X0, first, for each coordinate point on the image, set a forced unit offset X direction , Y direction , Z direction with a total length of kernal_size, which is composed of a series of coordinate points on the axis as the offset reference point:

[0039]

[0040] Wherein ， x c , y c , z c represent a voxel coordinate point, and the value of k is equal to the value of kernal_size, representing the total length of the axial convolution.

[0041] 9-2. Based on the coordinates of each point on the reference offset of 9-1, the remaining two axial free offsets Δm i and Δn i, single free offset value is fixed at the origin of 0, 1 length of the neighborhood, free offset has cumulative effect. By the accumulation of free offset, can get based on the original coordinate point of a certain axis of convolution position. X-axis feature extraction as an example:

[0042] For X direction Any point x c +i corresponding to the total offset Δy and Δz is:

[0043]

[0044] So the final x c +i sampling coordinate point is (x c +i, y c +Δy, z c +Δz), i represents the position of the convolution to take, offset is (0, Δy, Δz), traversal X direction All points of convolution sampling point coordinates, you can perform convolution operation.

[0045] For example: known horizontal coordinate point reference coordinate position (x c +2, y c , z c ), we know the x-axis coordinate selection position x c +2, we also extra get its corresponding Δy and Δz, through the previous calculation, we have obtained Δm0 and Δm1 and Δn0 and Δn1, at this time, through the convolution of Δm2 and Δn2, will be obtained Δm0 and Δm1 and Δm2 cumulative, get based on (x c +2, y c , z c ) about y c The offset corresponding to the axis, similarly will get Δn0 and Δn1 and Δn2 cumulative, get based on (x c +2, y c , z c ) about z c The offset corresponding to the axis. Finally, the reference coordinate position (x c +2, y c , z c ) convolution to take the position (x c +2, y c +Δm0+Δm1+Δm2, z c +Δn0+Δn1+Δn2)

[0046] 9-3. For the obtained points, non-integer cases may occur. To handle the non-integer positions that may occur due to dynamic adjustment, a trilinear interpolation method is used for numerical substitution, which can effectively estimate the pixel values at non-integer coordinates when calculating the convolution operation, ensuring the continuity and accuracy of feature extraction. The formula is as follows:

[0047]

[0048] Wherein ， f(p) represents the function value at point p, (x i ,y i ,z i ) represents the vertex coordinates in space, f(x i ,y i ,z i ) represents the function value at point (x i ,y i ,z i ). (x0, y0, z0) represents the left front lower coordinate vertex, and (x1, y1, z1) represents the right rear upper coordinate vertex.

[0049] The beneficial effects of the present application are:

[0050] The CBCT mandibular nerve canal fine segmentation method constructed based on multi-view feature fusion can more accurately capture the detailed structure of the mandibular nerve canal and reduce errors through a multi-stage segmentation strategy, rough segmentation first, and then gradual refinement. In the process of constructing the mandibular canal center line, noise interference is reduced by denoising processing and minimum connected subdomain merging, ensuring that the segmentation needs to focus on the target. In addition, by using the Unet and DIConv network model, combining position coding and multi-view feature extraction, the method effectively fuses various feature information, ensuring the accuracy and integrity of the segmentation result. This method is not only suitable for mandibular canal segmentation, but also suitable for the segmentation of various types of blood vessels or small long strip-shaped targets in medical images. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 The flowchart of the present application is;

[0052] Figure 2 The model architecture diagram of the present application is;

[0053] Figure 3 The mandibular canal convolution DIConv schematic diagram of the present application is;

[0054] Figure 4 The process of classifying left and right mandibular canals of the present application is;

[0055] Figure 5 The segmentation effect display of the present application on the public data set is. DETAILED DESCRIPTION

[0056] The application will be further described in detail below with reference to the accompanying drawings.

[0057] As Figure 1 With Figure 2 As shown in the figure, the application provides a CBCT mandibular nerve canal fine segmentation method based on multi-view feature fusion, which first selects CBCT images and mandibular nerve canal annotations to form a data set; the training set is cut into several patches to obtain input data, and the position coordinates of each patch are encoded. Secondly, each patch and its corresponding position code are sent to the pre-segmentation network model to obtain the pre-segmentation result, and the mandibular canal center line is obtained by using cubic spline interpolation. Finally, multi-view feature extraction is performed on the CBCT and the mandibular canal center line, and feature fusion is performed through 3DAttention, and the obtained features are decoded, and finally the segmentation result of the mandibular canal is obtained, and optimization training is performed. The application effectively fuses the feature information of multiple views, ensures the accuracy and integrity of the segmentation result, and thereby reduces the work burden of radiologists.

[0058] The specific implementation steps of the application are as follows:

[0059] Step (1), select samples containing cone beam computed tomography (CBCT) images and corresponding annotations {X, Y} to form a data set, and divide it into a training set, a test set and a validation set; wherein X and Y represent the CBCT images and the corresponding mandibular nerve canal annotations in the data set, respectively.

[0060] Step (2), for the obtained data set, in the training stage, it will be cut into (80, 80, 80) size patch blocks first, and the position coordinates of the patch blocks are recorded. Then the obtained patch blocks are subjected to data enhancement operations such as random affine transformation and random flip, and finally the data X0 with a size of (6, 1, 80, 80, 80) is obtained and input into the first module. Wherein ， 6 represents the size of batch size, 1 represents the number of input channels, and (80, 80, 80) represents the size of each patch block.

[0061] Step (3), for the data to be inferred, it will be first sent to the pre-segmentation model, and the pre-segmentation model will encode the position coordinates of the patch obtained by cutting the original data to obtain the position code P0. For the X0 obtained by data preprocessing, the X0 itself and the corresponding position code P0 are sent to the pre-segmentation network model, and the network model will first perform a series of convolution and pooling operations on X0 to obtain the final feature map Feature0. Wherein, the formula of convolution is as follows:

[0062] y = f(W * x + b)

[0063] where x is input, W is convolution kernel, * represents convolution operation, b represents bias value, f represents activation function, and (m, n, p) e window defines the size of the pooling window.

[0064] Step (4), the Feature0 obtained by final encoding is spliced with the position encoding P0 calculated before to perform feature fusion. The specific splicing formula is as follows:

[0065] Con_Feature0 = concat(Feature0, P0)

[0066] where Feature0 refers to the feature map obtained after CBCT encoding, and P0 refers to the position encoding of the patch block.

[0067] Step (5), the feature map Con_Feature0 after the fusion of CBCT image encoding and position encoding is decoded, and the corresponding layer output of encoding and decoding is continuously up-sampled and pooled, and a jump connection is performed to obtain the pre-segmentation result Y'0 corresponding to all data; the up-sampling operation and the jump connection formula are as follows:

[0068] y = f(W T *x+b)

[0069] z = concat(y encoder ,y decoder )

[0070] where ，x is input, W T represents a transposed convolution kernel, * represents convolution operation, b represents bias value, f represents activation function, and concat represents feature map connection operation, y encoder represents the feature map of CBCT input to the encoder part, and y decoder represents the feature map output by the decoder part.

[0071] Step (6), based on the pre-segmentation result Y'0 obtained in step (5), denoising is performed, and the obtained N minimum connected subdomains are respectively initialized as N classes. The center distance between all classes is calculated, and the two classes with the closest distance are selected for merging, and the center coordinates of the merged class are updated. If the number of classes is equal to 2 at last, the loop is exited, and if the number of classes is greater than 2, the merging operation is continued.

[0072] Step (7), through the merging of step (6), the mandibular canal is divided into left and right mandibular canals Y left and Y rightAt the same time, the starting point of the candidate box and the size of the candidate box of the left and right mandibular canal are obtained, and for each mandibular canal in the obtained candidate box, a cubic spline interpolation is used to fit the center line of the mandibular canal, and the obtained left and right mandibular canal curves are spliced back to the position of the original image, denoted as X Line .

[0073] Step (8), the obtained mandibular canal center line X Line and X0 are input into the second stage as a double-channel input into the DIConv network model as shown in Figure 3 , the generated mandibular canal center line is used as a prompt mask, and multi-view feature extraction is performed on the mandibular canal, and then 3D attention is used to fuse the features extracted by multi-view, to obtain a feature map X DSConv .

[0074] Step (9), the obtained X DSConv is spliced with the feature map obtained by ordinary convolution on X0, and is input into the subsequent encoder, and the operations in steps (3), (4) and (5) are the same, encoding is performed, then position encoding P0 is embedded, decoding is performed, and the final mandibular canal segmentation result Y1 is obtained.

[0075] Step (10), the model is optimized and trained by a loss function, and the performance is tested by a test set.

[0076] To test the performance of the method, qualitative and quantitative experiments were performed on a public data set. The qualitative experimental results are shown in Figure 4 , it can be seen that the present application can accurately segment the structure of the mandibular nerve canal, even in the case of discontinuous intervals in the training samples, compared with the segmentation of the ordinary traditional model, the present application has better performance in the continuity of the segmentation result. In addition, the method can also sample the center of the continuous blood vessels to generate the center line of the blood vessels, which is not only suitable for the extraction of the center line of the mandibular nerve canal, but also can be used for the extraction of the center line of other blood vessels.

[0077] In terms of quantitative analysis, the performance of the present application and other five commonly used methods for three-dimensional image segmentation are compared, and the experimental results are shown in Table 1. As can be seen from the table, the IoU and Dice coefficient show that the model of the present application provides superior overlap and similarity between the predicted and actual segmentation masks, and the accuracy is further improved compared with the traditional model.

[0078] Table 1 Performance comparison of data set (the best result is marked in bold)

[0079]

[0080] The model of the present application has the smallest Hausdorff distance, and the lower Hausdorff distance indicates that the maximum deviation between the prediction and the real boundary is small, indicating that the model performs well in accurately capturing the boundary details. The smoothness defined by the gradient reflects the smoothness of the model, which indicates that the segmentation contour generated by the model of the present application is smoother than that generated by other models.

[0081] In medical imaging, such reduced granularity is very advantageous, because jagged edges can mistakenly represent the true morphology of anatomical structures. Moreover, in the Small Volume Detection indicator, the model of the present application is also the best, indicating that our model produces almost no small noise in the segmentation process. In terms of continuity, the model of the present application is also the best in terms of the combination of Hausdorff distance and smoothness. This is mainly due to the Line Extractor submodule and the DIConv-Segmentation submodule designed in the present application, which are specifically designed for the segmentation task of the mandibular nerve canal and can better handle specific image features.

[0082] The Line Extractor optimizes the processing of connected domains through numerical analysis methods, extracts the center line of the mandibular canal, and inputs the center line and the CBCT image as a dual-channel into the next stage of the segmentation network. The center line of the mandibular canal plays a very important role as a mask prompt. Through the mask prompt of the center line, the segmentation network further focuses on feature extraction near the center line, strengthens the features around the mandibular canal, and has continuity, which is more consistent with the anatomical structure of the mandibular canal in medicine. The experimental results are shown in the diagram as Figure 5

[0083] The DIConv-Segmentation module strengthens the extraction ability of linear features through a custom mandibular canal convolution. The feature extraction process forces the use of linear candidate units for extraction. The design of this special module significantly improves the model's ability to handle complex structures and further strengthens the continuity of the features.

[0084] In the DIConv-Segmentation module, the attention mechanism is used to fuse features extracted from different angles, enhancing the model's ability to recognize key features. This feature fusion strategy ensures that information extracted from different perspectives and scales can be effectively integrated, improving the coherence and accuracy of the segmentation results.

[0085] In addition, the present experiment also conducts an ablation experiment on each model using multi-angle DIConv feature extraction, as shown in Table 2 below.

[0086] ​Table 2 Ablation experiment of DIConv module (whether to use the module DIConv of the application, the best result is marked in bold)

[0087]

[0088] From the ablation experiment in the table, after the application adopts the multi-view DIConv feature extraction, the segmentation accuracy is correspondingly improved, which proves that the module added by the application can further better extract features.

Claims

1. A method for fine segmentation of the mandibular nerve canal under CBCT based on multi-view feature fusion, characterized in that, Comprising the following steps: Step 1, select a sample containing CBCT image X and mandibular nerve canal annotation Y {X, Y} to constitute a data set, and divide it into a training set, a test set and a validation set; Step 2, cut the training set into several patch blocks, and record the position coordinates of each patch block, then perform data enhancement operation on the patch block to obtain input data X0; Step 3, based on Unet, a pre-segmentation model is constructed, the position coordinates of each patch block of CBCT are encoded to obtain position encoding P0, the input data X0 and its corresponding position encoding P0 are input into the pre-segmentation model to obtain the corresponding pre-segmentation result Y0; Step 4, use the training set and the validation set to train the pre-segmentation model, and save its weight; Then all the data sets are input into the pre-segmentation model for inference to obtain the pre-segmentation results Y'0 corresponding to all the data sets; Step 5, based on the pre-segmentation result Y'0, denoising is performed, and then merging operation is performed; The denoising is specifically based on the pre-segmentation result Y'0, and the number of voxels of each minimum connected domain is counted, and the connected domain with a number of voxels less than a set threshold is deleted; The merging operation is specifically: the N minimum connected domains after denoising are respectively initially assigned to N categories, the center distance between all categories is calculated, the two categories with the closest center distance are selected by clustering algorithm for merging, and the center coordinates of the merged categories are updated, if the number of categories is equal to 2, the next step is performed, if it is greater than 2, the merging operation is continued; Step 6, divide the mandibular canal into left mandibular canal Y left and right mandibular canal Y right , fit the mandibular canal center line, and respectively splice the obtained left mandibular canal center line and right mandibular canal center line according to the position of the original image to obtain a matrix with the same size as the original size, that is, the mandibular canal center line mask X Line ; Step 7, X Line and X0 as a dual-channel input to the multi-view inferior alveolar canal convolution DIConv network model, and then using 3D attention for multi-view feature fusion to obtain the feature map X DIConv ; Step 8, the feature map X DIConv is spliced with the feature map obtained by convolving X0, and then input to the DIConv network model to obtain a final mandibular canal segmentation result Y1. Step 9, the model is optimized and trained by the loss function, and the performance is tested by the test set.

2. The method of claim 1, wherein the method is a multi-view feature fusion based CBCT mandibular nerve canal fine segmentation method. The pre-segmentation model is specifically implemented as follows: Step 3.1, encode the position coordinates of each patch block of CBCT to obtain position encoding P0; input the input data X0 and its corresponding position encoding P0 into the pre-segmentation model, first perform convolution and pooling encoding operation on X0 to obtain feature map Feature0; Step 3.2, splice the feature map Feature0 and the position encoding P0 to obtain the feature map Con_Feature0; Step 3.3, decode the feature map Con_Feature0, constantly perform upsampling and pooling operation, and perform jump connection on the corresponding layer output of encoding and decoding, and obtain the corresponding pre-segmentation result Y0 through the full connection layer.

3. The method of claim 2, wherein the method is based on multi-view feature fusion. The step 6 is specifically implemented as follows: Step 6.1, for the left mandibular canal Y left and the right mandibular canal Y right , according to the size of the candidate box, divide the candidate box into eleven equal parts along the y-axis, extract ten xoz sections from the eleven parts, for each section, extract the center point coordinates of the pre-segmentation result Y'0 on the section, thereby obtaining ten center line coordinate points P1 to P 10 If there is a point without a value in the ten center line coordinate points, the center line coordinate point is discarded; Step 6.

2. Constructing the geometric model of the mandibular canal, using the centerline coordinate points P1 to P 10 Constructing a cubic spline interpolation curve C(t); each centerline coordinate point P i is given a corresponding parameter t i , as follows: where N=10 is the total number of centerline coordinate points; the coordinates of each point C(t) on the curve are defined by independent cubic spline functions S x (t), S y (t), S z (t) with the specific formulas: For x coordinate: S xi (t i ) = a i +b i (t-t i )+c i (t-t i ) 2 +d i (t-t i ) 3 , t e [t i , t i+1 ] For y coordinate: S yi (t i ) = e i +f i (t-t i )+g i (t-t i ) 2 +h i (t-t i ) 3 ,t∈[t i ,t i+1 ] For z coordinate: S zi (t i ) = m i + n i (t - t i ) + p i (t - t i ) 2 + q i (t - t i ) 3 , t e [t i , t i+1 ] where the coefficients a i ,b i ,c i ,d i ,e i ,f i ,g i ,h i ,m i ,n i ,p i ,q i are obtained by solving a system of linear equations that ensures the continuity of the first and second derivatives at the end points of each segment; moreover, the boundary conditions are that the second derivative of the curve is zero at the start and end points. The three-dimensional curve C(t) is a geometric model describing the center line of the mandibular canal, wherein the coordinates of each point are determined by three independent cubic polynomials, which are defined in each interval [t i ,t i+1 ] and can accurately depict the spatial course of the mandibular canal; Step 6.3, the left and right mandibular canal center lines are obtained by fitting the construction of the curve of step 6.2, respectively, and are dilated using inflation, and finally the obtained mandibular canal is put back into the original image according to the recorded starting position between them, and is restored to X Line Mandibular nerve canal center line mask.

4. The CBCT mandibular nerve canal fine segmentation method based on multi-view feature fusion according to claim 3, characterized in that, The multi-view mandibular canal convolution DIConv network model is specifically implemented as follows: Step 7.1, to X Line Each coordinate point of the image after merging with X0, set the double-end length of the mandatory unit offset X of kernal_size direction , Y direction , Z direction , which consists of a series of coordinate points on the axis as the offset reference point: wherein x c , y c , z c represent a voxel coordinate point, and the value of k is equal to the value of kernal_size; Step 7.2, based on offset X direction Y direction Z direction The coordinates of each point on the graph are used to predict the free offset Δm along the remaining two axes using convolution. i and Δn i Each free offset value is fixed within a neighborhood with 0 as the origin and 1 as the length. The free offsets have a cumulative effect. By accumulating the free offsets, the convolution position based on a certain axis of the original coordinate point is obtained. Step 7.3, process the non-integer position caused by dynamic adjustment, and replace the value by using the trilinear interpolation method, the formula is as follows: where f(p) represents the function value at point p, (x i ,y i ,z i ) represents the vertex coordinates in space, and f(x i ,y i ,z i ) represents the function value at point (x i ,y i ,z i ); (x0,y0,z0) represents the left-front-bottom coordinate vertex, and (x1,y1,z1) represents the right-rear-top coordinate vertex.

5. The method of claim 4, wherein the method is based on multi-view feature fusion. The loss function Jaccard Loss is: where N denotes the number of samples, y i is the true label, is the predicted label, and ∈ is a parameter to prevent the denominator from being zero.

Images