CBCT mandibular neural tube refined segmentation method based on multi-view feature fusion

Abstract

The invention discloses a CBCT (Cone Beam Computed Tomography) mandibular neural tube refined segmentation method based on multi-view feature fusion. The method comprises the following steps: firstly, selecting a CBCT image and a mandibular neural tube label to form a data set; cutting the training set into a plurality of Patches to obtain input data, and encoding the position coordinates of each Patch; secondly, sending each Patch and the corresponding position code into a pre-segmentation network model to obtain a pre-segmentation result, and obtaining a mandibular canal center line by adopting cubic spline interpolation; and finally, carrying out multi-view feature extraction on the CBCT and the mandibular canal center line, carrying out feature fusion through 3D Attention, decoding the obtained features, finally obtaining a segmentation result of the mandibular canal, and carrying out optimization training. According to the method, feature information of multiple visual angles is effectively fused, and the accuracy and integrity of a segmentation result are ensured.

Description

Technical Field

[0001] The present invention belongs to the fields of computer vision, medical image analysis and numerical analysis, and in particular relates to a CBCT mandibular nerve canal fine segmentation method based on multi-view feature fusion. Background Art

[0002] In the field of computer vision and image processing, cone beam computed tomography (CBCT) technology is widely used to obtain fine three-dimensional image data. CBCT can provide high-resolution three-dimensional reconstruction images and has a wide range of applications in industrial inspection, material analysis, and bioengineering. However, due to the complexity and high noise of CBCT image data, the accurate identification and segmentation of important structures has always been a technical challenge.

[0003] Traditional image segmentation methods rely on simple image processing techniques or manual annotation, which have the problems of low efficiency and poor accuracy. With the rapid development of deep learning technology, automatic segmentation methods based on convolutional neural networks (CNNs) have gradually become a research hotspot. These methods can process large-scale image data and show excellent performance in the field of image segmentation. However, relying solely on CNN for segmentation often makes it difficult to take into account both global information and local details, especially when faced with high-noise data, which is prone to mis-segmentation or missed segmentation. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention proposes a CBCT (cone beam computed tomography) mandibular canal fine segmentation method based on multi-view feature fusion. The method first selects samples containing CBCT images and corresponding annotations to form a training data set through multi-step processing, and performs data enhancement to improve the generalization ability of the model. Then, the Unet model is used as a pre-segmentation network to perform convolution and pooling operations on the CBCT data to generate a feature map, and after splicing with the position code, a preliminary segmentation result is obtained through a decoding operation. Subsequently, the pre-segmentation result is denoised, and the preliminary segmentation results and candidate boxes of the left and right mandibular canals are obtained by merging the minimum connected subdomains. Then, cubic spline interpolation is performed on the candidate boxes of the left and right mandibular canals, the center line of the mandibular canal is fitted, and the center line is spliced ​​back to the original image. Based on the obtained fitting results, the center line of the mandibular canal and the original CBCT image are input as dual channels into the second-stage dynamic mandibular canal convolution (DIConv) neural network model, and feature fusion is performed through multi-view feature extraction and a three-dimensional attention mechanism (3D attention) module. Finally, the fused feature map is concatenated with the feature map obtained by ordinary convolution and sent to the encoder for encoding and decoding operations. Finally, a fine mandibular canal segmentation result is obtained. Jaccard Loss is used as the loss function to optimize the model and improve the segmentation accuracy.

[0005] This method gradually improves the accuracy and robustness of the segmentation results by introducing a multi-stage segmentation strategy, combining pre-segmentation models, position encoding, curve fitting, and multi-view feature extraction. Specifically, the CBCT images are first roughly segmented to obtain the preliminary mandibular canal region; then the boundary and morphology of the mandibular canal are further optimized through curve fitting; finally, the multi-view dynamic mandibular canal convolution (DIConv) model is used for feature extraction and fusion to ensure the accuracy of the segmentation results.

[0006] The specific steps of the CBCT mandibular nerve canal fine segmentation method based on multi-view feature fusion are as follows:

[0007] Step (1), select samples {X, Y} containing cone beam computed tomography (CBCT) images and corresponding annotations to form a data set, and divide it into a training set, a test set, and a validation set; wherein X and Y represent the CBCT images in the data set and their corresponding mandibular nerve canal annotations, respectively.

[0008] Step (2): For the obtained data set, during the training phase, the training set is cut into several patch blocks, and the position coordinates of the patch blocks are recorded. Then, data enhancement operations are performed on the patch blocks to obtain input data X0.

[0009] Step (3), build a pre-segmentation model based on Unet, encode the position coordinates of each patch block of CBCT, obtain the position code P0, send the data X0 and its corresponding position code P0 into the pre-segmentation network model, and obtain the corresponding pre-segmentation result Y0. The pre-segmentation model will first perform a series of convolution and pooling encoding operations on X0 to obtain the final feature map Feature0.

[0010] Step (4): concatenate the finally encoded Feature0 with the previously calculated position code P0 to perform feature fusion.

[0011] Step (5), decode the feature map Con_Feature0 after the CBCT image encoding and position encoding are fused, continuously perform upsampling and pooling operations, and jump connect the corresponding layer outputs of encoding and decoding.

[0012] Step (6): Use the current training set and validation set to train the pre-segmentation model and save its weights. After obtaining the model and weights, all data sets need to be sent to the pre-segmentation network for inference to obtain the pre-segmentation results Y'0 corresponding to all data.

[0013] Step (7), based on the pre-segmentation result Y'0 obtained in step (6), denoising is performed: based on the pre-segmentation result Y'0, the number of voxels in each minimum connected domain is counted, and the connected domains that are less than the set threshold are deleted. The clustering algorithm is used to obtain the N minimum connected subdomains, and each is initially assigned to N categories. The center distance between all categories is calculated, and the two categories with the closest distance are selected for merging, and the center coordinates of the merged class are updated. If the final number of categories is equal to 2, proceed to the next step. If it is greater than 2, the merging operation is continued.

[0014] Step (8), by merging step (7), the mandibular canal is divided into the left and right mandibular canals Y left and Y right At the same time, the starting point and size of the candidate boxes of the left and right mandibular canals are obtained. For each mandibular canal in the obtained candidate box, the center line of the mandibular canal is fitted using cubic spline interpolation. The center lines of the left and right mandibular canals are spliced ​​back to the positions of the original images, respectively, and recorded as X Line .

[0015] Step (9), the center line X of the mandibular canal obtained by splicing Line X0 and X1 are input as dual channels into the second stage DIConv network model. The center line of the mandibular canal generated previously is used as a prompt mask to extract the multi-view features of the mandibular canal. Then, 3D attention is used to fuse the features extracted from multiple views to obtain the feature map X2. DIConv .

[0016] Step (10), obtain X DIConv It is then concatenated with the feature map obtained by performing ordinary convolution on X0 and input into the DIConv network model. The same operations as steps (3), (4), and (5) are performed for encoding. The position code P0 is then embedded and decoded to obtain the final mandibular canal segmentation result U1.

[0017] DIConv is based on the Unet network, uses dual-channel feature input, and simultaneously embeds the position encoding information of the patch block.

[0018] Step (11), optimize the model by the following loss function, and test the performance by the test set. The loss function is as follows:

[0019]

[0020] Where N represents the number of samples, y i is the true label, is the predicted label, ∈ is a parameter that prevents the denominator from being zero, thus avoiding numerical instability or computational errors.

[0021] Furthermore, the specific implementation process of the pre-segmentation model in step 3 is as follows:

[0022] Step 3.1, encode the position coordinates of each patch block of CBCT to obtain the position code P0; send the input data X0 and its corresponding position code P0 to the pre-segmentation model, first perform convolution and pooling encoding operations on X0 to obtain the feature map Feature0;

[0023] Step 3.2, concatenate the feature map Feature0 with the position code P0 to obtain the feature map Con_Feature0;

[0024] Step 3.3: Decode the feature map Con_Feature0, continuously perform upsampling and pooling operations, and jump-connect the corresponding layer outputs of encoding and decoding to obtain the corresponding pre-segmentation result Y0 after passing through the fully connected layer.

[0025] Furthermore, the denoising in step 7 is specifically as follows: based on the pre-segmentation result Y'0, the number of voxels in each minimum connected domain is counted, and the connected domains with a number of voxels less than a set threshold are deleted; the merging operation is specifically as follows: the N minimum connected domains after denoising are initially assigned to N categories, the center distances between all categories are calculated, the two categories with the closest center distances are selected through a clustering algorithm for merging, and the center coordinates of the merged class are updated. If the final number of categories is equal to 2, proceed to the next step; if it is greater than 2, continue the merging operation.

[0026] Furthermore, the curve fitting of the mandibular nerve canal is specifically implemented in step (8) as follows:

[0027] Step 8.1: For the left mandibular canal Y left and right mandibular canal Y right According to the size of the candidate box, the candidate box is divided into eleven equal parts along the y axis, and ten xoz sections are extracted. For each section, the center point coordinates of the pre-segmentation result Y'0 on the section are extracted, thereby obtaining ten center line coordinate points P1 to P 10 If there is a point without a value among the ten centerline coordinate points (i.e., a point without a predicted mask result in the xoz section), the centerline coordinate point is discarded;

[0028] Step 8.2: Construct the geometric model of the mandibular canal using the centerline coordinates P1 to P 10 Construct a cubic spline interpolation curve C(t); each center line coordinate point P i is assigned a corresponding parameter t i , the formula is as follows:

[0029]

[0030] Where N = 10 is the total number of center line coordinate points; the coordinates of each point C(t) on the curve are obtained by an independent cubic spline function S x (t), S y (t), S z (t) is defined, and the specific formula is:

[0031] For x coordinate: S xi (t i )=a i +b i (tt i )+c i (tt i ) 2 +d i (tt i ) 3 ,t∈[t i ,t i+1 ]

[0032] For the y coordinate: S yi (t i )=e i +f i (tt i )+g i (tt i ) 2 +h i (tt i ) 3 ,t∈[t i ,t i+1 ]

[0033] For z coordinate: S zi (t i )=m i +n i (tt i )+p i (tt i ) 2 +q i (tt i ) 3 ,t∈[t i ,t i+1 ]

[0034] Among them, the coefficient a of each component i ,b i ,c i ,d i ,e i ,f i ,g i ,h i ,mi ,n i ,p i ,q i It is obtained by solving a system of linear equations, ensuring the continuity of the first and second derivatives at the end point of each segment; in addition, the boundary condition is that the second derivative of the curve at the start and end points is zero;

[0035] The three-dimensional curve C(t) is a geometric model describing the centerline of the mandibular canal, where the coordinates of each point are determined by three independent cubic polynomials. These polynomials are in each interval [t i ,t i+1 ] can accurately depict the spatial direction of the mandibular canal;

[0036] Step 8.3: By constructing and fitting the curve in step 8.2, the center lines of the left and right mandibular canals are obtained respectively. Since the voxel proportion of the mandibular canal center line is small, expansion is used for expansion. Finally, the mandibular canal is generated according to the starting position recorded before, and is placed back into the original image to restore to X Line Mandibular nerve canal centerline mask.

[0037] Furthermore, the specific implementation of multi-view mandibular canal convolution DIConv in step (9) is as follows:

[0038] 9-1. For the centerline X of the mandibular canal inputted through two channels Line First, for each coordinate point on the image, set the mandatory unit offset X with a double-end length of kernal_size. direction , Y direction , Z direction , which are composed of a series of coordinate points on the axis with a total length of kernal_size, as the offset reference point:

[0039]

[0040] in ， x c ,y c , z c Represents a voxel coordinate point, the value of k is equal to the value of kernal_size, which represents the total length of the axial convolution.

[0041] 9-2. Based on the coordinates of each point on the reference offset in 9-1, the free offsets Δm of the remaining two axes are predicted by convolution. i and Δn i, the value of a single free offset is fixed in a neighborhood with 0 as the origin and 1 as the length, and the free offset has a cumulative effect. By accumulating the free offset, the convolution position of a certain axis based on the original coordinate point can be obtained. Take the x-axis feature extraction as an example:

[0042] For X direction Any point x c The total offsets Δy and Δz corresponding to +i are:

[0043]

[0044] So the final x c The sampling coordinate point of +i is (x c +i,y c +Δy,z c +Δz), i represents the position to be convolved, the offset is (0, Δy, Δz), traversing X direction The convolution operation can be performed by obtaining the convolution sampling point coordinates of all points in the image.

[0045] For example: the horizontal coordinate point reference coordinate position (x c +2,y c ,z c ), we know the x-axis coordinate at this time and select the position x c +2, we also additionally calculate the corresponding Δy and Δz. Through the previous calculation, we have obtained Δm0 and Δm1 as well as Δn0 and Δn1. At this time, we calculate Δm2 and Δn2 through convolution, and accumulate the obtained Δm0, Δm1 and Δm2 to obtain the value based on (x c +2,y c ,z c ) about y c The corresponding axial offset is similarly accumulated by adding the obtained Δn0, Δn1 and Δn2 to obtain the value based on (x c +2,y c ,z c ) about z c The corresponding axial offset is finally obtained. c +2,y c ,z c )The position (x) that the convolution should take c +2,y c +Δm0+Δm1+Δm2,z c +Δn0+Δn1+Δn2)

[0046] 9-3. For the points obtained, non-integer values ​​may appear. To handle non-integer positions that may appear due to dynamic adjustment, the trilinear interpolation method is used for numerical substitution. This method can effectively estimate the pixel values ​​at non-integer coordinates when calculating the convolution operation, ensuring the consistency and accuracy of feature extraction. The formula is as follows:

[0047]

[0048] in ， f(p) represents the function value at point p, (x i ,y i ,z i ) represents the vertex coordinates in space, f(x i ,y i ,z i ) represents the point (x i ,y i ,z i ). (x0,y0,z0) represents the coordinate vertex at the bottom left, and (x1,y1,z1) represents the coordinate vertex at the top right.

[0049] The beneficial effects of the present invention are:

[0050] The CBCT mandibular nerve canal fine segmentation method based on multi-view feature fusion uses a multi-stage segmentation strategy, first performs rough segmentation and then gradually refines it, which can more accurately capture the detailed structure of the mandibular nerve canal and reduce errors. In the process of constructing the centerline of the mandibular canal, denoising and minimum connected subdomain merging are used to reduce noise interference and ensure the segmentation of the target that needs to be focused. In addition, the Unet and DIConv network models are used, combined with position encoding and multi-view feature extraction, to effectively fuse multiple feature information, ensuring the accuracy and integrity of the segmentation results. This method is not only suitable for the segmentation of the mandibular canal, but also for the segmentation of various types of blood vessels or small and long strip targets in medical images. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flow chart of the present invention;

[0052] Figure 2 This is a model architecture diagram of the present invention;

[0053] Figure 3 Schematic diagram of mandibular canal convolution DIConv of the present invention;

[0054] Figure 4 The process of classifying the left and right mandibular canals for the present invention;

[0055] Figure 5 The segmentation effect of the present invention on the public data set is demonstrated. DETAILED DESCRIPTION

[0056] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0057] like Figure 1 and Figure 2 As shown, the present invention provides a CBCT mandibular canal refined segmentation method based on multi-view feature fusion. The present invention first selects CBCT images and mandibular canal annotations to form a data set; the training set is cut into several patches to obtain input data, and the position coordinates of each patch are encoded. Secondly, each patch and its corresponding position code are sent to the pre-segmentation network model to obtain a pre-segmentation result, and the mandibular canal centerline is obtained by cubic spline interpolation. Finally, multi-view feature extraction is performed on the CBCT and the mandibular canal centerline, and feature fusion is performed through 3D Attention, and the obtained features are decoded to finally obtain the segmentation result of the mandibular canal, and optimization training is performed. The present invention effectively integrates feature information from multiple perspectives, ensures the accuracy and completeness of the segmentation results, and thus reduces the workload of radiologists.

[0058] The specific implementation steps of the present invention are:

[0059] Step (1), select samples {X, Y} containing cone beam computed tomography (CBCT) images and corresponding annotations to form a data set, and divide it into a training set, a test set, and a validation set; wherein X and Y represent the CBCT images in the data set and their corresponding mandibular nerve canal annotations, respectively.

[0060] Step (2): For the obtained data set, during the training phase, it will be cut into patches of size (80, 80, 80) and the position coordinates of the patches will be recorded. Then, these obtained patches will be subjected to data enhancement operations such as random affine transformation and random flipping, and finally the data X0 input to the first module will be of size (6, 1, 80, 80, 80). ， 6 represents the batch size, 1 represents the number of input channels, and (80,80,80) represents the size of each patch block.

[0061] Step (3), for the data that needs to be inferred, it will first be sent to the pre-segmentation model. The pre-segmentation model uses Unet as the prototype, and encodes the position coordinates of the patch obtained by cutting the original data to obtain the position code P0. For X0 obtained by data preprocessing, X0 itself and its corresponding position code P0 are sent to the pre-segmentation network model. The network model will first perform a series of convolution and pooling operations on X0 to obtain the final feature map Feature0. The convolution formula is as follows:

[0062] y=f(W*x+b)

[0063] Among them, x is the input, W is the convolution kernel, * represents the convolution operation, b represents the bias value, f represents the activation function, and (m,n,p)∈window defines the size of the pooling window.

[0064] Step (4) concatenates the Feature0 obtained by the final encoding with the previously calculated position encoding P0 to perform feature fusion. The specific concatenation formula is as follows:

[0065] Con_Feature0=concat(Feature0,P0)

[0066] Among them, Feature0 refers to the feature map obtained after CBCT encoding, and P0 refers to the position encoding of the patch block.

[0067] Step (5), decode the feature map Con_Feature0 after encoding the CBCT image and fusing the position encoding, continuously perform upsampling and pooling operations, and perform jump connection on the corresponding layer outputs of encoding and decoding to obtain the pre-segmentation result Y'0 corresponding to all data; wherein the formulas for upsampling operation and jump connection are as follows:

[0068] y=f(W T *x+b)

[0069] z=concat(y encoder ,y decoder )

[0070] in ，x It is input, middle W T represents the transposed convolution kernel, * represents the convolution operation, b represents the bias value, f represents the activation function, concat represents the connection operation of the feature map, and y encoder Represents the feature map of CBCT input to the encoder part, y decoder Represents the feature map of the decoder part output.

[0071] Step (6): Based on the pre-segmentation result Y'0 obtained in step (5), denoise and initially assign the N smallest connected subdomains obtained to N categories. Calculate the center distance between all categories, select the two categories with the closest distance to merge, and update the center coordinates of the merged class. If the final number of categories is equal to 2, exit the loop. If it is greater than 2, continue the merging operation.

[0072] Step (7), by merging step (6), the mandibular canal is divided into the left and right mandibular canals Y left and Y rightAt the same time, the starting point and size of the candidate boxes of the left and right mandibular canals are obtained. For each mandibular canal in the obtained candidate box, the center line of the mandibular canal is fitted using cubic spline interpolation. The obtained left and right mandibular canal curves are spliced ​​back to the position of the original image, respectively, and recorded as X Line .

[0073] Step (8), the center line X of the mandibular canal obtained by splicing Line With X0 as the dual channel input to the second stage such as Figure 3 In the DIConv network model shown in the figure, the center line of the mandibular canal generated previously is used as a hint mask to extract the features of the mandibular canal from multiple perspectives, and then 3D attention is used to fuse the features extracted from multiple perspectives to obtain the feature map X DSConv .

[0074] Step (9), obtain X DSConv It is then concatenated with the feature map obtained by performing ordinary convolution on X0 and input into the subsequent encoder. The operation is the same as steps (3), (4), and (5). It is encoded and then embedded in the position code P0 for decoding to obtain the final mandibular canal segmentation result Y1.

[0075] Step (10): Optimize the model training through the loss function and test the performance with the test set.

[0076] To test the performance of the method of the present invention, qualitative and quantitative experiments were conducted on a public data set. Figure 4 As shown, it can be seen that the present invention can accurately segment the structure of the mandibular nerve canal. Even when there are discontinuous intervals in the training samples, the segmentation of the present invention has better performance in the continuity of the segmentation results compared with the segmentation of the ordinary traditional model. In addition, the method of the present invention can also perform center sampling on continuous blood vessels to generate the center line of the blood vessels. This method is not only suitable for extracting the center line of the mandibular nerve canal, but also can be used for extracting the center line of other blood vessels.

[0077] In terms of quantitative analysis, the performance of the present invention is compared with the other five most commonly used methods for 3D image segmentation, and the experimental results are shown in Table 1. As can be seen from the table, the IoU and Dice coefficients indicate that the model of the present invention provides superior overlap and similarity between the predicted and actual segmentation masks, and has further improved the accuracy compared with the traditional model.

[0078] Table 1 Performance comparison of data sets (the best results are marked in bold)

[0079]

[0080] The model of the present invention has the smallest Hausdorff distance. The lower Hausdorff distance indicates that the maximum deviation between the predicted and true boundaries is small, indicating that the model performs well in accurately capturing boundary details. Smoothness defined by gradient reflects the smoothness of the model, indicating that the segmentation contour generated by the model of the present invention is smoother than that generated by other models.

[0081] In medical imaging, this reduced granularity is very beneficial because jagged edges may incorrectly represent the true morphology of anatomical structures. In addition, the model of the present invention is also the best in the Small Volume Detection indicator, indicating that our model produces almost no small noise during the segmentation process. Combining the two indicators of Hausdorff distance and Smoothness, the model of the present invention also performs best in terms of continuity. This is mainly due to the Line Extractor submodule and DIConv-Segmentation submodule designed by the present invention. These modules specially customized for the mandibular nerve canal segmentation task can better handle specific image features.

[0082] Line Extractor optimizes the processing of connected domains through numerical analysis methods, extracts the centerline of the mandibular canal, and inputs the centerline and CBCT image as dual channels into the segmentation network of the next stage. The centerline of the mandibular canal plays a very important role as a mask prompt. Through the mask prompt of the centerline, the segmentation network further focuses on the vicinity of the centerline for feature extraction, strengthens the features around the mandibular canal, and has continuity, which is more in line with the medical anatomical structure of the mandibular canal. The schematic diagram of the experimental results is shown in the figure. Figure 5 shown.

[0083] The DIConv-Segmentation module enhances the ability to extract linear features through customized mandibular canal convolution. The feature extraction process forces the use of linear candidate units for extraction. The design of this dedicated module significantly improves the model's ability to handle complex structures and further enhances the continuity of features.

[0084] In the DIConv-Segmentation module, the features extracted from different angles are fused through the attention mechanism, which enhances the model's ability to recognize key features. This feature fusion strategy ensures that information extracted from different perspectives and scales can be effectively integrated, improving the consistency and accuracy of the segmentation results.

[0085] In addition, this experiment also uses multi-view DIConv feature extraction for each model to perform ablation experiments, as shown in Table 2 below.

[0086] Table 2 Ablation experiment of DIConv module (including whether the DIConv module of the present invention is used, and the best result is marked in bold)

[0087]

[0088] It can be seen from the ablation experiment in the table that after the present invention adopts multi-view DIConv feature extraction, the segmentation accuracy is correspondingly improved, proving that the module added by the present invention can further and better extract features.

Claims

1. A CBCT mandibular nerve canal fine segmentation method based on multi-view feature fusion, characterized in that: The steps include: Step 1: Select samples {X, Y} containing CBCT images X and mandibular nerve canal annotations Y to form a data set, and divide it into a training set, a test set, and a validation set; Step 2: Cut the training set into several patches, record the position coordinates of each patch, and then perform data enhancement on the patch to obtain the input data X0; Step 3: Build a pre-segmentation model based on Unet, encode the position coordinates of each patch block of CBCT, obtain the position code P0, and send the input data X0 and its corresponding position code P0 to the pre-segmentation model to obtain the corresponding pre-segmentation result Y0; Step 4: Use the training set and validation set to train the pre-segmentation model and save its weights; Then all data sets are sent to the pre-segmentation model for inference to obtain the pre-segmentation results Y'0 corresponding to all data sets; Step 5: De-noise based on the pre-segmentation result Y'0, and then perform a merging operation; Step 6: Divide the mandibular canal into the left mandibular canal Y left and right mandibular canal Y right , fit the mandibular canal centerline, and splice the left mandibular canal centerline and the right mandibular canal centerline according to the position of the original image to obtain a full zero matrix with the same size as the original size, that is, the mandibular canal centerline mask X Line ; Step 7: X Line X0 is used as a dual-channel input to the multi-view mandibular canal convolution DIConv network model, and then 3D attention is used to perform multi-view feature fusion to obtain the feature map X DIConv ; Step 8: Convert the feature map X DIConv It is concatenated with the feature map obtained by convolution of X0 and then input into the DIConv network model to obtain the final mandibular canal segmentation result Y1; Step 9: Optimize the model training through the loss function and test the performance with the test set.

2. The CBCT mandibular nerve canal refined segmentation method based on multi-view feature fusion according to claim 1, characterized in that: The specific implementation process of the pre-segmentation model is as follows: Step 3.1, encode the position coordinates of each patch block of CBCT to obtain the position code P0; send the input data X0 and its corresponding position code P0 to the pre-segmentation model, first perform convolution and pooling encoding operations on X0 to obtain the feature map Feature0; Step 3.2, concatenate the feature map Feature0 with the position code P0 to obtain the feature map Con_Feature0; Step 3.3: Decode the feature map Con_Feature0, continuously perform upsampling and pooling operations, and jump-connect the corresponding layer outputs of encoding and decoding to obtain the corresponding pre-segmentation result Y0 after passing through the fully connected layer.

3. The CBCT mandibular nerve canal refined segmentation method based on multi-view feature fusion according to claim 1, characterized in that: In step 5, the denoising is specifically to count the number of voxels of each minimum connected domain based on the pre-segmentation result Y'0, and delete the connected domain with the number of voxels less than the set threshold; The merging operation is specifically as follows: the N minimum connected domains after denoising are initially assigned to N categories, the center distances between all categories are calculated, the two categories with the closest center distances are selected for merging through a clustering algorithm, and the center coordinates of the merged class are updated. If the final number of categories is equal to 2, proceed to the next step; if it is greater than 2, continue the merging operation.

4. The CBCT mandibular nerve canal refined segmentation method based on multi-view feature fusion according to claim 3, characterized in that: The step 6 is specifically implemented as follows: Step 6.1: For the left mandibular canal Y left and right mandibular canal Y right According to the size of the candidate box, the candidate box is divided into eleven equal parts along the y axis, and ten xoz sections are extracted. For each section, the center point coordinates of the pre-segmentation result Y'0 on the section are extracted, thereby obtaining ten center line coordinate points P1 to P 10 , if there is a point without a value among the ten center line coordinate points, then the center line coordinate point will be discarded; Step 6.2: Construct the geometric model of the mandibular canal using the centerline coordinates P1 to P 10 Construct a cubic spline interpolation curve C(t); each center line coordinate point P i is assigned a corresponding parameter t i , the formula is as follows: Where N = 10 is the total number of center line coordinate points; the coordinates of each point C(t) on the curve are obtained by an independent cubic spline function S x (t), S y (t), S z (t) is defined, and the specific formula is: For x coordinate: S xi (t i )=a i +b i (tt i )+c i (tt i ) 2 +d i (tt i ) 3 ,t∈[t i ,t i+1 ] For the y coordinate: S yi (t i )=e i +f i (tt i )+g i (tt i ) 2 +h i (tt i ) 3 ,t∈[t i ,t i+1 ] For z coordinate: S zi (t i )=m i +n i (tt i )+p i (tt i ) 2 +q i (tt i ) 3 ,t∈[t i ,t i+1 ] Among them, the coefficient a of each component i ,b i ,c i ,d i ,e i ,f i ,g i ,h i ,m i ,n i ,p i ,q i It is obtained by solving a system of linear equations, ensuring the continuity of the first and second derivatives at the end point of each segment; in addition, the boundary condition is that the second derivative of the curve at the start and end points is zero; The three-dimensional curve C(t) is a geometric model describing the centerline of the mandibular canal, where the coordinates of each point are determined by three independent cubic polynomials. These polynomials are in each interval [t i ,t i+1 ] can accurately depict the spatial direction of the mandibular canal; Step 6.3: Through the curve construction and fitting of step 6.2, the center lines of the left and right mandibular canals are obtained respectively, and the expansion is performed. Finally, the obtained mandibular canals are placed back into the original image according to the starting position recorded before, and restored to X Line Mandibular nerve canal centerline mask.

5. The CBCT mandibular nerve canal refined segmentation method based on multi-view feature fusion according to claim 3, characterized in that: The specific implementation process of the multi-view mandibular canal convolution DIConv network model is as follows: Step 7.1: X Line For each coordinate point of the image after merging with X0, set the mandatory unit offset X with a total length of kernal_size on both ends direction , Y direction , Z direction , which are composed of a series of coordinate points on the axis, serving as offset reference points: Among them, x c ,y c , z c Represents a voxel coordinate point, the value of k is equal to the value of kernal_size; Step 7.2: Based on offset X direction , Y direction , Z direction The coordinates of each point on the axis are convolved to predict the free offset Δm of the remaining two axes. i and Δn i , the value of a single free offset is fixed in a neighborhood with 0 as the origin and 1 as the length, and the free offset has a cumulative effect; by accumulating the free offset, the convolution position of a certain axis based on the original coordinate point is obtained; Step 7.3: To handle non-integer positions due to dynamic adjustment, trilinear interpolation method is used for numerical substitution. The formula is as follows: in ， f(p) represents the function value at point p, (x i ,y i ,z i ) represents the vertex coordinates in space, f(x i ,y i ,z i ) represents the point (x i ,y i ,z i ) at the point where the function is located; (x0, y0, z0) represents the coordinate vertex at the leftmost front and bottom, and (x1, y1, z1) represents the coordinate vertex at the rightmost back and top.

6. The CBCT mandibular nerve canal refined segmentation method based on multi-view feature fusion according to claim 5, characterized in that: The loss function Jaccard Loss is: Where N represents the number of samples, y i is the true label, is the predicted label, ∈ is a parameter that prevents the denominator from being zero.

Images