A method for automatically segmenting the mandibular nerve canal based on multi-view guidance

Abstract

The application discloses a kind of automatic segmentation methods of mandibular nerve tube based on multi-view guidance, and the application relates to the field of oral medical technology.The application comprises the following steps: collecting original oral CBCT image data, and pre-processing;The pre-processed image data is sent into the mandibular bone segmentation network, and the mandibular bone region containing the mandibular nerve tube is segmented;The contrast enhancement technique is used to the segmented mandibular bone region;The processed mandibular bone image is projected along the coronal plane, the sagittal plane and the orthogonal cross section of the cross section;The key point detection network trained is used to detect the key points of the mandibular nerve tube on the three projection planes, and the shape information of the mandibular nerve tube is obtained;And these features are combined with texture direction features and input into the three-dimensional CBCT mandibular nerve tube segmentation network for accurate 3D segmentation.The space information missing in two dimensions is avoided, so that the mandibular nerve tube segmentation is more complete and accurate.

Description

Technical Field

[0001] This invention relates to the field of oral medical technology, specifically to an automatic segmentation method for the mandibular nerve canal based on multi-view guidance. Background Technology

[0002] According to the World Health Organization's "Global Status Report on Oral Health," approximately 3.5 billion people worldwide suffer from oral diseases. While largely preventable, oral diseases impose a significant health burden on many countries and impact people's lives, causing pain, discomfort, disfigurement, and even death. Existing research shows a close link between oral diseases and several systemic diseases, such as cardiovascular disease and diabetes. Therefore, oral health issues are receiving increasing attention.

[0003] With the rapid development of "artificial intelligence + healthcare," digital technology is changing the way oral diseases are treated and diagnosed. Among these advancements, the mandibular nerve canal is a crucial tissue to be carefully examined during maxillofacial surgery. Whether extracting the third molar or performing dental implant surgery, damage to the mandibular nerve canal must be avoided. Complete and accurate segmentation of the mandibular nerve canal not only facilitates the measurement of various intraoperative quantitative indicators but also significantly improves the efficiency and accuracy of the surgery, achieving "zero damage" to the mandibular nerve canal.

[0004] Traditional mandibular canal segmentation is generally based on prior knowledge of shape models, using statistical shape models (SSM) to segment the mandible, and then employing fast matching or tracking algorithms to segment the mandibular canal. However, these methods rely on prior knowledge and robust mandibular segmentation models.

[0005] Existing digital methods for automatic segmentation of the mandibular nerve canal mostly employ a coarse-to-fine segmentation approach, using a cascade of multiple segmentation networks to achieve the best segmentation results. However, this cascaded segmentation method has significant inter-network interference, and relies solely on the knowledge learned by the networks themselves during the segmentation process, without incorporating prior knowledge of shape to guide the segmentation, resulting in the final segmentation results failing to meet expectations.

[0006] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] The purpose of this invention is to provide an automatic segmentation method for the mandibular nerve canal based on multi-view guidance, so as to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A method for automatic segmentation of the mandibular nerve canal based on multi-view guidance, comprising the following steps:

[0010] Several raw oral CBCT image data with known mandibular nerve canal feature information were collected. The collected raw oral CBCT image data were preprocessed to obtain sample CBCT image data. Based on the obtained sample CBCT image data, the mandibular region was marked in the sample CBCT images by manual labeling to obtain training CBCT image data. The mandibular nerve canal feature information includes the location information of key points of the mandibular nerve canal and the corresponding heat map.

[0011] Based on the obtained training CBCT image data, a neural prediction network is established. The training CBCT images are used as the input of the neural prediction network, and the mandibular region in the images is used as a label to train the neural prediction network, resulting in a mandibular segmentation network.

[0012] The training CBCT image is segmented based on the mandibular segmentation network to obtain the mandibular tissue image. The contrast of the obtained mandibular tissue image is enhanced to obtain the target mandibular tissue image. The target mandibular tissue image is projected along three orthogonal planes: the coronal plane, the sagittal plane, and the cross-section to obtain the projection planes of the three orthogonal planes.

[0013] Based on the known features of the mandibular nerve canal from the training CBCT image data, the key points of the mandibular nerve canal in each of the three orthogonal projection planes are determined. The key points are marked in each projection plane, the location information is extracted, and the correspondence with the known key point heatmap is made.

[0014] Based on the location information of key points in each projection plane and the heat map of the corresponding key points, a key point detection network is established. The projection planes of three orthogonal tangents are used as input, and the location information of key points in each projection plane and the heat map of the corresponding key points are used as labels to train the key point detection network.

[0015] After passing the oral CBCT image to be segmented through the mandibular bone segmentation network, contrast enhancement, projection, and key point detection network, the heat map output by the key point detection network is used to extract shape feature information through an encoder. The shape feature information of the mandibular nerve canal, combined with the texture direction feature, is used as guidance information and input into the 3D CBCT mandibular nerve canal segmentation network to guide the network to perform the final accurate 3D segmentation of the mandibular nerve canal.

[0016] Furthermore, image preprocessing is performed on several raw oral CBCT image data collected, including signal denoising and enhancement processing.

[0017] Furthermore, the neural prediction network is trained to obtain a mandibular segmentation network, which is based on the 3D-Unet segmentation network. Preprocessed CBCT data is fed into the mandibular segmentation network, and redundant regions are removed. The mandible is segmented as the region of interest. Specific steps include: training the mandibular segmentation network using training CBCT image data as input and labeled mandibular regions as output labels, minimizing the loss during training to obtain the optimal parameters of the segmentation network; using the trained mandibular segmentation network to segment the mandibular tissue separately to obtain mandibular tissue images, where the optimal parameters of the segmentation network are obtained. Specific steps include: cropping the training CBCT image data and corresponding labeled data into 128... A 128-sized piece;

[0018] Data augmentation operations are performed on the data, including random rotation, translation, scaling, and flipping. The augmented data is then fed into the 3D-Unet segmentation network. The encoder extracts image features and reduces their spatial dimension step by step through a combination of multiple convolutional and pooling layers. The finest image features are extracted at the bottleneck layer. In the decoding stage, the transposed convolutional layer is used to upsample the feature map to restore the spatial resolution, and skip connections are used to concatenate the feature maps of the corresponding layers of the encoder with the feature maps of the decoder to restore spatial information.

[0019] The network parameters are trained using a weighted combination of cross-entropy loss and Dice loss to obtain the optimal parameters for the mandibular segmentation network.

[0020] Furthermore, the specific steps for obtaining the optimal parameters of the segmentation network include: defining the cross-entropy loss as... The Dice loss is defined as follows: The weighted combination of cross-entropy loss and Dice loss is defined as The loss of the mandibular segmentation network Specifically defined as:

[0021]

[0022] In the formula, This represents the weighting coefficient, with a value range of 100. [0,1]; Cross-entropy loss and Dice loss The definition of is:

[0023]

[0024]

[0025] In the formula, N represents the total number of voxels in the training CBCT image data; Indicates the first The true label value of the individual element, with a range of values ​​of: 0 represents the background region, and 1 represents the mandibular region; It is a segmentation model for the first The predicted value of individual elements, To train the voxel index in CBCT image data, .

[0026] Furthermore, the obtained mandibular tissue image is contrast-enhanced to obtain the target mandibular tissue image. The contrast enhancement technique used is adaptive histogram equalization, and the specific steps include: dividing the mandibular tissue image into... The image is divided into patches, and the pixel histogram of each patch is calculated. A contrast limit threshold is set within each patch. The histogram of each patch is equalized, that is, the cumulative distribution function of each pixel histogram is mapped to the range of [0,255] pixels. The patches are merged and the transition between the patches is smoothed by bilinear interpolation at the boundary to obtain the final target mandibular tissue image.

[0027] Furthermore, using perspective projection, each target mandibular tissue image is projected along the sagittal, coronal, and cross-sectional planes to obtain its respective projection section.

[0028] Using the projected plane images of three orthogonal tangents as input, and the location information of key points in each projected plane image and the corresponding heatmap of the key points as labels, the key point detection network is trained. The specific steps for training the key point detection network include:

[0029] The sagittal, coronal, and cross-sectional projection images corresponding to each target mandibular tissue image are paired, and the size of the three projection images is unified using the nearest neighbor interpolation algorithm;

[0030] The projected image is input into the HR-Net keypoint detection network, which consists of two layers. Convolutional layer + Batch Normalization + ReLU activation function are used to extract the initial feature map;

[0031] The category information of the projection image is introduced, that is, the sagittal, coronal and cross-sectional projection images have their own exclusive categories. The category information is used to obtain the category features of the projection image through the information embedding module. The category features of the projection image are fused into the initial feature map to obtain the advanced image feature map containing the category information of the projection image.

[0032] Advanced feature maps extract feature maps of different resolutions through parallel multi-branch paths. Low-resolution branches capture global contextual information, while high-resolution branches capture detailed information. Furthermore, high-resolution feature maps are fused with low-resolution feature maps to capture multi-scale information.

[0033] All low-resolution feature maps are upsampled and fused with high-resolution feature maps to output a predicted key point heatmap.

[0034] The training of the keypoint detection network is constrained by a weighted combination of mean squared error loss, heatmap loss, and focus loss, based on the predicted keypoint heatmap and the known real keypoint heatmap.

[0035] Furthermore, the loss function for training the keypoint detection network... Defined as:

[0036]

[0037] In the formula, It belongs to the weight hyperparameters, among which and , For mean square error loss, For heatmap loss, The focus loss includes the mean squared error loss. and heat map loss and focus loss The definition of is:

[0038]

[0039]

[0040]

[0041] In the formula, It is the total number of key points. It is the first The true location of each key point It is the first Predicted locations of key points; Indicates the first Real heat maps of key points No. The predicted heatmap of key points; M is the total number of pixels in each heatmap. It is the first The j-th pixel on the thermal map is predicted as either the foreground probability or the... The probability that the j-th pixel on the heatmap is predicted to be the background. It is a weighting factor that adjusts class imbalance. It is a moderating factor for focus loss, where For indexes of key points, , For the index of pixels on the heatmap, .

[0042] Furthermore, the heatmap output by the keypoint detection network is used to extract shape feature information through an encoder, and the shape feature information of the mandibular nerve canal, combined with texture direction features, is input into the 3D CBCT mandibular nerve canal segmentation network as guiding information. The formula used to calculate the texture direction features is as follows:

[0043]

[0044] In the formula, For texture direction feature vectors, For setting up voxel windows, The number of voxels in the window. Let be the gradient vector of the voxel. This is the transpose of the gradient vector. This represents the position coordinates of a voxel, where the gradient vector of the voxel is... The formula used for the calculation is:

[0045]

[0046] In the formula, This represents the rate of change of voxel values ​​along the X-axis. This represents the rate of change of voxel values ​​along the Y-axis. This represents the rate of change of voxel values ​​along the Z-axis.

[0047] Compared with the prior art, the beneficial effects of the present invention are:

[0048] This invention uses a segmentation network to segment the mandible first, obtaining the region of interest, thus reducing redundant information from the entire CBCT image. Compared to locating the mandibular nerve canal from a complete CBCT image, this method greatly saves the computational complexity of subsequent keypoint detection networks and 3D segmentation networks, and also lays the foundation for more accurate segmentation of the mandibular nerve canal.

[0049] A keypoint detection network is employed to detect keypoints on three orthogonal projection planes obtained from the sagittal, coronal, and cross-sectional planes. The network extracts primary, secondary, and high-level features to capture the basic shape information contained in each plane. Finally, the information from the three projection planes is combined to obtain the basic shape information of the entire mandibular canal. This method reduces the computational complexity of the model and minimizes the spatial information lost in keypoint detection in a 2D plane.

[0050] By incorporating basic shape information constraints into the 3D segmentation of the mandibular nerve canal, the segmentation network focuses more on extracting shape-related features of the mandibular nerve canal during feature extraction, thereby guiding the decoder to generate a more complete and accurate 3D mandibular nerve canal. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the overall method flow of the present invention;

[0052] Figure 2 This is a diagram of the mandibular segmentation network structure of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0054] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0055] Example:

[0056] Please see Figures 1-2 The present invention provides a technical solution:

[0057] A method for automatic segmentation of the mandibular nerve canal based on multi-view guidance, comprising the following steps:

[0058] Step 1: Collect several raw oral CBCT image data with known mandibular nerve canal feature information. Perform image preprocessing on the collected raw oral CBCT image data to obtain sample CBCT image data. Based on the obtained sample CBCT image data, use manual labeling to annotate the mandibular region in the sample CBCT images to obtain training CBCT image data. The mandibular nerve canal feature information includes the location information of key points of the mandibular nerve canal and the corresponding heat map.

[0059] Several raw oral CBCT image data were preprocessed, including signal denoising and enhancement.

[0060] The specific methods for signal denoising and enhancement are as follows: Denoising of the infrared spectral image is performed using wavelet transform. The specific steps of wavelet transform denoising include: decomposing the distortion-corrected image using wavelet transform to obtain wavelet coefficients at different scales and directions; thresholding the wavelet coefficients, setting low-amplitude wavelet coefficients to zero while retaining high-amplitude wavelet coefficients; performing inverse transform on the thresholded wavelet coefficients to reconstruct the image, thus completing the image denoising process; and using bilateral filtering to enhance the details of the original oral CBCT image.

[0061] The mandibular region was marked manually using the LabelImg tool.

[0062] Step 2: Based on the obtained training CBCT image data, establish a neural prediction network. Use the training CBCT images as input to the neural prediction network and use the mandibular region annotations in the images as labels to train the neural prediction network, thus obtaining the mandibular segmentation network.

[0063] A neural prediction network was trained to obtain a mandibular segmentation network, which was built based on the 3D-Unet segmentation network. Preprocessed CBCT data was fed into the mandibular segmentation network, and redundant regions were removed. The mandible was segmented as the region of interest. Specific steps included: training the mandibular segmentation network using CBCT image data as input and labeled mandibular regions as output labels, minimizing the loss during training to obtain the optimal parameters of the segmentation network; and using the trained mandibular segmentation network to segment the mandibular tissue separately to obtain mandibular tissue images, where the optimal parameters of the segmentation network were obtained by cropping the training CBCT image data and corresponding labeled data into 128 pixels. A 128-sized piece;

[0064] Data augmentation operations are performed on the data, including random rotation, translation, scaling, and flipping. The augmented data is then fed into the 3D-Unet segmentation network. The encoder extracts image features and reduces their spatial dimension step by step through a combination of multiple convolutional and pooling layers. The finest image features are extracted at the bottleneck layer. In the decoding stage, the transposed convolutional layer is used to upsample the feature map to restore the spatial resolution, and skip connections are used to concatenate the feature maps of the corresponding layers of the encoder with the feature maps of the decoder to restore spatial information.

[0065] The network parameters are trained using a weighted combination of cross-entropy loss and Dice loss to obtain the optimal parameters for the mandibular segmentation network.

[0066] The specific steps for obtaining the optimal parameters of the segmentation network include: defining the cross-entropy loss as... The Dice loss is defined as follows: The weighted combination of cross-entropy loss and Dice loss is defined as The loss of the mandibular segmentation network Specifically defined as:

[0067]

[0068] In the formula, This represents the weighting coefficient, with a value range of 100. [0,1]; Cross-entropy loss and Dice loss The definition of is:

[0069]

[0070]

[0071] In the formula, N represents the total number of voxels in the training CBCT image data; Indicates the first The true label value of the individual element, with a range of values ​​of: 0 represents the background region, and 1 represents the mandibular region; It is a segmentation model for the first The predicted value of individual elements, To train the voxel index in CBCT image data, .

[0072] Step 3: Segment the trained CBCT image based on the mandibular segmentation network to obtain the mandibular tissue image. Enhance the contrast of the obtained mandibular tissue image to obtain the target mandibular tissue image. Project the target mandibular tissue image along three orthogonal planes: the coronal plane, the sagittal plane, and the cross-section to obtain the projection planes of the three orthogonal planes respectively.

[0073] Using perspective projection, each target mandibular tissue image is projected along the sagittal, coronal, and cross-sectional planes to obtain its respective projection cross-section; the direction along the sagittal plane corresponds to the X-axis coordinate direction, the direction along the coronal plane corresponds to the Y-axis coordinate direction, and the direction along the cross-section corresponds to the Z-axis coordinate direction.

[0074] The obtained mandibular tissue image is contrast-enhanced to obtain the target mandibular tissue image. The contrast enhancement technique used is adaptive histogram equalization, and the specific steps include: dividing the mandibular tissue image into... The image is divided into patches, and the pixel histogram of each patch is calculated. A contrast limit threshold is set within each patch. The histogram of each patch is equalized, that is, the cumulative distribution function of each pixel histogram is mapped to the range of [0,255] pixels. The patches are merged and the transition between the patches is smoothed by bilinear interpolation at the boundary to obtain the final target mandibular tissue image.

[0075] Step 4: Based on the known mandibular nerve canal feature information from the training CBCT image data, determine the key points of the mandibular nerve canal in each of the three orthogonal projection planes. Mark the key points in each projection plane, extract their location information, and correlate them with the known key point heatmap.

[0076] The key points of the mandibular nerve canal typically include the origin and the end point of the mandibular nerve canal. The origin is usually located at the mandibular foramen, which is the entrance for the inferior alveolar nerve to enter the mandible from the base of the skull, and the end point is usually located at the mental foramen, which is the exit for the inferior alveolar nerve to exit the mandible and reach the face.

[0077] The path features of the mandibular nerve canal are as follows: the mandibular nerve canal curves along the mandible, and there may be significant bends or inflections along the path. These features reflect the spatial orientation of the nerve canal, such as the intermediate inflection point (intermediate path point) of the nerve canal. They are used to describe the overall three-dimensional curve shape of the nerve canal and structural change points, such as places where the inner diameter of the nerve canal suddenly narrows or widens.

[0078] Specific anatomical landmarks correspond to points where the mandibular nerve canal is associated with other important anatomical structures (such as tooth roots, the chin of the mandible) near certain specific anatomical locations. For example, points near tooth roots: points in the root apex region of the mandibular nerve canal near the mandibular posterior teeth (such as molars or premolars).

[0079] Step 5: Based on the location information of key points in each projection plane and the corresponding heat map of key points, establish a key point detection network. Use the projection planes of three orthogonal tangents as input and the location information of key points in each projection plane and the corresponding heat map of key points as labels to train the key point detection network.

[0080] Using the projected plane images of three orthogonal tangents as input, and the location information of key points in each projected plane image and the corresponding heatmap of the key points as labels, the key point detection network is trained. The specific steps for training the key point detection network include:

[0081] The sagittal, coronal, and cross-sectional projection images corresponding to each target mandibular tissue image are paired, and the size of the three projection images is unified using the nearest neighbor interpolation algorithm;

[0082] The projected image is input into the HR-Net keypoint detection network, which consists of two layers. Convolutional layer + Batch Normalization + ReLU activation function are used to extract the initial feature map;

[0083] The category information of the projection image is introduced, that is, the sagittal, coronal and cross-sectional projection images have their own exclusive categories. The category information is used to obtain the category features of the projection image through the information embedding module. The category features of the projection image are fused into the initial feature map to obtain the advanced image feature map containing the category information of the projection image.

[0084] Advanced feature maps extract feature maps of different resolutions through parallel multi-branch paths. Low-resolution branches capture global contextual information, while high-resolution branches capture detailed information. Furthermore, high-resolution feature maps are fused with low-resolution feature maps to capture multi-scale information.

[0085] All low-resolution feature maps are upsampled and fused with high-resolution feature maps to output a predicted key point heatmap.

[0086] The training of the keypoint detection network is constrained by a weighted combination of mean squared error loss, heatmap loss, and focus loss, based on the predicted keypoint heatmap and the known real keypoint heatmap.

[0087] Loss function for training keypoint detection network Defined as:

[0088]

[0089] In the formula, It belongs to the weight hyperparameter. For mean square error loss, For heatmap loss, The focus loss includes the mean squared error loss. and heat map loss and focus loss The definition of is:

[0090]

[0091]

[0092]

[0093] In the formula, It is the total number of key points. It is the first The true location of each key point It is the first Predicted locations of key points; Indicates the first Real heat maps of key points No. The predicted heatmap of key points; M is the total number of pixels in each heatmap. It is the first The j-th pixel on the thermal map is predicted as either the foreground probability or the... The probability that the j-th pixel on the heatmap is predicted to be the background. It is a weighting factor that adjusts class imbalance. It is a moderating factor for focus loss, where For indexes of key points, , For the index of pixels on the heatmap, Among them, due to focus loss The main function of heatmaps is to enhance focus on keypoint regions while suppressing background interference. This is crucial for keypoint detection tasks because keypoint regions only occupy a small portion of the image, while background regions are usually the main interference and therefore have the highest weight, resulting in a significant reduction in heatmap loss. Its purpose is to guide the model in generating probability heatmaps that match the true distribution, thereby enhancing the local detection capability of key points. In contrast, the mean squared error loss... This is a direct constraint on the final keypoint coordinates. Since the distribution information of the heatmap is more important for network training, the weight of the heatmap loss is usually higher than that of the mean squared error loss; therefore, it is set... and .

[0094] Step 6: After passing the oral CBCT image to be segmented through the mandibular segmentation network, contrast enhancement, projection and key point detection network, the heat map output by the key point detection network is used to extract shape feature information through the encoder. The shape feature information of the mandibular nerve canal combined with the texture direction feature is used as guidance information and input into the 3D CBCT mandibular nerve canal segmentation network to guide the network to perform the final accurate 3D segmentation of the mandibular nerve canal.

[0095] The heatmap output by the keypoint detection network is used to extract shape feature information through an encoder. The shape feature information of the mandibular nerve canal, combined with texture direction features, is then input into the 3D CBCT mandibular nerve canal segmentation network as guiding information. The formula used to calculate the texture direction features is as follows:

[0096]

[0097] In the formula, For texture direction feature vectors, For setting up voxel windows, The number of voxels in the window. Let be the gradient vector of the voxel. This is the transpose of the gradient vector. This represents the position coordinates of a voxel, where the gradient vector of the voxel is... The formula used for the calculation is:

[0098]

[0099] In the formula, This represents the rate of change of voxel values ​​along the X-axis. This represents the rate of change of voxel values ​​along the Y-axis. This represents the rate of change of voxel values ​​along the Z-axis.

[0100] The specific steps involved in the guidance network performing final, precise 3D segmentation of the mandibular canal include:

[0101] The sagittal, coronal, and cross-sectional heatmaps with key point location information output by the key point detection network are respectively input into three image encoders to extract shape information.

[0102] Shape information and texture direction feature information extracted from the projection surface are sent to the feature fusion module, so that the shape information is injected into the 3D CBCT image encoder;

[0103] The corresponding 3D training CBCT image data is fed into the image encoder based on Swin Transformer. The image encoder combines the injected basic shape information with the powerful global modeling capability of Transformer to extract the overall structural features focused on the mandibular canal.

[0104] In the decoder section, following the traditional U-Net decoding structure, the features extracted by the encoder are connected to the decoder at various scales through skip connections, ultimately resulting in a precisely segmented 3D mandibular neural canal.

[0105] The loss function for training the 3D CBCT mandibular canal segmentation network is defined as follows: Its expression is as follows:

[0106]

[0107] In the formula, It is a weighting factor, and its value range is... , It is Dice's loss. This is the clDice loss, which simultaneously considers the topology and connectivity of the mandibular canal, and is defined as follows:

[0108]

[0109] In the formula, It is the result of network prediction. It is the actual value. and It was obtained through the soft-skeleton method. and Defined as:

[0110]

[0111]

[0112] In the formula, during the segmentation of the mandibular nerve canal, Indicates the result of network prediction The extracted neural tube midline, From the true value The extracted skeletal information represents the central axis of the actual location of the neural tube. For topological accuracy, used for calculation Neutral with the true value The proportion of the overlapping part, Topological sensitivity is used to represent Results of network prediction The proportion of overlapping parts.

[0113] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0114] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0115] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0116] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for automatic segmentation of the mandibular nerve canal based on multi-view guidance, characterized in that, The specific steps include: Collecting a plurality of original oral CBCT image data with known mandibular nerve canal feature information, performing image preprocessing on the collected plurality of original oral CBCT image data to obtain sample CBCT image data, based on the obtained sample CBCT image data, using artificial marking to mark the mandibular bone region in the sample CBCT image to obtain training CBCT image data, and the mandibular nerve canal feature information includes mandibular nerve canal key point position information and corresponding heat map; Based on the obtained training CBCT image data, a neural prediction network is established, the training CBCT image is taken as the input of the neural prediction network, and the mandibular bone region marking in the image is taken as the label to train the neural prediction network, and a mandibular bone segmentation network is obtained; Based on the mandibular bone segmentation network, the training CBCT image is segmented to obtain a mandibular bone tissue image, the obtained mandibular bone tissue image is subjected to contrast enhancement to obtain a target mandibular bone tissue image, and the target mandibular bone tissue image is projected along three orthogonal sections of the coronal plane, the sagittal plane and the transverse plane to obtain three projection plane graphs of the orthogonal sections, respectively; According to the known mandibular nerve canal feature information of the training CBCT image data, the mandibular nerve canal key points in each projection plane graph are determined, the key points are marked in each projection plane graph, the position information is extracted, and the corresponding key point heat map is obtained; Based on the position information of the key points in each projection plane graph and the heat map of the corresponding key points, a key point detection network is established, the projection plane graphs of the three orthogonal sections are taken as the input, and the position information of the key points in each projection plane graph and the heat map of the corresponding key points are taken as the label to train the key point detection network; After the oral CBCT image to be segmented is subjected to the mandibular bone segmentation network, contrast enhancement, projection and key point detection network, the heat map output by the key point detection network is subjected to shape feature information extraction by an encoder, and the shape feature information of the mandibular nerve canal is combined with the texture direction feature as guidance information input into the three-dimensional CBCT mandibular nerve canal segmentation network to guide the network to perform final accurate 3D segmentation on the mandibular nerve canal.

2. The method of claim 1, wherein the method is a multi-view guided automatic segmentation method of the mandibular nerve canal. The collected plurality of original oral CBCT image data is subjected to image preprocessing, and the preprocessing includes signal denoising processing and enhancement processing.

3. The method of claim 2, wherein the method is a multi-view guided automatic segmentation method of the mandibular nerve canal. The neural prediction network is trained to obtain a mandible segmentation network, wherein the mandible segmentation network is established based on a 3D-Unet segmentation network, and the preprocessed CBCT data is input into the mandible segmentation network to remove redundant regions of the data and segment the mandible as a region of interest, wherein the specific steps include: taking the training CBCT image data as input and taking the labeled mandible region as output label to train the mandible segmentation network, and obtaining the optimization parameters of the segmentation network by minimizing the loss during the training process; the mandible tissue is separately segmented by the mandible segmentation network that has completed the training to obtain a mandible tissue image, wherein the optimization parameters of the segmentation network are obtained, and the specific steps include: the training CBCT image data and the corresponding labeled data are cropped into 128 128 size patches; The data is subjected to random rotation, translation, scaling and flipping data enhancement operation; the data subjected to the data enhancement operation is sent into the 3D-Unet segmentation network, the encoder gradually extracts image features and reduces the spatial dimensions through the combination of a plurality of convolution layers and pooling layers; the most fine image features are extracted at the bottleneck layer; in the decoding stage, the transposed convolution layer is used for upsampling the feature map, the spatial resolution is recovered, and the encoder corresponding layer feature map is spliced with the decoder feature map through the jump connection to recover the spatial information; The network parameters are trained using the weighted combination of cross-entropy loss and Dice loss to obtain the optimized parameters of the mandibular bone segmentation network.

4. The method of claim 3, wherein: The specific steps of obtaining the optimization parameters of the segmentation network include: defining the cross-entropy loss as defining the Dice loss as defining the weighted combination of the cross-entropy loss and the Dice loss as that is, the loss of the mandible segmentation network is specifically defined as: In the formula, denotes a weighting coefficient, and the value range is [0, 1]; cross-entropy loss [0, 1]; cross-entropy loss and Dice loss The definition is: In the formula, N represents the total number of voxels in the training CBCT image data; represents the true label value of the i-th voxel, and the value range is , 0 represents a background region, and 1 represents a mandible region; is the prediction value of the i-th voxel by the segmentation model, is the index of the voxel in the training CBCT image data, .​​ 5. The method of claim 1, wherein: The obtained mandibular tissue image is subjected to contrast enhancement to obtain a target mandibular tissue image, wherein an adaptive histogram equalization technique is adopted, and the specific steps include: dividing the mandibular tissue image into patches, calculating the pixel histogram of each patch; setting a contrast limit threshold in each patch; equalizing the histogram of each patch, i.e. mapping the cumulative distribution function of each pixel histogram to the pixel range of [0, 255]; merging the patches, and smoothing the transition between the patches at the boundary part by using a bilinear interpolation method to obtain the final target mandibular tissue image. The obtained mandibular tissue image is subjected to contrast enhancement to obtain a target mandibular tissue image, wherein an adaptive histogram equalization technique is adopted, and the specific steps include: dividing the mandibular tissue image into patches, calculating the pixel histogram of each patch; setting a contrast limit threshold in each patch; equalizing the histogram of each patch, i.e. mapping the cumulative distribution function of each pixel histogram to the pixel range of [0, 255]; merging the patches, and smoothing the transition between the patches at the boundary part by using a bilinear interpolation method to obtain the final target mandibular tissue image.

6. The method of claim 5, wherein: Using perspective projection method, each target mandibular tissue image is projected along sagittal plane, coronal plane and cross section to obtain respective projection sections; Taking the projection planar graphs of the three orthogonal sections as input, and taking the position information of the key points in each projection planar graph and the heat map of the corresponding key points as labels, the key point detection network is trained, and the specific steps of training the key point detection network include: The sagittal plane, coronal plane and cross section projection images corresponding to each target mandibular tissue image are paired, and the sizes of the three projection images are unified using the nearest neighbor interpolation algorithm; The projection image is input into a key point detection network HR-Net network, and initial feature maps are extracted through 2 layers of convolution + Batch Normalization + ReLU activation functions. convolution + Batch Normalization + ReLU activation functions. The category information of the projection images is introduced, that is, the sagittal plane, coronal plane and cross section projection images have their own exclusive categories, and the category information is obtained by an information embedding module to obtain projection image category features, which are fused into the initial feature map to obtain an advanced image feature map containing projection image category information; The advanced feature map extracts different resolution feature maps through a parallel multi-branch, the low resolution branch captures global context information, the high resolution branch captures detailed information, and the high resolution feature map and the low resolution feature map are fused to capture multi-scale information; The low resolution feature map is all up-sampled and fused with the high resolution feature map to output the predicted key point heat map; According to the predicted key point heat map and the known real key point heat map, the weighted combination of mean square error loss and heat map loss and focal loss is used to constrain the training of the key point detection network.

7. The method of claim 6, wherein: Loss function for training a keypoint detection network is defined as: wherein belongs to the weight hyperparameters, wherein and , is the mean squared error loss, is the heat map loss, is the focal loss, wherein the mean squared error loss and the heat map loss and the focal loss are defined as: wherein, is the total number of key points, is the true position of the th key point, is the predicted position of the th key point; represents the true heat map of the th key point, the predicted heat map of the th key point; M is the total number of pixel points in each heat map, is the probability that the th pixel on the th heat map is predicted to be foreground or the probability that the th pixel on the th heat map is predicted to be background, is a weight factor to adjust the class imbalance, is a modulation factor for focal loss, wherein is the index of the key point, is the index of the pixel on the heat map.

8. The method of claim 7, wherein: The heat map output by the key point detection network is subjected to shape feature information extraction by an encoder, and the shape feature information of the mandibular nerve canal is combined with the texture direction feature as guide information input into the three-dimensional CBCT mandibular nerve canal segmentation network, and the formula for calculating the texture direction feature is: wherein is the texture direction feature vector, is the set of voxel windows, is the number of voxels in the window, is the gradient vector of the voxel, is the transpose of the gradient vector, denotes the position coordinates of the voxel, wherein the gradient vector of the voxel The formula on which the calculation is based is: In the formula, represents the rate of change of the voxel value in the X-axis direction, represents the rate of change of the voxel value in the Y-axis direction, represents the rate of change of the voxel value in the Z-axis direction.

Images