Mandibular nerve canal segmentation method, system and device based on multi-scale feature fusion and strip deformation cascade enhancement and medium

Abstract

The present application belongs to the technical field of oral medical image processing and computer-aided diagnosis, and discloses a mandibular nerve canal segmentation method, system, device and medium based on multi-scale feature fusion and strip deformation cascade enhancement. An oral panoramic image is collected to construct a pixel-level supervised training dataset with CBCT three-dimensional labeling as a gold standard; an InternImage backbone network extracts multi-scale features, and multi-scale context information is integrated through a feature pyramid module; a strip deformation cascade enhancement module strengthens the feature expression of the slender tubular structure; an end-to-end training is performed using a composite loss function; and a panoramic image to be measured is input into the trained model, the mandibular nerve canal region is extracted, and the segmentation result is output. The present application effectively solves the problem of easy fracture and poor topological continuity of the mandibular nerve canal segmentation in the prior art, realizes high-precision automatic extraction in the oral panoramic image, and provides preoperative nerve anatomy risk assessment and surgical planning support for wisdom tooth extraction, implant restoration and other surgeries.

Description

Technical Field

[0001] This invention relates to the field of oral medical image processing and computer-aided diagnosis technology, and in particular to a method, system, device and medium for mandibular nerve canal segmentation based on multi-scale feature fusion and strip deformation cascade enhancement. Background Technology

[0002] The mandibular nerve canal is an intraosseous canal structure located beneath the cortical bone of the mandible. It is winding and narrow, and accurate identification of its spatial location and course is crucial for preoperative assessment in clinical procedures such as implant planning, impacted tooth extraction, and mandibular surgery. If an implant is mistakenly placed in the mandibular nerve canal region, it may lead to complications such as numbness and sensory abnormalities of the lower lip. Therefore, accurately determining the extent of the mandibular canal preoperatively is one of the key aspects of safe oral surgery.

[0003] Currently, the commonly used methods for locating the mandibular canal in clinical practice mainly include manual interpretation and delineation by physicians on panoramic oral radiographs, and three-dimensional reconstruction and layer-by-layer annotation based on cone-beam computed tomography (CBCT). Manual interpretation relies on physician experience, is labor-intensive and inefficient, and is subject to subjective differences among interpreters. While CBCT annotation offers high accuracy, it is costly, involves relatively high radiation doses, and is time-consuming, making it difficult to efficiently promote in large-scale screening or routine panoramic radiograph interpretation scenarios. Therefore, how to achieve automatic, accurate, and stable segmentation of the mandibular canal based on the widely used panoramic oral radiographs has become a pressing technical problem to be solved in intelligent oral diagnosis and treatment.

[0004] With the development of deep learning technology, semantic segmentation methods have been applied to organ and lesion extraction in medical images. However, directly applying existing segmentation methods to mandibular canal recognition still faces many challenges: First, the mandibular canal appears as a slender tubular structure in panoramic images, occupying only a small number of pixels in width, resulting in extreme imbalance between the foreground and background, making it difficult to achieve ideal segmentation results with conventional cross-entropy loss; Second, the mandibular canal has variable directions of travel, with local bends, bifurcations, and confusion with surrounding bone textures, making it difficult for ordinary isotropic convolution receptive fields to effectively model tubular topological features; Third, panoramic images have large pixel sizes, resulting in high GPU memory overhead for full-image training, typically requiring block training and sliding window inference, which places higher demands on multi-scale context fusion capabilities; Fourth, models that rely solely on pixel-level overlap index optimization are prone to topological errors such as tubular structure breaks and discontinuities, affecting clinical usability.

[0005] While existing segmentation networks based on U-Net and FPN possess some multi-scale feature fusion capabilities, they remain insufficient for directional feature extraction of slender tubular targets. Although some methods introduce attention mechanisms or additional loss functions, their ability to balance overall connectivity and boundary accuracy of the mandibular canal in high-resolution panoramic scenes still needs improvement. Therefore, it is necessary to design a mandibular canal segmentation scheme for oral panoramic radiographs that fully integrates multi-scale contextual information, strengthens the representation of tubular structural features, and combines it with a composite loss function suitable for slender targets for training, in order to achieve high-precision, high-connectivity automatic segmentation of the mandibular canal. Summary of the Invention

[0006] The technical problem this invention aims to solve is to overcome the shortcomings of existing mandibular nerve canal segmentation methods based on panoramic oral radiographs, such as insufficient extraction of elongated tubular structure features, difficulty in maintaining topological connectivity, and limited segmentation accuracy due to class imbalance. This invention provides a mandibular nerve canal segmentation method, system, device, and medium based on multi-scale feature fusion and strip deformation cascade enhancement. Based on the InternImage backbone network and the UPerHeadStrip decoding architecture, this invention effectively improves the accuracy and topological integrity of mandibular nerve canal segmentation, achieving accurate extraction of tubular structures.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a method for segmenting the mandibular nerve canal based on multi-scale feature fusion and strip deformation cascade enhancement, comprising the following steps: S1. Acquire panoramic oral cavity images and construct a pixel-level supervised training dataset using CBCT 3D annotation as the gold standard. S2, input the pixel-level supervised training dataset into the mandibular nerve tube semantic segmentation network; the mandibular nerve tube semantic segmentation network adopts an encoder-decoder structure, including an InternImage backbone network, a UPerHeadStrip main decoder head, a fully convolutional network auxiliary decoder head, and a classification output layer; The InternImage backbone network extracts multi-scale features sequentially through four cascaded stages, resulting in a four-layer feature map with decreasing resolution and increasing number of channels. The UPerHeadStrip main decoder includes a feature pyramid module and a strip deformation cascade enhancement module. Deep features are fused by multi-scale pooling and upsampling, and then horizontally connected and layer-by-layer added with shallow features to construct a feature pyramid module that integrates multi-scale contextual information, thus obtaining the fused features. The multi-fused features are input into the strip deformation cascade enhancement module; the strip deformation cascade enhancement module includes a fixed strip convolution submodule and a strip deformation compensation submodule cascaded in series, outputs the deformation-enhanced features, and then maps them through the classification output layer to obtain the main segmentation result; S3 employs a multi-component composite loss function to train the mandibular nerve tube semantic segmentation network end-to-end, and uses a fully convolutional network to assist the decoding head in supervising the intermediate layer features of multi-scale features with weighted cross-entropy. S4. After training, the panoramic oral cavity image to be tested is input into the trained mandibular nerve canal semantic segmentation network. The deformed and enhanced features are obtained through forward propagation in step S2, and then mapped to the category channel through the classification output layer and upsampled to output the final segmentation prediction map.

[0008] The InternImage backbone network is an InternImage network pre-trained and fine-tuned on ImageNet. It uses DCNv3 deformable convolution as the core operator and extracts multi-scale features through four cascaded stages, outputting four feature maps of layers C1, C2, C3, and C4 in sequence.

[0009] The feature pyramid module is as follows: Multi-scale pooling is performed with C4 as input. Each branch is convolutionally reduced in dimensionality and upsampled before being concatenated with the original C4. The resulting pyramid scene parsing and fusion features are then compressed through a bottleneck layer. C1, C2, and C3 are reduced in dimensionality through lateral convolution, and then fused with the pyramid scene parsing features layer by layer from top to bottom; After fusion, each layer undergoes convolution processing, upsampling alignment, channel splicing, and compression to obtain the FPN fused features.

[0010] The strip deformation cascaded enhancement module includes horizontal strip branches and vertical strip branches; The horizontal strip branch uses a 1×K depthwise separable convolution to extract horizontal features from the fused features; Vertical strip branches use K×1 depthwise separable convolutions to extract vertical features from the fused features; The outputs of the horizontal and vertical strip branches are added element by element, and then fused by 1×1 convolution, batch normalization, and activation function to obtain the strip enhancement feature S; Using the strip enhancement feature S as input, predict the offset of K sampling points in the horizontal strip branch and the vertical strip branch respectively; The offset of the sampling points is limited to obtain the offset increment. Taking the center sampling point of the strip convolution kernel as the anchor point, the offset at the anchor point is set to zero. The offset increment is accumulated point by point along the main axis of the strip to obtain the deformation offset of each sampling point. Based on the deformation offset, deformable strip sampling and weighted aggregation are performed by bilinear interpolation. The outputs of the horizontal strip branch and the vertical strip branch are added element by element to the strip enhancement feature S at the corresponding position and in the corresponding channel to form a residual connection. Then, the residual connection is fused by 1×1 convolution to output the deformation-enhanced feature.

[0011] In step S3, the UPerHeadStrip main decoder performs primary supervision by weighted sum of weighted cross-entropy loss, Dice loss, Lovász loss, and centerline Dice loss. At the same time, the FCN auxiliary decoder performs auxiliary supervision on the C3 features of the third output of the InternImage backbone network with weighted cross-entropy loss, and the auxiliary loss weight is 0.5.

[0012] A mandibular nerve canal segmentation system based on multi-scale feature fusion and strip deformation cascade enhancement includes: The image acquisition and preprocessing module is used to acquire panoramic oral radiographs and construct a pixel-level supervised training dataset. The feature extraction module is used to input panoramic oral radiographs into the InternImage backbone network to extract multi-scale features; The multi-scale fusion and strip deformation cascade enhancement module is used to fuse multi-scale features of the backbone network and enhance the tubular structure expression, including pyramid scene parsing fusion, feature pyramid network construction, and serial cascade processing of fixed strip convolution submodule and strip deformation compensation submodule. The model training module is used for end-to-end training using a multi-component composite loss function, supplemented by auxiliary supervision. The segmentation result output module is used to map the classification and upsample the output segmentation prediction map.

[0013] An electronic device for segmenting the mandibular nerve canal includes a processor, a memory, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described.

[0014] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described.

[0015] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art: (1) This method only requires panoramic images and corresponding mandibular nerve canal annotations during training, without the need for complex CBCT three-dimensional reconstruction or additional multimodal registration preprocessing, which greatly reduces the workload of data acquisition and preprocessing.

[0016] (2) To address the low contrast between the mandibular nerve canal and surrounding bone tissue, the slender tubular structure and its variable orientation in panoramic images, a strip deformation cascade enhancement module (SDCE) is designed on the FPN fusion feature of the decoder: The first stage uses horizontal 1×K and vertical K×1 depth separable convolutions to capture the responses of the slender tubular structure in different directions; the second stage introduces deformable sampling with cumulative offset constraint along the main axis of the strip on the strip enhancement feature, so that the receptive field can be adaptively adjusted with the bending of the tube wall, in order to solve the problem that the fixed receptive field is difficult to cover the curved tube segment and is prone to cause the prediction mask to break.

[0017] (3) The main supervision is performed by a composite loss consisting of weighted cross-entropy, Dice and Lovász loss, which alleviates class imbalance and improves the optimization effect of region overlap and cross-union ratio; and the centerline Dice loss is introduced to maintain the topological connectivity of the mandibular canal in a soft skeletonization manner, and suppresses fractures and voids.

[0018] (4) The proposed model is an end-to-end semantic segmentation model for the mandibular nerve canal. During the training phase, image patch cropping is used. During the inference phase, the segmentation results can be directly output from full-size panoramic images using a sliding window method without additional complex preprocessing. This enhances the model's adaptability to clinical images of different resolutions, improves the accuracy and stability of segmentation inference, and enhances the practicality and generalization ability of clinical deployment. Attached Figure Description

[0019] The accompanying drawings, as part of this invention, are provided to further illustrate the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention, but do not constitute an undue limitation thereof. Clearly, the drawings described below are merely some embodiments, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0020] Figure 1 This is a schematic diagram of a mandibular nerve canal segmentation method based on multi-scale feature fusion and strip deformation cascade enhancement according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall architecture of the method; Figure 3 This is a schematic diagram of the internal module structure of the UPerHeadStrip decoder according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the strip deformation cascaded enhancement module structure according to an embodiment of the present invention; Figure 5 This is a visualization diagram of the mandibular nerve canal segmentation result of a panoramic oral radiograph according to an embodiment of the present invention; (a)-(c) are different original images; (d)-(f) are the corresponding real label images of the original images; (g)-(i) are the images of the present method corresponding to the original images.

[0021] Figure 6 This is a schematic diagram of the segmentation system structure. Detailed Implementation

[0022] The specific embodiments of the present invention will now be described in further detail with reference to the accompanying drawings.

[0023] Figure 1 This invention illustrates a mandibular nerve canal segmentation method based on multi-scale feature fusion and strip deformation cascade enhancement, applicable to the automatic segmentation of the mandibular nerve canal in two-dimensional medical images such as panoramic oral radiographs. The method can be summarized into four steps, each step being sequentially connected and executed.

[0024] S1. Acquire panoramic oral radiographs and construct a CBCT gold standard dataset. Acquire panoramic oral radiographs of the subjects to be processed; these radiographs are RGB or grayscale medical images. For model training and evaluation, construct CBCT annotation data that corresponds to the panoramic radiographs in registration or spatial mapping: extract or project mandibular nerve canal annotations aligned with the panoramic radiographs from the CBCT data along a predetermined direction, and form a gold standard mask after expert review. Divide the panoramic radiographs and their gold standard masks into training, validation, and test sets at the case level; each case belongs to only one subset to avoid data leakage.

[0025] Furthermore, considering the large pixel size of panoramic images and the high memory overhead of direct full-image training, this embodiment performs image block cropping on the training and validation sets: First, the effective region containing the mandibular canal is determined based on the gold standard mask. Simultaneous cropping is then performed within the effective region and its neighborhood using a 512×512 fixed-size sliding window. Image blocks and mask blocks are generated one-to-one according to the same window position. Nearest neighbor interpolation is used for mask cropping to ensure pixel-level label consistency. Pure background blocks without the mandibular canal foreground are removed from the cropped blocks in the training set, while the cropped blocks in the validation set are retained. No cropping is performed on the test set cases, maintaining the original full-image size for subsequent full-image inference evaluation. During the training phase, the cropped blocks are also normalized and subjected to random flipping and photometric distortion data augmentation. After the above processing, 1695 512×512 cropped blocks were obtained from 113 cases in the training set, 480 cropped blocks were obtained from 32 cases in the validation set, and a 2943×1435 full-image was obtained from 17 cases in the test set. Table 1 provides information related to the dataset.

[0026] Table 1. Dataset Description

[0027] S2, the mandibular nerve canal semantic segmentation network is used to complete feature extraction, multi-scale fusion, and strip deformation cascade enhancement. The panoramic oral cavity image obtained in step S1 is input into the mandibular nerve canal semantic segmentation network. The mandibular nerve canal semantic segmentation network adopts an encoder-decoder structure, including an InternImage backbone network, a UPerHeadStrip main decoder head, a fully convolutional network auxiliary decoder head, and a classification output layer. Figure 2 The overall architecture flowchart of the method is shown. S2 includes the following steps: S21, Backbone network extracts multi-scale features. InternImage, based on deformable convolution DCNv3, is used as the backbone network. The network contains four feature extraction stages, with depth configurations of [4, 4, 18, 4] for each stage and a base number of 64 channels. After loading general visual pre-trained weights, it is fine-tuned on panoramic dental image data. Assuming the input image size is H×W, the InternImage backbone network outputs four-scale feature maps C1, C2, C3, and C4, with their respective number of channels and spatial dimensions: C1: 64×H / 4×W / 4; C2: 128×H / 8×W / 8; C3: 256×H / 16×W / 16; C4: 512×H / 32×W / 32.

[0028] During the training phase, a fully convolutional network auxiliary decoding head FCNHead is introduced from C3 for auxiliary supervision in step S3; during the inference phase, only the main decoding head is used to output the segmentation results.

[0029] S22, the feature pyramid module performs multi-scale fusion. C1 to C4 obtained in step S2 are input into the main decoding head UPerHeadStrip, and multi-scale fusion is completed through the feature pyramid module. Wherein, Figure 3 The diagram shows the internal module structure of the UPerHeadStrip main decoder. S22 includes the following steps: In the feature pyramid module, multi-scale pooling fusion is performed only on the deepest feature C4. C4 is processed in parallel by the identity path and four adaptive average pooling branches, with pooling output spatial sizes of 1×1, 2×2, 3×3, and 6×6, respectively. Each path is compressed to 256 channels by 1×1 convolution and upsampled to the original size of C4, and then concatenated with the identity path in the channel dimension to obtain a 1536-channel feature. Then, a 3×3 convolution bottleneck layer is used to output a 256-channel feature P, denoted as L4. C1, C2, and C3 are mapped to 256-channel features L1, L2, and L3 respectively through 1×1 horizontal convolution; P is used as L4, and they are fused in a top-down manner. Upsample is a bilinear upsampling algorithm.

[0030] The fused L1, L2, and L3 are convolved with 3×3 FPN to obtain F1′, F2′, and F3′ respectively. F2′, F3′, and L4 are upsampled to the same size as F1′ and then concatenated with F1′ in the channel dimension to obtain 1024-channel features. Among them, L4 does not undergo 3×3 FPN convolution but participates in the concatenation directly in a skip-connection manner. The 1024-channel features are mapped to 256 channels through a 3×3 convolutional bottleneck layer, which serves as the input for the subsequent strip deformation cascade enhancement module.

[0031] S23, the strip deformation cascade enhancement and classification output layer yield the main segmentation result. The 256-channel features output from step S22 are input into the strip deformation cascade enhancement module (SDCE). The SDCE module is positioned after the feature pyramid module fusion bottleneck and before the classification output. It includes a two-level structure: a fixed strip convolution submodule and a strip deformation compensation submodule. The two levels are cascaded in a preset order: the first-level submodule extracts the axis-aligned response of the slender tubular structure in the horizontal and vertical directions, and the second-level submodule deformably adjusts the sampling position along the main axis of the strip based on the output features of the first-level submodule to enhance the topological continuity of the curved tubular structure features. Figure 4 A schematic diagram of the strip deformation cascaded enhancement module structure is shown. S23 includes the following steps: Horizontal strip feature extraction: Let the size of the fused feature F be 256×H′×W′. In the horizontal strip branch of the fixed strip convolution submodule, a depthwise separable convolution operation with a kernel size of 1×K is applied to F to aggregate neighborhood information along the horizontal direction to obtain the horizontal strip response feature Fh; where K is the length of the strip convolution kernel in the direction of the strip principal axis, and K takes the value of an odd number greater than 1. In this embodiment, K=15 is preferred. Vertical strip feature extraction: In the vertical strip branch of the fixed strip convolution submodule, a depthwise separable convolution operation with a kernel size of K×1 is applied to F, and neighborhood information is aggregated along the vertical direction to obtain the vertical strip response feature Fv; Fixed strip feature fusion: Element-wise addition of corresponding positions of Fh and Fv is performed to obtain intermediate fused features; 1×1 convolution operation, batch normalization, and nonlinear activation are sequentially applied to the intermediate fused features to output strip enhancement features S; S and F have the same number of channels and spatial dimensions. Strip deformation compensation: Using S as the input feature of the strip deformation compensation submodule, horizontal and vertical deformation branches are constructed respectively. Deformable strip sampling and feature reconstruction are performed on S, specifically including: Offset prediction: In the horizontal deformation branch and the vertical deformation branch, a 3×3 convolutional layer is used to perform convolution operation on S. Each branch outputs the original offset corresponding to K sampling points, forming an offset map with size K×H′×W′. Offset limiting and cumulative constraint: The offset increment of each sampling point is limited to obtain the offset increment after limiting; taking the center sampling point of the strip convolution kernel as the anchor point, the offset at the anchor point is set to zero, and the offset increment after limiting is accumulated point by point along the main axis of the strip to obtain the deformation offset of each sampling point, so as to make the offset change between adjacent sampling points continuous. Deformable strip sampling: In the horizontal strip branch, fixed coordinates of each sampling point are determined at preset intervals along the horizontal strip main axis, and the deformation offset obtained by superimposing in the direction perpendicular to the strip main axis constitutes the deformable sampling coordinates of each sampling point; in the vertical strip branch, fixed coordinates of each sampling point are determined at preset intervals along the vertical strip main axis, and the deformation offset obtained by superimposing in the direction perpendicular to the strip main axis constitutes the deformable sampling coordinates of each sampling point; based on the deformable sampling coordinates, bilinear interpolation sampling is performed on S to obtain the feature values ​​corresponding to each sampling point; the feature values ​​of each sampling point are multiplied by the corresponding learnable strip weights and then summed in the sampling point dimension to obtain the horizontal deformation branch output h and the vertical deformation branch output v, respectively; Residual fusion output: h, v, and S are added element-wise at corresponding positions and channels to form residual connections. The addition results are then subjected to 1×1 convolution, batch normalization, and nonlinear activation to obtain the deformed enhanced features SS. The deformed enhanced features SS are then mapped to the category channels through the classification output layer and upsampled to obtain the main segmentation result.

[0032] S3, employing a composite loss algorithm supplemented by auxiliary supervision. Using the training set constructed in step S1, the mandibular neural tube semantic segmentation network described in step S2 is trained end-to-end. During training, 512×512 image blocks and their gold-standard masks are input into the network, and the pixel-wise classification scores (logits) output by the main decoder are obtained through forward propagation. The normalized exponential function (Softmax) is used to convert the probability into class probability. Using gold standard mask To monitor the signal, a composite loss function consisting of cross-entropy loss, Dice loss, Lovász loss, and centerline Dice loss is used to optimize the network parameters.

[0033] Among them, the weighted cross-entropy loss is used to alleviate the class imbalance caused by the extremely small foreground proportion of the mandibular canal, and its expression is: In the formula, and These represent the height and width of the feature map, respectively. For indicator functions, For the first The class weight, in this embodiment, is taken as... ; Dice loss is used to improve the overlap of foreground regions, and its basic form is as follows: In the formula, Predict probabilities for the foreground class. Foreground truth value, ; Lovász loss is used to directly optimize the intersection-union-ratio (IU) related objective, improving the segmentation effect of slender structure boundaries. Let the foreground class prediction error be... The pixel errors are sorted in descending order and denoted as follows: , ,but: In the formula, The discrete increment of the intersection-union ratio after sorting by error; The centerline Dice loss was calculated using a soft skeletonization method with 15 iterations to maintain the topological connectivity of the mandibular canal. Let... and Let's take the soft skeleton diagrams obtained after soft skeletonization of the predicted foreground and the gold standard foreground, respectively. Then: In the formula, ; Indicates topological accuracy. Indicates topology sensitivity.

[0034] The total loss function of the main decoder is: In this embodiment, , , , .

[0035] Furthermore, during the training phase, the backbone network... The features lead to an auxiliary head for the Fully Convolutional Network (FCN). This auxiliary head is supervised only by a cross-entropy loss with weights of the same class as the main head. Its expression is similar to... The loss weight is the same, with a weight of 0.5, and it is not included in the calculation of Dice, Lovász, and the centerline Dice. The total network loss is: in This is used to calculate the head cross-entropy loss.

[0036] Training uses the AdamW optimizer with an initial learning rate of The weights are decayed by 0.005, and a cosine annealing learning rate strategy is adopted. The warm-up iterations are 1500, with a maximum of 60000 iterations. The batch size per GPU is 8, and the gradient pruning threshold is 1.0. Every 300 iterations, the mean intersection over union (mIoU) is monitored on the validation set, and the model weights with the best validation performance are saved. After the above training, a mandibular neural tube segmentation model that can be used for inference in step S4 is obtained.

[0037] S4: Input the test sample into the trained mandibular nerve canal semantic segmentation model and output the mandibular canal segmentation prediction map. Use the mandibular nerve canal semantic segmentation model trained in step S3 for segmentation inference of the test panoramic oral radiograph. After performing normalization preprocessing on the test panoramic radiograph consistent with the training stage, input it into the network. After forward propagation through steps S2 to S4, the main decoder outputs two-class pixel-by-pixel classification scores, which are then converted into probabilities of each class by a normalized exponential function (Softmax). The foreground class probability is taken to obtain the foreground probability map, and then a binary segmentation mask is generated by threshold discrimination.

[0038] For a 512×512 block-level input, the class with the highest probability for each pixel can be directly used as the prediction result to obtain a block-level binary mask. For a 2943×1435 full-size panoramic oral radiograph, sliding inference is performed with a 512×512 sliding window and a stride=96. The prediction results of the overlapping areas are weighted and fused. Further enhancement is performed during the horizontal flip test. The prediction results of the original image and the horizontally flipped image are fused and then inversely transformed and aligned to finally obtain a binary segmentation mask of the mandibular canal with the same size as the input panoramic radiograph and registered pixel by pixel, which can be used for clinical implant planning or preoperative surgical evaluation.

[0039] Specifically, Figure 5This diagram illustrates the visualization of the mandibular nerve canal segmentation results from a panoramic oral radiograph according to an embodiment of the present invention. Each group sequentially displays the original panoramic oral radiograph, the manually annotated gold standard mask, and the predicted mask of the present invention. It can be seen that the predicted curve is basically consistent with the gold standard in the mandibular body, and can continuously depict the bilateral mandibular nerve canal paths, indicating that the method has good segmentation accuracy and clinical application potential.

[0040] This invention also provides a mandibular nerve canal segmentation system based on multi-scale feature fusion and strip deformation cascade enhancement, such as... Figure 6 As shown, it includes: The image acquisition and preprocessing module is used to acquire panoramic oral radiographs and construct a pixel-level supervised training dataset.

[0041] The feature extraction module is used to input images into the InternImage backbone network to extract multi-scale features.

[0042] A multi-scale fusion and strip deformation cascade enhancement module is used to perform multi-scale pooling fusion on deep features and laterally connect them with shallow features to construct a feature pyramid. It is also used to enhance the feature responses of slender and curved tubular structures through the serial cascade of a fixed strip convolution submodule and a strip deformation compensation submodule.

[0043] The model training module is used for end-to-end training using a multi-component composite loss function, supplemented by auxiliary supervision.

[0044] The segmentation result output module is used to map the classification and upsample the output segmentation prediction map.

[0045] The present invention also provides a mandibular nerve canal segmentation electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the method embodiment described above in Embodiment 1 of the present invention.

[0046] Furthermore, as an executable solution, the mandibular nerve canal segmentation electronic device can be a dental imaging workstation, a dental specialty PACS terminal, a medical graphics processing server, a desktop computer, a laptop, or a cloud server, etc. The mandibular nerve canal segmentation electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that the above-described composition of the mandibular nerve canal segmentation electronic device is merely an example and does not constitute a limitation on the electronic device of the present invention. It may include more or fewer components than described above, or combine certain components, or different components. For example, the electronic device may also include a medical imaging display device, input / output devices, network access devices, buses, etc., and the embodiments of the present invention do not limit this.

[0047] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the embodiments of the present invention.

[0048] Furthermore, if the modules or units integrated in the mandibular nerve canal segmentation electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.

[0049] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-described technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for segmenting the mandibular nerve canal based on multi-scale feature fusion and strip deformation cascade enhancement, characterized in that, Includes the following steps: S1. Acquire panoramic images of the oral cavity and construct a pixel-level supervised training dataset using CBCT 3D annotation as the gold standard. S2, input the pixel-level supervised training dataset into the mandibular nerve tube semantic segmentation network; the mandibular nerve tube semantic segmentation network adopts an encoder-decoder structure, including an InternImage backbone network, a UPerHeadStrip main decoder head, a fully convolutional network auxiliary decoder head, and a classification output layer; The InternImage backbone network extracts multi-scale features sequentially through four cascaded stages, resulting in a four-layer feature map with decreasing resolution and increasing number of channels. The UPerHeadStrip main decoder includes a feature pyramid module and a strip deformation cascade enhancement module. Deep features are fused by multi-scale pooling and upsampling, and then horizontally connected and layer-by-layer added with shallow features to construct a feature pyramid module that integrates multi-scale contextual information, thus obtaining the fused features. The multi-fused features are input into the strip deformation cascade enhancement module; the strip deformation cascade enhancement module includes a fixed strip convolution submodule and a strip deformation compensation submodule cascaded in series, outputs the deformation-enhanced features, and then maps them through the classification output layer to obtain the main segmentation result; S3 employs a multi-component composite loss function to train the mandibular nerve tube semantic segmentation network end-to-end, and uses a fully convolutional network to assist the decoding head in supervising the intermediate layer features of multi-scale features with weighted cross-entropy. S4. After training, the panoramic oral cavity image to be tested is input into the trained mandibular nerve canal semantic segmentation network. After forward propagation through steps S2 to S4, the deformed and enhanced features are obtained. Then, the features are mapped to the category channel through the classification output layer and upsampled to output the final segmentation prediction map.

2. The mandibular nerve canal segmentation method based on multi-scale feature fusion and strip deformation cascade enhancement according to claim 1, characterized in that, The InternImage backbone network is an InternImage network pre-trained and fine-tuned on ImageNet. It uses DCNv3 deformable convolution as the core operator and extracts multi-scale features through four cascaded stages, outputting four feature maps of layers C1, C2, C3, and C4 in sequence.

3. The mandibular nerve canal segmentation method based on multi-scale feature fusion and strip deformation cascade enhancement according to claim 2, characterized in that, The feature pyramid module is as follows: Multi-scale pooling is performed with C4 as input. Each branch is convolutionally reduced in dimensionality and upsampled before being concatenated with the original C4. The resulting pyramid scene parsing and fusion features are then compressed through a bottleneck layer. C1, C2, and C3 are reduced in dimensionality through lateral convolution, and then fused with the pyramid scene parsing features layer by layer from top to bottom; After fusion, each layer undergoes convolution processing, upsampling alignment, channel splicing, and compression to obtain the FPN fused features.

4. The mandibular nerve canal segmentation method based on multi-scale feature fusion and strip deformation cascade enhancement according to claim 1, characterized in that, The strip deformation cascaded enhancement module includes horizontal strip branches and vertical strip branches; The horizontal strip branch uses a 1×K depthwise separable convolution to extract horizontal features from the fused features; Vertical strip branches use K×1 depthwise separable convolutions to extract vertical features from the fused features; The outputs of the horizontal and vertical strip branches are added element by element, and then fused by 1×1 convolution, batch normalization, and activation function to obtain the strip enhancement feature S; Using the strip enhancement feature S as input, predict the offset of K sampling points in the horizontal strip branch and the vertical strip branch respectively; The offset of the sampling point is limited to obtain the offset increment. The center sampling point of the strip convolution kernel is used as the anchor point. After setting the offset at the anchor point to zero, the offset increment is accumulated point by point along the main axis of the strip to obtain the deformation offset of each sampling point. Based on the deformation offset, deformable strip sampling and weighted aggregation are performed by bilinear interpolation. The outputs of the horizontal and vertical strip branches are added element-wise to the strip enhancement feature S at corresponding positions and channels to form residual connections. Then, the residuals are fused by 1×1 convolution to output the deformed and enhanced features.

5. The mandibular nerve canal segmentation method based on multi-scale feature fusion and strip deformation cascade enhancement according to claim 1, characterized in that, In step S3, the UPerHeadStrip main decoder performs primary supervision by weighted sum of weighted cross-entropy loss, Dice loss, Lovász loss, and centerline Dice loss. At the same time, the FCN auxiliary decoder performs auxiliary supervision on the C3 features of the third output of the InternImage backbone network with weighted cross-entropy loss, and the auxiliary loss weight is 0.

5.

6. A mandibular nerve canal segmentation system based on multi-scale feature fusion and strip deformation cascade enhancement, characterized in that, include: The image acquisition and preprocessing module is used to acquire panoramic oral radiographs and construct a pixel-level supervised training dataset. The feature extraction module is used to input panoramic oral radiographs into the InternImage backbone network to extract multi-scale features; The multi-scale fusion and strip deformation cascade enhancement module is used to fuse multi-scale features of the backbone network and enhance the tubular structure expression, including pyramid scene parsing fusion, feature pyramid network construction, and serial cascade processing of fixed strip convolution submodule and strip deformation compensation submodule. The model training module is used for end-to-end training using a multi-component composite loss function, supplemented by auxiliary supervision. The segmentation result output module is used to map the classification and upsample the output segmentation prediction map.

7. An electronic device for segmenting the mandibular nerve canal, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images