Three-stage CBCT maxillofacial panoramic segmentation method based on Kolmogorov-Arnold representation theorem and U-Net architecture

Abstract

The invention discloses a three-stage CBCT (cone beam computed tomography) maxillofacial panoramic segmentation method based on a Kolmogorov-Arnold representation theorem and a U-Net architecture, and the method can learn a nonlinear activation function and multi-scale feature extraction through fusing position prior, and effectively enhances the feature expression capability. According to the method, a three-stage maxillofacial structure panoramic segmentation framework is designed, wherein in the first stage, coarse segmentation and tooth position calibration are carried out in a low-resolution image by utilizing MC-UKAN; in the second stage, on the original image, a target structure is positioned based on a coarse segmentation result, a K-means clustering algorithm is utilized to divide targets into five classes, and a corresponding network is adopted to perform fine segmentation; and in the third stage, the jawbone and the throat are segmented on the original image by using a lightweight network. And finally, through a probability graph fusion mechanism, generating a complete maxillofacial panoramic segmentation result containing tooth position calibration.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of jaw and facial image segmentation in medical image processing, and particularly relates to a three-stage CBCT jaw and facial panoramic segmentation method based on Kolmogorov-Arnold representation theorem and U-Net architecture. BACKGROUND

[0002] Cone beam CT (CBCT) is a three-dimensional imaging method using a cone-shaped X-ray beam in medical imaging technology. Traditional CBCT image segmentation methods represented by threshold segmentation, level set, region growing and watershed algorithm have the advantages of low computational complexity and strong real-time performance, but when dealing with complex topological structures, they often have over-segmentation or under-segmentation problems. The core defect of this kind of method is that it relies too much on artificial experience parameters, and it is not robust enough to deal with the diverse and complex image features of the jaw and facial region.

[0003] In recent years, deep learning technology has provided a new solution for jaw and facial structure segmentation. Convolutional neural networks (CNN) have made breakthrough progress in oral anatomy structure segmentation due to their strong feature extraction capability. For example, improved models based on U-Net architecture achieve high-precision segmentation of single structures such as teeth and jaw bones through encoder-decoder structure and skip connection; some studies combine attention mechanisms (such as squeeze-and-excitation modules) to enhance the feature representation of key structures, which to some extent improves the segmentation effect of fuzzy boundaries. However, there are still two major shortcomings in existing technology: first, most studies focus on a single or a few types of anatomical structures, lack of panoramic coverage of the jaw and face, and are difficult to support multi-step joint diagnosis and treatment scheme development; second, traditional CNNs are limited by local receptive fields and are difficult to effectively model long-range dependencies between jaw and facial structures, such as the spatial relationship between the molar region and the end of the inferior alveolar nerve canal, the symmetrical distribution relationship between the left and right maxillary sinuses and the midline, etc. Semantic information cannot be fully extracted, which not only leads to confusion between adjacent teeth (such as mislabeling the second molar as the first molar), but also affects the accuracy of subsequent quantitative analysis (such as tooth inclination angle calculation and nerve canal diameter measurement). SUMMARY

[0004] The purpose of the application is to provide a three-stage CBCT jaw and facial panoramic segmentation method based on Kolmogorov-Arnold representation theorem and U-Net architecture.

[0005] Technical solution: The application provides a three-stage CBCT jaw and facial panoramic segmentation method based on Kolmogorov-Arnold representation theorem and U-Net architecture, comprising the following steps:

[0006] S1. Obtain the dataset of oral CBCT images and perform preprocessing on the images in the dataset, including downsampling;

[0007] S2. Construct a multi-scale channel-separable KAN convolutional module MC-KAN;

[0008] S3. Construct a multi-scale feature fusion network MC-UKAN for the coarse segmentation stage based on the multi-scale channel separable KAN convolutional module MC-KAN;

[0009] S4. In the coarse segmentation stage, the downsampled image is input into the multi-scale feature fusion network MC-UKAN to obtain coarse segmentation results containing various structures, including maxilla / mandible, left / right neural canal, left / right maxillary sinus, teeth marked according to FDI tooth position representation, pharynx, and restorations.

[0010] S5. In the local high-resolution fine segmentation stage, based on the coarse segmentation results, the positions of various structures on the original image are located. The structures in the coarse segmentation results, excluding the maxilla / mandible and pharynx, are divided into five categories using K-means clustering analysis: incisors, molars, inferior alveolar nerve canal, maxillary sinus, and restorations. After locating the positions of these five categories of structures on the original image, a fixed block size is used, and the corresponding 3D Unet_SE model is used for segmentation. For structures whose actual size exceeds the set block size, intelligent sliders are used for inference, and stitching is performed based on the highest probability. The segmentation and stitching process yields the S5 segmentation result.

[0011] S6. In the large-scale structure segmentation stage, for the maxilla / mandible and pharyngeal structures, the 3DUnet_SE model is trained on the original image to segment the three types of structures and obtain the S6 segmentation result.

[0012] S7. The segmentation results of S5 and S6 are fused using a probabilistic graph fusion mechanism.

[0013] Furthermore, the multi-scale channel separable KAN convolutional module MC-KAN in S2 includes the following construction steps:

[0014] Channel dimensionality reduction is performed on the input feature vector using 1×1×1 point convolution;

[0015] KAN convolution kernels of [3,3,3] and [5,5,5] are applied to each channel respectively. The feature map output by the [3,3,3] convolution is reshaped into a two-dimensional feature vector of size [(D×H×W) / 27,27]; the feature map output by the [5,5,5] convolution is reshaped into a two-dimensional feature vector of dimension (C / 2)×[(D×H×W) / 125]×125.

[0016] The two types of two-dimensional feature vectors mentioned above are input into a two-layer KAN network for nonlinear spatial feature extraction.

[0017] The extracted features are processed by residual connection, and then channel fusion is achieved through point convolution and batch normalization.

[0018] Furthermore, the multi-scale feature fusion network MC-UKAN in S3 includes the following construction steps:

[0019] The input downsampled image is processed by two layers of standard convolution (Conv3d). The convolutional kernel size of the convolutional layer is 3×3×3, the stride is 1, and the padding is 1, and feature maps of 64 channels are extracted.

[0020] The feature map is explicitly injected with prior location information through a 3D location encoding module;

[0021] The feature map with injected location prior information is sequentially fed into 5 layers of MC-KAN convolutional layers as the model encoder. Each MC-KAN convolutional layer is followed by 2×2×2 max pooling, and a squeezing and activation module is introduced to enhance the feature representation using a global attention mechanism.

[0022] Five convolutional decoders are constructed corresponding to the model encoder. Each convolutional decoder includes two standard convolutional layers (Conv3d), one normalization layer, and a squeezing and excitation module. The convolutional kernel size of the convolutional layer is 3×3×3, the stride is 1, and the padding is 1. Each convolutional decoder first amplifies the input of the previous layer by a ratio of 2×2×2 through trilinear interpolation, and then fuses the features output by the corresponding encoder layer through skip connections.

[0023] Furthermore, in S4, the last layer decoder described in S3 maps the feature channels to the number of channels corresponding to the various structures described in S4 through pointwise convolution, and outputs the probability distribution map of various structures through the Sigmoid activation function. The maximum probability label is extracted through the argmax function, and the downsampled image is input into the multi-scale feature fusion network MC-UKAN to generate a coarse segmentation result.

[0024] Furthermore, in S3, the 3D position encoding module includes the following construction steps:

[0025] The physical coordinates of the 64-channel feature map obtained after initial feature extraction are normalized to the [0,1] interval to generate a three-dimensional mesh coordinate.

[0026] A multi-scale frequency band is generated using a log-interval frequency distribution, wherein the number of multi-scale frequency bands is determined according to the spatial scale range of the maxillofacial structure.

[0027] For each spatial dimension (x, y, z) of the three-dimensional grid coordinates, the sine and cosine components of the three-dimensional grid coordinates in each frequency band are calculated respectively, and all components are spliced ​​together according to the dimensions to form a 6D position code with a dimension of 6×N, where N is the number of frequency bands;

[0028] The 6D positional encoding is compressed to 64 channels using 1×1×1 convolution, and then fused with the original feature map element by element.

[0029] Furthermore, the method for constructing the 3D Unet_SE corresponding to the segmentation network of the five structures in S5 is as follows:

[0030] A corresponding 3D Unet_SE segmentation network is configured for each of the five structural categories. The segmentation network includes an encoder module and a decoder module connected in sequence.

[0031] The encoder module contains 5 encoding units. Each encoding unit includes 2 standard convolutional layers (Conv3d), one normalization layer, and a squeezing and excitation module. Downsampling is achieved through 2×2×2 max pooling. The convolutional layers use 3×3×3 convolutional kernels, and the stride and padding are set to keep the feature map size unchanged.

[0032] The decoder module contains 5 decoding units. Each decoding unit includes 2 standard convolutional layers (Conv3d), one normalization layer, and a squeezing and excitation module. It also uses 2×2×2 upsampling to enlarge the feature map. The convolutional layers use 3×3×3 convolutional kernels, and the stride and padding are set to keep the feature map size unchanged.

[0033] Each decoding unit of the decoder module fuses the features output by the corresponding encoding unit of the encoder module at the same layer through skip connections, and the number of feature channels of the encoding unit and the decoding unit at the same layer are the same.

[0034] Furthermore, in step S5, a smart slider is used for processing, including:

[0035] When the actual size of the structure exceeds the set block size, a sliding window is generated based on the set block size, and the step size of the sliding window is dynamically adjusted according to the edge features of the structure.

[0036] Within the structure's internal region, the sliding step size is set to 1 / 2 of the block size;

[0037] In the structural edge region, the edge contour is located using the coarse segmentation results, and the sliding step size is shortened to 1 / 4 of the block size;

[0038] During the sliding process, the overlapping areas of adjacent windows are fused using a probability-weighted method, with the weight being positively correlated with the distance of the area from the center of the window.

[0039] After all window inferences are completed, redundant predictions in overlapping regions are eliminated using a non-maximum suppression algorithm, and finally, the segments are stitched together to form a complete segmentation result.

[0040] Furthermore, the method for constructing the segmentation network of the maxilla / mandible and pharynx in S6 is as follows:

[0041] The same type of 3D Unet_SE network structure as the segmentation network described in S5 is adopted, and the 3D Unet_SE network structure described in S5 also includes an encoder module and a decoder module connected in sequence.

[0042] The output module maps the feature channels into three channels through pointwise convolution, corresponding to the segmentation results of the maxilla / mandible, pharynx, and background, respectively. The probability distribution map of the three categories is output through the Sigmoid activation function.

[0043] Furthermore, the probabilistic graph fusion mechanism in S7 is as follows:

[0044] Obtain the coarse segmentation label from S4, the five-category structural probability maps output from S6, and the maxilla / mandible and pharynx probability maps output from S7, and uniformly map them to the coordinate system of the original resolution CBCT image of the oral cavity CBCT image.

[0045] Probability thresholds are set for teeth, dental pulp, restorations, neural tube, maxillary sinus, maxilla / mandible, and pharynx respectively;

[0046] For a probability map of a single structure, the initial region of the corresponding structure is first located by combining the S4 coarse segmentation label. Then, voxels with probability values ​​higher than the corresponding threshold within the initial region are retained as high-confidence regions. For voxels with probability values ​​lower than the threshold within the initial region, they are corrected through a neighborhood voting mechanism.

[0047] For overlapping regions of different structural probability maps, the structural category of the overlapping region is first determined based on the S4 coarse segmentation label, and then the structural priority weighted fusion is adopted: the overlapping region of the neural tube and mandible is mainly based on the neural tube probability map, the overlapping region of the dental pulp and tooth is mainly based on the dental pulp probability map, the overlapping region of the maxillary sinus and maxilla is mainly based on the maxillary sinus probability map, and the remaining overlapping regions are weighted averaged according to the probability value. The weight is calculated as the probability value of the structure / the sum of the probability values ​​of all overlapping structures.

[0048] After fusion, isolated regions with areas smaller than a preset threshold are removed by connected component analysis. The preset threshold is determined according to the structure type. Finally, the FDI tooth position calibration information of the S4 coarse segmentation label is combined with the "tooth-pulp" anatomical correspondence to verify and correct the label consistency of the final segmentation result, generating a complete maxillofacial panoramic segmentation result.

[0049] Beneficial effects: 1. The multi-scale channel separable KAN convolutional module (MC-KAN) proposed in this invention combines the strong nonlinear feature extraction capability of the Kolmogorov-Arnold representation theorem with multi-scale convolutional kernel design. It can accurately capture the subtle gray-level differences of maxillofacial structures (such as the inferior alveolar nerve canal and restorations) and effectively extract global contour features. At the same time, it balances feature expression capability and computational efficiency through channel separation and residual connection, providing better feature input for subsequent segmentation networks and improving the basic accuracy of full maxillofacial structure segmentation.

[0050] 2. The three-stage framework of "coarse segmentation - local high-resolution fine segmentation - large-scale structural segmentation" proposed in this invention adopts a differentiated processing strategy for differences in the size and complexity of maxillofacial structures (such as small-sized teeth and large-sized jawbones): the coarse segmentation stage quickly locates the entire structure, the fine segmentation stage uses intelligent slider splicing to ensure the accuracy of small structural details, and the large-scale segmentation stage uses original size training to ensure complete coverage of large structures. This solves the contradiction of "large computational load for large structures and loss of details for small structures" in existing technologies, and achieves dual optimization of segmentation accuracy and efficiency.

[0051] 3. The 3D position coding module (PE3D) introduced in the Multi-Scale Feature Fusion Network (MC-UKAN) of this invention generates position codes through logarithmically spaced frequency bands, explicitly injecting long-distance position dependence and absolute position information of the maxillofacial structure, making up for the limitations of the local receptive field of traditional CNN, and improving the accuracy of tooth position calibration (such as avoiding confusion between adjacent teeth) and the spatial correlation of structure (such as matching the anatomical position of the maxillary sinus and the maxilla).

[0052] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method. Attached Figure Description

[0053] Figure 1 This is a three-stage flowchart of the present invention;

[0054] Figure 2 This is a diagram of the architecture of MC-UKAN proposed in this invention;

[0055] Figure 3 This is a diagram of the architecture of MC-KAN proposed in this invention;

[0056] Figure 4 This is a diagram showing the segmentation result of the present invention. Detailed Implementation

[0057] To more clearly reveal the purpose, technical solutions, and beneficial effects of the various embodiments of the present invention, the technical solutions of the present invention are explained in detail and completely below. It should be emphasized that the embodiments described below are only some embodiments of the present invention, not all embodiments. It is hereby stated that the content described herein is only used to further clarify the technical features and advantages of the present invention and does not constitute a limitation on the scope of protection of the claims of the present invention; other implementations that can be obtained by those skilled in the art based on the embodiments provided by the present invention without creative effort all fall within the protection scope of the present invention.

[0058] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0059] Figures 1 to 4 As shown, this invention proposes a three-stage CBCT maxillofacial panoramic segmentation method based on the Kolmogorov-Arnold representation theorem and the U-Net architecture. Figure 1 The three-stage maxillofacial segmentation framework proposed in this invention includes the following steps:

[0060] Step 1: Dataset Preparation

[0061] 532 CBCT datasets were collected from publicly available databases. In the preprocessing stage, the images were first uniformly downsampled to 160×160×108, and the CT values ​​were truncated to the range [-1000, 3500] and normalized to [0, 1]. To alleviate the resolution loss and difficulty in identifying fine structures caused by downsampling, feature enhancement processing was performed on the downsampled images. The average gradient magnitude was calculated using the 3D Sobel operator to generate edge feature maps. Adaptive histogram equalization was applied to the bone tissue region with CT values ​​between 800 and 1600 to enhance the contrast of bone microstructures such as trabeculae and cortical bone, generating bone tissue enhancement maps. The downsampled images, their edge feature maps, and the bone tissue enhancement maps were then fused at ratios of 0.5, 0.2, and 0.3.

[0062] Step 2: Construct a multi-scale channel-separable KAN convolutional module MC-KAN

[0063] The MC-KAN module is the core feature extraction unit of this invention. It combines the nonlinear fitting ability of the Kolmogorov-Arnold representation theorem with multi-scale convolution design, such as... Figure 3 As shown, the specific construction and parameter settings are as follows:

[0064] 2.1 Implementation of Channel Dimensionality Reduction Layer: The input feature vector has a dimension of C×D×H×W (where C is the number of channels, D is the voxel depth, H is the voxel height, and W is the voxel width). Channel dimensionality reduction is performed using 1×1×1 point convolution with C / 2 kernels, stride = 1, and no padding. After convolution, a ReLU activation function is used to introduce nonlinearity, and the output dimension is (C / 2)×D×H×W.

[0065] 2.2 Convolutional Kernel Configuration: Apply [3,3,3] and [5,5,5] KAN convolutional kernels for each channel respectively to ensure that the sum of the output channels of the two scale convolutional kernels is consistent with the number of channels before dimensionality reduction (C); convolution stride = 1, [3,3,3]KAN convolution padding = 1, [5,5,5]KAN convolution padding = 2 to ensure that the feature map size after convolution is consistent with the input;

[0066] 2.3 Feature Map Reshaping: Before convolution, the feature map output by the [3,3,3] convolution (dimension (C / 2)×D×H×W) is reshaped into a two-dimensional feature vector. The reshaping rule is as follows: the D×H×W voxels of each channel are divided into 3×3×3 blocks, and each block is flattened into a 27-dimensional vector. The final output dimension is (C / 2)×[(D×H×W) / 27]×27. Similarly, the feature map output by the [5,5,5] convolution is reshaped into a two-dimensional feature vector with a dimension of (C / 2)×[(D×H×W) / 125]×125.

[0067] 2.4 KAN Network Layer Implementation: The two types of two-dimensional feature vectors described above are input into two layers of the KAN network respectively. A K-layer KAN network can be represented as a nested network of multiple KAN layers:

[0068]

[0069] in This represents the KAN network at layer k. The computation results from layer k to layer (k+1) can be represented by a matrix. , where the activation function It can be represented as:

[0070]

[0071] In the formula: It is a learnable non-linear activation function, which can be represented as follows:

[0072]

[0073]

[0074]

[0075] In the formula: These are functions and The weighting factors are learnable. and It is a spline function The i-th learnable parameter is a univariate nonlinear function. This mechanism enables the KAN convolution kernel to dynamically adapt to the local feature distribution, modeling higher-order associations while extracting low-level textures.

[0076] 2.5 Residual Connection and Channel Fusion Implementation: The feature vectors output by the KAN network are concatenated by channel and reconstructed back into a 3D feature map (dimensions C×D×H×W). This map is then residually connected to the input feature map (dimensions C×D×H×W) to avoid gradient vanishing during model training. Subsequently, the number of channels is adjusted to the specified number of output channels using 1×1×1 point convolutions (the number of convolution kernels is equal to the number of output channels). Combined with batch normalization, multi-scale feature fusion is achieved, and finally, the feature map of the MC-KAN module is output.

[0077] Step 3: Construction and Implementation of the Multi-Scale Feature Fusion Network MC-UKAN

[0078] The MC-UKAN network is used in the coarse segmentation stage. By combining MC-KAN modules, 3D positional encoding (PE3D), and the U-Net architecture, rapid localization of the entire maxillofacial structure is achieved, such as... Figure 2 As shown, the specific construction and training parameters are as follows:

[0079] 3.1 Implementation of the initial feature extraction layer

[0080] The input is a downsampled CBCT image (dimensions 1×160×160×108), which is passed through two 3×3×3 convolutional layers: the first convolutional layer has 32 kernels, a stride of 1, padding of 1, and ReLU activation; the second convolutional layer has 64 kernels, a stride of 1, padding of 1, and ReLU activation; both convolutional layers are followed by a batch normalization (BN) layer to ensure the stability of the initial feature extraction.

[0081] 3.2 Implementation of the 3D Position Encoding Module (PE3D)

[0082] 3D mesh coordinate generation: 3D mesh coordinates (x, y, z) are generated based on the dimensions of the input feature map (160×160×108), where x∈[0,D-1], y∈[0,H-1], and z∈[0,W-1]. The coordinates are normalized to the [0,1] interval through linear transformation to eliminate the influence of size differences between different CBCT images, generating 3D mesh coordinates (x, y, z). Multi-scale frequency bands are generated using a log-interval frequency distribution, where the number of frequency bands is determined according to the spatial scale range of the maxillofacial structure. For each spatial dimension (x, y, z) of the 3D mesh coordinates, the sine and cosine components under each frequency band are calculated, and all components are concatenated according to dimensions to form a 6D positional code with a dimension of 6×N (where N is the number of frequency bands). The 6D positional code is compressed to the same number of channels as the original feature map (64 channels) through 1×1×1 convolution, and then fused with the original feature map element-wise.

[0083] 3.3 Encoder Implementation

[0084] The feature map with fused position encoding is sequentially fed into a 5-layer MC-KAN module as an encoder. The parameters of each MC-KAN layer are the same as in step two, and the following operations are performed after each layer: (1) Max pooling: 2×2×2 max pooling is used with a step size of 2 and no padding to achieve feature map downsampling (the feature map size is halved and the number of channels is doubled after each encoder layer); (2) SE module: The squeezing and activation module is introduced. First, the feature map is compressed to 1×1×1×C (C is the current number of channels) through global average pooling (GAP). Then, channel weights are generated through two fully connected layers (the number of neurons in the first layer = C / 16, and the number of neurons in the second layer = C) and the Sigmoid activation function. The weights are multiplied with the original feature map channel by channel to strengthen the feature representation of key structures of the maxillofacial region (such as the lower alveolar nerve canal).

[0085] 3.4 Decoder Implementation

[0086] Five convolutional decoders are constructed for the corresponding encoder. The operation process and parameters of each decoder are as follows: (1) Upsampling: The input feature map of the previous layer is enlarged by trilinear interpolation at a ratio of 2×2×2, and the size is restored to the feature map size of the corresponding encoder layer; (2) Skip connection: The upsampled feature map is concatenated with the feature map output by the corresponding encoder layer; (3) Convolution and normalization: After concatenation, it goes through two 3×3×3 convolutional layers (the number of convolutional kernels = 1 / 2 of the number of channels after concatenation, stride = 1, padding = 1, ReLU activation), one BN layer and one SE module in sequence;

[0087] 3.5 Output Layer Implementation

[0088] The output of the final decoder layer is mapped to a 42-channel feature map through a 1×1×1 pointwise convolution; the probability distribution of each structure is calculated by the Sigmoid activation function, and the maximum probability label is extracted by the argmax function to generate a coarse segmentation result.

[0089] Step 4: Implementation in the coarse segmentation stage

[0090] The downsampled image is input into the MC-UKAN network to obtain coarse segmentation results including the maxilla / mandible, left / right neural canals, left / right maxillary sinuses, teeth marked according to FDI tooth position representation, pharynx, and restorations (dental bridges, crowns, dental implants);

[0091] Step 5: Implementation of Local High-Resolution Fine Segmentation Stage

[0092] Based on the coarse segmentation results, local small structures are located. K-means clustering is used to classify the categories, and a 3D Unet_SE model is used to achieve high-resolution segmentation. The specific implementation is as follows:

[0093] 5.1 Input Data Preparation: The output labels from step four are divided into 5 categories using K-means clustering to determine the input block size. The input size for incisors, molars, and restorations is 48×48×72 (cropped from the bounding box of the original image; zero-padding is used to fill in any insufficient size). The input size for the maxillary sinus is 64×96×48, for the inferior alveolar nerve canal it is 120×200×200, and for restorations it is 48×48×48. The training samples for each category are the cropped blocks of the corresponding structures.

[0094] 5.2 Network Parameters: The encoder has 5 layers, and the number of convolutional kernels (channels) in each coding unit is configured according to category: incisors, maxillary sinuses, and restorations are [16, 32, 64, 128, 256], while molars and inferior alveolar nerve canals are [32, 64, 128, 256, 512]; the decoder has 5 layers, and the number of channels in the decoding unit is the same as that in the coding unit; the SE module parameters are the same as those in the SE module of MC-UKAN.

[0095] 5.3 Intelligent Slider Processing Implementation: When the actual size of structures such as the lower alveolar nerve canal exceeds the set block size, intelligent slider inference is adopted. The specific steps are as follows: In the internal region of the structure, the sliding step size is set to 1 / 2 of the block size to reduce redundant calculations; in the edge region of the structure, the edge contour is located through coarse segmentation results, and the sliding step size is shortened to 1 / 4 of the block size to preserve edge details; during the sliding process, the overlapping areas of adjacent windows are fused using probability weighting, and the weight is positively correlated with the distance from the region to the center of the window; after all windows are inferred, the redundant predictions of the overlapping areas are eliminated by the non-maximum suppression algorithm, and finally the segmentation results of the complete structure are stitched together.

[0096] Step Six: Implementation of the Large-Scale Structural Segmentation Stage

[0097] The same type of 3D Unet_SE network structure is used, including encoder modules and decoder modules connected in sequence, wherein:

[0098] The input data uses the original CBCT image size, requiring no block processing. The encoder module contains 5 coding units, each including two convolutional layers, one normalization layer, and a squeeze and activation (SE) module. Downsampling is achieved through 2×2×2 max pooling, and the convolutional layers use 3×3×3 kernels with stride and padding set to maintain the feature map size. The decoder module contains 5 decoding units, each including two convolutional layers, one normalization layer, and a squeeze and activation (SE) module. Feature map magnification is achieved through 2×2×2 upsampling. The convolutional layer parameters are consistent with the coding units, and the output features of the corresponding coding units are fused through skip connections. The output module maps the feature channels into 3 channels through pointwise convolution, corresponding to the segmentation results of the maxilla / mandible, pharynx, and background, respectively. The probability distribution maps of the three categories are output after passing through the Sigmoid activation function.

[0099] Step 7: Implementation of the Probabilistic Graphical Fusion Mechanism

[0100] Obtain the coarse segmentation labels from step four (containing 42 types of structures: 32 teeth labeled according to FDI tooth position, left / right nerve canals, left / right maxillary sinuses, maxilla / mandible, pharynx, and 3 types of restorations), the probability maps of the 5 categories of structures output from step five (probability maps of tooth-pulp binary classification corresponding to incisors and molars, probability maps of nerve canals, probability maps of maxillary sinuses, and probability maps of restorations), and the probability maps of maxilla / mandible and pharynx output from step six, and map them uniformly to the coordinate system of the original CBCT image;

[0101] Confidence thresholds were set for various structures: the probability thresholds for teeth, dental pulp, and restorations were set at 0.65-0.75; the probability thresholds for neural tube and maxillary sinus were set at 0.55-0.65; and the probability thresholds for maxillary / mandibular bones and pharynx were set at 0.60-0.70.

[0102] For a probability map of a single structure, the initial region of the corresponding structure is first located by combining the coarse segmentation label from step four (e.g., 32 teeth correspond to 32 initial regions, and the potential distribution region of dental pulp is synchronously associated within each tooth region). Then, voxels with probability values ​​higher than the corresponding threshold within the initial region are retained as high-confidence regions. For voxels with probability values ​​lower than the threshold within the initial region, a neighborhood voting mechanism is used for correction. The neighborhood voting mechanism is as follows: take high-confidence pixels within a 3×3×3 range around the pixel that belong to the tooth / dental pulp category corresponding to the coarse segmentation label from step four, and use the "specific tooth - corresponding dental pulp" label to which most high-confidence pixels belong as the correction result.

[0103] For overlapping regions of different structural probability maps, the structural category of the overlapping region is first determined based on the coarse segmentation label in step four, and then the structural priority weighted fusion is adopted: the overlapping region of the neural tube and mandible is mainly based on the neural tube probability map, the overlapping region of the dental pulp and tooth is mainly based on the dental pulp probability map, the overlapping region of the maxillary sinus and maxilla is mainly based on the maxillary sinus probability map, and the remaining overlapping regions are weighted averaged according to the probability value. The weight is calculated as "the probability value of the structure / the sum of the probability values ​​of all overlapping structures".

[0104] After fusion, isolated regions with areas smaller than a preset threshold are removed by connected component analysis. The preset threshold is determined according to the structure type: the preset threshold for teeth, pulp, and restorations is set to 30-50 voxels; the preset threshold for nerve canals and maxillary sinuses is set to 50-80 voxels; and the preset threshold for maxilla / mandible and pharynx is set to 100-150 voxels. Finally, the FDI tooth position calibration information of the coarse segmentation labels in step four is combined with the anatomical correspondence between "teeth and pulp" (each tooth corresponds to one pulp) to verify and correct the label consistency of the final segmentation results, generating a complete maxillofacial panoramic segmentation result containing 74 types of structures (32 teeth, 32 corresponding pulps, left / right mandibular nerve canals, left / right maxillary sinuses, maxilla / mandible, pharynx, and 3 types of restorations, namely crowns, bridges, and implants).

[0105] Step 8: Effects of the Implementation Example

[0106] This invention uses two common evaluation metrics to measure segmentation performance: the Dice similarity coefficient and the 95th percentile Hausdorff distance (HD95), calculated as follows:

[0107]

[0108]

[0109]

[0110] in: It is the prediction region. It is a real area. Represents a set The 95th percentile of the element in the middle, This represents the distance between points a and b.

[0111] Due to the significant scale difference between the dental pulp and jawbone in the original CBCT images, existing state-of-the-art (SOTA) models struggle to achieve satisfactory end-to-end panoramic segmentation results. Therefore, three SOTA methods were compared with the method of this invention within a three-stage framework. Except for 3D Unet_SE, the other two are based on the latest SOTA architecture from 2025. The results were averaged across seven anatomical structures, yielding the Dice coefficients and HD95 for the maxilla / mandible, left / right mandibular canal (IAC), left / right maxillary sinus, pharynx, restorations, teeth, and dental pulp (including dental landmark localization), as shown in Tables 1 and 2.

[0112] Table 1 Comparison of Dice coefficients using different methods

[0113]

[0114] Table 2 Comparison of HD95 coefficients using different methods

[0115]

[0116] As shown in Tables 1 and 2, the three-stage CBCT maxillofacial panoramic segmentation method proposed in this invention, based on the Kolmogorov-Arnold representation theorem and the U-Net architecture, demonstrates the best overall performance and segmentation of multiple key structures. In terms of overall segmentation accuracy, the proposed method achieves the highest accuracy of 88.3%, outperforming the baseline 3D Unet_SE model (83.67%) and two state-of-the-art (SOTA) models from 2025: 3D UKAN_SE (87.36%) and PMFSNet3D (86.64%). On the HD95 metric, which measures boundary segmentation accuracy, the proposed method also achieves the best average accuracy of 3.17 mm, further validating its superiority in segmenting complex structural edges, and its overall performance is better than the comparative models. This indicates that the three-stage framework and MC-KAN module design adopted in this invention can effectively improve the comprehensive accuracy of panoramic segmentation. Looking at the segmentation results for different anatomical structures, the proposed method excels in maxillary sinus segmentation, with a Dice coefficient of 82.09% and an HD95 of 2.33 mm, both superior to other comparative methods. In pulp segmentation, this method achieved the highest Dice value (87.55%) and the lowest HD95 value (2.11 mm). It also led in several metrics for tooth, restoration, and jawbone segmentation. Notably, all methods showed complete consistency in pharyngeal segmentation results. This is because this structural feature is distinct and does not overlap with other regions, and all methods used the same large-scale structural segmentation model. However, jawbone segmentation results may have interference with adjacent teeth, requiring minor corrections during the fusion stage, resulting in slight differences between methods. In summary, the method of this invention demonstrates superior overall performance in CBCT maxillofacial panoramic segmentation tasks, particularly in multi-scale feature fusion and complex structural processing.

[0117] Figure 4 shows the segmentation effect of the panoramic segmentation method for complex maxillofacial structures described in this invention, targeting 74 types of oral and maxillofacial structures. All sample segmentation results use the same color mapping table to ensure comparability of segmentation results for various structures. It can be seen that normal dentition, missing teeth, crowns, restorations, and root remnants are all accurately segmented and labeled, fully demonstrating the adaptability of this method to complex maxillofacial scenarios. Figure 4 (a) is a midsagittal slice of a CBCT image, on which the segmentation results of the maxilla, mandible and pharyngeal structures output by this method are superimposed; Figure 4 (b) is a sagittal section of a CBCT image located at approximately 1 / 4 of the position, with the segmentation results of the left and right maxillary sinuses and the inferior alveolar canal (IAC) superimposed on the section; Figure 4 (c) is a cross-sectional slice of the CBCT image at approximately 1 / 3 of its length, on which the segmentation results of the tooth and the internal dental pulp structure are superimposed; Figure 4(d) is a view of the three-dimensional segmentation results of the maxilla, mandible, and pharynx; Figure 4 (e) Three-dimensional segmentation results of the left and right maxillary sinuses and inferior alveolar canal (IAC); Figure 4 (f) is a three-dimensional segmentation result view of the right mandibular dentition. The red box shows the segmentation details of the restoration (crown and implant). It can be seen that the boundary between the restoration and the natural tooth is clear and there is no segmentation confusion. Figure 4 (g) is a three-dimensional segmentation result view of the complete dentition. The area marked by the red box shows a scene with a missing tooth on the right. This method can still maintain the consistency of tooth position numbering of the natural teeth on both sides of the missing tooth and has no misalignment labeling problem. Figure 4 (h) is a three-dimensional segmentation result view of the complete dentition, in which the area marked by the red box shows the coexistence of restorations (crowns) and residual roots. The segmentation boundary between the two is clear, and the shape and position of the residual roots are accurately restored. Figure (i) is a three-dimensional segmentation result view of the left mandibular dentition, in which the area marked by the red box shows the scene with two consecutive missing teeth. This method can still accurately locate the remaining teeth across the missing area without any numbering errors.

[0118] The above description focuses on preferred embodiments of the present invention, but the scope of protection of the present invention is not limited to the specific implementation methods described above. Within the scope of conventional technical knowledge possessed by those skilled in the art, various adjustments and improvements can be made as long as they do not deviate from the core design principles of the present invention. These variations and modifications based on the technical concept of the present invention should all be included within the scope of protection claimed by the present invention.

Claims

1. A three-stage CBCT maxillofacial panoramic segmentation method based on the Kolmogorov-Arnold representation theorem and U-Net architecture, characterized in that, Includes the following steps: S1. Obtain the dataset of oral CBCT images and perform preprocessing on the images in the dataset, including downsampling; S2. Construct a multi-scale channel-separable KAN convolutional module MC-KAN; S3. Construct a multi-scale feature fusion network MC-UKAN for the coarse segmentation stage based on the multi-scale channel separable KAN convolutional module MC-KAN; S4. In the coarse segmentation stage, the downsampled image is input into the multi-scale feature fusion network MC-UKAN to obtain coarse segmentation results containing various structures, including maxilla / mandible, left / right neural canal, left / right maxillary sinus, teeth marked according to FDI tooth position representation, pharynx, and restorations. S5. In the local high-resolution fine segmentation stage, based on the coarse segmentation results, the positions of various structures on the original image are located. The structures in the coarse segmentation results, excluding the maxilla / mandible and pharynx, are divided into five categories using K-means clustering analysis: incisors, molars, inferior alveolar nerve canal, maxillary sinus, and restorations. After locating the positions of these five categories of structures on the original image, a fixed block size is used, and the corresponding 3D Unet_SE model is used for segmentation. For structures whose actual size exceeds the set block size, intelligent sliders are used for inference, and stitching is performed based on the highest probability. The segmentation and stitching process yields the S5 segmentation result. S6. In the large-scale structure segmentation stage, for the maxilla / mandible and pharyngeal structures, a 3D Unet_SE model is trained on the original image to segment the three types of structures and obtain the S6 segmentation result. S7. The segmentation results of S5 and S6 are fused using a probabilistic graph fusion mechanism.

2. The three-stage CBCT maxillofacial panoramic segmentation method based on the Kolmogorov-Arnold representation theorem and U-Net architecture as described in claim 1, characterized in that, The multi-scale channel separable KAN convolutional module MC-KAN in S2 includes the following construction steps: Channel dimensionality reduction is performed on the input feature vector using 1×1×1 point convolution; KAN convolution kernels of [3,3,3] and [5,5,5] are applied to each channel respectively. The feature map output by the [3,3,3] convolution is reshaped into a two-dimensional feature vector of size [(D×H×W) / 27,27]; the feature map output by the [5,5,5] convolution is reshaped into a two-dimensional feature vector of dimension (C / 2)×[(D×H×W) / 125]×125. The two types of two-dimensional feature vectors mentioned above are input into a two-layer KAN network for nonlinear spatial feature extraction. The extracted features are processed by residual connection, and then channel fusion is achieved through point convolution and batch normalization.

3. The three-stage CBCT maxillofacial panoramic segmentation method based on the Kolmogorov-Arnold representation theorem and U-Net architecture as described in claim 2, is characterized in that... The multi-scale feature fusion network MC-UKAN in S3 includes the following construction steps: The input downsampled image is processed by two layers of standard convolution (Conv3d). The convolutional kernel size of the convolutional layer is 3×3×3, the stride is 1, and the padding is 1, and feature maps of 64 channels are extracted. The feature map is explicitly injected with prior location information through a 3D location encoding module; The feature map with injected location prior information is sequentially fed into 5 layers of MC-KAN convolutional layers as the model encoder. Each MC-KAN convolutional layer is followed by 2×2×2 max pooling, and a squeezing and activation module is introduced to enhance the feature representation using a global attention mechanism. Five convolutional decoders are constructed corresponding to the model encoder. Each convolutional decoder includes two standard convolutional layers (Conv3d), one normalization layer, and a squeezing and excitation module. The convolutional kernel size of the convolutional layer is 3×3×3, the stride is 1, and the padding is 1. Each convolutional decoder first amplifies the input of the previous layer by a ratio of 2×2×2 through trilinear interpolation, and then fuses the features output by the corresponding encoder layer through skip connections.

4. The three-stage CBCT maxillofacial panoramic segmentation method based on the Kolmogorov-Arnold representation theorem and U-Net architecture as described in claim 3, is characterized in that... In S4, the last decoder layer described in S3 is used to map the feature channels to the number of channels corresponding to the various structures described in S4 through pointwise convolution, and the probability distribution map of various structures is output through the Sigmoid activation function. The maximum probability label is extracted through the argmax function. The downsampled image is input into the multi-scale feature fusion network MC-UKAN to generate a coarse segmentation result.

5. The three-stage CBCT maxillofacial panoramic segmentation method based on the Kolmogorov-Arnold representation theorem and U-Net architecture as described in claim 3, is characterized in that... In S3, the 3D position encoding module includes the following construction steps: The physical coordinates of the 64-channel feature map obtained after initial feature extraction are normalized to the [0,1] interval to generate a three-dimensional mesh coordinate. A multi-scale frequency band is generated using a log-interval frequency distribution, wherein the number of multi-scale frequency bands is determined according to the spatial scale range of the maxillofacial structure. For each spatial dimension (x, y, z) of the three-dimensional grid coordinates, the sine and cosine components of the three-dimensional grid coordinates in each frequency band are calculated respectively, and all components are spliced ​​together according to the dimensions to form a 6D position code with a dimension of 6×N, where N is the number of frequency bands; The 6D positional encoding is compressed to 64 channels using 1×1×1 convolution, and then fused with the original feature map element by element.

6. The three-stage CBCT maxillofacial panoramic segmentation method based on the Kolmogorov-Arnold representation theorem and U-Net architecture as described in claim 1, characterized in that, The method for constructing the 3D Unet_SE corresponding to the segmentation network of the five structures in S5 is as follows: A corresponding 3D Unet_SE segmentation network is configured for each of the five structural categories. The segmentation network includes an encoder module and a decoder module connected in sequence. The encoder module contains 5 encoding units. Each encoding unit includes 2 standard convolutional layers (Conv3d), one normalization layer, and a squeezing and excitation module. Downsampling is achieved through 2×2×2 max pooling. The convolutional layers use 3×3×3 convolutional kernels, and the stride and padding are set to keep the feature map size unchanged. The decoder module contains 5 decoding units. Each decoding unit includes 2 standard convolutional layers (Conv3d), one normalization layer, and a squeezing and excitation module. It also uses 2×2×2 upsampling to enlarge the feature map. The convolutional layers use 3×3×3 convolutional kernels, and the stride and padding are set to keep the feature map size unchanged. Each decoding unit of the decoder module fuses the features output by the corresponding encoding unit of the encoder module at the same layer through skip connections, and the number of feature channels of the encoding unit and the decoding unit at the same layer are the same.

7. The three-stage CBCT maxillofacial panoramic segmentation method based on the Kolmogorov-Arnold representation theorem and U-Net architecture as described in claim 1, characterized in that, In step S5, a smart slider is used for processing, including: When the actual size of the structure exceeds the set block size, a sliding window is generated based on the set block size, and the step size of the sliding window is dynamically adjusted according to the edge features of the structure. Within the structure's internal region, the sliding step size is set to 1 / 2 of the block size; In the structural edge region, the edge contour is located using the coarse segmentation results, and the sliding step size is shortened to 1 / 4 of the block size; During the sliding process, the overlapping areas of adjacent windows are fused using a probability-weighted method, with the weight being positively correlated with the distance of the area from the center of the window. After all window inferences are completed, redundant predictions in overlapping regions are eliminated using a non-maximum suppression algorithm, and finally, the segments are stitched together to form a complete segmentation result.

8. The three-stage CBCT maxillofacial panoramic segmentation method based on the Kolmogorov-Arnold representation theorem and U-Net architecture as described in claim 1, characterized in that, The method for constructing the segmentation network of the maxilla / mandible and pharynx in S6 is as follows: The same type of 3D Unet_SE network structure as the segmentation network described in S5 is adopted, and the 3D Unet_SE network structure described in S5 also includes an encoder module and a decoder module connected in sequence. The output module maps the feature channels into three channels through pointwise convolution, corresponding to the segmentation results of the maxilla / mandible, pharynx, and background, respectively. The probability distribution map of the three categories is output through the Sigmoid activation function.

9. The three-stage CBCT maxillofacial panoramic segmentation method based on the Kolmogorov-Arnold representation theorem and U-Net architecture as described in claim 8, is characterized in that... The probabilistic graph fusion mechanism in S7 is as follows: Obtain the coarse segmentation label from S4, the five-category structural probability maps output from S6, and the maxilla / mandible and pharynx probability maps output from S7, and uniformly map them to the coordinate system of the original resolution CBCT image of the oral cavity CBCT image. Probability thresholds are set for teeth, dental pulp, restorations, neural tube, maxillary sinus, maxilla / mandible, and pharynx respectively; For a probability map of a single structure, the initial region of the corresponding structure is first located by combining the S4 coarse segmentation label. Then, voxels with probability values ​​higher than the corresponding threshold within the initial region are retained as high-confidence regions. For voxels with probability values ​​lower than the threshold within the initial region, they are corrected through a neighborhood voting mechanism. For overlapping regions of different structural probability maps, the structural category of the overlapping region is first determined based on the S4 coarse segmentation label, and then the structural priority weighted fusion is adopted: the overlapping region of the neural tube and mandible is mainly based on the neural tube probability map, the overlapping region of the dental pulp and tooth is mainly based on the dental pulp probability map, the overlapping region of the maxillary sinus and maxilla is mainly based on the maxillary sinus probability map, and the remaining overlapping regions are weighted averaged according to the probability value. The weight is calculated as the probability value of the structure / the sum of the probability values ​​of all overlapping structures. After fusion, isolated regions with areas smaller than a preset threshold are removed by connected component analysis. The preset threshold is determined according to the structure type. Finally, the FDI tooth position calibration information of the S4 coarse segmentation label is combined with the "tooth-pulp" anatomical correspondence to verify and correct the label consistency of the final segmentation result, generating a complete maxillofacial panoramic segmentation result.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

Images