Three-dimensional CBCT tooth image segmentation method and system based on state space modeling and storage medium

Abstract

The invention discloses a three-dimensional CBCT tooth image segmentation method and system based on state space modeling and a storage medium, and relates to the technical field of medical image processing. Comprising the following steps: acquiring a plurality of historical CBCT jaw region image data, and preprocessing the image data to obtain a sample jaw region image; manually marking the sample jaw region image to generate a model training image set; establishing a CBCT image tooth segmentation model, taking a model training image as input, taking a tooth contour as a label, and performing state space information modeling and multi-frequency domain analysis on the input image; carrying out multi-scale feature fusion on the feature output to generate a fused feature map, carrying out decoding operation based on the fused feature map, restoring to a full-size feature map, and carrying out Softmax function processing to generate a final segmentation result; and setting a loss function, continuously updating model parameters through gradient back propagation until the loss function converges, completing model training, and outputting a segmented tooth contour image.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a three-dimensional CBCT tooth image segmentation method, system and storage medium based on state space modeling. Background Art

[0002] Oral health is an important part of national health. With the development of social economy and the improvement of quality of life, more and more people pay attention to dental health and seek dental treatment to maintain oral function and improve facial appearance. The rapid progress of artificial intelligence has promoted the development of digital oral medicine, especially the widespread application of computer-assisted treatment systems, which has greatly improved the efficiency of oral treatment. In clinical dental treatment, 3D cone beam computed tomography (CBCT) images are the only means to provide complete three-dimensional information of teeth and alveolar bones. Therefore, accurate segmentation of teeth and alveolar bone structures from CBCT images to reconstruct three-dimensional models is crucial for digital dentistry.

[0003] However, dental CBCT image segmentation faces many challenges: due to the similar strength of teeth and alveolar bones, it is easy to misclassify or miss the disorder and structure of the three-dimensional data structure; there is adhesion between the adjacent teeth and the occlusal teeth at the crown, and the boundaries are blurred. The patient's dental condition is complex, and there are phenomena such as tooth misalignment, tooth loss, and brackets. The distribution and arrangement of teeth vary greatly among patients. The above difficulties bring huge challenges to the digital three-dimensional tooth segmentation of dental CBCT images. Traditional digital three-dimensional tooth segmentation methods for dental CBCT images rely on a lot of prior knowledge and require certain manual intervention, and cannot achieve fully automatic and intelligent tooth segmentation. The existing dental CBCT image segmentation methods based on deep learning have limited ability to extract local detail features. Digital three-dimensional tooth segmentation methods have poor results in tooth disorder and tooth-gingival boundary segmentation, often over-segmentation or under-segmentation of adjacent teeth, and unclear boundary segmentation; different tooth categories may have large differences in the same category and small differences between categories, resulting in confusion between categories. In addition, these methods have a large computational burden and are difficult to process three-dimensional CBCT data in real time in actual application scenarios.

[0004] In the prior art, the publication number CN114972360A discloses a method for segmenting a computed tomography image of a tooth, which comprises obtaining a first three-dimensional digital model representing a crown of an erupted tooth of a first jaw and a two-dimensional tomographic image sequence of the first jaw; using a local image classification model, based on the two-dimensional tomographic image sequence of the first jaw, selecting a two-dimensional tomographic image for each erupted tooth from the two-dimensional tomographic image sequence of the first jaw, wherein the local image classification model is a trained deep neural network for classifying the local two-dimensional tomographic image into categories including crowns and roots; based on the first three-dimensional digital model, obtaining the position and range information of each erupted tooth; and for each erupted tooth, using the position and range information, using the local image segmentation model, segmenting the local image of the erupted tooth from the corresponding two-dimensional tomographic image for the start segmentation in the direction of the crown and the root, and obtaining its binary mask image sequence. However, the deep neural network used in this method may not be able to fully extract important features when processing complex structures, especially when the tooth structure is complex or fuzzy, which may lead to reduced classification accuracy. At the same time, although a three-dimensional digital model is used, the segmentation process mainly depends on the two-dimensional image sequence. The lack of effective utilization of 3D structural information may lead to inaccurate segmentation results.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] The object of the present invention is to provide a three-dimensional CBCT tooth image segmentation method, system and storage medium based on state space modeling to solve the problems raised in the above background technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A three-dimensional CBCT tooth image segmentation method, system and storage medium based on state space modeling, the specific steps include:

[0009] S1, acquiring a number of historical CBCT jaw region image data, and preprocessing the acquired historical CBCT jaw region image data to obtain a sample jaw region image, wherein the preprocessing includes image denoising and image enhancement processing;

[0010] S2, based on the sample jaw region image, manually marking the sample jaw region image, marking the tooth contour in the sample jaw region image, recording the marked image as a model training image, and generating a model training image set;

[0011] S3, based on the image data in the model training image set, establish a CBCT image tooth segmentation model, take the image data in the model training image set as input, and the tooth contour in the model training image as a label, the CBCT image tooth segmentation model performs state space information modeling and multi-frequency domain analysis on the input image, and obtains a state space information modeling branch output and a multi-frequency domain analysis branch output, the CBCT image tooth segmentation model includes an encoder, a state frequency domain joint enhancement layer, a multi-scale feature fusion layer, a decoder and an output layer;

[0012] S4, performing multi-scale feature fusion processing on the state space information modeling branch output and the multi-frequency domain analysis output to generate a fused feature map, performing decoding operation based on the fused feature map to generate a feature map restored to the full size, inputting the full size feature map into the output layer of the CBCT image tooth segmentation model, and processing it through the Softmax function to generate the final segmentation result;

[0013] S5, setting the loss function, calculating the loss function value of the model based on the segmentation result, and continuously updating the model parameters through gradient back propagation until the loss function converges to a value less than a preset loss threshold, completing the model training, and finally obtaining a trained CBCT image tooth segmentation model, and outputting the segmented tooth contour image;

[0014] S6, obtaining CBCT jaw region image data of the jaw to be detected and preprocessing it, and then inputting it into the CBCT image tooth segmentation model to obtain a tooth contour image in the jaw to be detected.

[0015] Furthermore, a number of historical CBCT jaw region image data collected are preprocessed, and the preprocessing includes image denoising and image enhancement processing, wherein the image is denoised by adopting a wavelet transform denoising method, and the image is enhanced by adopting a bilateral filtering method, and the preprocessed image is recorded as a sample jaw region image.

[0016] Furthermore, the sample jaw region image is manually labeled, and the method for marking the tooth contours in the sample jaw region image is as follows: the tooth contours in a single jaw region image are annotated by the LabelImg tool using a manual labeling method, and the labeled image is recorded as a model training image to generate a model training image set.

[0017] Furthermore, the step S3 further comprises the following steps:

[0018] S31, performing state space information modeling on the input image to obtain a state space information modeling branch output;

[0019] S32, performing multi-frequency domain analysis to obtain multi-frequency domain analysis output;

[0020] Before performing state space information modeling and multi-frequency domain analysis on the input image, it is necessary to perform preliminary feature extraction on the model training image and use downsampling processing to obtain a preliminary feature map. The formula for obtaining the preliminary feature map is:

[0021]

[0022] Where I is the image data before downsampling, I * is the downsampled image data, R is the multidimensional tensor space on the real number domain, C is the number of feature channels, H, W, and D represent the height, width, and depth of the image respectively, and R C×H×W×D That is, it represents a multidimensional array with C channels, H height, W width and D depth. Conv represents the convolution operation, stride is the step size, and 3×3Conv (stride=2) means using a 3×3 convolution kernel and convolving the input with a stride of 2.

[0023] Furthermore, the step S31 further includes the following steps:

[0024] S311, flattening the image data after preliminary feature extraction into a one-dimensional vector, and the specific formula is:

[0025]

[0026] In the formula, X sequence is the flattened one-dimensional vector, HWD is the product of the length, width and depth of the model training image, and n2 represents the flattened one-dimensional vector X sequence The number of channels is n2;

[0027] S312, based on the flattened one-dimensional vector, the position embedding vector is added to compensate for the position information loss caused by the flattening operation to obtain a compensation vector, wherein the formula for calculating the compensation vector is:

[0028]

[0029] In the formula, X pos represents the compensation vector, PE is the position embedding vector;

[0030] S313, based on the compensation vector, obtain an input vector of the state space model by performing forward and backward convolution, wherein the input vector of the state space model includes a forward convolution vector and a backward convolution vector, wherein the formulas for obtaining the input vector of the state space model are respectively:

[0031]

[0032] In the formula, X for represents the forward convolution vector, Xback represents the backward convolution vector;

[0033] S314, forward convolution vector X for and the backward convolution vector X back Input into the state space model, model the global dependency information in the feature vector to obtain the forward state feature graph and the backward state feature graph, and their feature expressions are:

[0034]

[0035] Where, X forwarf represents the forward state feature map, X backward represents the backward state feature map, ※ represents the global convolution, and Represent the structured convolution kernels in the forward and backward state-space models, respectively;

[0036] S315, based on the obtained forward state feature map and backward state feature map combined with the compensation vector, a tensor reshaping operation is performed, and an inverse flattening is performed to obtain a state space information modeling branch output, wherein the expression of the state space information modeling branch output is:

[0037]

[0038] Where, X state The output of the state space information modeling branch, Reshape represents the tensor reshaping operation, Concat represents the concatenation operation, and σ is the SiLU activation function.

[0039] Furthermore, the step S32 further includes the following steps:

[0040] S321, input the feature map after preliminary feature extraction into the adaptive wavelet convolution layer to obtain the frequency domain information feature map, wherein the formula for obtaining the frequency domain information feature map is:

[0041]

[0042] Where, X w represents the frequency domain information feature map, n3 represents the frequency domain information feature map X w The number of channels is n3;

[0043] S322, based on frequency domain information feature map X w , through convolution and activation operations, the feature information in the high and low frequency domains is fused and compressed, nonlinear transformation is introduced to highlight high-frequency details, and finally the inverse wavelet transform convolution is performed to obtain the output of the multi-frequency domain analysis branch. The formula for obtaining the output of the frequency domain analysis branch is:

[0044]

[0045] In the formula, X f is the output of the frequency domain analysis branch, PointConv represents point-by-point convolution, which is implemented by a convolution operation with a convolution kernel size of 1×1, σ is the SiLU activation function, and Conv represents a convolution operation.

[0046] Furthermore, the step S4 comprises the following steps:

[0047] S41, performing multi-scale feature fusion processing on the state space information modeling branch output and the multi-frequency domain analysis output, combining the preliminary feature map, and generating a fused feature map, wherein the formula for generating the fused feature map is:

[0048]

[0049] In the formula, X o To fuse feature maps, Concat represents the concatenation operation. represents element-by-element multiplication, X score is the feature enhancement score map, n2 is the flattened one-dimensional vector X sequence The number of channels, that is, the state space information modeling branch output X state The number of channels, where the feature enhancement score map X score The formula is:

[0050] X score =Sigmoid(Conv(I * ))

[0051] In the formula, Sigmoid represents the Sigmoid activation function;

[0052] S42, the fusion feature map X o Input the decoder to generate a feature map X restored to full size out ∈R N×H×W×D ;

[0053] S43, based on feature map X restored to full size out ∈R N×H×W×D , the formula for obtaining the final segmentation result after using the Softmax function is:

[0054]

[0055] Where Y j is the final segmentation result, X j Represents the output corresponding to the semantics of the jth category, N represents the total number of categories, j is the index of the category, where j = 1, 2, …, N.

[0056] Furthermore, a loss function is set, where the loss function of the model includes a multi-class Dice loss function and a wavelet loss function. The multi-class Dice loss and the wavelet loss are summed to obtain the total loss function of the model. The expression of the multi-class Dice loss function is:

[0057]

[0058] where Y c,i and g c,i Represents the category of the i-th pixel prediction and ground truth, respectively, w c represents the weight of the cth class, ∈ is a constant for numerical stability; ground truth represents the label data used to supervise the training process, that is, the tooth contour;

[0059] The wavelet loss function expression is:

[0060]

[0061] Where k represents the position index of the convolution operation, a0 and a1 are the analysis vectors in the adaptive wavelet convolution, and s0 and s1 are the reconstruction vectors in the inverse wavelet convolution;

[0062] The final loss function is the sum of the two:

[0063] L=L MCD +L wavelet

[0064] In the formula, L represents the final loss function;

[0065] The model parameters are continuously updated through gradient back propagation until the loss function converges to a value less than the preset loss threshold yz, completing the model training, and finally obtaining the trained CBCT image tooth segmentation model, and outputting the segmented tooth contour image.

[0066] The present invention also provides a three-dimensional CBCT tooth image segmentation system based on state space modeling, wherein the three-dimensional CBCT tooth image segmentation system is used to execute the three-dimensional CBCT tooth image segmentation method based on state space modeling, comprising:

[0067] An image preprocessing module is used to obtain a number of historical CBCT jaw region image data, and preprocess the collected historical CBCT jaw region image data to obtain a sample jaw region image, wherein the preprocessing includes image denoising and image enhancement processing;

[0068] A training sample acquisition module is used to manually mark the sample jaw region image based on the sample jaw region image, mark the tooth contour in the sample jaw region image, record the marked image as a model training image, and generate a model training image set;

[0069] A tooth segmentation model establishment module is used to establish a CBCT image tooth segmentation model based on the image data in the model training image set, with the image data in the model training image set as input and the tooth contours in the model training image as labels. The CBCT image tooth segmentation model performs state space information modeling and multi-frequency domain analysis on the input image to obtain a state space information modeling branch output and a multi-frequency domain analysis branch output. The CBCT image tooth segmentation model includes an encoder, a state frequency domain joint enhancement layer, a multi-scale feature fusion layer, a decoder, and an output layer.

[0070] The feature fusion module is used to perform multi-scale feature fusion processing on the state space information modeling branch output and the multi-frequency domain analysis output to generate a fused feature map, perform decoding operations based on the fused feature map, generate a feature map restored to the full size, input the full size feature map into the output layer of the CBCT image tooth segmentation model, and generate the final segmentation result through Softmax function processing;

[0071] The loss analysis and adjustment module is used to set the loss function, calculate the loss function value of the model based on the segmentation result, and continuously update the model parameters through gradient back propagation until the loss function converges to a value less than the pre-set loss threshold, completing the model training, and finally obtaining the trained CBCT image tooth segmentation model and outputting the segmented tooth contour image;

[0072] The tooth image output module is used to obtain the CBCT jaw region image data of the jaw to be detected and, after preprocessing it, input it into the CBCT image tooth segmentation model to obtain the tooth contour image in the jaw to be detected.

[0073] The present invention also provides a storage medium, wherein a computer program stored in the storage medium can be executed by one or more processors, and when the computer program is executed by the processor, the above-mentioned three-dimensional CBCT tooth image segmentation method based on state space modeling can be implemented.

[0074] Compared with the prior art, the present invention has the following beneficial effects:

[0075] The present invention uses a 3D convolution-state space hybrid neural network that combines a state-frequency domain joint enhancement module and a multi-scale feature fusion module to segment dental CBCT images. It also combines state-space information modeling and multi-frequency domain analysis to ensure accurate modeling of long-range relationships of data on a large scale and accurate restoration of high-frequency details on a small scale. The multi-scale fusion module combines low-level features to retain the effective semantics of the target. It effectively improves the segmentation and classification accuracy of dental CBCT images and reduces the computational burden. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 It is a schematic diagram of the overall method flow of the present invention;

[0077] Figure 2 It is a schematic diagram of the overall system structure of the present invention. DETAILED DESCRIPTION

[0078] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0079] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0080] Example:

[0081] See also Figure 1 , the present invention provides a technical solution:

[0082] A three-dimensional CBCT tooth image segmentation method based on state space modeling, the specific steps include:

[0083] S1, acquiring a number of historical CBCT jaw region image data, and preprocessing the acquired historical CBCT jaw region image data to obtain a sample jaw region image, wherein the preprocessing includes image denoising and image enhancement processing;

[0084] Preprocessing is performed on several historical CBCT jaw region image data collected, and the preprocessing includes image denoising and image enhancement processing, wherein the image is denoised by adopting a wavelet transform denoising method, and the image is enhanced by adopting a bilateral filtering method, and the preprocessed image is recorded as a sample jaw region image.

[0085] The wavelet transform denoising method is used to denoise the jaw region image. The specific steps of the wavelet transform denoising method include: decomposing the jaw region image through wavelet transform to obtain wavelet coefficients of the image at different scales and directions; thresholding the wavelet coefficients to set the low-amplitude wavelet coefficients to zero and retain the high-amplitude wavelet coefficients; inversely transforming the wavelet coefficients after thresholding, reconstructing the processed coefficients into an image, and completing the image denoising;

[0086] Bilateral filtering is used to enhance the details of the jaw area image. The specific filter transformation is based on the formula:

[0087]

[0088] Where y is the coordinate vector in the image coordinate system, I y is the gray value at the coordinate vector y, B y is the gray value I y The gray value after bilateral filtering transformation, G d and G r are all Gaussian functions, where G d and G r The formula is:

[0089]

[0090] Where x is the coordinate vector in the image coordinate system, I x is the gray value at the coordinate vector x, σ d and σ r G d and G r The standard deviation of .

[0091] S2, based on the sample jaw region image, manually marking the sample jaw region image, marking the tooth contour in the sample jaw region image, recording the marked image as a model training image, and generating a model training image set;

[0092] The sample jaw region image is manually labeled, and the method for marking the tooth contours in the sample jaw region image is as follows: the tooth contours in a single jaw region image are manually labeled using the LabelImg tool, and the labeled image is recorded as a model training image to generate a model training image set. The LabelImg command is run through the Python software. After running LabelImg, a main window will appear on the interface, and labeling is performed through the main window.

[0093] S3, based on the image data in the model training image set, establish a CBCT image tooth segmentation model, take the image data in the model training image set as input, and the tooth contour in the model training image as a label, the CBCT image tooth segmentation model performs state space information modeling and multi-frequency domain analysis on the input image, and obtains a state space information modeling branch output and a multi-frequency domain analysis branch output, the CBCT image tooth segmentation model includes an encoder, a state frequency domain joint enhancement layer, a multi-scale feature fusion layer, a decoder and an output layer;

[0094] The step S3 further comprises the following steps:

[0095] S31, performing state space information modeling on the input image to obtain a state space information modeling branch output;

[0096] S32, performing multi-frequency domain analysis to obtain multi-frequency domain analysis output;

[0097] Before performing state space information modeling and multi-frequency domain analysis on the input image, it is necessary to perform preliminary feature extraction on the model training image and use downsampling processing to obtain a preliminary feature map. The formula for obtaining the preliminary feature map is:

[0098]

[0099] Where I is the image data before downsampling, I * is the downsampled image data, R is the multidimensional tensor space on the real number domain, C is the number of feature channels, H, W, and D represent the height, width, and depth of the image respectively, and R C×H×W×D That is, it represents a multidimensional array with C channels, H height, W width and D depth. Conv represents the convolution operation, stride is the step size, and 3×3Conv (stride=2) means using a 3×3 convolution kernel and convolving the input with a stride of 2.

[0100] The step S31 further comprises the following steps:

[0101] S311, flattening the image data after preliminary feature extraction into a one-dimensional vector, and the specific formula is:

[0102]

[0103] In the formula, X sequence is the flattened one-dimensional vector, HWD is the product of the length, width and depth of the model training image, and n2 represents the flattened one-dimensional vector X sequence The number of channels is n2;

[0104] S312, based on the flattened one-dimensional vector, the position embedding vector is added to compensate for the position information loss caused by the flattening operation to obtain a compensation vector, wherein the formula for calculating the compensation vector is:

[0105]

[0106] In the formula, X pos represents the compensation vector, PE is the position embedding vector; the acquisition and determination of the position embedding vector can be achieved by generating or training a fixed mathematical function. The selection of a suitable method depends on the specific application scenario and model requirements. The sine and cosine methods are suitable for capturing global position relationships, while learnable embedding is more flexible. The specific steps are mature existing technologies and will not be described here.

[0107] S313, based on the compensation vector, obtain an input vector of the state space model by performing forward and backward convolution, wherein the input vector of the state space model includes a forward convolution vector and a backward convolution vector, wherein the formulas for obtaining the input vector of the state space model are respectively:

[0108]

[0109] In the formula, X for represents the forward convolution vector, X back represents the backward convolution vector;

[0110] S314, forward convolution vector X for and the backward convolution vector X back Input into the state space model, model the global dependency information in the feature vector to obtain the forward state feature graph and the backward state feature graph, and their feature expressions are:

[0111]

[0112] In the formula, X forward represents the forward state feature map, X backward represents the backward state feature map, ※ represents the global convolution, and denote the structured convolution kernels in the forward and backward state space models respectively; they are composed of the parameter matrices in the discrete time state equation:

[0113]

[0114] Among them, the state matrix A, control matrix B and output matrix D are parameter matrices updated as the model iterates, and Δ is a manually set hyperparameter that controls the discrete step size. Represents a form of state transfer matrix, Where H represents the identity matrix, represents the adjusted control matrix, and t represents the number of historical states contained in the structured convolution kernel.

[0115] S315, based on the obtained forward state feature map and backward state feature map combined with the compensation vector, a tensor reshaping operation is performed, and an inverse flattening is performed to obtain a state space information modeling branch output, wherein the expression of the state space information modeling branch output is:

[0116]

[0117] In the formula, X state The output of the state space information modeling branch, Reshape represents the tensor reshaping operation, Concat represents the concatenation operation, and σ is the SiLU activation function.

[0118] The step S32 further comprises the following steps:

[0119] S321, input the feature map after preliminary feature extraction into the adaptive wavelet convolution layer to obtain the frequency domain information feature map, wherein the formula for obtaining the frequency domain information feature map is:

[0120]

[0121] In the formula, X w represents the frequency domain information feature map, n3 represents the frequency domain information feature map X w The number of channels is n3;

[0122] The adaptive wavelet convolution kernel is formed by combining the low-frequency, horizontal high-frequency, vertical high-frequency and diagonal high-frequency convolution operators generated by the analysis vector, which can give the model the ability to adaptively adjust the wavelet basis function and analyze and extract high-frequency detail features. Its specific expression is:

[0123] K w =cat(F ll ,F lh ,F hl ,F hh )

[0124] In the formula, K wrepresents the adaptive wavelet convolution kernel, cat represents connecting multiple matrices or arrays along a specific dimension, where F ll ,F lh ,F hl ,F hh Respectively represent low frequency, horizontal high frequency, vertical high frequency and diagonal high frequency convolution operators, and their specific expressions are:

[0125]

[0126] in and are analysis vectors derived from supervised learning of the optimal reconstruction principle, and their specific expressions are:

[0127]

[0128] Where ψ(·) and φ(·) represent the wavelet function and the scaling function respectively, k represents the position index of the convolution operation, and t represents a continuous time or space variable, which is the input of the wavelet function and the scaling function;

[0129] The expression of the optimal reconstruction principle is:

[0130] A0(-z)S0(z)+A1(-z)S1(z)=0

[0131] A0(z)S0(z)+A1(z)S1(z)=2

[0132] Where A0(z), A1(z), S0(z), S1(z) are the analysis vectors respectively. and the reconstruction vector The Z transform (Ztransform), the optimal reconstruction principle can be expressed as the wavelet loss function L wavelet Apply deep learning to supervise the analysis of vectors and the reconstruction vector learning process.

[0133] S322, based on frequency domain information feature map X w , through convolution and activation operations, the feature information in the high and low frequency domains is fused and compressed, nonlinear transformation is introduced to highlight high-frequency details, and finally the inverse wavelet transform convolution is performed to obtain the output of the multi-frequency domain analysis branch. The formula for obtaining the output of the frequency domain analysis branch is:

[0134]

[0135] In the formula, X fis the output of the frequency domain analysis branch, PointConv represents point-by-point convolution, which is implemented by a convolution operation with a convolution kernel size of 1×1, σ is the SiLU activation function, and Conv represents a convolution operation.

[0136] S4, performing multi-scale feature fusion processing on the state space information modeling branch output and the multi-frequency domain analysis output to generate a fused feature map, performing decoding operation based on the fused feature map to generate a feature map restored to the full size, inputting the full size feature map into the output layer of the CBCT image tooth segmentation model, and processing it through the Softmax function to generate the final segmentation result;

[0137] The step S4 comprises the following steps:

[0138] S41, performing multi-scale feature fusion processing on the state space information modeling branch output and the multi-frequency domain analysis output, combining the preliminary feature map, and generating a fused feature map, wherein the formula for generating the fused feature map is:

[0139]

[0140] Where, X o To fuse feature maps, Concat represents the concatenation operation. represents element-by-element multiplication, X score is the feature enhancement score map, where the feature enhancement score map X score The formula is:

[0141] X score =Sigmoid(Conv(I * ))

[0142] In the formula, Sigmoid represents the Sigmoid activation function;

[0143] S42, the fusion feature map X o Input the decoder to generate a feature map X restored to full size out ∈R N×H×W×D ;

[0144] S43, based on feature map X restored to full size out ∈R N×H×W×D , the formula for obtaining the final segmentation result after using the Softmax function is:

[0145]

[0146] Where Y j is the final segmentation result, X j Represents the output corresponding to the semantics of the jth category, N represents the total number of categories, j is the index of the category, where j = 1, 2, …, N.

[0147] The resolution of the feature map can also be adjusted by performing multiple downsampling to obtain feature maps of different resolutions, and state space information modeling and multi-frequency domain analysis can be performed based on the feature maps of different resolutions. The output results of the analysis can be upsampled to adjust the feature maps to the same length, width, and depth, and then the feature maps can be fused to obtain a fused feature map.

[0148] S5, set the loss function, calculate the loss function value of the model based on the segmentation result, and continuously update the model parameters through gradient back propagation until the loss function converges to a value less than the pre-set loss threshold, completing the model training, and finally obtaining the trained CBCT image tooth segmentation model, and outputting the segmented tooth contour image.

[0149] Set the loss function, where the loss function of the model includes the multi-class Dice loss function and the wavelet loss function. The multi-class Dice loss and the wavelet loss are summed to obtain the total loss function of the model. The expression of the multi-class Dice loss function is:

[0150]

[0151] where Y c,i and g c,i Represents the category of the i-th pixel prediction and ground truth, respectively, w c represents the weight of the cth class, ∈ is a constant for numerical stability; ground truth represents the label data used to supervise the training process, that is, the tooth contour;

[0152] The wavelet loss function expression is:

[0153]

[0154] Where k represents the position index of the convolution operation, a0 and a1 are the analysis vectors in the adaptive wavelet convolution, and s0 and s1 are the reconstruction vectors in the inverse wavelet convolution;

[0155] The final loss function is the sum of the two:

[0156] L=L MCD +L wavelet

[0157] In the formula, L represents the final loss function;

[0158] The model parameters are continuously updated through gradient back propagation until the loss function converges to a value less than the preset loss threshold yz, completing the model training, and finally obtaining a trained CBCT image tooth segmentation model, and outputting the segmented tooth contour image;

[0159] The loss threshold yz can be set based on expert experience and adjusted according to actual needs.

[0160] S6, obtaining CBCT jaw region image data of the jaw to be detected and preprocessing it, and then inputting it into the CBCT image tooth segmentation model to obtain a tooth contour image in the jaw to be detected.

[0161] See also Figure 2 The present invention also provides a three-dimensional CBCT tooth image segmentation system based on state space modeling, wherein the three-dimensional CBCT tooth image segmentation system is used to execute the three-dimensional CBCT tooth image segmentation method based on state space modeling, comprising:

[0162] An image preprocessing module is used to obtain a number of historical CBCT jaw region image data, and preprocess the collected historical CBCT jaw region image data to obtain a sample jaw region image, wherein the preprocessing includes image denoising and image enhancement processing;

[0163] A training sample acquisition module is used to manually mark the sample jaw region image based on the sample jaw region image, mark the tooth contour in the sample jaw region image, record the marked image as a model training image, and generate a model training image set;

[0164] A tooth segmentation model establishment module is used to establish a CBCT image tooth segmentation model based on the image data in the model training image set, with the image data in the model training image set as input and the tooth contours in the model training image as labels. The CBCT image tooth segmentation model performs state space information modeling and multi-frequency domain analysis on the input image to obtain a state space information modeling branch output and a multi-frequency domain analysis branch output. The CBCT image tooth segmentation model includes an encoder, a state frequency domain joint enhancement layer, a multi-scale feature fusion layer, a decoder, and an output layer.

[0165] The feature fusion module is used to perform multi-scale feature fusion processing on the state space information modeling branch output and the multi-frequency domain analysis output to generate a fused feature map, perform decoding operations based on the fused feature map, generate a feature map restored to the full size, input the full size feature map into the output layer of the CBCT image tooth segmentation model, and generate the final segmentation result through Softmax function processing;

[0166] The loss analysis and adjustment module is used to set the loss function, calculate the loss function value of the model based on the segmentation result, and continuously update the model parameters through gradient back propagation until the loss function converges to a value less than the pre-set loss threshold, completing the model training, and finally obtaining the trained CBCT image tooth segmentation model and outputting the segmented tooth contour image;

[0167] The tooth image output module is used to obtain the CBCT jaw region image data of the jaw to be detected and, after preprocessing it, input it into the CBCT image tooth segmentation model to obtain the tooth contour image in the jaw to be detected.

[0168] The present invention also provides a storage medium, wherein a computer program stored in the storage medium can be executed by one or more processors, and when the computer program is executed by the processor, the above-mentioned three-dimensional CBCT tooth image segmentation method based on state space modeling can be implemented.

[0169] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0170] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0171] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0172] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A three-dimensional CBCT tooth image segmentation method based on state space modeling, characterized in that: The specific steps include: S1, acquiring a number of historical CBCT jaw region image data, and preprocessing the acquired historical CBCT jaw region image data to obtain a sample jaw region image, wherein the preprocessing includes image denoising and image enhancement processing; S2, based on the sample jaw region image, manually marking the sample jaw region image, marking the tooth contour in the sample jaw region image, recording the marked image as a model training image, and generating a model training image set; S3, based on the image data in the model training image set, establish a CBCT image tooth segmentation model, take the image data in the model training image set as input, and the tooth contour in the model training image as a label, the CBCT image tooth segmentation model performs state space information modeling and multi-frequency domain analysis on the input image, and obtains a state space information modeling branch output and a multi-frequency domain analysis branch output, the CBCT image tooth segmentation model includes an encoder, a state frequency domain joint enhancement layer, a multi-scale feature fusion layer, a decoder and an output layer; S4, performing multi-scale feature fusion processing on the state space information modeling branch output and the multi-frequency domain analysis output to generate a fused feature map, performing decoding operation based on the fused feature map to generate a feature map restored to the full size, inputting the full size feature map into the output layer of the CBCT image tooth segmentation model, and processing it through the Softmax function to generate the final segmentation result; S5, setting the loss function, calculating the loss function value of the model based on the segmentation result, and continuously updating the model parameters through gradient back propagation until the loss function converges to a value less than a preset loss threshold, completing the model training, and finally obtaining a trained CBCT image tooth segmentation model, and outputting the segmented tooth contour image; S6, obtaining CBCT jaw region image data of the jaw to be detected and preprocessing it, and then inputting it into the CBCT image tooth segmentation model to obtain a tooth contour image in the jaw to be detected.

2. The three-dimensional CBCT tooth image segmentation method based on state space modeling according to claim 1, characterized in that: Preprocessing is performed on several historical CBCT jaw region image data collected, and the preprocessing includes image denoising and image enhancement processing, wherein the image is denoised by adopting a wavelet transform denoising method, and the image is enhanced by adopting a bilateral filtering method, and the preprocessed image is recorded as a sample jaw region image.

3. The three-dimensional CBCT tooth image segmentation method based on state space modeling according to claim 1, characterized in that: The sample jaw region image is manually labeled, and the method for marking the tooth contour in the sample jaw region image is based on: the tooth contour in a single jaw region image is annotated by the Label Img tool using the manual labeling method, and the labeled image is recorded as a model training image to generate a model training image set.

4. The three-dimensional CBCT tooth image segmentation method based on state space modeling according to claim 1, characterized in that: The step S3 further comprises the following steps: S31, performing state space information modeling on the input image to obtain a state space information modeling branch output; S32, performing multi-frequency domain analysis to obtain multi-frequency domain analysis output; Before performing state space information modeling and multi-frequency domain analysis on the input image, it is necessary to perform preliminary feature extraction on the model training image and use downsampling processing to obtain a preliminary feature map. The formula for obtaining the preliminary feature map is: Where I is the image data before downsampling, I * is the downsampled image data, R is the multidimensional tensor space representing the real number domain, C is the number of feature channels, H, W, and D represent the height, width, and depth of the image respectively, and R C×H×W×D That is, it represents a multidimensional array with C channels, H height, W width, and D depth. Conv represents the convolution operation, stride is the step size, 3× 3Conv(stride=2) means using a 3×3 convolution kernel and convolving the input with a stride of 2.

5. The three-dimensional CBCT tooth image segmentation method based on state space modeling according to claim 4, characterized in that: The step S31 further comprises the following steps: S311, flattening the image data after preliminary feature extraction into a one-dimensional vector, and the specific formula is: Where, X sequence is the flattened one-dimensional vector, HWD is the product of the length, width and depth of the model training image, and n2 represents the flattened one-dimensional vector X sequence The number of channels is n2; S312, based on the flattened one-dimensional vector, the position embedding vector is added to compensate for the position information loss caused by the flattening operation to obtain a compensation vector, wherein the formula for calculating the compensation vector is: Where, X pos represents the compensation vector, PE is the position embedding vector; S313, based on the compensation vector, obtain an input vector of the state space model by performing forward and backward convolution, wherein the input vector of the state space model includes a forward convolution vector and a backward convolution vector, wherein the formulas for obtaining the input vector of the state space model are respectively: Where, X for represents the forward convolution vector, X back represents the backward convolution vector; S314, forward convolution vector X for and the backward convolution vector X back Input into the state space model, model the global dependency information in the feature vector to obtain the forward state feature graph and the backward state feature graph, and their feature expressions are: Where, X forward represents the forward state feature map, X backward represents the backward state feature map, ※ represents the global convolution, and Represent the structured convolution kernels in the forward and backward state-space models, respectively; S315, based on the obtained forward state feature map and backward state feature map combined with the compensation vector, a tensor reshaping operation is performed, and an inverse flattening is performed to obtain a state space information modeling branch output, wherein the expression of the state space information modeling branch output is: Where, X state The output of the state space information modeling branch, Reshape represents the tensor reshaping operation, Concat represents the concatenation operation, and σ is the SiLU activation function.

6. The three-dimensional CBCT tooth image segmentation method based on state space modeling according to claim 4, characterized in that: The step S32 further comprises the following steps: S321, input the feature map after preliminary feature extraction into the adaptive wavelet convolution layer to obtain the frequency domain information feature map, wherein the formula for obtaining the frequency domain information feature map is: Where, X w represents the frequency domain information feature map, n3 represents the frequency domain information feature map X w The number of channels is n3; S322, based on frequency domain information feature map X w , through convolution and activation operations, the feature information in the high and low frequency domains is fused and compressed, nonlinear transformation is introduced to highlight high-frequency details, and finally the inverse wavelet transform convolution is performed to obtain the output of the multi-frequency domain analysis branch. The formula for obtaining the output of the frequency domain analysis branch is: Where, X f is the output of the frequency domain analysis branch, PointConv represents point-by-point convolution, which is implemented by a convolution operation with a convolution kernel size of 1×1, σ is the SiLU activation function, and Conv represents a convolution operation.

7. The three-dimensional CBCT tooth image segmentation method based on state space modeling according to claim 6, characterized in that: The step S4 comprises the following steps: S41, performing multi-scale feature fusion processing on the state space information modeling branch output and the multi-frequency domain analysis output, combining the preliminary feature map, and generating a fused feature map, wherein the formula for generating the fused feature map is: Where, X o To fuse feature maps, Concat represents the concatenation operation. represents element-by-element multiplication, X score is the feature enhancement score map, n2 is the flattened one-dimensional vector X sequence The number of channels, that is, the state space information modeling branch output X state The number of channels, where the feature enhancement score map X score The formula is: X score =Sigmoid(Conv(I * )) In the formula, Sigmoid represents the Sigmoid activation function; S42, the fusion feature map X o Input the decoder to generate a feature map X restored to full size out ∈R N×H×W×D ; S43, based on feature map X restored to full size out ∈R N×H×W×D , the formula for obtaining the final segmentation result after using the Softmax function is: Where Y j is the final segmentation result, X j represents the output corresponding to the semantics of the jth category, N represents the total number of categories, j is the index of the category, where j = 1, 2, …, N.

8. The three-dimensional CBCT tooth image segmentation method based on state space modeling according to claim 1, characterized in that: Set the loss function, where the loss function of the model includes the multi-class Dice loss function and the wavelet loss function. The multi-class Dice loss and the wavelet loss are summed to obtain the total loss function of the model. The expression of the multi-class Dice loss function is: where Y c,i and g c,i Represents the category of the i-th pixel prediction and ground truth, respectively, w c represents the weight of the cth class, ∈ is a constant for numerical stability; ground truth represents the label data used to supervise the training process, that is, the tooth contour; The wavelet loss function expression is: Where k represents the position index of the convolution operation, a0 and a1 are the analysis vectors in the adaptive wavelet convolution, and s0 and s1 are the reconstruction vectors in the inverse wavelet convolution; The final loss function is the sum of the two: L=L MCD +L wavelet In the formula, L represents the final loss function; The model parameters are continuously updated through gradient back propagation until the loss function converges to a value less than the preset loss threshold yz, completing the model training, and finally obtaining the trained CBCT image tooth segmentation model, and outputting the segmented tooth contour image.

9. A three-dimensional CBCT tooth image segmentation system based on state space modeling, characterized in that: The three-dimensional CBCT tooth image segmentation system based on state space modeling is used to execute the three-dimensional CBCT tooth image segmentation method based on state space modeling according to any one of claims 1 to 8, comprising: An image preprocessing module is used to obtain a number of historical CBCT jaw region image data, and preprocess the collected historical CBCT jaw region image data to obtain a sample jaw region image, wherein the preprocessing includes image denoising and image enhancement processing; A training sample acquisition module is used to manually mark the sample jaw region image based on the sample jaw region image, mark the tooth contour in the sample jaw region image, record the marked image as a model training image, and generate a model training image set; A tooth segmentation model establishment module is used to establish a CBCT image tooth segmentation model based on the image data in the model training image set, with the image data in the model training image set as input and the tooth contours in the model training image as labels. The CBCT image tooth segmentation model performs state space information modeling and multi-frequency domain analysis on the input image to obtain a state space information modeling branch output and a multi-frequency domain analysis branch output. The CBCT image tooth segmentation model includes an encoder, a state frequency domain joint enhancement layer, a multi-scale feature fusion layer, a decoder, and an output layer. The feature fusion module is used to perform multi-scale feature fusion processing on the state space information modeling branch output and the multi-frequency domain analysis output to generate a fused feature map, perform decoding operations based on the fused feature map, generate a feature map restored to the full size, input the full size feature map into the output layer of the CBCT image tooth segmentation model, and generate the final segmentation result through Softmax function processing; The loss analysis and adjustment module is used to set the loss function, calculate the loss function value of the model based on the segmentation result, and continuously update the model parameters through gradient back propagation until the loss function converges to a value less than the pre-set loss threshold, completing the model training, and finally obtaining the trained CBCT image tooth segmentation model and outputting the segmented tooth contour image; The tooth image output module is used to obtain the CBCT jaw region image data of the jaw to be detected and, after preprocessing it, input it into the CBCT image tooth segmentation model to obtain the tooth contour image in the jaw to be detected.

10. A storage medium, characterized in that: The computer program stored in the storage medium can be executed by one or more processors. When the computer program is executed by the processor, the three-dimensional CBCT tooth image segmentation method based on state space modeling described in any one of claims 1 to 8 can be implemented.

Images