Method and system for medical image segmentation and uncertainty estimation based on diffusion principle
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING UNIV
- Filing Date
- 2026-06-10
- Publication Date
- 2026-08-07
AI Technical Summary
而以卷积编码器解码器为代表的分割方案,虽然能够借助残差结构和注意力机制改善部分分割效果,但其特征提取方式整体上仍偏重局部卷积建模,对于长程空间依赖、二维空间连续性保持等方面仍显不足
首先,本发明通过视觉状态空间模块与PDE Operator偏微分轻量算子的协同耦合构建了偏微分视觉状态空间模块PVSS Block,实现了全局特征建模与局部特征优化的深度兼容。视觉状态空间模块具备优异的长程依赖建模能力,能够有效捕捉医学图像中跨区域、远距离的全局上下文关联特征,解决了传统卷积网络感受野受限、全局信息提取不足,以及Transformer模型计算复杂度高、医学图像适配性差的问题,可精准建模人体器官、病灶等目标的整体形态、位置关联与语义分布特征,大幅提升复杂解剖结构下全局语义分割的准确性。同时,耦合的PDE Operator偏微分轻量算子弥补了状态空间模型局部细节刻画能力薄弱的缺陷,让特征更新过程同时兼顾全局整体性与局部精细化,实现特征表达的全方位优化。
Smart Images

Figure CN122530186A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of medical image processing technology, and relates to a method and system for medical image segmentation and uncertainty estimation based on the diffusion principle. Background Technology
[0002] Medical image segmentation is a crucial foundational technology in intelligent diagnosis and treatment, surgical navigation, lesion analysis, and robot-assisted surgery. In scenarios such as robot-assisted surgery, gastrointestinal endoscopy, and chest X-rays, systems typically require real-time segmentation of organs, lesions, and surgical instruments to support navigation planning and intraoperative decision-making. However, medical images often suffer from issues such as similar tissue textures, blurred boundaries, specular reflections, blood occlusion, and noise artifacts. This causes segmentation models that simply output category probabilities to easily make overconfident erroneous judgments in high-risk areas. Therefore, in addition to segmentation accuracy, it is also necessary to estimate the uncertainty of the prediction results.
[0003] Currently, medical image segmentation has developed relatively mature processing methods. Existing solutions typically rely on encoders. The decoder, a deep neural network, acts as the main component, representing the input image at multiple scales through multi-layer convolutions or other feature extraction modules. It then utilizes upsampling and skip connections to recover the spatial contours and boundary details of the target region, thereby outputting segmentation results for targets such as organs, lesions, or surgical instruments. To further improve the reliability of the results, some techniques add an uncertainty estimation module to the segmentation framework, expressing the credibility of the segmentation result through statistical analysis of multiple candidate segmentation results.
[0004] Patent CN113362272A utilizes a trained machine learning network to generate multiple candidate segmentation masks from an input medical image. Based on these candidate masks, a final segmentation mask is determined, and an uncertainty metric corresponding to the final segmentation mask is also determined. Patent CN111798458A uses a deep neural network to obtain initial segmentation results and uncertainty estimates. It then automatically recommends two-dimensional slices requiring correction based on the uncertainty, allowing the operator to provide interactive information on the recommended slices. Subsequently, an interactive level set is used to correct the initial segmentation results. Patent WO2020168934A1 acquires medical image data and weakly supervised annotation information of the target object. Based on this annotation information, a pseudo-segmentation mask is determined. Then, based on the pseudo-segmentation mask, medical image data, and a pre-defined mapping model, the final segmentation result of the target object is obtained, thereby reducing reliance on large amounts of manually labeled data. Patent CN114066866B employs a deep U-shaped convolutional neural network model that integrates residual modules and attention mechanisms. It is trained using a hybrid loss function of Dice loss and binary cross-entropy to achieve automatic segmentation of the target region. Patent CN117152168A constructs an image segmentation model, including an image decomposition module, a channel adaptation module, and a backbone network, by acquiring source and target domain datasets. It improves cross-domain medical image segmentation performance by learning domain-invariant features. However, its focus is on frequency band decomposition and cross-domain generalization, primarily addressing domain offset issues caused by different devices, hospitals, or populations, rather than modeling segmentation uncertainty, boundary reliability, and two-dimensional spatial continuity.
[0005] From the common performance of existing technologies in practical applications, although some schemes can output segmentation results and others can provide uncertainty information, the two are mostly sequential rather than unified and collaborative. Existing schemes typically complete segmentation first, then statistically analyze multiple candidate masks at the result level, or introduce subsequent manual corrections and additional judgments based on the uncertainty results. This makes it difficult for uncertainty information to participate in the feature propagation and boundary refinement processes within the network in a timely manner. For situations commonly found in medical images, such as weak boundaries, similar tissue textures, and reflective occlusion, existing technologies often result in a disconnect between segmentation results and reliability representation in practical applications. While a segmentation result and an uncertainty value can be obtained, this uncertainty information cannot be used to specifically adjust high-risk regions during the feature update stage.
[0006] While the aforementioned existing technologies have improved medical image segmentation from different perspectives, they still have significant shortcomings compared to this invention. Their uncertainty mainly stems from the statistical dispersion among multiple candidate outputs, essentially remaining an uncertainty estimation method at the output or posterior level, and does not provide direct constraints on feature updates during the encoding and decoding processes. In contrast, convolutional encoders... While segmentation schemes represented by decoders can improve some segmentation results by leveraging residual structures and attention mechanisms, their feature extraction methods still largely rely on local convolutional modeling, which is insufficient for long-range spatial dependencies and maintaining two-dimensional spatial continuity. Existing technologies generally lack a unified approach that can simultaneously achieve global context modeling, local boundary preservation, and interpretable uncertainty estimation within the same framework. Therefore, in complex medical image scenarios, it remains difficult to simultaneously achieve segmentation accuracy, boundary refinement, and result reliability. Summary of the Invention
[0007] In view of this, the purpose of this invention is to provide a medical image segmentation and uncertainty estimation method and system based on the diffusion principle, and to provide a segmentation method that can simultaneously realize global context information modeling, local boundary detail preservation and pixel-level uncertainty estimation in the medical image segmentation process, so that the obtained uncertainty information can not only reflect the reliability of the segmentation result, but also participate in the feature update regulation in the segmentation process, thereby improving the segmentation stability and result reliability in complex scenarios.
[0008] To achieve the above objectives, the present invention provides the following technical solution: On the one hand, a medical image segmentation and uncertainty estimation method based on the diffusion principle is provided, which includes the following steps: S1. Acquire the medical image to be segmented and perform preprocessing; S2. Multi-scale features are extracted by a multi-scale encoder with several alternating visual state space modules (VSS Block) and an image patch merging layer (Patch Merging). S3. The final decoded features are obtained by multi-scale decoding and skip connection fusion through a multi-stage decoder with several Partial Differential Visual State Space (PVSS) Blocks. Resolution is restored through Patch Expanding, and the PVSS Blocks simultaneously handle long-range dependency modeling and local structure refinement during the decoding stage. The PVSS Blocks have two branches: The first branch obtains gated features through linear layers and activation functions; The second branch processes the enhanced features through linear mapping, depthwise separable convolution, activation functions, SS2D modules, and PDE Operator partial differential lightweight operators. The features output by the first branch and the second branch are multiplied element by element and then fused to obtain the output features. S4. Input the final decoded features into the projection head and the evidence output head to obtain the segmentation probability map and the uncertainty estimate, respectively. S5. Calculate the segmentation loss based on the segmentation probability map and the labeled mask, calculate the negative log marginal likelihood term and the KL divergence regularization term based on the evidence distribution and the true category, and calculate the consistency loss based on the multi-scale gating map and the final uncertainty map. Perform joint optimization training through each loss. S6. Use the trained network architecture for medical image segmentation and uncertainty estimation.
[0009] Further, step S1 includes: Acquiring raw medical images and the corresponding annotation mask during the training phase Input medical images are denoted as , ,in Indicates the image height. Indicates the image width. This indicates the number of input channels; during the training phase, the corresponding labeled mask is also input. , ,in Indicates the number of categories; Preprocessing of raw medical images includes size normalization, intensity normalization, and data augmentation. Size normalization refers to resampling, cropping, or padding the raw image to a preset size. Intensity normalization refers to normalizing pixel values to a preset range. Data augmentation refers to performing random cropping, rotation, flipping, brightness perturbation, Gaussian noise perturbation, and intensity transformation. The processed images are organized into batch tensors and input into the network: multiple samples are organized into batches and input into the network to form tensors. , ,in This refers to the batch size.
[0010] Furthermore, in step S2, the input image is divided into non-overlapping image blocks according to a preset block size, and each image block is linearly projected to obtain initial image features. :
[0011] in, This indicates an image patch embedding operation. , Indicates the initial number of feature channels; Will The input is a multi-scale encoder, and multi-scale features are extracted through a state space module and a patch merging layer. , , and Each stage includes at least two Visual State Space Blocks (VSS Blocks). Spatial downsampling and channel expansion are achieved between adjacent stages through a Patch Merging layer, as shown below:
[0012]
[0013]
[0014]
[0015] in, to This represents the feature extraction process of a four-level encoder. This indicates an image block merging operation.
[0016] Furthermore, in step S3, from the deepest feature layer... The image patch expanding layer is executed, and skip connections are used to fuse the features with the corresponding coding layer features. Within each decoding stage, the fused features are enhanced by multiple Partial Differential Visual State Space (PVSS) Blocks, and the upsampling and fusion process is repeated until the target image resolution is restored. The first three decoding stages each contain at least two PVSS Blocks, and the last decoding stage contains at least one PVSS Block. Each decoding stage restores the resolution through patch expanding and combines it with skip connections at the corresponding scale.
[0017]
[0018]
[0019]
[0020] in, This indicates upsampling or image patch expansion. to This indicates the decoding process at the corresponding stage. This indicates the final decoded features.
[0021] Furthermore, in each partial differential visual state space block (PVSS Block) at each stage, let the input features be... , ,in This indicates the number of channels in that layer; the processing procedure is as follows: For input features After performing layer normalization, we get:
[0022] In the first branch, Input a linear layer and an activation function to obtain gated features. :
[0023] in, Indicates a linear projection layer. Used to adjust the enhanced features extracted from the second branch; In the second branch, enhanced features are obtained by sequentially processing linear mapping, depthwise separable convolution, activation functions, SS2D modules, and the PDEOperator partial differential lightweight operator. :
[0024] in, This indicates a depthwise separable convolution used to input local neighborhood features. This represents the two-dimensional state-space modeling process, used to extract long-range dependency features; This describes the partial differential-based feature update process of the PDE Operator's lightweight partial differential operator. Multiply the first branch output and the second branch output element-wise, then perform linear mixing and add the residuals to obtain the output features of the Partial Differential Visual State Space Block (PVSS Block):
[0025] in, This represents element-wise multiplication. This represents a random depth path used for regularized training. This represents the output feature of the current partial differential vision state space module (PVSS Block).
[0026] Furthermore, a two-dimensional hyperparameter network adjustment module, 2DMam Block, is set in the second branch of the partial differential visual state space module (PVSS Block) to adjust the diffusion intensity parameter of the PDE Operator's partial differential lightweight operator. and biharmonic intensity parameters Pixel-by-pixel adaptive adjustment is performed, and the characteristics of the input two-dimensional hyperparameter network adjustment module 2DMam Block are given by... The parameter adjustment process is as follows: First, the channel dimension is expanded using linear projection, and the features are decoupled into branch one and branch two. Depthwise separable convolution and activation are then performed on branch one, resulting in:
[0027] in, , This indicates the expanded channel dimension; Features Rearrange the sequences into one-dimensional sequences along both the row and column directions. Then, while preserving the two-dimensional spatial structure, perform long-range dependency modeling to obtain:
[0028]
[0029] in, This indicates a row-by-row expansion operation. This indicates an operation to expand by column. This represents the state space scan operator in the row direction. This represents the column-direction state-space scan operator. and This represents restoring a one-dimensional sequence to a two-dimensional feature map; The row and column scan results are fused in two-dimensional space, and a pixel-level adjustment coefficient is generated by combining the two branches. :
[0030] in, This represents a graph showing the diffusion intensity adjustment coefficient. This represents the bitone intensity adjustment coefficient diagram. This represents the auxiliary branch feature used for gated fusion; Contextual features Perform global average pooling, normalization, and linear transformation to generate the basic strength parameters at the global semantic level:
[0031] in, Indicates global average pooling. This indicates that the scalar or vector is expanded to the current feature map space resolution. Indicates the basic diffusion intensity parameter, Indicates the basic biharmonic strength parameter; Combining uncertainty gating graphs Generate the pixel-wise parameters for the final PDE Operator partial differential lightweight operator:
[0032] in, and All of these are parameter diagrams consistent with the current decoding layer spatial resolution.
[0033] Furthermore, the current features are obtained in the PDE Operator of the Partial Differential Visual State Space Module (PVSS Block). and guide images Learning diffusion coefficient , And construct edge suppression weights based on the guide image gradient. Finally, calculate the diffusion term after edge suppression correction. ,pass Perform adaptive diffusion update, and then through Applying bimodal enhancement yields the final output features. The process is represented as follows: First, the discrete states in the visual state space module are expanded into a spatially distributed state field, thereby performing feature updates based on the principle of partial differential equations within the feature domain; let the first... The first channel or the first Each state component is at position The state is Then its basic state is updated as follows:
[0034] in, This indicates the state characteristics of the current time and current position. Indicates the previous state. Indicates the current input features. and Represents the state transition parameters after discretization; Let the guiding image be ,Will and The splicing is performed in the channel dimension, through... Convolution and the Sigmoid activation function learn the local anisotropic diffusion coefficient:
[0035] The diffusion coefficient matrix is then constructed as follows:
[0036] in, express Directional diffusion coefficient, express Directional diffusion coefficient, This represents the diagonal matrix construction operation. Indicates a splicing operation; For the guide image Calculate gradient information Construct edge suppression weights :
[0037] in, Indicates the location of the guide image. gradient at, This represents the squared magnitude of the gradient, used to characterize edge strength. This represents the threshold parameter after processing by the positive value constraint function. These are numerically stable terms greater than 0, used to avoid the denominator being zero. Represents an exponential function; The diffusion coefficient is adjusted using edge suppression weights:
[0038] in, This represents the diffusion coefficient matrix that ultimately participates in the diffusion calculation; Then calculate the diffusion term:
[0039] in, Represents the flux term. Denotes the divergence operator, The feature vector representing the current position; Using the diffusion intensity parameters output by the 2D Dam Block Modulating the diffusion term yields the diffusion-enhanced features:
[0040] in, The diffusion intensity parameter indicates the current location; A bitonic operator is superimposed after adaptive diffusion. High-order smoothing and structure enhancement are performed on the feature map using a biharmonic operator. Defined as:
[0041] in, This represents the biharmonic intensity parameter at the current position. This represents the result after applying a bimodal enhancement to the current feature. This represents the final output of the PDE Operator, a lightweight partial differential operator.
[0042] Furthermore, in step S4, the final decoded features are input into the projection head for... Perform a linear mapping to obtain the segmented logits. The segmentation probability map is obtained through the Softmax function. :
[0043]
[0044] in, Indicates the projection layer. ; The final decoded features are input into the evidence header to obtain the evidence value. and Dirichlet parameters And calculate the prediction probability. Accidental uncertainty Cognitive uncertainty Total uncertainty First, the non-negative evidence value is output. The non-negativity is ensured by the softplus transformation, and then according to... Constructing Dirichlet distribution parameters :
[0045]
[0046] in, Indicates the first The class's raw output logit, Indicates the first The evidence value of a class, Indicates the first The Dirichlet parameter of the class, , This represents the number of categories; a Dirichlet distribution can be constructed based on the parameters of each category:
[0047] in, Represents the category probability vector, Represents a multivariate beta function. Let the Dirichlet parameter vector be represented; then the total evidence quantity is... :
[0048] Then the first The class prediction probability is rewritten as:
[0049] in, Indicates the first The predicted probability of a class; Therefore, the following uncertainty information is obtained: Total uncertainty :
[0050] This includes random uncertainty. and cognitive uncertainty random uncertainty Used to indicate the ambiguity of the data itself:
[0051] in, This can be viewed as the information entropy of the predicted probability distribution; the more dispersed the probability distribution, the more... The larger; Cognitive uncertainty Used to represent the uncertainty caused by insufficient model knowledge:
[0052] in, The smaller the value, the less total evidence there is, and the less certain the model is about the current pixel. The larger.
[0053] Furthermore, in step S5, a composite loss function is employed. At least including the loss of division Loss of evidence and consistency loss :
[0054] in, , , These represent the weighting coefficients of the three loss terms; The segmentation loss consists of Dice Loss and Focal Loss:
[0055] in, and These represent the weights of Dice Loss and Focal Loss, respectively. The preferred loss of evidence includes a negative log-marginal likelihood term and a KL divergence regularization term:
[0056] in, The weights are KL regularization terms; for pixels If its true category is Then the negative logarithmic marginal likelihood term is:
[0057] in, Represents the Digamma function. Indicates the first The total amount of evidence per pixel Indicates the first The Dirichlet parameter of each pixel in the true class; the KL divergence regularization term is expressed as:
[0058] Consistency loss is used to constrain the consistency between the uncertainty gating maps generated at different scale decoding stages and the final uncertainty map at the corresponding scale, and it is expressed as:
[0059] in, Indicates the total number of decoding stages. Indicates batch size, Indicates the first A set of pixels at a scale This indicates the total number of pixels at this scale. Indicates the first The sample at the th Layer position The gating graph values, This indicates scaling from the final total uncertainty plot to the [missing information - likely a specific plot or scale]. Reference values after the layer.
[0060] On the other hand, a system for performing the aforementioned diffusion-based medical image segmentation and uncertainty estimation method is also provided. This system includes an image patch embedding layer, an encoder, a multi-level decoder, a final projection head, and an evidence output head. The image patch embedding layer divides the input medical image into several non-overlapping image patches and maps them to an initial feature sequence. The encoder extracts multi-scale contextual semantic features. The decoder progressively restores spatial resolution and fuses shallow details with deep semantics. The final projection head outputs a segmentation mask. The evidence output head outputs pixel-level evidence values and uncertainty results. The encoder incorporates several VSS Block modules and Patch Merging modules, while the decoder includes several Partial Differential Visual State Space (PVSS) Block modules and Patch Expanding layers. The VSSS Block module in the encoder extracts image features and models long-range dependencies to capture global contextual information. The Patch Merging layer downsamples the feature map, reducing spatial resolution and increasing the number of channels. The PVSS Block in the decoder models long-range dependencies and refines local structures during decoding, while the Patch Expanding layer restores resolution and incorporates skip connection features at the corresponding scale. The PVSS Block has two branches; one branch incorporates an uncertainty-gated branch that adjusts the feature update intensity at each decoding stage using uncertainty information. A PDE Operator (partial differential lightweight operator) performs adaptive diffusion and smoothing enhancement in the feature space, and a two-dimensional hyperparameter network adjustment module adjusts the diffusion intensity parameter in the PDE Operator. and biharmonic intensity parameters Perform pixel-by-pixel adaptive adjustment.
[0061] The beneficial effects of this invention are as follows: First, this invention constructs a Partial Differential Visual State Space (PVSS) Block through the synergistic coupling of a visual state space module and a PDE Operator lightweight partial differential operator, achieving deep compatibility between global feature modeling and local feature optimization. The visual state space module possesses excellent long-range dependency modeling capabilities, effectively capturing cross-regional and long-distance global contextual features in medical images. This solves the problems of limited receptive field and insufficient global information extraction in traditional convolutional networks, as well as the high computational complexity and poor adaptability to medical images of Transformer models. It can accurately model the overall morphology, positional relationships, and semantic distribution features of human organs, lesions, and other targets, significantly improving the accuracy of global semantic segmentation under complex anatomical structures. Simultaneously, the coupled PDE Operator lightweight partial differential operator compensates for the weak local detail depiction capability of the state space model, allowing the feature update process to simultaneously consider global integrity and local refinement, achieving comprehensive optimization of feature representation.
[0062] Secondly, the PDE Operator's lightweight partial differential operator significantly improves the ability to preserve boundaries and refine structures in medical images through its designed adaptive diffusion submodule and biharmonic submodule, effectively improving the accuracy of segmentation details. The adaptive diffusion submodule, relying on features and the guiding image to jointly learn anisotropic diffusion coefficients, can adaptively adjust the diffusion intensity according to the differences in image region features. Simultaneously, it precisely suppresses cross-boundary feature diffusion through edge suppression weights, fundamentally avoiding problems such as blurred target boundaries, confusion of foreground and background features, and annihilation of fine structures caused by traditional diffusion algorithms, accurately preserving the delicate boundary structures of lesions, blood vessels, and thin-walled organs. The biharmonic submodule can perform fine smoothing of feature maps without destroying the target boundary contour, effectively eliminating feature perturbations caused by image noise and artifacts, enhancing the smoothness of the target's internal structure and the ability to express details in boundary regions, perfectly adapting to the complex imaging characteristics of medical images, such as blurred edges, fragmented structures, and uneven grayscale.
[0063] Finally, the pixel-level uncertainty information output by this invention can not only intuitively quantify the confidence level of each pixel segmentation result, providing doctors with a reliable reference for clinical diagnosis and making up for the shortcomings of traditional segmentation methods that only output segmentation results without risk assessment basis; more importantly, this uncertainty information can participate in the feature update regulation of the segmentation process, adaptively optimize feature extraction and iterative update strategies for low-confidence regions, dynamically correct segmentation deviations, and effectively solve the problems of misclassification and omission that easily occur in complex scenarios such as blurred lesion boundaries, overlapping organs, and small gray-scale differences in medical images.
[0064] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description
[0065] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the network architecture of the medical image segmentation and uncertainty estimation method based on the diffusion principle in an embodiment of the present invention; Figure 2 This is a simplified schematic diagram of the Partial Differential Visual State Space Block (PVSS Block) according to an embodiment of the present invention. Figure 3 This is a schematic diagram of the structure of the PDE Operator partial differential lightweight operator and a schematic diagram of the row and column scan parameter adjustment of the 2DMamBlock according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the result of lesion or instrument segmentation in an endoscopic image according to an embodiment of the present invention, wherein... Figure 4 (a) is the original image of the endoscope. Figure 4 (b) is the predicted mask image of the endoscope image. Figure 4 (c) is an image representing the uncertainty estimate of the endoscopic image; Figure 5 This is a schematic diagram illustrating the result of organ region segmentation in a chest X-ray image according to an embodiment of the present invention, wherein... Figure 5 (a) is the original image of the chest X-ray. Figure 5 (b) is the predicted mask image of the chest X-ray. Figure 5 (c) is an image showing the uncertainty estimate of a chest X-ray; Figure 6 This is a schematic diagram illustrating the result of target region segmentation in thyroid or other ultrasound images according to an embodiment of the present invention, wherein... Figure 6 (a) is the original ultrasound image. Figure 6 (b) is the prediction mask of the ultrasound image. Figure 6 (c) is the uncertainty estimation image of the ultrasound image. Detailed Implementation
[0066] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0067] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0068] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.
[0069] Please see Figures 1-6 This paper presents a method and system for medical image segmentation and uncertainty estimation based on the diffusion principle.
[0070] Example 1 This embodiment first describes a medical image segmentation and uncertainty estimation method based on the diffusion principle, such as... Figure 1 As shown, it specifically includes the following steps: S1. Acquire the medical image to be segmented and perform preprocessing, specifically including: Acquiring raw medical images and the corresponding annotation mask during the training phase In this implementation, the input medical image is denoted as... , ,in Indicates the image height. Indicates the image width. Indicates the number of input channels. For RGB images, For grayscale medical images, During the training phase, the corresponding labeled mask is also input. , ,in Indicates the number of categories.
[0071] The input image size is standardized and normalized. Random cropping, rotation, flipping, brightness perturbation, or Gaussian noise enhancement are then performed. Specifically, the preprocessing steps include: size standardization: resampling, cropping, or padding the original image to a preset size; intensity normalization: normalizing pixel values to a preset range, or subtracting the mean and dividing by the standard; and data augmentation: performing random cropping, rotation, flipping, brightness perturbation, Gaussian noise perturbation, and intensity transformation to improve model robustness.
[0072] The processed images are organized into batch tensors and input into the network: multiple samples are organized into batches and input into the network to form tensors. , ,in This refers to the batch size.
[0073] S2. Perform image patch embedding and multi-scale encoding to extract multi-scale semantic features. Specifically, this includes dividing the input image into non-overlapping image patches according to a preset patch size, and performing linear projection on each image patch to obtain initial image features. ; Preferably, in this embodiment, the following is adopted: The non-overlapping block strategy maps the input image to the initial feature representation. :
[0074] in, This indicates an image patch embedding operation. , This represents the initial number of feature channels. The above process can be achieved through convolution with a stride equal to the block size or linear projection, with the aim of mapping the original pixel domain information into embedded features that are more suitable for subsequent feature modeling.
[0075] Will The input is a multi-scale encoder, and multi-scale features are extracted through a state space module and a patch merging layer. , , and In this embodiment, the encoder preferably includes four stages, each outputting multi-scale features. , , , Each stage includes at least two Visual State Space Blocks (VSS Blocks). Spatial downsampling and channel expansion are achieved between adjacent stages through a patch merging layer. This can be represented as:
[0076]
[0077]
[0078]
[0079] in, to This represents the feature extraction process of a four-level encoder. This represents the image patch merging operation. The encoder's role is to progressively expand the receptive field and extract multi-scale semantic features containing rich contextual information.
[0080] S3: Perform multi-scale decoding and skip connection fusion to restore the target image resolution; specifically including: from the deepest feature layers... Patch Expanding is initiated and fused with the corresponding coding layer features via skip connections. Within each decoding stage, the fused features are enhanced through multiple Partial Differential Visual State Space (PVSS) Blocks, and the upsampling and fusion process is repeated until the target image resolution is restored.
[0081] In this embodiment, the decoder preferably adopts an asymmetric structure, including four decoding stages. , , , The first three decoding stages each contain at least two Partial Differential Visual State Space (PVSS) Blocks, while the final decoding stage contains at least one PVSS Block. Each decoding stage recovers resolution through patch expanding and incorporates skip connectivity features at the corresponding scale.
[0082]
[0083]
[0084]
[0085] in, This indicates upsampling or image patch expansion. to This indicates the decoding process at the corresponding stage. This represents the final decoded features. The decoder can simultaneously preserve deep semantic information and shallow boundary details.
[0086] Specifically, in this embodiment, the Partial Differential Visual State Space (PVSS) Block is used to simultaneously handle long-range dependency modeling and local structure refinement during the decoding stage, such as... Figure 2 As shown, it includes at least two branches, where the input features are... , ,in This indicates the number of channels in this layer. The processing procedure of the Partial Differential Visual State Space Block (PVSS Block) is as follows: Feature normalization: normalization of input features After performing layer normalization, we get:
[0087] In the first branch, Input a linear layer and an activation function to obtain gated features. :
[0088] in, Indicates a linear projection layer. Used to adjust the enhanced features extracted from the second branch.
[0089] In the second branch, enhanced features are obtained by sequentially processing linear mapping, depthwise separable convolution, activation functions, SS2D modules, and the PDEOperator partial differential lightweight operator. :
[0090] in, This indicates a depthwise separable convolution used to input local neighborhood features. This represents the two-dimensional state-space modeling process, used to extract long-range dependency features; This represents the partial differential-based feature update process of the PDE Operator's lightweight partial differential operator.
[0091] In each decoding stage, a multi-scale uncertainty gating map is generated and the strength parameters of the PDE Operator are constructed. Specifically, this includes: to enable the network to adjust the feature update strength using uncertainty information, layer normalization is performed on the input features of each decoding stage. Convolution and Sigmoid activation yield pixel-wise gated maps. :
[0092] in, Indicates the first The previous level features are the input to each decoding stage. The representation layer normalization operation is used to stabilize the feature distribution. express Convolution is used to reshape features along the channel dimension. , and They represent the first Height and width of the layer feature map. Gated map. The partial differential visual state space module (PVSS Block) is input into the same stage to control the diffusion intensity and biharmonic intensity of the partial differential update operator in that stage.
[0093] Multiply the first branch output and the second branch output element-wise, then perform linear mixing and add the residuals to obtain the output features of the Partial Differential Visual State Space Block (PVSS Block):
[0094] in, This represents element-wise multiplication. This represents a random depth path used for regularized training. This represents the output feature of the current Partial Differential Visual State Space Block (PVSS Block). Through this structure, the PVSS Block can model long-distance semantic relationships using a state space model, while simultaneously preserving boundary and structural details within the feature domain using partial differential operators.
[0095] Preferably, when the uncertainty in a certain region is high, the network changes the local diffusion intensity and structural enhancement intensity through a gating mechanism; when the uncertainty in a certain region is low, the network maintains relatively stable feature propagation, enabling the model to have stronger adaptive adjustment capabilities in high-risk boundaries, weak contrast regions, and reflective occlusion regions. Specifically, in order to solve the problem that fixed diffusion parameters are difficult to adapt to different spatial locations and different sample structural differences, this embodiment sets a two-dimensional hyperparameter network adjustment module 2DMam Block in the second branch of the partial differential visual state space module PVSS Block to adjust the diffusion intensity parameter in the PDEOperator partial differential lightweight operator. and biharmonic intensity parameters Perform pixel-by-pixel adaptive adjustment. For example... Figure 3 As shown, let the input features of a 2D Dam Block be... , The current context features are fed into a 2D Damp Block to generate pixel-level parameter adjustment coefficients. and And in combination with the basic strength parameters , and gating diagram Generate final parameters and Adaptive adjustment of diffusion intensity parameters per pixel and biharmonic intensity parameters The specific process is as follows: Channel expansion and dual-branch decoupling: First, the channel dimension is expanded using linear projection, and the features are decoupled into branch one and branch two. Depthwise separable convolution and activation are performed on branch one, resulting in:
[0096] in, , This indicates the expanded channel dimension.
[0097] Row and column sequence modeling: In order to perform long-range dependency modeling while preserving the two-dimensional spatial structure, Rearrange the data into a one-dimensional sequence along both the row and column directions, respectively, to obtain:
[0098]
[0099] in, This indicates a row-by-row expansion operation. This indicates an operation to expand by column. This represents the state space scan operator in the row direction. This represents the column-direction state-space scan operator. and This means restoring a one-dimensional sequence to a two-dimensional feature map.
[0100] Pixel-level adjustment coefficient generation: The row and column scan results are fused in two-dimensional space, and the pixel-level adjustment coefficient is generated by combining the second branch. :
[0101] in, This represents a graph showing the diffusion intensity adjustment coefficient. This represents the bitone intensity adjustment coefficient diagram. This represents the auxiliary branch feature used for gating fusion.
[0102] Basic strength parameter generation: In order to generate basic strength parameters at the global semantic level, context features are processed. Performing global average pooling, normalization, and linear transformation yields:
[0103] in, Indicates global average pooling. This indicates that the scalar or vector is expanded to the current feature map space resolution. Indicates the basic diffusion intensity parameter, This represents the basic biharmonic strength parameter.
[0104] Final parameter generation: combining uncertainty gating graphs Generate the pixel-wise parameters for the final PDE Operator partial differential lightweight operator:
[0105] in, and All parameters are consistent with the spatial resolution of the current decoding layer. The aforementioned two-dimensional hyperparameter network is not used to directly output the segmentation result, but rather to adaptively control the pixel-level feature update intensity of the PDEOperator partial differential lightweight operator while maintaining the continuity of the two-dimensional spatial structure.
[0106] In another preferred embodiment, the two-dimensional long-range modeling method of the 2D Dam Block is not limited to the specific implementation of row-direction scanning and column-direction scanning. It can also employ bidirectional scanning or other two-dimensional state-space modeling methods that maintain the continuity of the two-dimensional spatial structure. This is as long as pixel-level parameter adjustment coefficients can be output to control the diffusion intensity parameter. and biharmonic intensity parameters This can be considered an alternative implementation of the present invention.
[0107] Then, based on the generated parameters, feature updates are performed in the Partial Differential Visual State Space Block (PVSS Block) and the PDE Operator lightweight partial differential operator. First, linear projection, depthwise separable convolution, and SS2D modeling are applied to the current input features to obtain contextual features with long-range dependencies; then, the current features are obtained. and guide images Learning diffusion coefficient , And construct edge suppression weights based on the guide image gradient. Finally, calculate the diffusion term after edge suppression correction. ,pass Perform adaptive diffusion update, and then through Applying bimodal enhancement yields the final output features. .like Figure 3 As shown, the specific process includes: Feature State Representation and State Update: To perform feature updates based on the partial differential principle within the feature domain, this invention extends the discrete states in the visual state space module into a spatially distributed state field. Let the first state be... The first channel or the first Each state component is at position The state is Then its basic state is updated as follows:
[0108] in, This indicates the state characteristics of the current time and current position. Indicates the previous state. Indicates the current input features. and This represents the state transition parameters after discretization.
[0109] Diffusion coefficient learning: Let the guiding image be... The guide image can be obtained from the original input image or its downsampled result. In another preferred embodiment, the guide image... The image is not limited to direct downsampling from the original input medical image; it can also be obtained by fusing the original input image with shallow edge features, or by constructing a multi-scale image pyramid. As long as the guiding image can provide structural prior information for diffusion coefficient learning, edge suppression, and adaptive diffusion update in the feature space, it can achieve the same or similar technical objectives as this invention and can be considered as an alternative implementation of this invention.
[0110] Will and The splicing is performed in the channel dimension, through... Convolution and the Sigmoid activation function learn the local anisotropic diffusion coefficient:
[0111] The diffusion coefficient matrix is then constructed as follows:
[0112] in, express Directional diffusion coefficient, express Directional diffusion coefficient, This represents the diagonal matrix construction operation. This indicates the stitching operation. Through this step, the network can adaptively determine the horizontal and vertical diffusion intensity based on the image structure and feature state at the current location.
[0113] Edge suppression weight construction: To prevent the diffusion process from crossing the target boundary and causing boundary blurring, the guide image is... Calculate gradient information Construct edge suppression weights :
[0114] in, Indicates the location of the guide image. gradient at, This represents the squared magnitude of the gradient, used to characterize edge strength. This represents the threshold parameter after processing by the positive value constraint function. These are numerically stable terms greater than 0, used to avoid the denominator being zero. This represents an exponential function. A larger gradient indicates that the current position is more likely to be at a structural boundary. The gradient will decrease to suppress diffusion; a smaller gradient indicates that the current location is more likely to be in a uniform region. Relatively large to allow for more complete and smooth propagation.
[0115] Adaptive diffusion update: Adjusting the diffusion coefficient using edge suppression weights.
[0116] in, This represents the diffusion coefficient matrix that ultimately participates in the diffusion calculation. The diffusion term is then calculated:
[0117] in, Represents the flux term. Denotes the divergence operator, The feature vector representing the current position. This is achieved using the diffusion intensity parameters output from the 2D Dam Block. Modulating the diffusion term yields the diffusion-enhanced features:
[0118] in, This represents the diffusion intensity parameter at the current location. The above steps ensure that the feature propagates smoothly within the region while preserving structural differences at the edges.
[0119] Biharmonic Enhancement: To further improve the feature update capability near edges and in detailed regions, this invention superimposes a biharmonic operator after adaptive diffusion. High-order smoothing and structure enhancement are performed on the feature map. Preferably, a biharmonic operator is used. Defined as:
[0120] in, This represents the biharmonic intensity parameter at the current position. This represents the result after applying a bimodal enhancement to the current feature. This represents the final output of the PDE Operator's lightweight partial differential operator. By simultaneously using adaptive diffusion terms and biharmonic enhancement terms, this invention achieves a balance between intra-regional smoothing, boundary preservation, and refinement of complex structures.
[0121] In another preferred embodiment, the discrete implementation of the PDE Operator's lightweight partial differential operator is not limited to the specific convolution approximation form described herein. Its adaptive diffusion term can be implemented using other anisotropic diffusion methods, and the biharmonic term can be implemented using other higher-order smoothing or higher-order partial differential discretization operators. Any implementation that achieves edge preservation, intra-region smoothing, and structure enhancement can be considered an alternative embodiment of the present invention.
[0122] S4. Output segmentation results and evidence results; input the final decoded features into the projection head and output segmentation logits. The segmentation probability map is obtained through Softmax or Sigmoid. : In a preferred embodiment, the final projection head pair Perform a linear mapping to obtain the segmented logits. The segmentation probability map is obtained through the Softmax function. :
[0123]
[0124] in, Indicates the projection layer. When performing a binary classification task, the Sigmoid function can also be used to obtain the foreground probability map.
[0125] The final decoded features are input into the evidence header to obtain the evidence value. and Dirichlet parameters And further calculate the prediction probability. Accidental uncertainty Cognitive uncertainty Total uncertainty : This invention incorporates an evidence output header at the final output end. Unlike traditional methods that directly output category probabilities, the evidence output header first outputs a non-negative evidence value. The non-negativity is ensured by the softplus transformation, and then according to... Constructing Dirichlet distribution parameters :
[0126]
[0127] in, Indicates the first The class's raw output logit, Indicates the first The evidence value of a class, Indicates the first The Dirichlet parameter of the class, , This represents the number of categories. A Dirichlet distribution can be constructed based on the category parameters:
[0128] in, Represents the category probability vector, Represents a multivariate beta function. This represents the Dirichlet parameter vector. Further, the total amount of evidence is defined. :
[0129] Then the first The class prediction probability can be written as:
[0130] in, Indicates the first The predicted probability of the class. Preferably, the output of this invention includes at least the following uncertainty information: (1) Random uncertainty , used to indicate the ambiguity of the data itself:
[0131] in, This can be viewed as the information entropy of the predicted probability distribution; the more dispersed the probability distribution, the more... The larger.
[0132] (2) Cognitive uncertainty This is used to represent the uncertainty caused by insufficient model knowledge:
[0133] in, The smaller the value, the less total evidence there is, and the less certain the model is about the current pixel. The larger.
[0134] (3) Total uncertainty :
[0135] Based on the above definition, the present invention can simultaneously output segmentation results and pixel-level uncertainty maps, and the uncertainty information can also be used in reverse to affect the gating control in the decoding stage.
[0136] In another preferred embodiment, the evidence output head is not limited to a single-structure linear mapping plus softplus form; other implementations capable of outputting non-negative evidence values and constructing Dirichlet parameters can also be used. Any alternative implementation of the present invention can be considered as long as pixel-level prediction probabilities, random uncertainties, cognitive uncertainties, and total uncertainties can be obtained, and this uncertainty can be used for control during the decoding stage.
[0137] S5. Joint optimization is performed during the training phase. The segmentation loss is calculated based on the segmentation probability map and the labeled mask. The negative log-marginal likelihood term and the KL divergence regularization term are calculated based on the evidence distribution and the true class. The consistency loss is calculated based on the multi-scale gating map and the final uncertainty map. The total loss is obtained by weighted summation of the above loss terms. The network parameters are updated using the backpropagation algorithm. Specifically: To simultaneously improve segmentation accuracy and uncertainty estimation quality, this invention employs a composite loss function. At least including the loss of division Loss of evidence and consistency loss :
[0138] in, , , These represent the weighting coefficients of the three loss terms.
[0139] Preferably, the segmentation loss consists of Dice Loss and Focal Loss:
[0140] in, and These represent the weights of Dice Loss and Focal Loss, respectively.
[0141] The preferred loss of evidence includes a negative log-marginal likelihood term and a KL divergence regularization term:
[0142] in, These are the weights for the KL regularization term. For pixels... If its true category is Then the negative logarithmic marginal likelihood term can be written as:
[0143] in, Represents the Digamma function. Indicates the first The total amount of evidence per pixel Indicates the first The Dirichlet parameter of each pixel in the true class. To avoid overconfident predictions, a KL divergence with the uniform prior Dir(1) is introduced:
[0144] Consistency loss is used to constrain the consistency between the uncertainty gating maps generated at different scale decoding stages and the final uncertainty map at the corresponding scale, thereby enabling the multi-scale decoding process to share a unified uncertainty representation. Its formula is:
[0145] in, Indicates the total number of decoding stages. Indicates batch size, Indicates the first A set of pixels at a scale This indicates the total number of pixels at this scale. Indicates the first The sample at the th Layer position The gating graph values, This indicates scaling from the final total uncertainty plot to the [missing information - likely a specific plot or scale]. Reference values after the layer. Through joint optimization of the above three loss terms, this invention ensures both segmentation accuracy and the interpretability and multi-scale consistency of uncertainty estimation.
[0146] S6. Perform medical image segmentation and uncertainty estimation using the trained network architecture. Input the medical image to be tested and perform the same preprocessing and forward propagation as in the training phase; output the segmentation result. or final mask And simultaneously output uncertainty graph High-risk areas are marked based on uncertainty maps for use in clinical decision support, outcome screening, or risk control in automated systems.
[0147] Example 2 This embodiment provides a system for executing the method of Embodiment 1, which includes at least: an image patch embedding layer, an encoder, a multi-level decoder, a final projection head, and an evidence output head. The image patch embedding layer is used to divide an input medical image into several non-overlapping image patches and map them to an initial feature sequence. The encoder is used to extract multi-scale contextual semantic features. The decoder is used to progressively restore spatial resolution and fuse shallow details with deep semantics. The final projection head is used to output a segmentation mask. The evidence output head is used to output pixel-level evidence values and uncertainty results. The encoder incorporates several VSS Block modules and Patch Merging modules, while the decoder includes several Partial Differential Visual State Space (PVSS) Block modules and Patch Expanding layers. The VSS Block modules extract image features and model long-range dependencies to capture global contextual information. The Patch Merging layer downsamples the feature maps, reducing spatial resolution and increasing the number of channels. The PVSS Block modules in the decoder model long-range dependencies and refine local structures during decoding, while the Patch Expanding layer restores resolution and incorporates skip connection features at corresponding scales. The PVSS Block modules have two branches; one branch incorporates an uncertainty-gated branch that adjusts the feature update intensity at each decoding stage using uncertainty information. A PDE Operator lightweight partial differential operator performs adaptive diffusion and smoothing enhancement in the feature space, and a two-dimensional hyperparameter network adjustment module adjusts the diffusion intensity parameter in the PDE Operator lightweight partial differential operator. and biharmonic intensity parameters Perform pixel-by-pixel adaptive adjustment.
[0148] Example 3 This embodiment applies the methods and systems proposed in Embodiments 1 and 2 to a scenario involving lesion or instrument segmentation in endoscopic images. The endoscopic image is input into the method of this invention, and the result is as follows: Figure 4 As shown, where, Figure 4 (a) is the original image of the endoscope. Figure 4 (b) is the predicted mask image of the endoscope image. Figure 4 (c) is an uncertainty estimation image of the endoscopic image. The encoder extracts instrument and tissue context features, and the decoder automatically enhances diffusion control and boundary refinement capabilities in high uncertainty regions, outputting lesion or instrument segmentation results and corresponding uncertainty maps to assist doctors in judging boundary reliability.
[0149] Example 4 This embodiment applies the methods and systems proposed in Embodiments 1 and 2 to organ region segmentation in chest X-ray images. The chest X-ray image is input into the method of this invention, and the result is as follows: Figure 5 As shown, Figure 5 (a) is the original image of the chest X-ray. Figure 5 (b) is the predicted mask image of the chest X-ray. Figure 5(c) Uncertainty estimation image of chest X-ray. Spatial continuity of chest structure is maintained by using a two-dimensional hypernetwork and PDE Operator partial differential lightweight operator, outputting more stable segmentation results in low-contrast and blurred edge regions, and simultaneously indicating high uncertainty regions.
[0150] Example 5 This embodiment applies the methods and systems proposed in Embodiments 1 and 2 to target region segmentation in thyroid or other ultrasound images, and the results are as follows: Figure 6 As shown, Figure 6 (a) is the original ultrasound image. Figure 6 (b) is the prediction mask of the ultrasound image. Figure 6 (c) is the uncertainty estimation image of the ultrasound image. Due to the presence of speckle noise and weak boundaries in ultrasound images, this invention reduces cross-boundary diffusion by constructing an edge suppression term through guided image gradients and improves structural continuity through biharmonic enhancement, making it suitable for tissue or nodule segmentation in ultrasound scenarios.
[0151] Example 6 This embodiment validates the methods and systems proposed in Embodiments 1 and 2 on publicly available datasets such as Kvasir-SEG, Kvasir-Instrument, Montgomery County CXR, and DDTI. The results show that, compared with existing methods, the present invention achieves improvements in metrics such as DSC, PPV, HD95, and ASD. The improvement in DSC / PPV ranges from approximately 0.43% to 4.40%, and the improvement in HD95 / ASD ranges from approximately 7.41% to 46.16%.
[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A medical image segmentation and uncertainty estimation method based on the diffusion principle, characterized in that: The method includes the following steps: S1. Acquire the medical image to be segmented and perform preprocessing; S2. Multi-scale features are extracted by a multi-scale encoder with several alternating visual state space modules (VSS Block) and an image patch merging layer (PatchMerging). S3. The final decoded features are obtained by multi-scale decoding and skip connection fusion through a multi-stage decoder with several Partial Differential Visual State Space (PVSS) Blocks. Resolution is restored through Patch Expanding, and the PVSS Blocks simultaneously handle long-range dependency modeling and local structure refinement during the decoding stage. The PVSS Blocks have two branches: The first branch obtains gated features through linear layers and activation functions; The second branch processes the enhanced features through linear mapping, depthwise separable convolution, activation functions, SS2D modules, and PDE Operator partial differential lightweight operators. The features output by the first branch and the second branch are multiplied element by element and then fused to obtain the output features. S4. Input the final decoded features into the projection head and the evidence output head to obtain the segmentation probability map and the uncertainty estimate, respectively. S5. Calculate the segmentation loss based on the segmentation probability map and the labeled mask, calculate the negative log marginal likelihood term and the KL divergence regularization term based on the evidence distribution and the true category, and calculate the consistency loss based on the multi-scale gating map and the final uncertainty map. Perform joint optimization training through each loss. S6. Use the trained network architecture for medical image segmentation and uncertainty estimation.
2. The medical image segmentation and uncertainty estimation method based on the diffusion principle according to claim 1, characterized in that: Step S1 includes: Acquiring raw medical images and the corresponding annotation mask during the training phase Input medical images are denoted as , ,in Indicates the image height. Indicates the image width. This indicates the number of input channels; during the training phase, the corresponding labeled mask is also input. , ,in Indicates the number of categories; Preprocessing of raw medical images includes size normalization, intensity normalization, and data augmentation. Size normalization refers to resampling, cropping, or padding the raw image to a preset size. Intensity normalization refers to normalizing pixel values to a preset range. Data augmentation refers to performing random cropping, rotation, flipping, brightness perturbation, Gaussian noise perturbation, and intensity transformation. The processed images are organized into batch tensors and input into the network: multiple samples are organized into batches and input into the network to form tensors. , ,in This refers to the batch size.
3. The medical image segmentation and uncertainty estimation method based on the diffusion principle according to claim 1, characterized in that: In step S2, the input image is divided into non-overlapping image blocks according to a preset block size, and each image block is linearly projected to obtain initial image features. : in, This indicates an image patch embedding operation. , Indicates the initial number of feature channels; Will Input a multi-scale encoder, and extract multi-scale features through a state space module and Patch Merging. , , and Each stage includes at least two Visual State Space Blocks (VSS Blocks). Spatial downsampling and channel expansion are achieved between adjacent stages through a Patch Merging layer, as shown below: in, to This represents the feature extraction process of a four-level encoder. This indicates an image block merging operation.
4. The medical image segmentation and uncertainty estimation method based on the diffusion principle according to claim 1, characterized in that: In step S3, from the deepest feature The process begins with Patch Expanding, which fuses the image patch features with the corresponding coding layer features via skip connections. Within each decoding stage, multiple Partial Differential Visual State Space (PVSS) Blocks enhance the fused features, and the upsampling and fusion process is repeated until the target image resolution is restored. The first three decoding stages each contain at least two PVSS Blocks, and the final decoding stage contains at least one PVSS Block. Each decoding stage restores the resolution through Patch Expanding and combines it with skip connection features at the corresponding scale. in, This indicates upsampling or image patch expansion. to This indicates the decoding process at the corresponding stage. This indicates the final decoded features.
5. The medical image segmentation and uncertainty estimation method based on the diffusion principle according to claim 4, characterized in that: In each partial differential visual state space block (PVSS) at each stage, let the input features be... , ,in This indicates the number of channels in that layer; the processing procedure is as follows: For input features After performing layer normalization, we get: In the first branch, Input a linear layer and an activation function to obtain gated features. : in, Indicates a linear projection layer. Used to adjust the enhanced features extracted from the second branch; In the second branch, enhanced features are obtained by sequentially processing linear mapping, depthwise separable convolution, activation functions, SS2D modules, and the PDEOperator partial differential lightweight operator. : in, This indicates a depthwise separable convolution used to input local neighborhood features. This represents the two-dimensional state-space modeling process, used to extract long-range dependency features; This describes the partial differential-based feature update process of the PDE Operator's lightweight partial differential operator. Multiply the first branch output and the second branch output element-wise, then perform linear mixing and add the residuals to obtain the output features of the Partial Differential Visual State Space Block (PVSS Block): in, This represents element-wise multiplication. This represents a random depth path used for regularized training. This represents the output feature of the current partial differential vision state space module (PVSS Block).
6. The medical image segmentation and uncertainty estimation method based on the diffusion principle according to claim 5, characterized in that: A two-dimensional hyperparameter network adjustment module, 2DMam Block, is set in the second branch of the Partial Differential Visual State Space (PVSS) Block to adjust the diffusion intensity parameter of the PDE Operator's lightweight partial differential operator. and biharmonic intensity parameters Pixel-by-pixel adaptive adjustment is performed, and the characteristics of the input two-dimensional hyperparameter network adjustment module 2DMam Block are given by... The parameter adjustment process is as follows: First, the channel dimension is expanded using linear projection, and the features are decoupled into branch one and branch two. Depthwise separable convolution and activation are then performed on branch one, resulting in: in, , This indicates the expanded channel dimension; Will Rearrange the sequences into one-dimensional sequences along both the row and column directions. Then, while preserving the two-dimensional spatial structure, perform long-range dependency modeling to obtain: in, This indicates a row-by-row expansion operation. This indicates an operation to expand by column. This represents the state space scan operator in the row direction. This represents the column-direction state-space scan operator. and This represents restoring a one-dimensional sequence to a two-dimensional feature map; The row and column scan results are fused in two-dimensional space, and a pixel-level adjustment coefficient is generated by combining the two branches. : in, This represents a graph showing the diffusion intensity adjustment coefficient. This represents the bitone intensity adjustment coefficient diagram. This represents the auxiliary branch feature used for gated fusion; Contextual features Perform global average pooling, normalization, and linear transformation to generate the basic strength parameters at the global semantic level: in, Indicates global average pooling. This indicates that the scalar or vector is expanded to the current feature map space resolution. Indicates the basic diffusion intensity parameter, Indicates the basic biharmonic strength parameter; Combining uncertainty gating graphs Generate the pixel-wise parameters for the final PDE Operator partial differential lightweight operator: in, and All of these are parameter diagrams consistent with the current decoding layer spatial resolution.
7. The medical image segmentation and uncertainty estimation method based on the diffusion principle according to claim 6, characterized in that: In the PDE Operator of the Partial Differential Visual State Space Module (PVSS Block), the current feature is obtained. and guide images Learning diffusion coefficient , And construct edge suppression weights based on the guide image gradient. Finally, calculate the diffusion term after edge suppression correction. ,pass Perform adaptive diffusion update, and then through Applying bimodal enhancement yields the final output features. The process is represented as follows: First, the discrete states in the visual state space module are expanded into a spatially distributed state field, thereby performing feature updates based on the principle of partial differential equations within the feature domain; let the first... The first channel or the first Each state component is at position The state is Then its basic state is updated as follows: in, This indicates the state characteristics of the current time and current position. Indicates the previous state. Indicates the current input features. and Represents the state transition parameters after discretization; Let the guiding image be ,Will and The splicing is performed in the channel dimension, through... Convolution and the Sigmoid activation function learn the local anisotropic diffusion coefficient: The diffusion coefficient matrix is then constructed as follows: in, express Directional diffusion coefficient, express Directional diffusion coefficient, This represents the diagonal matrix construction operation. Indicates a splicing operation; For the guide image Calculate gradient information Construct edge suppression weights : in, Indicates the location of the guide image. gradient at, This represents the squared magnitude of the gradient, used to characterize edge strength. This represents the threshold parameter after processing by the positive value constraint function. These are numerically stable terms greater than 0, used to avoid the denominator being zero. Represents an exponential function; The diffusion coefficient is adjusted using edge suppression weights: in, This represents the diffusion coefficient matrix that ultimately participates in the diffusion calculation; Then calculate the diffusion term: in, Represents the flux term. Denotes the divergence operator, The feature vector representing the current position; Using the diffusion intensity parameters output by the 2D Dam Block Modulating the diffusion term yields the diffusion-enhanced features: in, The diffusion intensity parameter indicates the current location; A bitonic operator is superimposed after adaptive diffusion. High-order smoothing and structure enhancement are performed on the feature map using a biharmonic operator. Defined as: in, This represents the biharmonic intensity parameter at the current position. This represents the result after applying a bimodal enhancement to the current feature. This represents the final output of the PDE Operator, a lightweight partial differential operator.
8. The medical image segmentation and uncertainty estimation method based on the diffusion principle according to claim 1, characterized in that: In step S4, the final decoded features are input into the projection head for... Perform a linear mapping to obtain the segmented logits. The segmentation probability map is obtained through the Softmax function. : in, Indicates the projection layer. ; The final decoded features are input into the evidence header to obtain the evidence value. and Dirichlet parameters And calculate the prediction probability. Accidental uncertainty Cognitive uncertainty Total uncertainty First, the non-negative evidence value is output. The non-negativity is ensured by the softplus transformation, and then according to... Constructing Dirichlet distribution parameters : in, Indicates the first The class's raw output logit, Indicates the first The evidence value of a class, Indicates the first The Dirichlet parameter of the class, , This represents the number of categories; a Dirichlet distribution can be constructed based on the parameters of each category: in, Represents the category probability vector, Represents a multivariate beta function. Let the Dirichlet parameter vector be represented; then the total evidence quantity is... : Then the first The class prediction probability is rewritten as: in, Indicates the first The predicted probability of a class; Therefore, the following uncertainty information is obtained: Total uncertainty : This includes random uncertainty. and cognitive uncertainty random uncertainty Used to indicate the ambiguity of the data itself: in, This can be viewed as the information entropy of the predicted probability distribution; the more dispersed the probability distribution, the more... The larger; Cognitive uncertainty Used to represent the uncertainty caused by insufficient model knowledge: in, The smaller the value, the less total evidence there is, and the less certain the model is about the current pixel. The larger.
9. The medical image segmentation and uncertainty estimation method based on the diffusion principle according to claim 1, characterized in that: In step S5, a composite loss function is used. At least including the loss of division Loss of evidence and consistency loss : in, , , These represent the weighting coefficients of the three loss terms; The segmentation loss consists of Dice Loss and Focal Loss: in, and These represent the weights of Dice Loss and Focal Loss, respectively. The preferred loss of evidence includes a negative log-marginal likelihood term and a KL divergence regularization term: in, The weights are KL regularization terms; for pixels If its true category is Then the negative logarithmic marginal likelihood term is: in, Represents the Digamma function. Indicates the first The total amount of evidence per pixel Indicates the first The Dirichlet parameter of each pixel in the true class; the KL divergence regularization term is expressed as: Consistency loss is used to constrain the consistency between the uncertainty gating maps generated at different scale decoding stages and the final uncertainty map at the corresponding scale, and it is expressed as: in, Indicates the total number of decoding stages. Indicates batch size, Indicates the first A set of pixels at a scale This indicates the total number of pixels at this scale. Indicates the first The sample at the th Layer position The gating graph values, This indicates scaling from the final total uncertainty plot to the [missing information - likely a specific plot or scale]. Reference values after the layer.
10. A system for performing the diffusion-based medical image segmentation and uncertainty estimation method according to any one of claims 1-9, characterized in that: The system includes an image patch embedding layer, an encoder, a multi-level decoder, a final projection head, and an evidence output head. The image patch embedding layer is used to divide the input medical image into several non-overlapping image patches and map them to an initial feature sequence. The encoder is used to extract multi-scale contextual semantic features. The decoder is used to progressively restore spatial resolution and fuse shallow details with deep semantics. The final projection head is used to output a segmentation mask. The evidence output head is used to output pixel-level evidence values and uncertainty results. The encoder incorporates several VSS Block modules and Patch Merging modules, while the decoder includes several Partial Differential Visual State Space (PVSS) Block modules and Patch Expanding layers. The VSSS Block module in the encoder extracts image features and models long-range dependencies to capture global contextual information. The Patch Merging module downsamples the feature map, reducing spatial resolution and increasing the number of channels. The PVSS Block in the decoder models long-range dependencies and refines local structures during decoding, while the Patch Expanding layer restores resolution and incorporates skip connection features at the corresponding scale. The PVSS Block has two branches; one branch incorporates an uncertainty-gated branch that adjusts the feature update intensity at each decoding stage using uncertainty information. A PDE Operator lightweight partial differential operator performs adaptive diffusion and smoothing enhancement in the feature space, and a two-dimensional hyperparameter network adjustment module adjusts the diffusion intensity parameter in the PDE Operator. and biharmonic intensity parameters Perform pixel-by-pixel adaptive adjustment.
Citation Information
Patent Citations
Interactive medical image segmentation method based on uncertainty guidance
CN111798458A
Medical image segmentation with uncertainty estimation
CN113362272A
A method for automatic segmentation of medical images based on deep learning
CN114066866B
Medical image segmentation method based on frequency band decomposition and deep learning
CN117152168A
Medical image segmentation method, apparatus, computer device, and storage medium
WO2020168934A1