Cascade network-based pancreatic tumor segmentation method

Abstract

The invention belongs to the technical field of pancreatic tumor segmentation, and relates to a pancreatic tumor segmentation method based on a cascade network, which comprises a two-stage deep learning framework, in the first stage, coarse segmentation is performed on the whole pancreas based on a multi-scale U-Net backbone network to suppress background interference, and in the second stage, under the guidance of the output of the first stage, the pancreatic tumor segmentation is performed on the whole pancreas based on the multi-scale U-Net backbone network. Focusing in a pancreas area to carry out tumor fine segmentation; the two stages are tightly coupled through an interaction enhancement module, and the interaction enhancement module not only dynamically cuts and optimizes an input region of fine segmentation by using a coarse segmentation result, but also transmits space weight information to realize end-to-end joint optimization; in the second stage, an inter-class shared boundary measurement mechanism is introduced, and a loss function is fused, so that the sensitivity of the model to small-scale tumors and fuzzy boundaries thereof is enhanced; according to the method, it can be ensured that the training process is completely consistent with the reasoning process during actual deployment, performance fluctuation caused by process splitting is effectively avoided, and the stability and reproducibility of the model in a real clinical scene are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of pancreatic tumor segmentation technology, specifically relating to a pancreatic tumor segmentation method based on a cascaded network. Background Technology

[0002] In the field of pancreatic tumor segmentation, existing technologies primarily rely on deep learning algorithms to improve automation. Significant progress has been made using 2D convolutional neural network (CNN) methods, such as combining cascaded CNN structures with local directional gradient coding (RDC) algorithms for efficient liver boundary identification, and applying pyramid dilatation networks to lung and colon segmentation tasks through multi-parallel dilatation convolution strategies. 3D CNN schemes have been developed to address the 3D spatial characteristics of pancreatic tumors, such as using a binary learning framework to integrate tumor feature information from MRI and CT images to enhance feature extraction capabilities from multimodal data. Simultaneously, a pseudo-3D method has been proposed to balance computational efficiency and segmentation accuracy. This method uses stacked adjacent CT slices as input to capture continuity information between slices, avoiding the computational burden of directly processing high-dimensional 3D volumetric data, thus providing a lightweight solution for pancreatic tumor segmentation.

[0003] A two-stage segmentation framework, from coarse to fine, has become the mainstream technique for pancreatic tumor segmentation. In the first stage, a coarse segmentation network generates preliminary localization of the pancreatic region or some target pixels, effectively reducing background interference. The second stage utilizes a fine segmentation network to extract the tumor boundary with high precision. For example, in the coarse segmentation stage, spatial and semantic information is used to generate a small input region to improve the focusing ability of subsequent segmentation; wide-channel convolution and downsampling operations are used to achieve rapid localization of the pancreas; and then a fine segmentation of the tumor is completed using a fine enhancement network model. These existing technologies, through staged collaborative processing, progressively optimize the segmentation results, providing systematic and automated support for the clinical diagnosis of pancreatic tumors.

[0004] However, existing technologies have the following drawbacks: Inconsistent training and testing processes: Existing coarse-to-fine two-stage segmentation methods typically optimize the coarse and fine segmentation networks independently during the training phase, while the two phases need to be cascaded or even iteratively run during the testing phase to obtain the final result. This inconsistency between the training and inference processes makes it difficult for the model to stably reproduce the performance of the training phase in practical applications, affecting the reliability of the segmentation results.

[0005] The lack of an effective information exchange mechanism between stages: The existing two-stage framework only uses the segmentation mask of the first stage as the input or masking condition of the second stage, which cannot make full use of the semantic and spatial context information learned in the coarse segmentation stage. This leads to the fine segmentation stage easily ignoring the global structure or over-focusing on local regions. Especially when pancreatic tumors have blurred boundaries and small volume, the integrity of the segmentation is difficult to guarantee.

[0006] Insufficient ability to model small-scale targets and ambiguous boundaries: Pancreatic tumors are characterized by small size, large morphological variation and unclear boundaries with surrounding tissues. Existing methods generally use standard segmentation loss functions (such as Dice loss and cross-entropy), which make it difficult to assign sufficient learning weights to boundary pixels and small target regions. This results in insufficient sensitivity of the model to fine structures, which can easily lead to missed segmentation or boundary shift.

[0007] Therefore, a pancreatic tumor segmentation method is needed that can eliminate the differences between training and testing processes, construct an efficient inter-stage information transmission mechanism, and enhance global semantic guidance and local focusing capabilities to solve the above-mentioned technical problems. Summary of the Invention

[0008] This invention aims to overcome problems in existing technologies, such as training-test inconsistency, information fragmentation between stages, and insufficient segmentation accuracy for small-scale targets, by using an interactive enhancement module and an inter-class shared boundary measurement mechanism. It is particularly suitable for accurately extracting small-volume, poorly defined pancreatic tumor regions from computed tomography (CT) images.

[0009] This invention provides the following technical solution: a pancreatic tumor segmentation method based on cascaded networks, comprising a two-stage deep learning framework. The first stage of the two-stage deep learning framework performs coarse segmentation of the entire pancreas based on a multi-scale U-Net backbone network to suppress background interference. The second stage of the two-stage deep learning framework, guided by the output of the first stage, focuses on fine segmentation of the tumor within the pancreatic region. The first and second stages are tightly coupled through an interactive enhancement module. The interactive enhancement module not only dynamically trims and optimizes the input region of the fine segmentation using the coarse segmentation results, but also transmits spatial weight information to achieve end-to-end joint optimization. The second stage introduces an inter-class shared boundary metric mechanism and incorporates a loss function to enhance the model's sensitivity to small-scale tumors and their ambiguous boundaries.

[0010] The segmentation process of a two-stage deep learning framework includes the following steps: Step 1: Construct a multi-scale U-Net backbone network to extract multi-level contextual features.

[0011] Step 2: Introduce an interaction enhancement module to achieve efficient collaborative optimization and regional adaptive focusing.

[0012] Step 3: Introduce an inter-class shared boundary metric method to enhance the learning ability of small-scale tumor boundaries.

[0013] Step 4: Define the loss function and the phased model training strategy to achieve efficient collaborative optimization.

[0014] Preferably, the multi-scale U-Net backbone network simultaneously serves the first stage of pancreatic overall segmentation and the second stage of pancreatic tumor fine segmentation; this not only ensures the consistency of the two stages at the feature extraction level, but also lays the structural foundation for subsequent joint optimization and parameter sharing.

[0015] In step 1, the multi-scale U-Net backbone network constructs a feature extraction path with multiple receptive fields by introducing three different sizes of convolutional kernels (1×1, 3×3, and 5×5) in parallel at each convolutional level.

[0016] The multi-scale U-Net backbone network systematically enhances the encoder-decoder structure of the standard U-Net: First, a dual encoder branch structure is adopted in the encoder's contraction path; second, the skip connection mechanism is optimized in the decoder part, integrating the outputs of the two encoding branches into different stages of the decoding process.

[0017] More preferably, the dual-encoder branch structure employed in the encoder's contraction path specifically includes: for the first... There are coded blocks, and the outputs of the two branches are denoted as follows: and The first branch performs conventional feature abstraction through stacked 3×3 convolutions; the second branch fuses 1×1 and 5×5 convolutions and integrates multi-scale information through a concatenation operation; The output of each coded block is:

[0018] in, Indicates use The convolution operation of the convolution kernel, This represents feature concatenation along the channel dimension. This indicates a pooling operation.

[0019] At the deepest layer of the network, the features of the two branches are further fused to form a more discriminative global representation; the central layer output... for:

[0020] in, This is the output of the two branches corresponding to the fourth coded block.

[0021] Integrating the outputs of the two encoding branches into different stages of the decoding process specifically includes: Let... For the first The output of each upsampling block, then the output of its previous stage Refactor as follows:

[0022] in, This indicates the deconvolution operation.

[0023] Preferably, in step 2, the interaction enhancement module uses the pancreas segmentation probability map obtained in the first stage. P As crucial prior information for the second-stage tumor segmentation; the interaction enhancement module constructs an interest graph that effectively integrates spatial attention mechanisms. I This guides the subsequent network to identify and segment the tumor region; interest graph I The generative formula is:

[0024] in, The original input CT image, For parameters Controlled spatial weight transformation function, This indicates an element-wise multiplication operation.

[0025] An adaptive pruning strategy limits the treatment scope to the pancreas and its adjacent regions. This adaptive pruning strategy is achieved by defining a pruning function. To implement, among which The cropped interest map represents the width of the extended edge; it is then fed into a multi-scale U-Net network for final tumor segmentation prediction. The cropped interest map is as follows:

[0026] in, For parameters A multi-scale U-Net network.

[0027] Step 2's entire process starts from the probability map in the first stage, goes through the transformation of the interaction enhancement module, and finally reaches precise segmentation, forming a closed-loop optimization system.

[0028] Preferably, in step 3, an inter-class shared boundary measurement method is introduced and a lightweight boundary-aware module ISBencode is designed to explicitly model the spatial adjacency relationships between different categories, and the measurement results are integrated into the end-to-end training process, thereby enhancing the model's sensitivity to small-scale tumor boundaries.

[0029] ISBencode takes the segmentation probability map output by the multi-scale U-Net in the second stage as input. ISBencode is based on the VGG-16 architecture and is simplified and adapted. It contains three convolutional blocks followed by max pooling layers to progressively downsample the feature map.

[0030] More preferably, in step 4, the overall loss function is:

[0031]

[0032]

[0033] in: These are the balance coefficients, , These represent the losses of the pancreatic segmentation network and the pancreatic tumor segmentation network, respectively. For the loss of ISBencode, Indicates the prediction result. This indicates accurate labeling. This represents the number of pixels that intersect between the predicted result and the actual annotation. It is the inter-class shared boundary metric matrix predicted by ISBencode. It is the true boundary metric matrix calculated based on the actual annotations. Indicates the number of categories.

[0034] Preferably, in step 4, the phased training strategy includes: In the early stages: the interaction enhancement module was turned off, and the pancreatic segmentation network and pancreatic tumor segmentation network were optimized independently; the input images were cropped based on real annotations.

[0035] Mid-term phase: Enable the interaction enhancement module to transfer spatial weight information between the pancreatic segmentation network and the pancreatic tumor segmentation network.

[0036] Later stage: The interactive enhancement module is fully activated, and the prediction results of the pancreatic segmentation network are used as a reference to guide the selection and cropping of the input region of the pancreatic tumor segmentation network.

[0037] The beneficial effects of this invention are: 1. This invention achieves end-to-end joint optimization, eliminates training-test inconsistency, and improves model stability and generalization ability. By designing an interaction enhancement module, this invention introduces the spatial weights and tumor prior information output from the first stage (coarse pancreas segmentation) during the training phase, dynamically adjusts the input features and region range of the second stage (fine tumor segmentation), and achieves end-to-end joint optimization of the two stages. Therefore, this invention can ensure that the training process is completely consistent with the inference process during actual deployment, effectively avoid performance fluctuations caused by process fragmentation, and significantly improve the stability and reproducibility of the model in real clinical scenarios.

[0038] 2. This invention constructs an efficient inter-stage information transmission mechanism to enhance global semantic guidance and local focusing capabilities: The interactive enhancement module of this invention not only transmits segmentation results but also integrates spatial attention weights and location-aware priors to generate an enhanced interest map with tumor perception capabilities, serving as input guidance for the second-stage network. Therefore, this invention enables the fine-grained segmentation network to focus on potential tumor regions under the constraints of the global pancreatic structure, effectively suppressing background interference, especially when the tumor is small and its location is variable, significantly improving the completeness of segmentation and the accuracy of localization.

[0039] 3. This invention enhances the modeling ability of small-scale blurred boundaries, significantly improving the accuracy of edge segmentation: Pancreatic tumors are generally characterized by small volume (often <1% of image area), blurred boundaries, and low contrast with the pancreatic parenchyma. The inter-class shared boundary measurement method proposed in this invention, along with a lightweight ISBencode module, explicitly models the spatial adjacency relationships between tumor and pancreas, and tumor and background, constructing a class-aware boundary refinement loss function. This loss term assigns higher learning weights to shared boundary regions during training, effectively alleviating the gradient smothering problem for small targets, enabling the model to accurately capture blurred boundary details. Experimental results show that this invention can improve DSC from 49.13% to 51.07%; combined with multi-scale features, DSC further reaches 60.24%, with boundary matching significantly better than existing methods.

[0040] 4. This invention balances high accuracy and high efficiency to meet clinical deployment needs: This invention employs a 2D multi-scale U-Net as the unified backbone network. While avoiding the high computational overhead of 3D models, it achieves multi-receptive field feature fusion through parallel 1×1, 3×3, and 5×5 convolutional kernel design, taking into account both local details and global context. Experimental results show that the model of this invention has only 24,175,093 parameters and a training time of 95,252.81 seconds, significantly lower than mainstream 3D methods. While maintaining minimal resource consumption, the model of this invention achieves state-of-the-art performance in both DSC (0.6024) and SPE (0.9998), fully demonstrating its excellent balance between accuracy and efficiency, making it particularly suitable for primary healthcare institutions or real-time assisted diagnostic systems with limited computational resources.

[0041] 5. This invention achieves a leap in performance through systematic synergistic optimization: This invention is not a mere accumulation of single technologies, but rather achieves synergistic gains among modules through a closed-loop design of multi-scale feature extraction → inter-stage interaction enhancement → boundary-aware loss optimization. Ablation experiments show that introducing multi-scale U-Net (M) alone can improve DSC to 53.34%; adding boundary metric (I) reaches 55.96%; and finally, after fusing interaction enhancement (P), it reaches 60.24%, verifying the necessity and advancement of the overall architecture. Compared with the existing best method DB-Net+FB-Net, this invention improves DSC by 1.8%; compared with the basic U-Net, it improves by as much as 11.11%, fully demonstrating the systematic advantages and significant technological progress of this invention. Attached Figure Description

[0042] Figure 1 This is a diagram illustrating the overall architecture of a pancreatic tumor segmentation method based on a cascaded network according to the present invention. Figure 2 This is the multi-scale U-Net diagram of the present invention; Figure 3 This is a diagram of the inter-class shared boundary measurement method of the present invention; Figure 4 Examples of pancreatic tumor segmentation results using different methods of the present invention are shown in the figure, where the red outline represents the predicted result and the green outline represents the actual annotation; Figure 5 This is a visualization of the ablation experiment results of the present invention. Detailed Implementation

[0043] The related technologies of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0044] like Figures 1-5 As shown, the two-stage deep learning framework of this implementation performs coarse segmentation of the pancreas as a whole based on a multi-scale U-Net backbone network in the first stage to effectively suppress background interference. The second stage, guided by the output of the first stage, focuses on fine segmentation of the tumor within the pancreatic region. The two stages are tightly coupled through an interactive enhancement module. This module not only dynamically trims and optimizes the input region for fine segmentation using the coarse segmentation results but also passes spatial weight information to achieve end-to-end joint optimization. Simultaneously, an inter-class shared boundary metric mechanism is introduced in the second stage and incorporated into the loss function to enhance the model's sensitivity to small-scale tumors and their ambiguous boundaries. The overall solution significantly improves the accuracy and robustness of pancreatic tumor segmentation and is suitable for clinical auxiliary diagnostic scenarios.

[0045] The specific steps for dividing the entire model architecture in this implementation method are as follows: Step 1: Construct a multi-scale U-Net backbone network to extract multi-level contextual features.

[0046] To address the challenges posed by pancreatic tumors in CT images, such as their small size, irregular shape, blurred boundaries, and low contrast with surrounding tissues, this invention designs and implements an improved multi-scale U-Net network structure as the shared backbone network for a coarse-to-fine two-stage segmentation framework. This backbone network simultaneously serves the first stage of overall pancreatic segmentation and the second stage of fine segmentation of the pancreatic tumor, ensuring consistency in feature extraction between the two stages and laying a structural foundation for subsequent joint optimization and parameter sharing.

[0047] Traditional U-Net or its variants, based on fixed kernel sizes, are often limited by a single receptive field when processing small-scale objects, making it difficult to balance local details with global semantics. To address this issue, such as... Figure 2 As shown, the multi-scale U-Net proposed in this invention constructs a feature extraction path with multiple receptive fields by introducing three different sizes of convolutional kernels (1×1, 3×3, and 5×5) in parallel at each convolutional layer. This design enables the network to simultaneously capture fine texture, moderate structure, and large-scale contextual information within the same layer, significantly enhancing its ability to represent small tumor regions and their blurred boundaries. Especially in scenarios with extremely low foreground coverage, such as pancreatic tumors, multi-scale feature fusion effectively alleviates the problem of detail loss caused by downsampling and improves the model's sensitivity to key regions.

[0048] In terms of overall architecture, this invention systematically enhances the encoder-decoder structure of the standard U-Net. Firstly, in the encoder's contraction path, the traditional single-branch design is abandoned in favor of a dual-encoder branch structure. Specifically, for the first... The outputs of the two branches of the coded block are denoted as follows: and The first branch primarily uses stacked 3×3 convolutions for conventional feature abstraction, while the second branch fuses 1×1 convolutions (for channel compression and feature recalibration) and 5×5 convolutions (for expanding the receptive field), and integrates multi-scale information through a concatenation operation. The output of a coded block is thus defined as:

[0049] in, Indicates use The convolution operation of the convolution kernel, This represents feature concatenation along the channel dimension. This indicates a pooling operation (such as max pooling or average pooling) used to implement spatial downsampling.

[0050] At the deepest layer of the network (i.e., the bottleneck layer), the features of the two branches are further fused to form a more discriminative global representation. This central layer outputs... The calculation method is as follows:

[0051] in As the output of the two branches corresponding to the fourth coding block, this design not only preserves semantic information from different scale paths, but also enhances the diversity and robustness of features through multi-path convolution combination.

[0052] In the decoder section, this invention further optimizes the skip connection mechanism. Unlike the standard U-Net, which only passes a single encoded feature to the corresponding decoding layer, this invention passes the outputs of two encoding branches... and These are incorporated into different stages of the decoding process. Specifically, let's set... For the first The output of each upsampling block, then the output of its previous stage Refactor as follows:

[0053] in, This indicates a deconvolution (or transposed convolution) operation used to achieve spatial upsampling of the feature map. Through this bipath skip connection, the decoder is able to recover not only high-resolution spatial details (from...) It can also introduce context-enhanced features (from multi-scale fusion) This allows for more precise boundary positioning and structural integrity during the reconstruction process.

[0054] Step 2: Introduce an interaction enhancement module to achieve efficient collaborative optimization and regional adaptive focusing.

[0055] In step one, the present invention has already performed preliminary processing of the input CT image, including basic morphological analysis of the pancreas and coarse localization of the pancreatic tumor. However, to improve the accuracy and robustness of the final segmentation result, it is necessary to further utilize detailed structural information of the pancreas to guide fine segmentation of the pancreatic tumor. Therefore, in this step, the present invention discloses a specially designed interactive enhancement module, which aims to enhance the effect of pancreatic tumor segmentation by optimizing the information flow between stages.

[0056] Specifically, considering that the location of pancreatic tumors is highly dependent on the overall morphology of the pancreas, this invention uses the pancreatic segmentation probability map obtained in the first stage. This serves as crucial prior information for the second-stage tumor segmentation. The core of this step lies in constructing an interest graph that effectively integrates spatial attention mechanisms. This guides subsequent networks to more accurately identify and segment tumor regions.

[0057] Interest Map The formula for generating it is as follows:

[0058] in, It is the original input CT image. Indicates by parameters The controlled spatial weight transformation function, and This indicates an element-wise multiplication operation. This process not only preserves the grayscale information of the original image, but also weights and emphasizes potential tumor regions based on the pancreas segmentation probability map, enabling the model to focus more on the target region.

[0059] Furthermore, to reduce computational burden and improve efficiency, this invention proposes an adaptive pruning strategy that limits the processing scope to the pancreas and its adjacent regions. This strategy is achieved by defining a pruning function. To implement, among which This represents the expanded edge width. The cropped interest map is fed into a multi-scale U-Net network for final tumor segmentation prediction, and its mathematical expression is:

[0060] here, Refers to parameters The multi-scale U-Net network is used to perform pixel-level tumor segmentation tasks. The entire process starts from the probabilistic graph in the first stage, is transformed through the interaction enhancement module, and finally achieves accurate segmentation, forming a closed-loop optimization system that ensures consistency between training and inference and significantly improves the accuracy of pancreatic tumor segmentation.

[0061] Step 3: Introduce an inter-class shared boundary metric to enhance the learning ability of small-scale tumor boundaries.

[0062] In step one, the present invention constructs a multi-scale U-Net backbone network, effectively fusing contextual information from different receptive fields, providing rich feature representations for the segmentation of the pancreas and pancreatic tumors. In step two, the interaction enhancement module realizes semantic connectivity and adaptive focusing of the input region between the two stages, significantly reducing background interference and improving stage synergy. However, even with high-quality feature input and a reasonable region cropping mechanism, pancreatic tumors, due to their small size and blurred boundaries, still struggle to obtain sufficient gradient attention in conventional segmentation loss functions (such as Dice loss or cross-entropy). This leads to insufficient learning of small target boundaries during model training, often resulting in missed detections or edge shifts in the segmentation results.

[0063] To systematically solve this problem, such as Figure 3 As shown, after the second-stage fine segmentation network, this invention further introduces an inter-class shared boundary metric method and designs a lightweight boundary-aware module—ISBencode—to explicitly model the spatial adjacency relationships between different classes. The metric results are then integrated into the end-to-end training process, thereby enhancing the model's sensitivity to small-scale tumor boundaries.

[0064] Specifically, the ISBencode module takes the segmentation probability map output by the multi-scale U-Net in the second stage as input, which has One channel (corresponding to) Categories, such as background, pancreas, pancreatic tumor), with a spatial dimension of ISBencode is a simplified and adapted version of the VGG-16 architecture, containing three convolutional blocks (each block contains two 3×3 convolutional layers and ReLU activation), followed by max pooling layers to progressively downsample the feature maps. To control model complexity, a bottleneck layer is introduced after the third convolutional block, using 1×1 convolutions to compress the number of channels from 256 to 32, effectively reducing the computational burden on subsequent fully connected layers. The feature maps are then flattened and fed into three fully connected layers (FC5, FC6, FC7), with the last layer, FC7, outputting a single... The prediction matrix, denoted as This is used to characterize the pairwise boundary-sharing relationships between each category. Each element of this matrix... Defined as:

[0065] in: This indicates a category index (e.g., 1=background, 2=pancreas, 3=pancreatic tumor). Indicates the category in the true annotation or prediction results. With category The number of pixels that share a boundary between the two regions (i.e., the length of the boundary between two spatially adjacent regions). Indicate category The perimeter of the target area (i.e., its total outline length).

[0066] therefore, Essentially reflects the category What percentage of the boundary corresponds to the category? Adjacency is a normalized inter-class adjacency measure.

[0067] During training, the ISBencode module simultaneously receives data calculated from the actual annotations. As a monitoring signal, the difference between the predicted matrix and the true matrix is ​​calculated using the mean squared error (MSE) loss function:

[0068] This loss term is integrated into the overall optimization objective, along with the pancreas segmentation loss. and pancreatic tumor segmentation loss Joint training forms an end-to-end differentiable optimization closed loop. Because... It is highly sensitive to boundary pixels, especially to the interaction between small targets (such as pancreatic tumors) and their neighboring structures (such as pancreatic parenchyma). Therefore, this mechanism can effectively guide the network to strengthen its ability to model blurred boundaries during training and avoid gradient flooding caused by the low proportion of foreground.

[0069] Step 4: Define the loss function and the phased model training strategy to achieve efficient co-optimization.

[0070] In the preceding steps, this invention detailed the design of the multi-scale U-Net backbone network (Step 1) for extracting rich feature representations of the pancreas and its tumors; the introduction of the interaction enhancement module (Step 2) to realize information flow and region adaptive focusing between the two stages; and the application of the inter-class shared boundary metric method (Step 3) to enhance the segmentation accuracy of small target boundaries by explicitly modeling the spatial adjacency relationships between categories. To ensure that the entire model can effectively learn and utilize the advantages of the above components, this step will elaborate on the specific loss function design and staged training strategy.

[0071] Loss function design The parameter updates of the entire model depend on the loss contributions of three main parts: the loss of the pancreas segmentation network. Loss of pancreatic tumor segmentation network And the loss of the ISBencode module. For segmentation tasks involving the pancreas and pancreatic tumors, Dice loss is used as the optimization objective, and its mathematical expression is as follows:

[0072] in: Indicates the prediction result. This indicates accurate labeling. It calculates the number of pixels that intersect between the predicted result and the true annotation. The 2x in the numerator is to balance the difference in the ratio of positive and negative samples, thereby improving the adaptability to imbalanced datasets.

[0073] Loss of the ISBencode module Defined as predictive Matrix and Reality Mean square error between matrices:

[0074] in: It is the inter-class shared boundary metric matrix predicted by ISBencode. It is the true boundary metric matrix calculated based on the actual annotations. This represents the number of categories. The final overall loss function combines the three loss terms mentioned above and uses a balancing coefficient. , , Perform a weighted summation:

[0075] These balance coefficients allow for adjusting the importance of different loss components according to actual needs to achieve optimal performance.

[0076] Phased training strategy Given that the quality of pancreatic segmentation results directly affects the effectiveness of subsequent pancreatic tumor segmentation, pancreatic prediction may be inaccurate in the early stages of training, potentially introducing excessive noise interference if the interactive enhancement module is applied directly. Therefore, this invention divides the entire training process into three phases for gradual optimization: Early Stage: In this stage, the interaction enhancement module is disabled, and the pancreas segmentation network and pancreatic tumor segmentation network are optimized independently. Input images are cropped based on ground truth annotations to ensure that the pancreas segmentation network obtains high-quality training samples and reduce the negative impact of inaccurate initial predictions.

[0077] Mid-stage: The interaction enhancement module is enabled to transfer spatial weight information between the pancreatic segmentation network and the pancreatic tumor segmentation network, even though the input is still based on real-label clipping at this stage, in order to control potential interference effects and promote collaborative work between the two stages.

[0078] Later stage: The interaction enhancement module is fully activated. At this point, the prediction results of the pancreatic segmentation network are used as a reference to guide the selection and cropping of the input region for the pancreatic tumor segmentation network. This not only enhances the model's adaptability to real-world application scenarios but also makes the training process closer to the testing environment, ensuring consistency between training and inference.

[0079] Through this progressive training strategy, the model can gradually learn feature representations of the pancreas and its internal micro-tumors from coarse to fine, significantly improving overall segmentation performance. Furthermore, this strategy ensures effective integration of all components, enabling the model to reach its maximum potential in complex medical image analysis tasks.

[0080] Step 5: Experimental Verification To verify the effectiveness and superiority of the model of this invention, 13 sets of experiments were designed to compare the model with several of the most advanced pancreatic tumor segmentation methods currently available. The segmentation results are as follows: Figure 4 As shown. These methods include classic 2D and 3D segmentation models such as UNet++, Attention Unet, ResNet50, Unet, nnUnet, and some models specifically for tumor segmentation such as C2FNAS, V-NAS, HyperSegNAS, U-Shiftformer, Aggregation-UNet, DB-Net+FB-Net, 2.5D slice interaction network, and Pranet.

[0081] In all evaluation metrics, the model of this invention demonstrated superior performance. In particular, regarding the key metric of Dice similarity coefficient (DSC), compared to the traditional U-Net model, the model of this invention achieved an improvement of up to 11.11%, with the DSC value increasing from 0.4913 to 0.6024; compared to the current best DB-Net+FB-Net, the DSC value also showed a significant improvement of 1.8%. This significant improvement indicates that the model of this invention has higher accuracy in small-scale pancreatic tumor segmentation. Furthermore, in terms of specificity (SPE), compared to U-Net, the model of this invention improved by 0.27%, reaching nearly 0.9998, indicating that the model of this invention can effectively reduce false positives and more accurately identify background regions.

[0082]

[0083] Table 1 details the performance of different models on the three metrics of DSC, SEN, and SPE. The comparison reveals that while some 3D models, such as nnUnet and HyperSegNAS, perform well in SEN, their performance in DSC and SPE, which better reflect the overall segmentation effect, is inferior to the model of this invention. For example, HyperSegNAS has a DSC of 0.5488, while the model of this invention reaches 0.6024, showing an improvement of approximately 5.36%. Furthermore, although nnUnet performs best in SEN (0.7268), its DSC (0.5456) is still lower than that of the model of this invention, and the model of this invention also outperforms it in SPE.

[0084] Further analysis shows that, in addition to its performance advantages, the model of this invention also has the fewest parameters (24,175,093) and the shortest training time (95,252.81 seconds), significantly lower than mainstream 3D methods (such as HyperSegNAS, which has over 51 million parameters and a training time of nearly 348,000 seconds) (see Table 2). This means that it not only provides higher segmentation accuracy but also leads in computational efficiency, making it very suitable for deployment in resource-constrained clinical environments.

[0085]

[0086] Ablation experiment analysis: like Figure 5 As shown in the visualization results, compared with the basic U-Net model, the method introducing multi-scale feature extraction and inter-class shared boundary measurement mechanism performs significantly better in the pancreatic tumor segmentation task. Among them, the MIP method achieves 60.24% on the DSC index, which is significantly better than other comparison schemes.

[0087] Table 3 presents the ablation study results under different multi-scale convolutional kernel configurations. When using a combination of (1×1, 3×3, 5×5) convolutional kernels, the model achieved optimal performance in all three metrics: DSC (60.24%), SEN (0.7042), and SPE (0.9988). In contrast, configurations of (1×1, 3×3, 7×7) or (1×1, 5×5, 7×7) resulted in performance degradation because excessively large receptive fields over-smooth high-frequency details (such as edges, textures, and microstructures) in the image, weakening the model's sensitivity to small-scale pancreatic tumors. Therefore, the multi-scale design of (1×1, 3×3, 5×5) effectively balances preserving local details with capturing global semantics, thereby improving segmentation accuracy.

[0088]

[0089] Furthermore, as shown in Table 4, simply introducing inter-class shared boundary metric into the U-Net architecture can improve the DSC from 49.13% to 51.07%, verifying the promoting effect of this metric mechanism on small target segmentation—by enhancing the weight of boundary regions in the loss function, it effectively improves the model's ability to identify small tumor structures. Further, the MI method, which combines multi-scale U-Net with inter-class shared boundary metric, further improves the DSC to 55.96%. This indicates that multi-scale feature fusion significantly enhances the model's spatial localization ability for small-scale tumors, providing strong support for high-precision pancreatic tumor segmentation.

[0090]

[0091] Compared with the prior art, this implementation method makes the following improvements in overall architecture design and key module implementation: (1) Introduce an interaction enhancement module to achieve two-stage joint optimization: Existing coarse-to-fine segmentation methods typically train the coarse and fine segmentation stages independently, then cascade them during testing. This leads to a discrepancy between the training objective and the actual inference process, affecting model stability and generalization ability. This implementation design incorporates an interactive enhancement module between the first stage (coarse pancreatic segmentation) and the second stage (fine pancreatic tumor segmentation). This module not only transmits the tumor candidate regions generated in the first stage but also integrates their spatial attention weights and semantic prior information, dynamically adjusting the input feature representation of the second stage. Through this mechanism, end-to-end joint optimization is achieved between the two stages during training, ensuring consistency between training and testing processes and significantly improving model robustness.

[0092] (2) A multi-scale U-Net backbone network is adopted to fuse multi-level contextual information: Unlike traditional single-scale 2D U-Nets or computationally expensive 3D CNNs, this implementation uses a multi-scale U-Net as a unified backbone network, extracting and fusing contextual features at different scales in two stages. This design effectively captures local details of the pancreas and its tumors while modeling long-range spatial dependencies, balancing segmentation accuracy and computational efficiency. It avoids the high hardware resource requirements of 3D methods and outperforms the performance instability of approximation strategies such as pseudo-3D.

[0093] (3) A method for measuring inter-class shared boundaries and a corresponding loss function are proposed to enhance the sensitivity of small target boundaries: Given the small size and indistinct boundaries of pancreatic tumors, existing segmentation loss functions (such as cross-entropy and Dice loss) are insufficient to effectively guide models to focus on subtle boundary regions. Inspired by inter-class boundary modeling, this implementation proposes an inter-class shared boundary metric. By explicitly modeling the shared boundary regions between the tumor, pancreas, and background, a class-aware boundary refinement loss function is constructed. This loss function strengthens the model's learning weights for small-scale boundary pixels during training, effectively improving its ability to discriminate indistinct boundaries, thereby significantly improving the segmentation integrity and edge accuracy of pancreatic tumors.

[0094] (4) Technological innovation of phased and progressive training strategies: The three-stage progressive training strategy in this implementation serves as a key support for the two-stage segmentation architecture. In the early stage, the coarse pancreas segmentation and fine tumor segmentation networks are trained independently, using real-labeled cropped inputs to ensure basic performance. In the mid-stage, the interaction enhancement module is activated, but cropping is still based on real pancreas masks, initially introducing inter-stage collaboration. In the late stage, the prediction results from the first stage are used entirely for adaptive cropping, ensuring consistency between training and inference processes. This strategy constructs a smooth transition mechanism from supervised to self-supervised training and from independent to joint training, effectively avoiding error propagation caused by inaccurate early coarse segmentation, ensuring the stable integration of high-level modules (such as interaction enhancement and boundary metrics), and significantly improving model convergence and generalization ability.

[0095] The key point of this invention is: 1. Design and implementation of the interaction enhancement module: This invention proposes an innovative interactive enhancement module to tightly couple the two stages of coarse pancreatic segmentation and fine pancreatic tumor segmentation. This module not only uses the pancreatic segmentation probability map generated in the first stage as a region prior, but also extracts its spatial attention weights to generate an enhanced interest map with tumor perception capabilities, and dynamically trims the region range input to the second-stage network. Crucially, this module achieves end-to-end joint optimization of the two sub-networks during the training phase, allowing the information output from the coarse segmentation (including location, weights, and semantic cues) to directly participate in the feature construction and loss calculation for fine segmentation. This design completely eliminates the inconsistency problem caused by the "independent training, cascaded testing" process in traditional two-stage methods, significantly improving the model's stability, robustness, and performance reproducibility in actual inference. This is the core mechanism for achieving high-precision segmentation in this invention.

[0096] 2. Structural innovation of the multi-scale U-Net backbone network: To address the challenges posed by the small size, blurred boundaries, and low contrast of pancreatic tumors, this invention designs an improved multi-scale U-Net as a two-stage shared backbone network. This network integrates 1×1, 3×3, and 5×5 convolutional kernels in parallel within each encoding block, constructing multiple receptive field feature extraction paths to simultaneously capture local details, mesoscopic structure, and global contextual information at the same level. Simultaneously, the encoder employs a dual-branch structure, and the decoder introduces dual-path skip connections, further enhancing the fusion capability of multi-scale features. This design effectively alleviates the problem of information loss in small targets due to downsampling without introducing 3D convolutions, significantly improves sensitivity to small tumor regions, balances segmentation accuracy and computational efficiency, and avoids the excessive reliance on hardware resources inherent in high-dimensional 3D models.

[0097] 3. Inter-class shared boundary measurement methods and boundary-aware loss functions: To address the problem that existing segmentation loss functions are insufficient to effectively guide models to focus on small-scale blurred boundaries, this invention proposes an inter-class shared boundary metric method, along with a lightweight ISBencode module. This module takes a segmentation probability map as input, explicitly models the spatial adjacency relationships between tumors and categories such as pancreas and background, and outputs a normalized inter-class boundary sharing matrix. This matrix serves as a supervisory signal, jointly optimized with the main segmentation task through mean squared error loss, enabling the model to actively strengthen the learned weights for boundary pixels during training. This mechanism effectively alleviates the gradient drowning phenomenon for small targets, significantly improves the ability to discriminate blurred boundaries of pancreatic tumors and the accuracy of segmentation edges. Experiments verify that it has a significant gain on the DSC index, making it a key technical means to improve the performance of small target segmentation.

[0098] 4. Phased and progressive training strategy: To ensure stable convergence and efficient learning of the complex two-stage architecture, this invention designs a three-stage progressive training strategy. In the early stage, the interaction enhancement module is disabled, and the pancreas and tumor segmentation networks are trained independently with real-world labeled inputs to ensure basic performance. In the mid-stage, the interaction module is enabled, but the actual pancreas region is still used as the cropping basis, initially establishing inter-stage collaboration. In the late stage, adaptive cropping is performed entirely based on the prediction results of the first stage, achieving complete alignment between training and testing processes. This strategy effectively avoids noise interference caused by inaccurate coarse segmentation in the early stages, ensures the effective integration of advanced mechanisms such as interaction enhancement and boundary measurement, and is a crucial support for the overall high-performance model.

[0099] In summary, this invention ensures that the training process is completely consistent with the inference process during actual deployment, effectively avoiding performance fluctuations caused by process fragmentation, and significantly improving the stability and reproducibility of the model in real clinical scenarios.

[0100] It should be emphasized that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A pancreatic tumor segmentation method based on cascaded networks, characterized in that, The system includes a two-stage deep learning framework. The first stage of this framework performs coarse segmentation of the entire pancreas based on a multi-scale U-Net backbone network to suppress background interference. The second stage, guided by the output of the first stage, focuses on fine segmentation of the tumor within the pancreatic region. The first and second stages are tightly coupled through an interactive enhancement module. This module not only dynamically trims and optimizes the input region for fine segmentation using the coarse segmentation results but also transmits spatial weight information to achieve end-to-end joint optimization. The second stage introduces an inter-class shared boundary metric mechanism and incorporates a loss function to enhance the model's sensitivity to small-scale tumors and their ambiguous boundaries. The segmentation process of the two-stage deep learning framework includes the following steps: Step 1: Construct a multi-scale U-Net backbone network to extract multi-level contextual features; Step 2: Introduce an interaction enhancement module to achieve efficient collaborative optimization and region adaptive focusing; Step 3: Introduce an inter-class shared boundary metric method to enhance the learning ability of small-scale tumor boundaries; Step 4: Define the loss function and the phased model training strategy to achieve efficient collaborative optimization.

2. The pancreatic tumor segmentation method based on a cascaded network according to claim 1, characterized in that, The multi-scale U-Net backbone network simultaneously serves the first stage of overall pancreatic segmentation and the second stage of fine segmentation of pancreatic tumors. In step 1, the multi-scale U-Net backbone network constructs a feature extraction path with multiple receptive fields by introducing three different sizes of convolutional kernels (1×1, 3×3, and 5×5) in parallel at each convolutional layer. The multi-scale U-Net backbone network systematically enhances the encoder-decoder structure of the standard U-Net: first, it adopts a dual encoder branch structure in the encoder's contraction path; second, it optimizes the skip connection mechanism in the decoder part, integrating the outputs of the two encoding branches into different stages of the decoding process.

3. The pancreatic tumor segmentation method based on a cascaded network according to claim 2, characterized in that, The dual-encoder branch structure employed in the encoder's contraction path specifically includes: for the first... There are coded blocks, and the outputs of the two branches are denoted as follows: and The first branch performs conventional feature abstraction through stacked 3×3 convolutions; the second branch fuses 1×1 and 5×5 convolutions and integrates multi-scale information through a concatenation operation; The output of each coded block is: in, Indicates use The convolution operation of the convolution kernel, This represents feature concatenation along the channel dimension. Indicates pooling operation; At the deepest layer of the network, the features of the two branches are further fused to form a more discriminative global representation; the central layer output... for: in, This is the output of the two branches corresponding to the fourth coded block; The specific steps of integrating the outputs of the two encoding branches into different stages of the decoding process include: setting... For the first The output of each upsampling block, then the output of its previous stage Refactor as follows: in, This indicates the deconvolution operation.

4. The pancreatic tumor segmentation method based on a cascaded network according to claim 1, characterized in that, In step 2, the interaction enhancement module uses the pancreas segmentation probability map obtained in the first stage. P As important prior information for the second stage of tumor segmentation; The interaction enhancement module constructs an interest graph that can effectively integrate spatial attention mechanisms. I This guides the subsequent network to identify and segment the tumor region; the interest map I The generative formula is: in, The original input CT image, For parameters Controlled spatial weight transformation function, This represents an element-wise multiplication operation; An adaptive pruning strategy limits the processing scope to the pancreas and its adjacent regions. This adaptive pruning strategy defines a pruning function. To implement, among which The cropped interest map represents the width of the extended edge; it is then fed into a multi-scale U-Net network for final tumor segmentation prediction. in, For parameters Multi-scale U-Net network; The entire process of step 2 starts from the probability map in the first stage, goes through the transformation of the interaction enhancement module, and finally reaches the precise segmentation, forming a closed-loop optimization system.

5. The pancreatic tumor segmentation method based on a cascaded network according to claim 1, characterized in that, In step 3, an inter-class shared boundary measurement method is introduced and a lightweight boundary-aware module ISBencode is designed to explicitly model the spatial adjacency relationship between different categories and integrate the measurement results into the end-to-end training process, thereby enhancing the model's sensitivity to small-scale tumor boundaries. The ISBencode takes the segmentation probability map output by the multi-scale U-Net in the second stage as input. The ISBencode is based on the VGG-16 architecture and is simplified and adapted. It contains three convolutional blocks followed by a max pooling layer to progressively downsample the feature map.

6. The pancreatic tumor segmentation method based on a cascaded network according to claim 5, characterized in that, In step 4, the overall loss function is: in: These are the balance coefficients, , These represent the losses of the pancreatic segmentation network and the pancreatic tumor segmentation network, respectively. For the loss of ISBencode, Indicates the prediction result. This indicates accurate labeling. This represents the number of pixels that intersect between the predicted result and the actual annotation. It is the inter-class shared boundary metric matrix predicted by ISBencode. It is the true boundary metric matrix calculated based on the actual annotations. Indicates the number of categories.

7. The pancreatic tumor segmentation method based on a cascaded network according to claim 1, characterized in that, In step 4, the phased training strategy includes: Early stage: The interaction enhancement module was turned off, and the pancreatic segmentation network and pancreatic tumor segmentation network were optimized independently; the input image was cropped based on the real annotation; Mid-term phase: Enable the interaction enhancement module to transfer spatial weight information between the pancreatic segmentation network and the pancreatic tumor segmentation network; Later stage: The interactive enhancement module is fully activated, and the prediction results of the pancreatic segmentation network are used as a reference to guide the selection and cropping of the input region of the pancreatic tumor segmentation network.

Images