CT image segmentation and classification system based on segmentation feature guidance

Abstract

The invention belongs to the technical field of medical image processing, and discloses a CT image segmentation and classification system based on segmentation feature guidance, and the specific technical scheme is as follows: the system adopts a shared encoder to extract general features, realizes collaborative optimization of segmentation and classification through a double decoding path, adopts a partial decoder in a segmentation path, and adopts a partial decoder in the segmentation path; in combination with a local feature attention module, through multi-scale feature fusion and a boundary perception mechanism, the region consistency of a global segmentation map is gradually optimized, edge detail information is supplemented, and a classification path generates a space attention weight through a segmentation feature guide module by utilizing segmentation prediction; the classification network is guided to focus on a focus area and suppress background interference, a self-adaptive loss weighting strategy based on multi-task learning is adopted, the double-task gradient flow is dynamically balanced, and the gradient competition problem in the multi-task learning is effectively relieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to a CT image segmentation and classification system guided by segmentation features. Background Art

[0002] Gastrointestinal malignancies are common clinical malignancies, primarily including gastric cancer and colorectal cancer. Among gastrointestinal mesenchymal tumors, gastrointestinal stromal tumors (GISTs) are the most common potentially malignant mesenchymal tumors, exhibiting significant heterogeneity in their biological behavior. These tumors originate from interstitial cells of Cajal and have a distinct anatomical distribution pattern: they are most commonly found in the gastric body and fundus, accounting for 50-70% of cases; followed by the small intestine, accounting for 20-30% of cases; the colorectum, accounting for approximately 10% of cases; and the esophagus, with a lowest incidence of only 3-5%. Notably, GISTs must be differentiated from benign mesenchymal tumors such as gastric leiomyoma and gastric schwannoma. Gastric leiomyoma (GL), a representative benign mesenchymal tumor, arises from the smooth muscle tissue of the muscularis propria or muscularis mucosa and accounts for the highest proportion of benign gastric tumors. Gastric schwannoma (GS), a rare tumor arising from Schwann cells of the nerve sheath, accounts for only 0.2% of all schwannomas. Because GS's imaging findings are highly similar to other mesenchymal tumors, the clinical misdiagnosis rate is high. Furthermore, ectopic pancreas (EP), a congenital developmental abnormality, occurs in the stomach and duodenum in over 70% of cases. Due to the small size of the lesions and the lack of specific symptoms, EP is easily misdiagnosed as a stromal tumor or leiomyoma.

[0003] In clinical practice, the diagnosis of gastrointestinal tumors relies on the application of medical imaging technologies. Currently, mainstream imaging modalities include computed tomography (CT), magnetic resonance imaging (MRI), endoscopic ultrasound (EUS), and positron emission tomography (PET). CT has become a routine imaging modality for gastrointestinal tumor screening due to its high image density and resolution, ease of examination, speed, and non-invasive nature. In clinical practice, physicians base their diagnoses and treatment plans on CT reports produced by radiologists. Even experienced radiologists can only diagnose an average of seven chest CT scans per hour. However, this manual, slice-by-slice review process is time-consuming and heavily reliant on the radiologist's experience. Developing a noninvasive, rapid method for accurately localizing gastrointestinal tumor types and tumor regions would facilitate early assessment of gastrointestinal tumors, enabling patients with these tumors to receive treatment earlier and more quickly. Accurate tumor segmentation is not only helpful for diagnosis but also valuable for assessing disease severity and prognosis. Therefore, effective computer-aided diagnosis (CAD) systems for CT images are crucial for assisting physicians in selecting appropriate surgical approaches and prognostic strategies.

[0004] The current field of medical image analysis faces the following key challenges: First, data acquisition and annotation bottlenecks. Medical imaging data has strong privacy implications and is expensive to collect, resulting in a scarcity of available training samples. More critically, segmentation model training relies heavily on pixel-level annotated data, which requires layer-by-layer delineation by experienced radiologists, consuming significant time and labor. This dual limitation directly hinders the quality and quantity of training data required for parameter optimization in deep learning models. Second, the complexity of tumor representation and the uncertainty of imaging features. Tumors of different pathological types are highly similar in morphology, volume, and anatomical location; tumors of the same type can vary significantly between individuals. Partial volume effects, motion artifacts, and low signal-to-noise ratios, common in CT imaging, lead to discontinuous tumor boundaries, significantly increasing segmentation uncertainty. Third, there is a lack of multi-task coordination mechanisms. Existing research often employs independent segmentation or classification task architectures, ignoring the inherent correlation between the two in feature space. In reality, tumor morphology is strongly correlated with pathological grade, but traditional single-task models fail to effectively leverage this cross-task semantic consistency, resulting in limited model generalization.

[0005] Research on intelligent tumor analysis technologies based on medical imaging has significant clinical value. First, automated and precise segmentation of CT images can significantly improve diagnostic and treatment efficiency. Deep learning methods enable automatic identification and delineation of tumor regions, not only assisting physicians in rapidly localizing lesions and quantitatively assessing tumor morphological characteristics, but also effectively reducing the workload of manual annotation, providing a reliable quantitative basis for subsequent treatment planning. Second, the development of effective computer-assisted diagnosis systems for CT images plays a crucial role in supporting clinical decision-making. Intelligent analysis systems that integrate radiomics features with tumor classification information can provide data-driven decision support for key aspects such as surgical procedure selection and prognosis assessment through multidimensional feature analysis, which is of great significance for improving the scientific nature of diagnosis and treatment plans. It is worth emphasizing that the collaborative optimization of tumor segmentation and classification tasks can yield significant performance gains. A multi-task learning framework is constructed to mutually enhance the two tasks through their inherent connection: on the one hand, spatial tumor features such as texture and edges extracted by the segmentation network provide key anatomical evidence for type discrimination; on the other hand, the semantic features and category information learned by the classification network effectively guide the segmentation network to focus on regions of greater diagnostic value.

[0006] Traditional medical image segmentation methods mainly rely on low-level features at the pixel level. When faced with complex scenes with low contrast between lesions and normal tissues and blurred boundaries, it is often difficult to achieve ideal diagnostic performance. This limitation stems from the fact that traditional methods lack the ability to understand the deep semantic information of images, making it difficult to effectively model the complex anatomical structure relationships in medical images. The effective design of the feature extraction module is a key factor in improving model performance, but this method also leads to a significant increase in the number of network parameters due to the introduction of a dense connection mechanism, which affects the lightweight nature of the model. Existing methods generally have the limitation of single task and fail to effectively explore the inherent consistency between classification and segmentation tasks to achieve synergistic enhancement of the two.

[0007] In the serial collaboration method of non-end-to-end serial collaboration, a phased optimization strategy is usually adopted to improve model performance through iterative feedback. In the serial architecture, each task is configured with an independent network module and connected in series through information flow. Specifically, the basic model is first trained for the priority task, and its output primary feature map and prediction results are used as the enhanced input of the secondary task. For example, a coarse segmentation network is used to generate a coarse lesion mask to enhance the lesion localization and differentiation ability of the classification network, and then the improved localization ability is transferred from the classification network to the enhanced segmentation network to achieve accurate lesion segmentation. Although this approach has achieved good performance, since each model in the network needs to be trained separately, the model training is complicated and the computational cost is high.

[0008] End-to-end parallel collaborative approaches simultaneously improve classification and segmentation performance by constructing a joint optimization objective. Parallel architectures typically employ a shared encoder and multiple decoders. The encoder network extracts common features across multiple tasks and simultaneously feeds these features into the segmentation and classification branches. Each branch then extracts differentiated representations through task-specific feature transformation layers, outputting segmentation and classification results. This parallel feature interaction strategy effectively improves the feature representation capabilities of multi-task learning while maintaining task independence. However, this approach only implicitly models the segmentation and classification tasks within the multi-task network, resulting in limited performance for multi-task models.

[0009] In summary, current multi-task collaborative algorithms based on deep learning for medical image segmentation and classification have achieved remarkable results, fully demonstrating the effectiveness of the collaborative nature of segmentation and classification tasks. However, existing multi-task methods have not fully explored the potential benefits of segmentation and classification results for gastrointestinal tumor classification and segmentation tasks, and have often adopted an isolated task paradigm, resulting in insufficient cross-task feature interaction. Summary of the Invention

[0010] In order to solve the technical problems existing in the prior art, the present invention provides a CT image segmentation and classification system based on segmentation feature guidance, which adopts a parallel task architecture to realize multi-task joint optimization of gastrointestinal tumor classification and segmentation, and fully utilizes the intrinsic correlation between segmentation and classification tasks in the feature space to improve the performance of both. The network model extracts common features through a shared encoder, designs a segmentation feature guidance module to inject predicted mask information into the classification sub-network, effectively suppresses background noise interference, introduces a local feature attention mechanism in the segmentation branch, combines partial decoders to progressively optimize boundary features, and balances the multi-task optimization process through an adaptive loss weight strategy, achieving better segmentation and classification performance on the gastrointestinal tumor dataset.

[0011] To achieve the above objectives, the technical solution adopted by the present invention is: based on the segmentation feature-guided CT image segmentation and classification system, a feature-guided multi-task segmentation network is constructed, a dual-path collaborative learning paradigm is adopted, and the joint segmentation and classification of gastric tumor CT images is realized through feature sharing and task interaction mechanism. The constructed feature-guided multi-task segmentation network consists of a tumor segmentation subnet based on an encoder-decoder, a local feature attention module with enhanced boundary perception, a segmentation feature guidance module with cross-task feature interaction, and a tumor classification subnet that integrates multi-level tumor regional features.

[0012] The tumor segmentation subnet uses the VGG19 architecture as a shared encoder, and uses its multi-level feature extraction mechanism to effectively extract the multi-scale features of gastrointestinal tumors. The encoder contains 5 hierarchical convolution stages, and the output feature map of each stage is recorded as , whose spatial resolution is Decreasing step by step. Deep features Contains high-level semantic information, while shallow features Rich texture details are preserved. Compared to shallow features containing detailed information, as the network depth increases, the network can extract deep features containing more semantic information. At the encoder end, a multi-branch dilated pyramid pooling module with dilation rates of 6 / 12 / 18 is used to enhance feature representation capabilities: based on the standard dilated convolution branch, a non-local operation branch is added. The parallel dilated convolution branch constructs multi-scale contextual information, and the non-local branch captures long-range dependencies of feature maps through a self-attention mechanism.

[0013] The segmentation decoder adopts a progressive feature fusion architecture, which consists of 4 cascaded decoding units and the final segmentation prediction head. The initial decoding stage uses the deep semantic features output by the Atrous Spatial Pyramid Pooling module (ASPP) As the initial input, the spatial resolution is restored step by step through the feature pyramid. Decoding Unit Receive current layer features and the corresponding encoding stage characteristics As input: First Perform bilinear interpolation upsampling to The spatial resolution is then achieved through channel splicing to achieve feature fusion, and then the fusion features are extracted through 3×3 convolution stacking to finally output the upper layer decoding features. After four stages of decoding, the network reconstructs a high-level feature map with the same resolution as the original input image. The final segmentation prediction head uses a two-stage processing flow: first, the number of feature channels is compressed to 1 dimension through a 1×1 convolution layer, and then a sigmoid activation function is applied to generate a spatial probability map. , where H and W correspond to the height and width of the input image, respectively. This prediction map can be thresholded to obtain pixel-level tumor segmentation results, fully realizing the end-to-end mapping from deep semantic features to pixel-level positioning.

[0014] In order to effectively improve the boundary positioning accuracy and regional consistency of medical image segmentation, a local feature attention module is designed as a feature enhancement mechanism. The local feature attention module systematically fuses multi-scale decoding features with global context information by constructing a dual-path attention mechanism. Specifically, the initial global segmentation map is first generated by aggregating deep features through a partial decoder. , and then adopt a progressive feature fusion strategy to gradually integrate decoding features of different levels through multiple cascaded LFA modules , to achieve fine-grained correction of the segmentation results.

[0015] The LFA module achieves feature enhancement through a dual-branch attention mechanism: first, based on the global segmentation map or multi-level LFA module output Generate spatial attention weights. The uncertain region detection branch strengthens the decision-ambiguous region with a segmentation probability close to 0.5, while the boundary attention branch mines the tumor information ignored in the background region of the coarse segmentation result through a flipping operation:

[0016] (1)

[0017] (2)

[0018] Where, is the upsampling operation, is the Sigmoid function, is the inverse operation, and Represent the generated uncertainty perception area and boundary perception area attention weight map, Represents the global segmentation map or the output of the previous LFA module. In the feature fusion stage, the two attention weight maps are respectively combined with the current layer decoding features. Perform spatial weighted operations to generate enhanced features with spatial selectivity. Finally, feature fusion is achieved through channel splicing and 1×1 convolution, and residual connection is performed with the coarse segmentation result of the previous level to obtain the optimized segmentation prediction map:

[0019] (3)

[0020] Where, represents a 1×1 convolution, Indicates channel cascade, This design enables the model to adaptively focus on fuzzy boundary areas and low-confidence areas, significantly improving segmentation accuracy through multi-level feature iterative optimization.

[0021] The segmentation feature guidance module embeds the segmentation features into the classification network through a multi-level feature fusion mechanism, effectively enhancing the classification feature's ability to focus on the lesion area.

[0022] The SFG module is a dual-input architecture: segmentation prediction and multi-level encoding features The output end generates a multi-level classification feature with enhanced characteristics of the lesion area Specifically, for the multi-level features extracted by the shared encoder First, channel normalization is performed through 1×1 convolution to generate basic classification features with unified channel dimensions , ensuring that multi-scale features have equal importance in the classification subnetwork.

[0023] The SFG module consists of two core stages: Downsample to The spatial dimension is then multiplied element by element to enhance the spatial features, and the enhanced features are fused with the original features:

[0024] (4)

[0025] Where, represents the downsampling operation, represents element-wise product, It is a channel dimension splicing operation, and conv represents the feature fusion of 3×3 convolution kernel.

[0026] Construct a spatial-channel collaborative attention mechanism to further optimize feature expression: capture spatial statistical features and channel fusion information through parallel global average pooling and global maximum pooling, and finally achieve feature refinement through residual connection. The specific formula is as follows:

[0027] (5)

[0028] (6)

[0029] (7)

[0030] Where, represents the Sigmoid function, ⊙ is the element-by-element multiplication, and represents global average pooling and maximum pooling in the channel dimension, and Represents global average pooling and maximum pooling in the spatial dimension. This dual attention mechanism significantly improves the ability of classification features to identify tiny lesions through collaborative optimization of the spatial-channel dimension.

[0031] This subnet adopts a top-down progressive feature fusion strategy, and the input features are multi-level classification features processed by the segmentation feature guidance module. Specifically, we first use a 3×3 convolutional layer with a stride of 2 to Downsample by a factor of two to make its spatial dimension equal to Keep it consistent. Then it is achieved by feature concatenation and 1×1 convolution operation. and The feature fusion of the cascaded layer is expanded downward layer by layer, and the fusion process is integrated in sequence. and After multi-level feature fusion, the feature of multi-granularity classification information is finally generated. This feature contains both deep semantic abstract information and shallow fine-grained anatomical features to form a discriminative multi-scale feature representation. Finally, the global average pooling layer is used to The spatial dimension is compressed, and after mapping through the fully connected layer, the tumor classification probability distribution is output through the softmax function.

[0032] A joint supervision strategy of Dice loss and binary cross entropy loss (BCE) is adopted to constrain the four side outputs through a deep supervision mechanism. , which is bilinearly upsampled to the gold standard annotated by clinical experts The same size as , the segmentation loss is composed of the Dice similarity coefficient loss and the pixel-level cross entropy loss, and its expression is:

[0033] (8)

[0034] Where, represents the side output segmentation mask after upsampling adjustment, By calculating the pixel distribution overlap between the predicted result and the true mask, the class imbalance problem is alleviated. It provides pixel-level classification constraints. For classification tasks, the standard cross entropy loss is used to measure the difference between the classification prediction and the true label:

[0035] (9)

[0036] Where, represents the total number of tumor categories for the gastrointestinal tumor classification task, and Respectively represent the samples as The true label of the class and the network prediction Class probability estimates.

[0037] To dynamically coordinate the optimization process of segmentation and classification tasks, an uncertainty-based adaptive weight adjustment mechanism is introduced. This strategy automatically adjusts the loss weight of each task through learnable parameters. The total loss function for combined gastrointestinal tumor classification and segmentation is:

[0038] (10)

[0039] Where, and are the learnable weight parameters for the segmentation and classification tasks, respectively, and their initial values ​​are set to 1. Through the automatic optimization of the parameters during back propagation, the network can autonomously adjust the loss weight ratio according to the learning difficulty of the two tasks, achieving end-to-end multi-task balanced optimization. The logarithmic regularization term It can effectively prevent the weight parameter from becoming 0.

[0040] The present invention proposes a feature-guided multi-task segmentation network, which adopts a parallel task architecture to realize the joint optimization of multi-tasks in gastrointestinal tumor classification and segmentation, and fully utilizes the intrinsic correlation between segmentation and classification tasks in the feature space to improve the performance of both. Specifically, the network model extracts common features through a shared encoder, designs a segmentation feature guidance module to inject the predicted mask information into the classification sub-network, and effectively suppresses background noise interference. By introducing a local feature attention mechanism in the segmentation branch, combining the progressive optimization of boundary features with a partial decoder, and balancing the multi-task optimization process through an adaptive loss weight strategy, the SFGNet proposed in the present invention achieves better segmentation and classification performance on the gastrointestinal tumor dataset. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Schematic diagram of the structure of channel attention and spatial attention, Figure 1 (a) is the structural diagram of channel attention, Figure 1(b) is the structural diagram of spatial attention.

[0042] Figure 2 Schematic diagram for class activation mapping.

[0043] Figure 3 Schematic diagram of two forms of multi-task learning framework, Figure 3 (a) is the hard parameter sharing diagram, Figure 3 (b) is the soft parameter sharing diagram.

[0044] Figure 4 This figure shows data on four types of gastrointestinal tumors.

[0045] Figure 5 This is the overall structure diagram of the SFGNet network.

[0046] Figure 6 Structural diagram of the local feature attention module.

[0047] Figure 7 This is the structural diagram of the segmentation feature guidance module.

[0048] Figure 8 The figure shows the comparison of the segmentation performance of this system and existing methods in GISTS.

[0049] Figure 9 A visualization of the segmentation results of different algorithms on the GISTS dataset.

[0050] Figure 10 Comparison of the classification confusion matrix of SFGNet and other methods on the GISTS dataset. Figure 10 (a) is a comparison chart with SE-ResNet, Figure 10 (b) is a comparison chart with DenseNet. Figure 10 (c) is a comparison chart with EfficientNet. Figure 10 (d) is the result graph of SFGNet.

[0051] Figure 11 This is the ablation comparison diagram of each module of SFGNet on the GISTS dataset. DETAILED DESCRIPTION

[0052] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0053] CT image segmentation and classification systems based on segmentation feature guidance, such as Figure 1 As shown, the structure of channel attention is as follows Figure 1As shown in (a), a channel-dimensional feature calibration strategy is adopted. Channel statistical features are extracted through dual-path global pooling consisting of maximum pooling and average pooling. The obtained global channel information is input into a multi-layer perceptron with shared parameters for inter-channel interaction. The channel weight matrix is ​​generated by the Sigmoid function to achieve adaptive recalibration of the feature map channel dimension.

[0054] The structure of spatial attention is as follows Figure 1 As shown in (b), the feature selection focuses on the spatial dimension, and spatial statistical features are obtained through maximum pooling and average pooling in the channel dimension. After splicing the two-channel feature maps, a convolutional layer is used to learn spatial correlations. Finally, a spatial weight matrix is ​​generated through the Sigmoid function to achieve adaptive weighting of the spatial position of the feature map.

[0055] The self-attention mechanism establishes global dependencies based on the feature graph itself, and calculates the long-range correlation between features by constructing a query matrix (Query) and a similarity measure of key-value pairs (Key-Value).

[0056] Various types of attention mechanisms achieve information screening through feature interactions in different dimensions. Channel attention focuses on distinguishing the importance of feature channels, spatial attention emphasizes modeling the correlation of spatial positions, and self-attention is good at establishing global context dependencies.

[0057] like Figure 2 As shown in , the high-level semantic features of the convolutional neural network are fused with the classifier weights in the spatial domain to establish a visual association based on pixel-level classification. Indicates the Feature channels are located in space The activation values ​​at these locations are non-negative after being processed by ReLU. Global Average Pooling (GAP) compresses them into channel-dimensional feature vectors:

[0058] (1)

[0059] For the target category c, its unnormalized prediction score can be expressed as:

[0060] (2)

[0061] In the formula, the weight parameter Indicates the first Feature channel pair categories The discriminant contribution of , expand the expression to get:

[0062] (3)

[0063] Where, Defined as spatial position For categories The discriminant significance of is obtained by linearly combining the feature map with the classification weight, and the mathematical expression of the class activation map is derived:

[0064] (4)

[0065] This formula describes the generation mechanism of a class activation map: by performing a class-dependent weighted summation of the feature maps of each channel, this method explicitly quantifies the influence of each pixel position on the final classification decision while preserving spatial information. After normalization, this map generates a corresponding heat map, where high-response areas are the locations of the key visual features driving the model's classification, providing an interpretable basis for the classification decision.

[0066] In the multi-task learning (MTL) framework, given input data , whose core goal is to use deep neural network models accomplish Related tasks The collaborative learning process is expressed as a composite mapping relationship:

[0067] (5)

[0068] Model Able to output the prediction targets corresponding to each task at the same time , for each task , and define its corresponding loss function as , then the optimization objective of multi-task joint learning is composed of weighted linear combinations:

[0069] (6)

[0070] Where, is a task-specific adaptive weight coefficient. In addition, the gradient back propagation is optimized To update the network weights in the layers of the deep learning model. The composite loss function is Gradient optimization can synchronously update the weight parameters of each layer in the deep neural network, thereby realizing knowledge sharing and transfer of representation learning between different tasks.

[0071] The multi-task learning framework achieves collaborative optimization through parameter sharing mechanism. Its core architecture can be divided into two paradigms: hard parameter sharing and soft parameter sharing. Figure 3As shown in (a), hard parameter sharing adopts a hierarchical parameter sharing strategy, which extracts common features across tasks by sharing the underlying network, while configuring an independent top-level network for each task to learn task-specific representations.

[0072] Compared with the rigid architecture of hard parameter sharing, soft parameter sharing adopts a more adaptive parameter constraint mechanism. Figure 3 As shown in (b), this paradigm maintains an independent network structure for each task while introducing regularization constraints, such as the L² norm and parameter discrepancy penalty, to promote progressive alignment of the network parameter space. This architectural design maintains model flexibility while enabling knowledge transfer through soft regularization of the parameter space, making it particularly suitable for multi-task learning scenarios with low task correlation.

[0073] The gastrointestinal stromal tumor segmentation dataset used contains 5019 abdominal CT cross-sectional images of 74 patients with pathologically confirmed gastrointestinal tumors, with a size of 512×512 pixels, including 410 positive slices of tumor lesions.

[0074] The dataset covers four types of gastrointestinal tumors, including: 233 images of gastrointestinal stromal tumors (GIST), accounting for 56.83%; 110 images of gastric schwannoma (GL), accounting for 26.83%; 45 images of gastric leiomyoma (GS), accounting for 10.98%; and 22 images of ectopic pancreas (EP), accounting for 5.37%. Figure 4 As shown, each column shows the typical CT images and segmentation and annotation results of four types of tumors, GIST, GL, GS, and EP, from left to right. The red lines represent the tumor boundaries confirmed by expert consensus.

[0075] In medical image analysis, a multi-dimensional quantitative evaluation system was constructed for the dual tasks of gastrointestinal tumor CT image segmentation and pathological subtype classification.

[0076] (1) Segmentation index:

[0077] The confusion matrix is ​​used as the basic evaluation framework. The matrix fully characterizes the spatial correspondence between the prediction results and the annotation results corresponding to the gold standard through four basic statistics, namely true positive (TP), false negative (FN), false positive (FP), and true negative (TN). A multidimensional evaluation system is introduced to comprehensively quantify the segmentation performance. The clinical significance of these four basic statistics is shown in Table 1.

[0078]

[0079] The Dice coefficient (DI) indicates the overlap ratio between the segmentation predicted tumor area and the segmentation label. This indicator is robust to the imbalanced situation between tumor area and background area in gastrointestinal tumor segmentation.

[0080] (7)

[0081] Where P represents the predicted tumor area, and G represents the actual labeled tumor area.

[0082] The Jaccard index (JA) represents the ratio of the intersection and union of the segmentation prediction tumor area and the segmentation label tumor area, reflecting the tumor localization accuracy. The specific formula is:

[0083] (8)

[0084] Sensitivity (SE) indicates the number of pixels in the tumor region of the segmentation label that are simultaneously predicted as tumors by the segmentation network. It is used to evaluate the ability of the model to identify the tumor region. The specific formula is:

[0085] (9)

[0086] Specificity (SP) refers to the number of pixels in the background area of ​​the segmentation label that the segmentation network predicts as background at the same time. It is used to measure the ability of the model to exclude the background area. The specific formula is:

[0087] (10)

[0088] Accuracy (AC) represents the ratio of the sum of the number of correctly predicted tumor pixels and correctly predicted background pixels to the total number of pixels, and comprehensively evaluates the global foreground-background classification performance of the segmentation network. The specific formula is:

[0089] (11)

[0090] (2) Classification indicators:

[0091] The following classification indicators are used to evaluate the model: Overall accuracy (OA) represents the proportion of samples whose predicted categories are consistent with the true categories to the total number of samples, which directly reflects the overall classification ability of the model. The specific formula is:

[0092] (12)

[0093] Cohen's Kappa score (CK) measures the difference in consistency between the classification results and random classification. The specific formula is:

[0094] (13)

[0095] Where, Represents the expected consistency probability, which is calculated from the actual and predicted distributions of each category. Precision (PRE) represents the proportion of samples predicted to be positive that are actually positive. This indicator is used to measure the ability of the model to avoid false positives (FP). The specific formula is:

[0096] (14)

[0097] Recall (RE) is the proportion of samples that are actually positive that are correctly predicted. It is used to measure the ability of the model to detect positive samples (reduce false negatives). The specific formula is:

[0098] (15)

[0099] The F1 coefficient (F1) is the harmonic mean of precision and recall, balancing the advantages of both. This indicator is used to comprehensively evaluate model performance and is particularly suitable for scenarios with imbalanced categories. The specific formula is:

[0100] (16)

[0101] like Figure 5 As shown in the figure, a feature-guided multi-task segmentation network is constructed. The proposed SFGNet adopts a dual-path collaborative learning paradigm to achieve joint segmentation and classification of gastric tumor CT images through feature sharing and task interaction. The network architecture consists of four core components: an encoder-decoder-based tumor segmentation subnet, a local feature attention module (LFA) for enhanced boundary perception, a segmentation feature guidance module (SFG) for cross-task feature interaction, and a tumor classification subnet that integrates multi-level tumor region features.

[0102] The tumor segmentation subnet uses the VGG19 architecture as a shared encoder, and uses its multi-level feature extraction mechanism to effectively extract the multi-scale features of gastrointestinal tumors. The encoder contains 5 hierarchical convolution stages, and the output feature map of each stage is recorded as , whose spatial resolution is Decreasing step by step. Deep features Contains high-level semantic information, while shallow features Rich texture details are preserved. At the encoder end, a multi-branch dilated pyramid pooling module with dilation rates of 6 / 12 / 18 is used to enhance feature representation capabilities. Based on the standard dilated convolution branch, a non-local operation branch is added. The parallel dilated convolution branch constructs multi-scale contextual information, while the non-local branch captures long-range dependencies of feature maps through a self-attention mechanism.

[0103] The segmentation decoder adopts a progressive feature fusion architecture, which consists of 4 cascaded decoding units and the final segmentation prediction head. The initial decoding stage uses the deep semantic features output by the Atrous Spatial Pyramid Pooling module (ASPP) As the initial input, the spatial resolution is restored step by step through the feature pyramid. Decoding Unit Receive current layer features and the corresponding encoding stage characteristics As input: First Perform bilinear interpolation upsampling to The spatial resolution is then achieved through channel splicing to achieve feature fusion, and then the fusion features are extracted through 3×3 convolution stacking to finally output the upper layer decoding features. After four stages of decoding, the network reconstructs a high-level feature map with the same resolution as the original input image. The final segmentation prediction head uses a two-stage processing flow: first, the number of feature channels is compressed to 1 dimension through a 1×1 convolution layer, and then a sigmoid activation function is applied to generate a spatial probability map. , where H and W correspond to the height and width of the input image respectively.

[0104] The local feature attention module systematically fuses multi-scale decoded features with global context information by building a dual-path attention mechanism. Specifically, the initial global segmentation map is generated by aggregating deep features through the partial decoder. , and then adopt a progressive feature fusion strategy to gradually integrate decoding features of different levels through multiple cascaded LFA modules , to achieve fine-grained correction of the segmentation results.

[0105] like Figure 6 As shown, the LFA module achieves feature enhancement through a dual-branch attention mechanism: first, based on the global segmentation map Or the output of the previous LFA module Generate spatial attention weights. The uncertain region detection branch strengthens the decision-ambiguous region with a segmentation probability close to 0.5, while the boundary attention branch mines the tumor information ignored in the background region of the coarse segmentation result through a flipping operation:

[0106] (17)

[0107] (18)

[0108] Where, is the upsampling operation, is the Sigmoid function, is the inverse operation, and Represent the generated uncertainty area and boundary perception area attention weight map respectively, Represents the global segmentation map or the output of the previous LFA module. In the feature fusion stage, the two attention weight maps are respectively combined with the current layer decoding features. Perform spatial weighted operations to generate enhanced features with spatial selectivity. Finally, feature fusion is achieved through channel splicing and 1×1 convolution, and residual connection is performed with the coarse segmentation result of the previous level to obtain the optimized segmentation prediction map:

[0109] (19)

[0110] Where, represents a 1×1 convolution, Indicates channel cascade, This design enables the model to adaptively focus on fuzzy boundary areas and low-confidence areas, significantly improving segmentation accuracy through multi-level feature iterative optimization.

[0111] A segmentation feature guidance module is proposed. The segmentation feature is embedded into the classification network through a multi-level feature fusion mechanism, which effectively enhances the ability of the classification feature to focus on the lesion area. The architecture design of this module is as follows: Figure 7 shown.

[0112] The SFG module is a dual-input architecture: segmentation prediction and multi-level encoding features The output end generates a multi-level classification feature with enhanced characteristics of the lesion area Specifically, for the multi-level features extracted by the shared encoder First, channel normalization is performed through 1×1 convolution to generate basic classification features with unified channel dimensions , ensuring that multi-scale features have equal importance in the classification subnetwork.

[0113] The SFG module consists of two core stages: the first stage: feature fusion enhancement, which uses bilinear interpolation to Downsample to The spatial dimension is then multiplied element by element to enhance the spatial features, and the enhanced features are fused with the original features:

[0114] (20)

[0115] Where, represents the downsampling operation, represents element-wise product, It is a channel dimension splicing operation, and conv represents the feature fusion of 3×3 convolution kernel.

[0116] Construct a spatial-channel collaborative attention mechanism to further optimize feature expression: capture spatial statistical features and channel fusion information through parallel global average pooling and global maximum pooling, and finally achieve feature refinement through residual connection. The specific formula is as follows:

[0117] (twenty one)

[0118] (twenty two)

[0119] (twenty three)

[0120] Where, represents the Sigmoid function, ⊙ is the element-by-element multiplication, and represents global average pooling and maximum pooling in the channel dimension, and represents global average pooling and maximum pooling in the spatial dimension. This dual attention mechanism significantly improves the ability of classification features to identify small lesions through the coordinated optimization of the spatial and channel dimensions.

[0121] like Figure 7 As shown in the green part, this subnet adopts a top-down progressive feature fusion strategy, and the input features are multi-level classification features processed by the segmentation feature guidance module. Specifically, we first use a 3×3 convolutional layer with a stride of 2 to Downsample by a factor of two to make its spatial dimension equal to Keep it consistent, then implement it through feature concatenation and 1×1 convolution operation and The fusion process is extended downward layer by layer in a cascade manner, integrating and After multi-level feature fusion, the feature of multi-granularity classification information is finally generated. Finally, the global average pooling layer is used to The spatial dimension is compressed, and after mapping through the fully connected layer, the tumor classification probability distribution is output through the softmax function.

[0122] A joint supervision strategy of Dice loss and binary cross entropy loss (BCE) is adopted to constrain the four side outputs through a deep supervision mechanism. , which is bilinearly upsampled to the gold standard annotated by clinical experts The same size as The segmentation loss is composed of the Dice similarity coefficient loss and the pixel-level cross entropy loss, and its expression is:

[0123] (twenty four)

[0124] Where, represents the side output segmentation mask after upsampling adjustment, By calculating the pixel distribution overlap between the predicted result and the true mask, the class imbalance problem is alleviated. This provides pixel-level classification constraints. For classification tasks, the standard cross entropy loss is used to measure the difference between the classification prediction and the true label:

[0125] (25)

[0126] Where, represents the total number of tumor categories for the gastrointestinal tumor classification task, and Respectively represent the samples as The true label of the class and the network prediction The class probability estimate automatically adjusts the loss weight of each task through learnable parameters. The total loss function for combined gastrointestinal tumor classification and segmentation is:

[0127] (26)

[0128] Where, and are the learnable weight parameters for the segmentation and classification tasks, respectively, and their initial values ​​are set to 1. Through the automatic optimization of the parameters during back propagation, the network can autonomously adjust the loss weight ratio according to the learning difficulty of the two tasks, achieving end-to-end multi-task balanced optimization. The logarithmic regularization term It can effectively prevent the weight parameter from becoming 0.

[0129] The performance evaluation system adopts multi-dimensional quantitative indicators: for classification tasks, five indicators are selected: PRE, RE, F1, CK and OA; for segmentation tasks, comprehensive evaluation is carried out through AC, JA, SE, DI and SP.

[0130] As shown in Table 2, SFGNet achieves optimal performance in all four metrics: Dice coefficient, Jaccard index, sensitivity, and accuracy, and approaches the highest value in specificity, significantly outperforming the other compared algorithms. Its low standard deviation also demonstrates the model's robustness. Specifically, SFGNet's DI and JA are 0.7780 and 0.6933, respectively, significantly outperforming other methods and achieving improvements of approximately 5.7% and 9.3%, respectively, compared to the next-best MSCA-Net. This demonstrates higher segmentation accuracy in tumor regions. SFGNet's SE is 0.7968, surpassing PraNet's 0.7762 and MSCA-Net's 0.7513, indicating a lower miss rate for tumor regions, making it particularly suitable for scenarios requiring high recall, such as medical diagnosis. SFGNet achieves high performance with AC and SP of 0.9988 and 0.9994, respectively, and minimizes standard deviation, demonstrating its stability and reliability in identifying background regions.

[0131]

[0132] In SFGNet, the LFA module continuously supplements boundary features and uncertainty region features through a multi-task learning mechanism, effectively improving the model's ability to identify subtle lesions. This allows the segmentation task to maintain high accuracy while significantly improving specificity. Furthermore, the SFG module establishes a collaborative optimization mechanism for segmentation and classification tasks: the lesion localization information provided by the segmentation mask can guide the classification branch to focus more accurately on the lesion area, while the semantic features obtained by classification are optimized through backpropagation to the feature expression of the shared encoder, thereby improving the segmentation network's ability to discriminate heterogeneous lesions.

[0133] like Figure 8 As shown in the figure, DI, JA, and SE are selected to represent the consistency indicators between the segmentation results and the segmentation labels to compare different methods. The various indicators of SFGNet on the GISTS dataset are higher than those of other comparison methods, indicating that it has strong performance in the gastrointestinal tumor segmentation task.

[0134] like Figure 9 As shown in the visualization, the green outline represents the gold standard tumor annotated by radiologists, and the red outline represents the predicted region of each model. The three methods, DeepLabv3+, LDNet, and PraNet, shown in the second column, each have significant segmentation flaws: DeepLabv3+ misses tumor regions, LDNet misjudges anatomical structures, and PraNet exhibits significant over-segmentation. In contrast, SFGNet demonstrates good performance in key metrics such as tumor localization accuracy, morphological restoration, and boundary conformance, and its predicted contours are closer to the gold standard annotated by doctors.

[0135]

[0136] SFGNet was compared with classification methods such as ResNet, SE-ResNet, DenseNet and EfficientNet. The experimental results are shown in Table 3. Each row represents a different classification index of the five image classification methods; each column represents the classification performance of different methods under the same classification index. The CK value of SFGNet is 0.8215, which far exceeds other methods, indicating that it performs better in category consistency and is more robust to category imbalance problems. OA reaches 0.9024, which is about 5.7% higher than the second-best DenseNet, verifying its significant advantage in overall classification ability. The F1 score of SFGNet in the stromal tumor category is 0.9324, the highest among all methods, and 7.3% higher than DenseNet, indicating that the model performs well in reducing misdiagnosis and missed diagnosis.

[0137] like Figure 10 As shown, SFGNet can accurately distinguish different types of tumors. Compared with other classification methods, SFGNet can achieve more accurate classification of various gastrointestinal tumor data. SFGNet's classification accuracy for gastrointestinal stromal tumors is 92%, higher than SE-ResNet's 81.33% and EfficientNet's 88%. In terms of misclassification, SFGNet has a lower misclassification rate for stromal tumors and schwannomas, indicating that it can better capture subtle feature differences.

[0138] In order to verify the effectiveness of each module in the network, the SFGNet network and its variants including LFA module and SFG module are compared. The experimental results are shown in Table 4 and Figure 11 As shown. The experimental results in Table 4 show that the introduction of the LFA module and the SFG module has a differential impact on the performance of classification and segmentation tasks. When only the LFA module is introduced, the segmentation index DI is improved from 0.7353 to 0.7793, indicating that the network's learning of edge features effectively enhances the segmentation accuracy; while the SFG module alone guides the classification task through the segmentation mask, which increases the classification index OA from 0.8780 to 0.8862, an increase of 1.0%, verifying the auxiliary role of the segmentation result in classification. When the two modules are introduced at the same time, the classification index OA of SFGNet is 0.9024, achieving the best result, but the segmentation index DI is 0.7780, slightly lower than the result of using the LFA module alone.

[0139]

[0140] Figure 11The synergy and trade-offs between modules are further demonstrated through visualization. B represents the baseline classification and segmentation network, consisting of only a shared encoder, segmentation decoder, and classification subnetwork; L and S represent the introduction of the LFA module and the SFG module, respectively, based on baseline network B; and M represents SFGNet. After introducing the LFA or SFG module on the basis of the baseline model, the metrics shift toward higher values ​​along the broken line. The bubbles of SFGNet are smaller and closer to the high-value areas of classification, indicating optimal classification performance and strong stability. However, the segmentation variance is slightly higher than that of the model using only LFA, indicating the complexity of multi-task optimization. Notably, the LFA module improves the stability of the segmentation results by focusing on edge uncertainty regions, while the SFG module enhances classification robustness through cross-task information exchange. In summary, SFGNet achieves a significant breakthrough in classification performance through a dual-module synergy, while also maintaining suboptimal performance in segmentation tasks.

[0141] In order to verify the superiority of the proposed multi-task network over the single-task network, the present invention conducted an ablation study on the single tasks of segmentation and classification, and the experimental results are shown in Table 5. The experimental results show that compared with single tasks, the multi-task model SFGNet achieves knowledge transfer between tasks by sharing the feature extraction layer. At the same time, the classification and segmentation performance of the equal weight strategy are better than those of single tasks, reflecting that feature complementarity is beneficial to both types of tasks. In addition, the present invention adopts different multi-task weight setting strategies such as equal weight, segmentation first and then classification, and uncertain weight for comparison. The results show that the segmentation first and then classification strategy performs the worst, indicating that the quality of the segmentation task directly affects the classification performance. If there is a deviation in the segmentation result, the subsequent classification task may make decisions based on incorrect features, resulting in a decrease in network performance. This phenomenon may be due to the low proportion of lesion areas in the gastrointestinal tumor dataset. The uncertainty weight dynamically adjusts the weight of each task and assigns weights to the loss of each task by quantifying the uncertainty of the task, avoiding the limitations of fixed weights and significantly improving the generalization ability of the model.

[0142]

[0143] The present invention proposes a feature-guided multi-task network for the segmentation and classification of gastrointestinal tumors. In view of the shortcomings of existing methods in multi-task collaborative optimization, on the one hand, a local feature attention module is designed in the segmentation subnetwork, and by establishing a multi-scale feature interaction mechanism, the contextual information fusion of the tumor area and the boundary is effectively enhanced; on the other hand, a segmentation feature guidance module is constructed, which uses the spatial attention of the segmentation mask to guide the classification features to focus on the lesion area, significantly suppressing background noise interference. Experimental results show that compared with existing deep learning-based segmentation and classification methods, SFGNet has achieved excellent performance in the segmentation and classification of gastrointestinal tumors, confirming the effectiveness of the multi-task collaborative learning strategy.

[0144] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims

1. A CT image segmentation and classification system based on segmentation feature guidance, characterized by: Construct a feature-guided multi-task segmentation network, which consists of an encoder-decoder-based tumor segmentation subnet, a local feature attention module for enhanced boundary perception, a segmentation feature guidance module for cross-task feature interaction, and a tumor classification subnet that integrates multi-level tumor region features. The tumor segmentation subnet uses the VGG19 architecture as a shared encoder, leveraging its multi-level feature extraction mechanism to extract multi-scale features of gastrointestinal tumors. The segmentation decoder adopts a progressive feature fusion architecture consisting of four cascaded decoding units and a final segmentation prediction head. After four stages of decoding, the network reconstructs a high-level feature map with the same resolution as the original input image. The final segmentation prediction head uses two-stage processing to obtain a prediction map, which is then thresholded to obtain pixel-level tumor segmentation results, achieving end-to-end mapping from deep semantic features to pixel-level localization. The local feature attention module builds a dual-path attention mechanism to fuse multi-scale decoding features with global context information to refine the segmentation results. The segmentation feature guidance module embeds lesion area information into multi-level classification features through a spatial-channel-aware feature selection mechanism and generates spatial attention weights using segmentation predictions. The classification subnet structure adopts a top-down progressive feature fusion strategy. The input features are multi-level classification features processed by the segmentation feature guidance module. The spatial dimension is compressed, and after mapping through the fully connected layer, the tumor classification probability distribution is output through the softmax function.

2. The CT image segmentation and classification system based on segmentation feature guidance according to claim 1, characterized in that: In the tumor segmentation subnet, the shared encoder consists of five hierarchical convolution stages, and the output feature maps of each stage are denoted as , whose spatial resolution is Gradually decreasing, deep features Contains high-level semantic information and shallow features To preserve texture details, a multi-branch dilated pyramid pooling module with dilation rates of 6, 12, and 18 is used at the encoder end. Based on the standard dilated convolution branch, a non-local operation branch is added. The parallel dilated convolution branch constructs multi-scale context information, while the non-local branch captures long-range dependencies of feature maps through a self-attention mechanism. The segmentation decoder adopts a progressive feature fusion architecture. The initial decoding stage uses the deep semantic features output by the hollow spatial pyramid pooling module ASPP. As the initial input, the spatial resolution is gradually restored through the feature pyramid. Decoding Unit Receive current layer features and the corresponding encoding stage characteristics As input: First Perform bilinear interpolation upsampling to The spatial resolution is then achieved through channel splicing to achieve feature fusion, and then the fusion features are extracted through 3×3 convolution stacking to finally output the upper layer decoding features. ; After four stages of decoding, the network reconstructs a high-level feature map with the same resolution as the original input image. , the number of feature channels is compressed to 1 dimension through a 1×1 convolution layer, and then the sigmoid activation function is applied to generate a spatial probability map , where H and W correspond to the height and width of the input image respectively.

3. The CT image segmentation and classification system based on segmentation feature guidance according to claim 2, characterized in that: In the local feature attention module, first based on the global segmentation map Or the output of the previous level local feature attention module Generate spatial attention weights, the uncertain region detection branch strengthens the decision-making fuzzy area, and the boundary attention branch mines the tumor information ignored in the background area of ​​the coarse segmentation result through flipping operation: （1） （2） Where, is the upsampling operation, is the Sigmoid function, is the inverse operation, and Represent the generated attention weight maps of the decision fuzzy area and the boundary perception area, respectively. Represents the global segmentation map or the output of the previous LFA module; In the feature fusion stage, the two attention weight maps are respectively combined with the current layer decoding features Perform spatial weighted operations, achieve feature fusion through channel splicing and 1×1 convolution, and perform residual connection with the coarse segmentation result of the previous level to obtain the optimized segmentation prediction map: （3） Where, represents a 1×1 convolution, Indicates channel cascade, Represents element-wise product.

4. The CT image segmentation and classification system based on segmentation feature guidance according to claim 3, characterized in that: Segmentation feature guidance module for segmentation prediction and multi-level encoding features The dual-input architecture consists of a multi-level classification feature generated at the output , for the multi-level features extracted by the shared encoder First, channel normalization is performed through 1×1 convolution to generate basic classification features ; By bilinear interpolation Downsample to The spatial dimension is then multiplied element by element to fuse the enhanced features with the original features: （4） Where, represents the downsampling operation, represents element-wise product, It is a channel dimension splicing operation, and conv represents the feature fusion of 3×3 convolution kernel; A spatial-channel collaborative attention mechanism is constructed to capture spatial statistical features and channel fusion information through parallel global average pooling and global maximum pooling, and feature refinement is achieved through residual connections. The specific formula is as follows: （5） （6） （7） Where, represents the Sigmoid function, ⊙ is the element-wise product, and represents global average pooling and maximum pooling in the channel dimension, and Represents global average pooling and maximum pooling in the spatial dimension.

5. The CT image segmentation and classification system based on segmentation feature guidance according to claim 4, characterized in that: In the classification subnet, the input features are multi-level classification features processed by the segmentation feature guidance module , first use a 3×3 convolutional layer with a stride of 2 Downsample by a factor of two to make its spatial dimension equal to Maintain consistency, achieved through feature concatenation and 1×1 convolution operations and The feature fusion of and Feature expression, generating features that integrate multi-granularity classification information , through the global average pooling layer The spatial dimension is compressed, and after mapping through the fully connected layer, the tumor classification probability distribution is output through the softmax function.

6. The CT image segmentation and classification system based on segmentation feature guidance according to claim 5, characterized in that: Constraining the four side outputs through a deep supervision mechanism , and bilinearly upsample it to the gold standard with the annotation The same size as , the segmentation loss is composed of the Dice similarity coefficient loss and the pixel-level cross entropy loss, and its expression is: （8） Where, represents the side output segmentation mask after upsampling adjustment, Calculate the degree of pixel distribution overlap between the predicted result and the true mask, Provide pixel-level classification constraints; For classification tasks, the standard cross entropy loss is used to measure the difference between the classification prediction and the true label: （9） Where, represents the total number of tumor categories for the gastrointestinal tumor classification task, and Respectively represent the samples as The true label of the class and the network prediction The class probability estimate automatically adjusts the loss weight of each task, and the total loss function for combined gastrointestinal tumor classification and segmentation is: （10） Where, and are the learnable weight parameters for the segmentation and classification tasks, respectively. The initial value is set to 1. Through the automatic optimization of the parameters during backpropagation, the network autonomously adjusts the loss weight ratio according to the learning difficulty of the two tasks, achieving end-to-end multi-task balanced optimization; The deep semantic features output by the atrous spatial pyramid pooling module ASPP in the initial decoding stage As the initial input, the spatial resolution is restored step by step through the feature pyramid, feature fusion is achieved through channel splicing, and the fusion features are extracted through 3×3 convolution stacking to output the upper layer decoding features. .

Images