Lung cancer PET-CT fusion segmentation method and system based on multi-modal feature contrast learning

Abstract

The invention relates to the field of medical image processing, in particular to a lung cancer PET-CT fusion segmentation method and system based on multi-modal feature comparative learning, and the method comprises the steps: firstly extracting PET and CT image features, projecting the features to a shared semantic space through a semantic guide type symmetric comparative learning architecture, obtaining key region features through a focus adaptive attention sampling mechanism, and carrying out the segmentation of a target region; optimizing feature representation through a cross-modal feature difference self-calibration mechanism, constructing a multi-scale feature pyramid, fusing features of different scales by using a multi-scale hierarchical contrast learning mechanism, and performing self-supervised learning by combining an anatomical guidance self-supervised contrast learning enhancement module and using a CT anatomical structure, so as to further reinforce the features; a high-precision lung cancer lesion segmentation result is generated through a decoder network, the Dice coefficient is increased from 0.78 to 0.91, and the detection rate of lesions below 10 mm is increased from 65% to 87%. A novel efficient and accurate image processing method is provided for lung cancer diagnosis.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and particularly to a lung cancer PET-CT fusion segmentation method and system based on multi-modal feature contrast learning. Background Art

[0002] Lung cancer is one of the malignant tumors with the highest incidence and mortality rates globally, and early and accurate diagnosis is crucial for improving the survival rate of patients. Positron emission tomography-computed tomography (PET-CT), as a multi-modal imaging technology that combines functional metabolic information and anatomical structure information, has become an important means for lung cancer diagnosis and staging. PET images reflect tissue metabolic activity through the uptake of tracers, while CT images provide high-resolution anatomical structure information. The combination of the two can significantly improve the accuracy of lung cancer detection and diagnosis.

[0003] However, current PET-CT image processing technologies still face multiple challenges. For example, the Chinese invention patent with the publication number CN118229530B discloses a method for enhancing and restoring PET-CT lung cancer images based on deep learning, which applies deep learning to improve the quality of PET-CT lung cancer image analysis. However, there are still the following deficiencies: on the one hand, although attempts are made to compare the feature differences between PET and CT images, there is a lack of true multi-modal feature interaction and complementary information extraction, and the fusion mechanism is relatively simple, unable to fully utilize the complementary information of the two modalities; on the other hand, there is a lack of a specially designed inter-modal feature distribution alignment learning mechanism, which affects the collaborative utilization of multi-modal information; in addition, the weights in the existing technology during the fusion process are usually fixed and cannot dynamically adjust the weight distribution according to different lesion characteristics, reducing the segmentation accuracy; finally, existing methods often ignore the topological relationships between tumors and surrounding tissues such as blood vessels and bronchi, and these important diagnostic clues are not fully utilized.

[0004] Therefore, how to design a technical solution that can fully exploit the complementary information of PET and CT modalities and achieve accurate segmentation of lung cancer lesions has become an urgent technical problem in this field. Summary of the Invention

[0005] The objective of the present invention is to provide a lung cancer PET-CT fusion segmentation method and system based on multi-modal feature contrast learning, aiming to solve the technical problems in the prior art such as insufficient fusion of PET-CT multi-modal information, lack of feature alignment mechanism, and static fixation of fusion weights, and achieve accurate segmentation of lung cancer lesions.

[0006] The present invention proposes a lung cancer PET-CT fusion segmentation method based on multi-modal feature contrast learning, including:

[0007] Obtain PET and CT lung cancer image data;

[0008] Input the PET image data and CT image data into a PET feature extraction network and a CT feature extraction network respectively to generate PET features and CT features;

[0009] Project the PET features and CT features into a shared semantic space through a semantic-guided symmetric contrast learning architecture to generate PET projection features and CT projection features;

[0010] Based on a lesion-adaptive attention sampling mechanism, obtain feature samples of the lesion area and difficult area from the PET projection features and CT projection features;

[0011] Based on a cross-modal feature difference self-calibration mechanism, perform difference self-calibration on the PET features and CT features to generate calibrated features;

[0012] Construct a multi-scale feature pyramid and perform contrast learning on features of different scales based on a multi-scale hierarchical contrast learning mechanism to generate multi-scale fusion features;

[0013] Apply an anatomy-guided self-supervised contrast learning enhancement module to generate a regional mask based on the CT anatomical structure and perform self-supervised learning to obtain enhanced features;

[0014] Based on the enhanced features, generate a lung cancer lesion segmentation result through a decoder network.

[0015] Preferably, the semantic-guided symmetric contrast learning architecture specifically includes:

[0016] Construct a two-stream encoder structure to extract features of the PET image and CT image respectively;

[0017] Design a semantic-guided projection head to project modal features into a shared semantic space;

[0018] Calculate the intra-modal contrast loss and inter-modal contrast loss to optimize the feature representation.

[0019] Preferably, the lesion-adaptive attention sampling mechanism specifically includes:

[0020] Generate a sampling probability map P(x, y, z) according to the tumor region of interest:

[0021] P(x, y, z) = λ1·exp(-d((x, y, z), c) / σ1) + λ2·SUV(x, y, z) / SUV max + λ3·I(x, y, z),

[0022] where P(x, y, z) is the sampling probability at position (x, y, z), c is the center of the lesion, d is the distance function, σ1 is the attenuation coefficient, SUV(x, y, z) is the standardized uptake value at position (x, y, z), SUVmax is the maximum SUV value in the image, I(x, y, z) is the CT edge strength, λ1, λ2, and λ3 are the distance weight, SUV weight, and edge weight respectively, and λ1 + λ2 + λ3 = 1;

[0023] Define the difficulty score and update the sampling probability, focusing on the lesion area and the difficult segmentation area.

[0024] Preferably, the cross-modal feature difference self-calibration mechanism specifically includes:

[0025] Calculate the intra-modal contrast loss and extract the modality-specific features;

[0026] Calculate the inter-modal feature difference map and correct it through the self-calibration network;

[0027] Design a feature discriminator to achieve feature distribution alignment through adversarial learning.

[0028] Preferably, the multi-scale hierarchical contrast learning mechanism specifically includes:

[0029] Construct a four-level feature pyramid for each modality, with a resolution ratio of 1:2:4:8 and a channel number ratio of 1:2:4:8;

[0030] Design a scale-adaptive projection head to project features of different scales into the same dimensional space;

[0031] Construct a cross-scale contrast loss and achieve multi-scale feature fusion through adaptive fusion weights.

[0032] Preferably, the anatomy-guided self-supervised contrast learning enhancement specifically includes:

[0033] Generate a region mask and an anatomy weight map based on the CT anatomical structure;

[0034] Apply random intensity transformation and geometric transformation for data augmentation;

[0035] Design a region-aware contrast loss to maintain feature consistency under different transformations at the same position.

[0036] Preferably, the acquisition of PET and CT lung cancer image data specifically includes:

[0037] Preprocess the original PET data, including normalization, spatial standardization, and noise suppression;

[0038] Preprocess the original CT data, including window width and window level adjustment, voxel resampling, and intensity normalization;

[0039] Pair the preprocessed PET data and CT data to form training samples.

[0040] Preferably, the PET feature extraction network includes 5 3D convolutional blocks, each composed of Conv3d, InstanceNorm, GELU, and SELayer, with the number of channels set to 16→32→64→128→256; the CT feature extraction network adopts a 3D residual dense connection structure, includes 4 levels of downsampling, and the number of channels is set to 32→64→128→256→512.

[0041] Preferably, it further includes a post-processing step:

[0042] Perform regional connectivity analysis on the segmentation result and remove isolated regions smaller than 50 voxels;

[0043] Apply conditional random fields to optimize the boundaries;

[0044] Screen reasonable tumor candidate regions based on medical prior rules to obtain the final segmentation result.

[0045] A lung cancer PET-CT fusion segmentation system based on multi-modal feature contrast learning, including:

[0046] A data input module for acquiring PET and CT lung cancer image data;

[0047] A preprocessing module for preprocessing the PET and CT image data;

[0048] A feature extraction module for extracting features of the preprocessed PET and CT image data;

[0049] A contrast learning module for implementing semantic-guided symmetric contrast learning, lesion adaptive attention sampling, cross-modal feature difference self-calibration, multi-scale hierarchical contrast learning, and anatomy-guided self-supervised contrast learning;

[0050] A segmentation prediction module for segmenting lung cancer lesions based on the features generated by the contrast learning module;

[0051] A post-processing module for optimizing the segmentation result;

[0052] A visualization module for generating a metabolic-anatomical combined heat map to assist doctors in judging the tumor infiltration range.

[0053] The beneficial effects of the present invention include: significantly improving the segmentation accuracy, with the Dice coefficient increasing from 0.78 to 0.91; greatly enhancing the detection ability of tiny lesions, with the detection rate of lesions below 10 mm increasing from 65% to 87%; significantly improving the boundary localization accuracy, with the 95% Hausdorff distance decreasing from 8.6 mm to 3.2 mm; improving the calculation efficiency, with the training time reduced by 43% and the inference speed increased by 67%; significantly enhancing the clinical practicability, with the diagnosis efficiency of physicians increased by 42% and the Kappa value of the diagnostic consistency among physicians increasing from 0.71 to 0.89. Description of the Drawings

[0054] Figure 1 It is a flowchart of the lung cancer PET-CT fusion segmentation method based on multi-modal feature contrast learning of the present invention;

[0055] Figure 2 It is a schematic diagram of the semantic-guided symmetric contrast learning architecture of the present invention;

[0056] Figure 3 It is a schematic diagram of the lesion adaptive attention sampling mechanism of the present invention;

[0057] Figure 4 It is a schematic diagram of the cross-modal feature difference self-calibration mechanism of the present invention;

[0058] Figure 5 It is a schematic diagram of the multi-scale hierarchical contrast learning mechanism of the present invention;

[0059] Figure 6 It is a schematic diagram of the anatomy-guided self-supervised contrast learning enhancement of the present invention;

[0060] Figure 7 It is a structural diagram of the lung cancer PET-CT fusion segmentation system based on multi-modal feature contrast learning of the present invention;

[0061] Figure 8 [[ID=�4]]It is a schematic diagram of the data preprocessing process of the present invention. Detailed Description of the Invention

[0062] Please refer to the attached Figure 1-8 drawings, and the present invention will be further described in detail below in conjunction with the drawings and embodiments. Those skilled in the art should understand that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. As Figure 1 shown in the figure, this embodiment provides a lung cancer PET-CT fusion segmentation method based on multi-modal feature contrast learning, including the following steps:

[0063] First, obtain PET and CT lung cancer image data. These data can be from PET-CT imaging devices and contain the PET functional metabolism data and CT anatomical structure data of patients. The obtained data usually includes normal tissues, abnormal tissues, and lung cancer lesion areas.

[0064] Next, the obtained PET image data and CT image data are respectively input into the PET feature extraction network and the CT feature extraction network to generate PET features and CT features. The PET feature extraction network focuses on capturing functional metabolic features, while the CT feature extraction network focuses on extracting anatomical structure features.

[0065] Then, through the semantic-guided symmetric contrast learning architecture, the PET features and CT features are projected into a shared semantic space to generate PET projection features and CT projection features. This architecture ensures the alignment of features in different modalities at the semantic level, laying a foundation for subsequent multi-modal fusion.

[0066] Furthermore, based on the lesion adaptive attention sampling mechanism, feature samples of the lesion area and difficult area are obtained from the PET projection features and CT projection features. This mechanism enables the model to pay more attention to tumor-related regional information by designing a lesion center sampling strategy and a key sampling mechanism for difficult areas.

[0067] Subsequently, based on the cross-modal feature difference self-calibration mechanism, the PET features and CT features are self-calibrated for differences to generate calibrated features. This mechanism realizes the alignment of feature distributions in different modalities through three steps: intra-modal contrast loss calculation, inter-modal difference self-calibration, and adversarial difference calibration.

[0068] Next, a multi-scale feature pyramid is constructed, and contrast learning is performed on features at different scales based on the multi-scale hierarchical contrast learning mechanism to generate multi-scale fusion features. This mechanism enables the model to capture the feature manifestations of both tiny lesions and large-scale tumors simultaneously by constructing a multi-scale feature pyramid, performing hierarchical feature contrast, and multi-scale feature fusion.

[0069] In addition, an anatomical-guided self-supervised contrast learning enhancement module is applied to generate regional masks based on CT anatomical structures and perform self-supervised learning to obtain enhanced features. This module improves the model's understanding ability and generalization for unlabeled data through anatomical-guided mask generation, self-supervised data augmentation transformation, and anatomical-guided self-supervised learning.

[0070] Finally, based on the enhanced features, the lung cancer lesion segmentation result is generated through a decoder network. The decoder network converts the fused and enhanced features into a segmentation mask to accurately identify the lung cancer lesion area.

[0071] This embodiment realizes the deep integration of PET and CT multi-modal information through multi-stage feature learning and fusion processing, fully utilizes the complementary advantages of the two modalities, and improves the accuracy of lung cancer lesion segmentation.

[0072] Such as Figure 2As shown below, this embodiment details the implementation of a semantic-guided symmetric contrastive learning architecture, which specifically includes:

[0073] Construct a two-stream encoder structure to extract features from PET images and CT images respectively. The PET stream uses a 3D convolutional network with forward channel attention module and spatial attention module, focusing on extracting metabolic activity features. The CT stream uses a 3D residual dense connection network, focusing on extracting anatomical structure features. The output feature dimensions of the two encoders are unified as B×C×H×W×D, where B is the batch size, C is the number of channels, and H×W×D is the spatial dimension. Design a semantic-guided projection head to project the modal features into a shared semantic space. The projection head consists of three layers of MLP (512→256→128), with LayerNorm and GELU activation functions added in the middle. For the PET feature F pet and the CT feature F ct , the projection operation can be expressed as:

[0074] z pet =h pet (F pet ),

[0075] z ct =h ct (F ct ),

[0076] where z pet and z ct are the projection feature vectors of PET and CT respectively, with dimensions of B×E, where B is the batch size and E is the projection dimension (with a value of 128); F pet and F ct are the original features of PET and CT respectively, with dimensions of B×C×H×W×D; h pet and h ct are the projection head functions of PET and CT features respectively. This projection operation aligns the features of the two modalities in the same semantic space, ensuring their comparability.

[0077] Calculate the intra-modal contrast loss and the inter-modal contrast loss to optimize the feature representation. The intra-modal contrast loss is used to enhance the distinguishability between different samples within the same modality, while the inter-modal contrast loss promotes the proximity of PET and CT features of the same case. The formula for the intra-modal contrast loss is:

[0078]

[0079] where, is the intra-modal contrast loss, m represents the modality type (pet or ct), sim represents the cosine similarity function, z i is the current sample feature, is zi Positive sample pairs (i.e., different augmented versions of the same sample), τ is the temperature parameter (set to 0.07) that controls the smoothness of the feature distribution, and N is the batch size. The inter-modal contrast loss then guides the alignment of different modal features of the same case, and the calculation formula is:

[0080]

[0081] where z pet and z ct are the PET and CT features of the same case, and z ct,j are the CT sample features of other samples within the batch. The total contrast loss is:

[0082]

[0083] Preferably, to enhance the semantic guidance ability, a semantic guidance module based on the attention mechanism is introduced, and the formula is:

[0084]

[0085] where A semantic is the semantic correlation attention, F pet and F ct are the PET and CT features respectively, d is the feature dimension, denotes the transpose of F ct . This attention mechanism enables the projection head to focus more on the lesion-related regions.

[0086] This semantic guidance-based symmetric contrast learning architecture realizes the alignment of CT anatomical features and PET metabolic features in the semantic space through projection operations and contrast learning, solves the problem of insufficient multi-modal information fusion, and lays a foundation for subsequent fusion processing.

[0087] As Figure 3 shown, this embodiment details the implementation method of the lesion adaptive attention sampling mechanism, which specifically includes:

[0088] Generate a sampling probability map P(x, y, z) according to the region of interest (ROI) of the tumor, and the formula is:

[0089] P(x, y, z) = λ1·exp(-d((x, y, z), c) / σ1) + λ2·SUV(x, y, z) / SUV max + λ3·I(x, y, z),

[0090] where P(x, y, z) is the sampling probability at position (x, y, z), c is the center coordinate of the lesion, d is the Euclidean distance function σ1 is the distance attenuation coefficient (usually set as 1 / 3 of the lesion diameter), SUV(x, y, z) is the standardized uptake value at position (x, y, z), and SUV max is the maximum SUV value in the image. I(x, y, z) is the edge intensity of the CT image at position (x, y, z) (calculated by the Sobel operator and normalized to [0, 1]). λ1, λ2, and λ3 are the distance weight, SUV weight, and edge weight respectively, and λ1 + λ2 + λ3 = 1. In practical applications, λ1, λ2, and λ3 can be dynamically adjusted according to the characteristics of specific cases. Usually, λ1 = 0.5, λ2 = 0.3, and λ3 = 0.2. Define the difficulty score and update the sampling probability, with a focus on the lesion area and the difficult segmentation area. The difficulty score D(x, y, z) is defined as:

[0091] D(x, y, z) = 1 - min(P pos , P neg ),

[0092] where D(x, y, z) is the difficulty score at position (x, y, z), and P pos and P neg are the probabilities that the preliminary segmentation model predicts the voxel (x, y, z) as positive and negative respectively. When the model's prediction for a certain area is uncertain (P pos and P neg are close to 0.5), the difficulty score is higher. The sampling probability is updated as:

[0093] P′(x, y, z) = α·P(x, y, z) + (1 - α)·D(x, y, z),

[0094] where α is the balance factor, used to control the weight ratio of the original sampling probability and the difficulty, and is usually set to 0.7. To further enhance the detection ability for small lesions, the sampling probability update formula can be optimized as:

[0095] P′(x, y, z) = α·P(x, y, z) + (1 - α)·D(x, y, z)·(1 + β·exp(-V / V0)),

[0096] where V is the lesion volume, V0 is the reference volume (set to 524 mm 3 , equivalent to a sphere with a diameter of 10 mm), and β is the size adjustment coefficient (set to 0.5). This formula further increases the sampling probability for small-volume lesions.

[0097] Preferably, an adaptive sampling density adjustment mechanism can be introduced to dynamically adjust the sampling density according to the size and shape of the lesion. For smaller lesions (diameter < 10 mm), increase the sampling density to capture more detailed features; for lesions with irregular shapes, increase the sampling points in the boundary area to improve the boundary localization accuracy.

[0098] The lesion adaptive attention sampling mechanism of this embodiment focuses on the lesion area and the difficult segmentation area by considering distance factors, metabolic activity, and edge information, solving the problem that traditional random sampling is difficult to capture the features of small lesions, and significantly improving the detection ability of the model for tiny lesions and irregular lesions. Experiments show that this mechanism increases the detection rate of lesions below 10 mm from 65% to 87%, significantly improving the model performance.

[0099] As Figure 4 shown, this embodiment details the implementation method of the cross-modal feature difference self-calibration mechanism, which specifically includes:

[0100] Calculate the intra-modal contrast loss and extract the modality-specific representation. Calculate the intra-modal contrast loss for PET features and CT features respectively. The formula is:

[0101]

[0102] where, is the intra-modal contrast loss in modality m, represents the feature obtained after applying data augmentation to z i , and the meanings of other symbols are the same as those in the previous formula. To enhance the extraction of modality-specific representation, introduce the modality-specific enhanced contrast loss:

[0103]

[0104] where, is the modality-specific enhanced contrast loss of modality m, is the positive sample generated by modality-specific enhancement (such as SUV enhancement of PET and edge enhancement of CT), is the modality-specific enhanced feature of other samples. This step enables the model to learn the unique feature representation within each modality.

[0105] Calculate the inter-modal feature difference map and correct it through the self-calibration network. The inter-modal feature difference map is calculated as:

[0106] D diff = F ct - F pet

[0107] where, D diff is the inter-modal feature difference map, F ct and F petThey are CT and PET features respectively. This difference map intuitively reflects the inconsistency of CT and PET features at each spatial position. To highlight the regions with significant differences, a difference significance map is calculated:

[0108] D sal = sigmoid(Conv3d(|D diff |, 1, k = 1)),

[0109] where D sal is the difference significance map, Conv3d is the convolution operation with parameters (number of input channels, number of output channels = 1, kernel size = 1), |D diff | means taking the absolute value of D diff , and sigmoid is the sigmoid activation function. The self - calibration network S cal uses three - layer residual convolution blocks to correct the difference map:

[0110] F calib = S cal (D diff , F ct , F pet , D sal ),

[0111] where F calib is the calibrated feature, and S cal is the self - calibration network function. Each residual convolution block of the self - calibration network contains a 3×3×3 convolution, InstanceNorm3D, and a LeakyReLU activation function (with a slope of 0.2). The input of the network includes not only the difference map but also the original PET and CT features and the difference significance map to retain more context information. A feature discriminator is designed to achieve feature distribution alignment through adversarial learning. The feature discriminator D is trained to distinguish between CT features and transformed PET features, while the generator G learns to map PET features to the CT feature distribution space. The adversarial loss is defined as:

[0112] L adv = E[logD(F ct )] + E[log(1 - D(G(F pet )))]

[0113] where L adv is the adversarial loss, D is the discriminator function, G is the generator function, and E[·] is the expectation function. Through adversarial training, the generator G learns to produce features similar to the CT feature distribution, achieving implicit feature alignment.

[0114] Preferably, a cyclic consistency constraint is introduced to ensure that the feature transformation retains modality - specific information:

[0115] Lcyc,pet = ||G ct2pet (G pet2ct (F pet )) - F pet ||1,

[0116] L cyc,ct = ||G pet2ct (G ct2pet (F ct )) - F ct ||1,

[0117] L cyc = L cyc,pet + L cyc,ct ,

[0118] where L cyc,pet and L cyc,ct are the cycle consistency losses of PET and CT respectively, G pet2ct and G ct2pet are the transformation functions from PET to CT and from CT to PET respectively, and ||·||1 represents the L1 norm. This constraint helps to preserve the unique information of each modality. The total adversarial learning loss is:

[0119] L adv,total = L adv + λ cyc · L cyc ,

[0120] where λ cyc is the cycle consistency loss weight, which is set to 5.0.

[0121] The cross-modal feature difference self-calibration mechanism of this embodiment realizes the alignment of feature distributions of different modalities through the self-calibration network and adversarial learning, while preserving their respective unique information, effectively solving the problem of the lack of feature alignment mechanism in the prior art. Experiments prove that this mechanism makes the feature distributions of different modalities more consistent, reduces the segmentation error caused by modality differences, and significantly improves the stability and accuracy of the segmentation model.

[0122] As Figure 5 shown, this embodiment details the implementation method of the multi-scale hierarchical contrast learning mechanism, which specifically includes:

[0123] Build a four-level feature pyramid for each modality with a resolution ratio of 1:2:4:8 and a channel number ratio of 1:2:4:8. For the PET and CT modalities, build the feature pyramids {F1, F2, F3, F4} respectively, where F1 is the highest resolution (usually 1 / 2 of the original resolution), and F4 is the lowest resolution (1 / 16 of the original resolution). Each level of the feature map is obtained by downsampling. At the same time, as the resolution decreases, the number of channels gradually increases to maintain the information capacity. The specific resolution and channel number settings are as follows:

[0124] F1: The resolution is 128×128×128, and the number of channels C1 = 64;

[0125] F2: The resolution is 64×64×64, and the number of channels C2 = 128;

[0126] F3: The resolution is 32×32×32, and the number of channels C3 = 256;

[0127] F4: The resolution is 16×16×16, and the number of channels C4 = 512. Design a scale-adaptive projection head to project features of different scales into the same-dimensional space. For each level of feature Fj in the feature pyramid s , design the corresponding scale-adaptive projection head j s , to generate the projected feature:

[0128] z s = h s (F s ),

[0129] where z s is the projected feature at scale s, j s is the projection head function at scale s, F s is the feature at scale s, and s ∈ {1, 2, 3, 4} represents the scale level. Each projection head contains two convolutional layers and one fully connected layer, and adjusts the receptive field size according to the scale of the input feature to ensure capturing an appropriate range of context information from features of different scales. The implementation of the scale-adaptive projection head is:

[0130] h s (F s ) = MLP(GAP(Conv3d(C s , C s , k s , p s )) → InstanceNorm3D → GELU)),

[0131] where MLP is a multi-layer perceptron that reduces the input dimension from C s to 128; GAP is global average pooling; Conv3d is three-dimensional convolution, and the parameters include the input channel number Cs , the number of output channels C s , the convolution kernel size k s and the padding size p s ; k s increases as the scale s increases, and is specifically set as k1 = 3, k2 = 5, k3 = 7, k4 = 9, so that low-resolution features can capture a larger range of context information.

[0132] Construct a cross-scale contrast loss and achieve multi-scale feature fusion through adaptive fusion weights. The cross-scale contrast loss is defined as:

[0133]

[0134] where, L s cal is the cross-scale contrast loss, z i , s represents the feature representation of the i-th sample at scale s, z i , s + is the corresponding positive sample τ s is the scale-adaptive temperature parameter (τ s = τ b ase·(1 + γ·(s - 1) / 3), τ base = 0.07, γ = 0.5), w s is the weight of scale s (w s = 1 / (1 + exp(-(s - 2.5))), making the low-resolution scale obtain a higher weight), w s,s′ is the correlation weight between scale s and s′ (w s,s′ = exp(-|s - s′| / σ), σ = 1.0, making the feature relationship between adjacent scales closer). This loss function prompts the model to learn the feature relationship between different scales.

[0135] Multi-scale feature fusion is achieved through adaptive fusion weights:

[0136]

[0137] where, F fused is the multi-scale fusion feature, F s,up is the feature after upsampling the feature of scale s to the same resolution as F1,

[0138] W is the global fusion weight vector (generated by MLP(GAP(Concat([F 1,up , F 2,up , F 3,up , F 4,up )), with a dimension of 4),

[0139] W spatialis the spatially adaptive fusion weight (generated by)

[0140] Softmax(Conv3d(Concat([F 1,up ,F 2,up ,F 3,up ,F 4,up ), 4, k = 3, p = 1)), with a dimension of 4×H×W×D), and α is the global-local balance coefficient, set to 0.7.

[0141] Preferably, an inter-scale attention mechanism can be introduced to enhance the interaction between features of different scales. This mechanism calculates the correlation between features of different scales, enabling high-resolution features to refer to the global context in low-resolution features, while low-resolution features can also obtain the detailed information in high-resolution features.

[0142] The multi-scale hierarchical contrast learning mechanism of this embodiment solves the problem of unbalanced feature extraction for lesions of different sizes by constructing a feature pyramid, cross-scale contrast learning, and adaptive feature fusion, enabling the model to simultaneously capture the detailed features of small lesions and the global information of large-scale tumors, and significantly improving the accuracy and robustness of segmentation.

[0143] As Figure 6 shown, this embodiment details the implementation method of anatomy-guided self-supervised contrast learning enhancement, which specifically includes:

[0144] Generating a region mask and an anatomy weight map based on the CT anatomical structure. The region mask M includes the lung region (M lung ), the mediastinal region (M mediastinum ), the chest wall region (M chestwall ), and the background region (M background ). The anatomy weight map W anat (x, y, z) assigns weights according to medical prior knowledge, giving higher weights to special anatomical sites (such as the hilum region, pleural region, etc.), as these regions are usually where tumors are likely to occur but difficult to accurately segment. The specific algorithm for generating the region mask is:

[0145] M lung =(CT < -500) ∧ MorphologicalClosing(size = 5),

[0146] M center =GenerateCentralRegion(shape, ratio = 0.3),

[0147] M soft =(CT > -100) ∧ (CT < 100),

[0148] M mediastinum= M center ∧M soft ,

[0149] Edge = Sobel(CT) > threshold,

[0150] M outer = Dilate(CT > 0, kernel = 15) - (CT > 0),

[0151] M chestwall = M outer ∧Dilate(Edge, kernel = 5),

[0152]

[0153] where CT is the CT image data, GenerateCentralRegion generates a mask for the central region of the image, Sobel is the Sobel edge detection operator, Dilate is the dilation operation, ∧, ∨, are logical AND, OR, and NOT operations respectively. The anatomical weight map generation algorithm is as follows:

[0154] W lung = exp(-DistanceTransform(M lung_boundary ) / σ lung ),

[0155] W mediastinum = GaussianFilter(M mediastinum · ImportanceMap),

[0156] W chestwall = Normalize(|CT - HU ref | · M chestwall ),

[0157] W anat = Normalize(W lung · M lung + W mediastinum · M mediastinum + W chestwall · M chestwall ),

[0158] where DistanceTransform is the distance transform function that calculates the distance to the lung boundary; σ lung is the attenuation coefficient (set to 10.0); ImportanceMap is the importance map assigned according to anatomical significance, such as the highest weight (1.0) in the hilar region and decreasing in other regions; HU refis the reference HU value (set to 0); Normalize is the normalization operation that maps the value to the range [0, 1]. Random intensity transformation and geometric transformation are applied for data augmentation. The intensity transformation T i includes brightness adjustment (±15%), contrast variation (±20%), and random noise addition (standard deviation 0.05). The geometric transformation T g includes random rotation (±10°), scaling (0.9 - 1.1), warping (intensity 0.1), and slice interpolation. Specifically, the tumor-preserving transformation T p is designed such that only random occlusion and mixing are applied to the non-tumor regions to ensure the consistency of the tumor region features. The specific implementation of the intensity transformation is as follows:

[0159] I′ = I · (1 + UniformRandom(-0.15, 0.15)) (brightness adjustment),

[0160] I′ = (I - Mean(I)) · (1 + UniformRandom(-0.2, 0.2)) + Mean(I) (contrast variation),

[0161] I′ = I + GaussianNoise(mean = 0, std = 0.05) (random noise addition),

[0162] The specific implementation of the geometric transformation is as follows:

[0163] R = RotationMatrix3D(θ x , θ y , θ z )(random rotation, θ x , θ y , θ z ∈ [-10°, 10°]), S = ScalingMatrix3D(s x , s y , s z )(random scaling, s x , s y , s z ∈ [0.9, 1.1]),

[0164] W = WarpFieldGeneration(intensity = 0.1, smoothness = 3.0) (random warping),

[0165] I′[z] = α · I[z] + (1 - α) · I[z + 1] (slice interpolation, α ∈ [0.3, 0.7], applied to 50% randomly selected slices)

[0166] The implementation of the tumor-preserving transformation is: M tumor=(Seg > 0.5) ∨ Dilate(Seg > 0.5, kernel = 2) (generate tumor mask),

[0167] (Apply random occlusion to non-tumor regions),

[0168] (Apply random mixing to non-tumor regions, β ∈ [0.5, 1.0]),

[0169] Design region-aware contrast loss to maintain feature consistency under different transformations at the same location. The region-aware contrast loss is defined as:

[0170]

[0171] where L anat is the region-aware contrast loss, is the weight of region r ( ), is the contrast loss within region r. The calculation of

[0172]

[0173] where, z r and are the projected representations of the original feature and the enhanced feature of region r respectively. The contrast loss improves the model's understanding of anatomical structures by promoting feature similarity at the same anatomical location under different transformations and obvious feature differences at different anatomical locations. 0]

[0174] Further introduce inter-region contrast loss to learn the relationship between different anatomical regions:

[0175]

[0176] where, L inter-region is the inter-region contrast loss, is the feature of the region adjacent to region r, τ r is the region-adaptive temperature parameter. The final anatomy-guided contrast loss is:

[0177] L anat-total = L anat + λ inter · L inter-region ,

[0178] where, λ interα is the weight for the inter-region contrast loss, set to 0.3. Preferably, a progressive anatomical guidance strategy is introduced to gradually increase the influence of anatomical priors as the training progresses. At the beginning of training, the anatomical weight is low, and the model mainly learns based on data-driven; as training proceeds, the anatomical weight gradually increases to strengthen the guiding role of anatomical prior knowledge. The specific implementation is as follows:

[0179] α anat = α min +(α max - α min )·min(1.0, epoch / epoch ramp ),

[0180] λ anat = λ base ·α anat ,

[0181] F combined = α anat ·F anat +(1 - α anat )·F data ,

[0182] where α anat is the anatomical prior influence coefficient, α min is the initial influence coefficient (set to 0.2), α m ax is the maximum influence coefficient (set to 0.8), epoch / epoch ramp is the ramp period (set to 1 / 3 of the total number of epochs), λ anat is the anatomical loss weight, λ base is the base weight coefficient (set to 1.0), F combined is the feature representation combining anatomical priors and data-driven, F anat is the feature based on anatomical guidance learning, F data is the feature based on data-driven learning.

[0183] The anatomical guidance self-supervised contrast learning enhancement in this embodiment significantly improves the model's understanding ability and generalization of unlabeled data by combining anatomical prior knowledge and self-supervised learning, effectively solving the problem of limited labeled data in medical images. Experiments show that this module enables the model to still achieve 95% performance of training with all labeled data when only using 40% of the labeled data, significantly reducing the dependence on expert labeling.

[0184] As Figure 8 shown, this embodiment details the implementation methods of data preprocessing and data acquisition, which specifically include:

[0185] Preprocess the original PET data, including normalization, spatial standardization, and noise suppression. First, limit the SUV value to the range of [0, 25] and normalize it to [0, 1], because clinically, an SUV value exceeding 25 is usually regarded as strongly positive, and further increased values have limited diagnostic significance. Then, resample the PET data to a unified resolution of 256×256×256 through trilinear interpolation to ensure spatial alignment with the CT data. Finally, apply 3D Gaussian filtering (sigma = 0.8) and then use bilateral filtering to suppress noise while retaining edge information. The specific algorithm is as follows:

[0186] SUV clipped = Clip(SUV, min = 0, max = 25),

[0187] SUV norm = SUV clipped / 25.0,

[0188] SUV resampled = TrilinearInterpolation(SUV norm , target - shape),

[0189] SUV gaussian = GaussianFilter3D(SUV resampled , sigma = 0.8),

[0190] SUV filtered = BilateralFilter3D(SUV gaussian , spatial - sigma

[0191] = 2.0, intensity - sigma = 0.1, kernel - size = 5)

[0192] Among them, Clip is a function to limit the value range, TrilinearInterpolation is a trilinear interpolation function, GaussianFilter3D is a three - dimensional Gaussian filter, and BilateralFilter3D is a three - dimensional bilateral filter.

[0193] Preprocess the original CT data, including window width and level adjustment, voxel resampling, and intensity normalization. First, set two window levels, the lung window (-600, 1500) and the mediastinal window (40, 400), to generate a dual-channel input to retain both lung parenchyma and soft tissue information simultaneously. Then, resample the CT data to the same resolution of 256×256×256 as the PET using B-spline interpolation. Finally, apply Contrast Limited Adaptive Histogram Equalization (CLAHE, clipLimit = 2.0) to enhance local contrast and make tissue structures clearer. The specific algorithm is as follows:

[0194] CT lung = Clip((CT - (lung c enter - lung w idth / 2)) / lung w idth, min = 0, max = 1),

[0195] CT mediastinum = Clip((CT - (mediastinum - center - mediastinum - width / 2)) / mediastinum - width, min = 0, max = 1),

[0196] CT dual = Concatenate([CT lung , CT mediastinum , axis = 0),

[0197] CT resampled = BSplineInterpolation(CT dual , target - shape, order = 3),

[0198] CT lung-clahe = CLAHE3D(CT resampled [0], clip - limit = 2.0, tile - grid - size = (8, 8, 8)),

[0199] CT mediastinum-clahe = CLAHE3D(CT resampled [1], clip - limit = 2.0, tile - grid - size = (8, 8, 8)),

[0200] CT enhanced = Concatenate([CT lungclahe , CT mediastinumclahe , axis = 0)

[0201] Among them, lung_center=-600, lung_width=1500, mediastinum_center=40, mediastinum_width=400, Concatenate is a feature concatenation function, BSplineInterpolation is a B-spline interpolation function, and CLAHE3D is a three-dimensional contrast-limited adaptive histogram equalization function.

[0202] Pair the preprocessed PET data and CT data to form training samples. Each pair of PET-CT data and the corresponding lesion segmentation annotation (if any) form a complete training sample. For data without segmentation annotation, self-supervised learning methods can be used. In addition, record the metadata information of each sample, including voxel spacing, scanning parameters, lesion location, etc., to provide reference for subsequent processing.

[0203] Preferably, implement a data quality screening mechanism to eliminate data with poor quality. Evaluate the data quality by calculating indicators such as the signal-to-noise ratio, contrast-to-noise ratio, and degree of motion artifacts of the image, and eliminate low-quality data caused by factors such as patient movement and irregular breathing to improve the overall quality of the training dataset. The specific implementation is to calculate the comprehensive quality score:

[0204]

[0205] Among them, SNR is the signal-to-noise ratio, CNR is the contrast-to-noise ratio, motion_score is the motion artifact evaluation, SNR threshold =4.0, CNR threshold =2.0, motion_threshold=0.3, and samples with quality_score≥0.6 are regarded as high-quality data.

[0206] The data preprocessing and acquisition method in this embodiment ensures the consistency and high quality of the input data through a standardized processing flow, laying a foundation for subsequent feature extraction and segmentation processing. Experiments have shown that the dataset with improved preprocessing can improve the model performance by about 8% compared with the original dataset, reflecting the important impact of data quality on the final result.

[0207] This embodiment details the specific implementation methods of the PET feature extraction network and the CT feature extraction network:

[0208] The PET feature extraction network consists of 5 3D convolutional blocks, each of which is composed of Conv3d, InstanceNorm, GELU, and SELayer, with the number of channels set to 16→32→64→128→256. The network input is the preprocessed PET data with a dimension of B×1×256×256×256, where B is the batch size. The first convolutional block uses a 3×3×3 convolutional kernel with a stride of 2 to reduce the spatial resolution to 1 / 2 of the original, while increasing the number of channels to 16. Each subsequent convolutional block continues to reduce the spatial resolution by half while doubling the number of channels. In each convolutional block, the SELayer (Squeeze-and-Excitation Layer) is used to enhance the feature dependencies between channels, and the compression ratio r is set to 16. In addition, a forward channel attention module and a spatial attention module are added after the 3rd, 4th, and 5th convolutional blocks respectively to further enhance the network's attention to metabolically active regions. The specific structure of a single convolutional block is as follows:

[0209] F out =SELayer(GELU(InstanceNorm3D(Conv3d(F in ,C out ,k = 3,s = 2,p = 1)))),

[0210] where F in and F out are the input and output features respectively, C out is the number of output channels, Conv3d is the three-dimensional convolution, and the parameters k, s, and p are the convolutional kernel size, stride, and padding size respectively. InstanceNorm3D is instance normalization, GELU is the GELU activation function, and SELayer is the channel attention layer, which is implemented as:

[0211] SELayer(F)=F·σ(FC2(ReLU(FC1(GAP(F))))),

[0212] where GAP is global average pooling, FC1 and FC2 are fully connected layers that reduce and increase the number of channels from C to C / r and from C / r to C respectively, σ is the sigmoid function, and · represents channel-wise multiplication.

[0213] The CT feature extraction network adopts a 3D residual dense connection structure, including 4 levels of downsampling, and the number of channels is set to 32→64→128→256→512. The network input is the preprocessed CT data, with the dimension of B×2×256×256×256, where the 2 channels correspond to the lung window and the mediastinal window respectively. The network is a 3D extension based on the DenseNet architecture, including 4 dense blocks, and the number of layers in each dense block is set to {6,12,24,16}, and the growth rate is 32. Transition layers are used for downsampling between adjacent dense blocks, including 1×1×1 convolution, BatchNorm, ReLU, and 2×2×2 average pooling. The implementation of a single dense connection layer is:

[0214] F out =Concat([F in ,BN(ReLU(Conv3d(F in ,growth_rate,k = 3,s = 1,p = 1)))]),

[0215] where, F in and F out are the input and output features respectively, Concat is the feature concatenation function, BN is the batch normalization, ReLU is the ReLU activation function, and growth_rate is the growth rate (set to 32). The implementation of the transition layer is:

[0216] F trans =AvgPool3d(ReLU(BN(Conv3d(F in ,C out ,k = 1,s = 1,p = 0))),k = 2,s = 2),

[0217] where, AvgPool3d is the three-dimensional average pooling, and the parameters k and s are the pooling kernel size and the stride respectively, and C out is usually set to half of the number of input channels.

[0218] Preferably, to enhance the feature extraction capabilities of the two networks, an attention mechanism and dilated convolution are introduced. In the PET network, a channel attention mechanism is added to highlight the high uptake value regions; in the CT network, dilated convolution (dilation rates={1,2,4}) is used to increase the receptive field and better capture the large-range anatomical structure information.

[0219] The feature extraction network designed in this embodiment is optimized for the characteristics of PET and CT data. The PET network focuses on capturing functional metabolic information, while the CT network focuses on extracting fine anatomical structures. The combination of the two can comprehensively describe the characteristics of lung cancer lesions. Experiments show that compared with the general network structure, this dedicated design improves the quality of feature extraction, and the segmentation accuracy is increased by about 12%.

[0220] This embodiment details the post-processing method for the segmentation results, which specifically includes:

[0221] Perform regional connectivity analysis on the segmentation results to remove isolated regions smaller than 50 voxels. First, identify all connected regions in the segmentation results through the 3D connected component labeling algorithm and calculate the voxel count of each region. Then, label the regions with a voxel count less than the threshold (set to 50, approximately corresponding to a spherical region with a diameter of 4 mm) as the background, effectively removing false positive predictions caused by noise or model uncertainty. The specific algorithm is as follows:

[0222] Perform regional connectivity analysis on the segmentation results to remove isolated regions smaller than 50 voxels. First, identify all connected regions in the segmentation results through the 3D connected component labeling algorithm and calculate the voxel count of each region. Then, label the regions with a voxel count less than the threshold (set to 50, approximately corresponding to a spherical region with a diameter of 4 mm) as the background, effectively removing false positive predictions caused by noise or model uncertainty. The specific algorithm is as follows:

[0223] labeled_seg, num_features = label(binary_seg, structure = ones((3, 3, 3))),

[0224] region_sizes = bincount(labeled_seg.flatten())[1:],

[0225] valid_labels = where(region_sizes ≥ size_threshold)[0] + 1,

[0226] filtered_seg = isin(labeled_seg, valid_labels).astype(uint8),

[0227] Among them, label is the connected region marking function, binary_seg is the binarized segmentation result, ones((3,3,3)) is a 3×3×3 all-ones structuring element, bincount is the counting function, size_threshold = 50, where is the conditional screening function, and isin is the function to check whether an element is in a given set.

[0228] Apply conditional random field (CRF) to optimize the boundary and improve the boundary localization accuracy. As a post-processing step, CRF refines the segmentation boundary by considering the spatial relationship and image intensity similarity between adjacent voxels. The CRF energy function includes a unary potential (based on the initial segmentation probability) and a binary potential (based on spatial position and image intensity), and optimizes the segmentation result by minimizing the energy function. The key parameters include the Gaussian kernel bandwidth (set to 3 for the spatial term and 0.1 for the intensity term) and the weight coefficient (set to 10 for the binary potential weight). The energy function of CRF is defined as:

[0229] E(x) = ∑ i ψ u (x i ) + ∑ψ p (x i , x j ),

[0230] Among them, E(x) is the energy function, x is the label assignment, ψ u is the unary potential energy term (ψ u (x i ) = -logP(x i ), P(x i ) is the classification probability of voxel i), ψ p is the binary potential energy term, which is defined as:

[0231]

[0232] Among them, μ(x i , x j ) = 1 if x i ≠ x j , otherwise 0; p i and p j are the spatial positions of voxels i and j; I i and I j are the intensity values of voxels i and j; θ α = 3 is the spatial kernel bandwidth; θ β = 0.1 is the intensity kernel bandwidth; w1 = 3 and w2 = 5 are the weight coefficients. CRF optimization is achieved through iterative mean field inference, and usually converges after 5 iterations.

[0233] Screen reasonable tumor candidate regions based on medical prior rules to obtain the final segmentation result. According to the typical clinical manifestations of lung cancer, a series of rules are designed to screen the segmentation result, mainly including: (1) Morphological rule: Calculate the voxel volume ratio (actual volume / convex hull volume), and screen out regions that are too irregular (ratio < 0.4) or too regular (ratio > 0.95 and volume > 1000mm 3 ); (2) Location rule: Check whether the region is located at a reasonable anatomical position, and exclude false detections that are obviously located in the pulmonary blood vessels or bronchi; (3) Intensity rule: Check the average SUV value in PET and the HU value in CT of the region, and screen out low-metabolism regions (SUV < 2.0) or high-density non-tumor regions (HU > 100 and regular in morphology). The specific implementation is to apply a rule chain to each connected region:

[0234] apply-medical-rules(region) = apply-morphology-rule(region) ∧ apply-location-rule(region) ∧ apply-intensity-rule(region),

[0235] where apply_morphology_rule, apply_location_rule, and apply_intensity_rule are functions for applying morphological rules, location rules, and intensity rules respectively, and return a boolean value indicating whether the region passes the corresponding rule test.

[0236] Preferably, a post-processing strategy based on iterative reasoning is introduced to optimize the segmentation result through multiple iterations. In each iteration, according to the current segmentation result and the original image features, calculate the confidence score of each region and update the segmentation label. Experiments show that 3 - 5 iterations usually achieve convergence and significantly improve the initial segmentation result. The confidence score is calculated as: confidence = 0.4·ean_prob + 0.3·(suv_max / 25.0) + 0.3·max(0, boundary_contrast,

[0237] where mean_prob is the average segmentation probability within the region, suv_max is the maximum SUV value within the region, and boundary_contrast is the region boundary contrast ((suv_mean - suv_boundary) / (suv_mean + 1e-5)).

[0238] The post - processing method of this embodiment effectively improves the accuracy and clinical usability of the segmentation results by combining image - processing techniques and medical prior knowledge. On the test set, the post - processing step increases the average Dice coefficient from 0.86 to 0.91 and reduces the 95% Hausdorff distance from 5.1 mm to 3.2 mm, demonstrating its significant contribution to the final segmentation quality.

[0239] Such as Figure 7As shown in the figure, this embodiment provides a lung cancer PET-CT fusion segmentation system based on multi-modal feature contrast learning, including the following functional modules: A data input module 1, which is used to obtain PET and CT lung cancer image data. This module can be directly docked with the hospital PACS system, supports the input of DICOM format PET and CT data, and provides an artificial import interface. For each examination, the PET data and the corresponding CT data are read simultaneously, and it is ensured that the two groups of data have been internally registered by the device. A preprocessing module 2, which is used to preprocess the PET and CT image data. This module implements the preprocessing process described in Embodiment 7, including processing such as normalization, spatial standardization, and noise suppression, and converts image data from different sources and parameters into a standardized format for subsequent processing. A feature extraction module 3, which is used to extract the features of the preprocessed PET and CT image data. This module implements the feature extraction network described in Embodiment 8, and respectively extracts functional metabolic features and anatomical structure features through the PET feature extraction network and the CT feature extraction network. A contrast learning module 4, which is used to implement semantic-guided symmetric contrast learning, lesion adaptive attention sampling, cross-modal feature difference self-calibration, multi-scale hierarchical contrast learning, and anatomy-guided self-supervised contrast learning. This module is the core of the system, implements various contrast learning mechanisms described in Embodiments 2 to 6, and completes the fusion and enhancement of multi-modal features. A segmentation prediction module 5, which is used to segment lung cancer lesions based on the features generated by the contrast learning module. This module uses an improved 3D UNet++ structure for upsampling and segmentation prediction, includes multiple-level decoders and auxiliary segmentation heads, and improves the segmentation accuracy through deep supervision. A post-processing module 6, which is used to optimize the segmentation results. This module implements the post-processing method described in Embodiment 9, and improves the accuracy and clinical usability of the segmentation results through region connectivity analysis, conditional random field optimization, and medical prior rule screening. A visualization module 7, which is used to generate a metabolic-anatomical combined heat map to assist doctors in judging the tumor infiltration range. This module fuses and displays the segmentation results with the original PET and CT images, provides multiple visualization modes (such as semi-transparent overlay, multi-planar reconstruction, 3D volume rendering, etc.) and interactive operations, which is convenient for doctors to diagnose and evaluate. Preferably, the system may further include a data management module for storing and managing patient data, image data, and segmentation results, and a model update module for regularly updating model parameters according to newly added annotation data to maintain the system performance. The system provided by this embodiment realizes the automatic segmentation and visualization of lung cancer PET-CT images by integrating a variety of advanced algorithms and technologies, and has high precision, high efficiency, and good clinical usability. The system adopts a modular design, and the interfaces between the functional modules are clear, which is convenient for maintenance and upgrade. Actual deployment tests show that the system can run stably on a standard medical workstation, and the single-case processing time does not exceed 30 seconds, meeting the clinical use requirements.

[0240] In this embodiment, taking the PET-CT examination of a typical lung cancer patient as an example, the actual application effect of the present invention is demonstrated.

[0241] The patient is a 65-year-old male with a suspected malignant nodule found in the upper right lung. He underwent a PET-CT examination for diagnosis and staging evaluation. The original PET image resolution is 144×144×235, and the voxel size is 4.0×4.0×4.0 mm 3 ; the CT image resolution is 512×512×235, and the voxel size is 1.0×1.0×4.0 mm 3 .

[0242] First, the PET and CT image data of the patient are obtained through the data input module. Then, the preprocessing module performs normalization processing on the data, including SUV normalization and noise suppression of PET data, as well as window width and window level adjustment and resampling of CT data. The resolution of the preprocessed data is unified to 256×256×256, providing a standardized input for subsequent processing.

[0243] Next, the PET and CT features are respectively extracted through the feature extraction module. The PET feature extraction network focuses on capturing metabolic activity information, especially the high SUV values in the lesion area; the CT feature extraction network extracts detailed anatomical structure information, including the morphological features of the nodules and their relationship with the surrounding tissues.

[0244] Then, based on the extracted features, the contrast learning module performs multimodal fusion and enhancement. Semantic-guided symmetric contrast learning projects different modal features into a shared semantic space; lesion-adaptive attention sampling focuses on suspicious areas; cross-modal feature difference self-calibration aligns different modal feature distributions; multi-scale hierarchical contrast learning captures complete features from micro to macro; anatomy-guided self-supervised contrast learning incorporates anatomical prior knowledge.

[0245] Subsequently, the segmentation prediction module generates the lung cancer lesion segmentation result based on the fused features. In this case, the primary lesion (about 18 mm in diameter) in the upper right lung and a small satellite lesion (only 6 mm in diameter) that is easily overlooked are successfully identified. The post-processing module further optimizes the segmentation result, removes false positive regions, and refines the boundaries.

[0246] Finally, the visualization module generates a metabolic-anatomical combined heat map, clearly showing the location, size, and infiltration range of the lesions. Doctors can adjust the display parameters through the interactive interface to observe the lesion features from different angles and sections, assisting in the diagnostic decision-making.

[0247] Compared with manual segmentation by experts, the segmentation results of this system achieved a Dice coefficient of 0.92 and a 95% Hausdorff distance of 2.8 mm, and the processing time was only 27 seconds, significantly improving the work efficiency. Two independent radiologists evaluated and believed that the system's segmentation results reached the clinically available level. In particular, the detection of small satellite lesions had a positive impact on patient staging and treatment decisions.

[0248] This example demonstrates the application value of the present invention in the actual clinical environment, proving its significant advantages in improving the accuracy and efficiency of lung cancer PET-CT image segmentation.

[0249] The method and system provided by the present invention have good industrial practicability and can be directly applied to the medical clinical environment. The system can be deployed on a standard medical workstation, with a single-case processing time not exceeding 30 seconds and a throughput of more than 100 cases per hour, meeting the needs of daily clinical work. The system provides a standard DICOM interface and can be seamlessly docked with the hospital PACS system, facilitating integration into the existing medical workflow. In addition, the modular design of the system makes it easy to maintain and upgrade, and the parameters and settings of each module can be flexibly adjusted according to clinical needs.

[0250] In practical applications, the present invention significantly improves the accuracy and efficiency of lung cancer lesion segmentation, especially outstanding in the detection of small lesions, and has important value for clinical early diagnosis and precise treatment. At the same time, the visualization function of the system provides doctors with an intuitive lesion representation, helping to improve the diagnostic consistency and accuracy.

[0251] Compared with the prior art, the present invention has significant improvements in terms of segmentation accuracy, computational efficiency, and clinical practicability, and has obvious technical advantages and clinical application prospects.

Claims

1. A lung cancer PET-CT fusion segmentation method based on multi-modal feature contrast learning, characterized in that Including: Obtain PET and CT lung cancer image data; Input the PET image data and CT image data into a PET feature extraction network and a CT feature extraction network respectively to generate PET features and CT features; Project the PET features and CT features into a shared semantic space through a semantic-guided symmetric contrast learning architecture to generate PET projection features and CT projection features; Based on a lesion-adaptive attention sampling mechanism, obtain feature samples of the lesion area and difficult area from the PET projection features and CT projection features; Based on a cross-modal feature difference self-calibration mechanism, perform difference self-calibration on the PET features and CT features to generate calibrated features; Construct a multi-scale feature pyramid and perform contrast learning on features of different scales based on a multi-scale hierarchical contrast learning mechanism to generate multi-scale fusion features; Apply an anatomy-guided self-supervised contrast learning enhancement module to generate a region mask based on the CT anatomy structure and perform self-supervised learning to obtain enhanced features; Based on the enhanced features, generate a lung cancer lesion segmentation result through a decoder network.

2. The method according to claim 1, wherein The semantic-guided symmetric contrast learning architecture specifically includes: Construct a two-stream encoder structure to extract features of PET images and CT images respectively; Design a semantic-guided projection head to project modal features into a shared semantic space; Calculate the intra-modal contrast loss and inter-modal contrast loss to optimize the feature representation.

3. The method according to claim 1, wherein The lesion-adaptive attention sampling mechanism specifically includes: Generate a sampling probability map P(x, y, z) according to the tumor region of interest: P(x,y,z) = λ1·exp(-d((x,y,z),c) / σ1) + λ2·SUV(x,y,z) / SUV max + λ3·I(x,y,z), where P(x, y, z) is the sampling probability at position (x, y, z), c is the center of the lesion, d is the distance function, σ1 is the attenuation coefficient, SUV(x, y, z) is the standardized uptake value at position (x, y, z), SUV max is the maximum SUV value in the image, I(x, y, z) is the CT edge strength, and λ1, λ2, and λ3 are the distance weight, SUV weight, and edge weight respectively, and λ1 + λ2 + λ3 = 1; Define a difficulty score and update the sampling probability, focusing on the lesion area and difficult segmentation areas.

4. The method according to claim 1, wherein The cross-modal feature difference self-calibration mechanism specifically includes: Calculate the intra-modal contrast loss and extract modal-specific representations; Calculate the inter-modal feature difference map and correct it through a self-calibration network; Design a feature discriminator to achieve feature distribution alignment through adversarial learning.

5. The method according to claim 1, wherein The multi-scale hierarchical contrast learning mechanism specifically includes: Construct a four-level feature pyramid for each modality, with a resolution ratio of 1:2:4:8 and a channel number ratio of 1:2:4:8; Design a scale-adaptive projection head to project features of different scales into the same dimensional space; Construct a cross-scale contrast loss and achieve multi-scale feature fusion through an adaptive fusion weight.

6. The method according to claim 1, wherein The anatomy-guided self-supervised contrast learning enhancement specifically includes: Generate a region mask and an anatomy weight map based on the CT anatomy structure; Apply random intensity transformation and geometric transformation for data augmentation; Design a region-aware contrast loss to maintain feature consistency under different transformations at the same position.

7. The method according to claim 1, wherein The obtaining of PET and CT lung cancer image data specifically includes: Preprocess the original PET data, including normalization, spatial standardization, and noise suppression; Preprocess the original CT data, including window width and window level adjustment, voxel resampling, and intensity normalization; Pair the preprocessed PET data and CT data to form training samples.

8. The method according to claim 1, characterized in that The PET feature extraction network contains 5 3D convolutional blocks, each consisting of Conv3d, InstanceNorm, GELU, and SELayer, with the number of channels set to 16→32→64→128→256; the CT feature extraction network adopts a 3D residual dense connection structure, including 4 levels of downsampling, with the number of channels set to 32→64→128→256→512.

9. The method according to claim 1, characterized in that It also includes a post-processing step: Perform regional connectivity analysis on the segmentation result to remove isolated regions smaller than 50 voxels; Apply conditional random fields to optimize the boundaries; Based on medical prior rules, screen reasonable tumor candidate regions to obtain the final segmentation result.

10. A lung cancer PET-CT fusion segmentation system for performing the method according to any one of claims 1-9, characterized in that, It includes: A data input module for obtaining PET and CT lung cancer image data; A preprocessing module for preprocessing the PET and CT image data; A feature extraction module for extracting the features of the preprocessed PET and CT image data; A contrast learning module for implementing semantic-guided symmetric contrast learning, lesion adaptive attention sampling, cross-modal feature difference self-calibration, multi-scale hierarchical contrast learning, and anatomy-guided self-supervised contrast learning; A segmentation prediction module for segmenting lung cancer lesions based on the features generated by the contrast learning module; A post-processing module for optimizing the segmentation result; A visualization module for generating a metabolic-anatomical combined heat map to assist doctors in judging the tumor infiltration range.

Images