A lung cancer PET-CT fusion segmentation method and system based on multimodal feature comparison learning

Abstract

This invention relates to the field of medical image processing, specifically to a lung cancer PET-CT fusion segmentation method and system based on multimodal feature contrastive learning. First, features from PET and CT images are extracted. These features are then projected into a shared semantic space using a semantically guided symmetric contrastive learning architecture. A lesion-adaptive attention sampling mechanism is employed to acquire key region features, and a cross-modal feature difference self-calibration mechanism optimizes feature representation. A multi-scale feature pyramid is constructed, and features at different scales are fused using a multi-scale hierarchical contrastive learning mechanism. An anatomically guided self-supervised contrastive learning enhancement module is combined with self-supervised learning using CT anatomical structures to further strengthen the features. A decoder network generates high-precision lung cancer lesion segmentation results, improving the Dice coefficient from 0.78 to 0.91 and the detection rate of lesions smaller than 10 mm from 65% to 87%. This provides a new, efficient, and accurate image processing method for lung cancer diagnosis.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, specifically to a lung cancer PET-CT fusion segmentation method and system based on multimodal feature contrast learning. Background Technology

[0002] Lung cancer is one of the leading causes of cancer-related deaths worldwide, and early, accurate diagnosis is crucial for improving patient survival. Positron emission tomography-computed tomography (PET-CT), a multimodal imaging technique that integrates functional metabolic and anatomical information, has become an important tool for lung cancer diagnosis and staging. PET images reflect tissue metabolic activity through tracer uptake, while CT images provide high-resolution anatomical information; the combination of the two can significantly improve the accuracy of lung cancer detection and diagnosis.

[0003] However, current PET-CT image processing technology still faces several challenges. For example, Chinese invention patent CN118229530B discloses a deep learning-based method for enhancing and restoring PET-CT lung cancer images. While applying deep learning to improve the quality of PET-CT lung cancer image analysis, it still has the following shortcomings: Firstly, although it attempts to compare the differences in features between PET and CT images, it lacks truly meaningful multimodal feature interaction and complementary information extraction. The fusion mechanism is relatively simple and cannot fully utilize the complementary information of the two modalities. Secondly, it lacks a specially designed intermodal feature distribution alignment learning mechanism, affecting the collaborative utilization of multimodal information. Furthermore, the weights in the fusion process in existing technologies are usually fixed, unable to dynamically adjust the weight allocation according to different lesion characteristics, reducing segmentation accuracy. Finally, existing methods often ignore the topological relationship between the tumor and surrounding blood vessels, bronchi, and other tissues, leaving these important diagnostic clues underutilized.

[0004] Therefore, how to design a technical solution that can fully utilize the complementary information of PET and CT modalities to achieve precise segmentation of lung cancer lesions has become an urgent technical problem to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a lung cancer PET-CT fusion segmentation method and system based on multimodal feature contrast learning, aiming to solve the technical problems in the existing PET-CT, such as insufficient multimodal information fusion, lack of feature alignment mechanism, and static fixed fusion weight, so as to achieve accurate segmentation of lung cancer lesions.

[0006] This invention proposes a lung cancer PET-CT fusion segmentation method based on multimodal feature contrast learning, including:

[0007] Acquire PET and CT lung cancer image data;

[0008] PET image data and CT image data are input into the PET feature extraction network and CT feature extraction network respectively to generate PET features and CT features;

[0009] The PET and CT features are projected onto a shared semantic space using a semantically guided symmetric contrastive learning architecture to generate PET projection features and CT projection features.

[0010] Based on the lesion adaptive attention sampling mechanism, feature samples of lesion areas and difficult areas are obtained from the PET projection features and CT projection features;

[0011] Based on the cross-modal feature difference self-calibration mechanism, the PET features and CT features are self-calibrated to generate calibration features;

[0012] A multi-scale feature pyramid is constructed, and features at different scales are compared and learned based on a multi-scale hierarchical contrastive learning mechanism to generate multi-scale fused features.

[0013] An anatomically guided self-supervised contrastive learning enhancement module is used to generate region masks based on CT anatomical structures and perform self-supervised learning to obtain enhancement features;

[0014] Based on the enhanced features, lung cancer lesion segmentation results are generated through a decoder network.

[0015] Preferably, the semantically guided symmetric contrastive learning architecture specifically includes:

[0016] A dual-stream encoder structure was constructed to extract features from PET and CT images respectively;

[0017] Design a semantically guided projection head to project modal features into a shared semantic space;

[0018] Calculate intra-modal contrast loss and inter-modal contrast loss, and optimize feature representation.

[0019] Preferably, the lesion adaptive attention sampling mechanism specifically includes:

[0020] A sampling probability map P(x,y,z) is generated based on the tumor region of interest:

[0021] P(x,y,z)=λ1·exp(-d((x,y,z),c) / σ1)+λ2·SUV(x,y,z) / SUV max +λ3·I(x,y,z),

[0022] Where P(x,y,z) is the sampling probability at location (x,y,z), c is the lesion center, d is the distance function, σ1 is the attenuation coefficient, SUV(x,y,z) is the standard uptake value at location (x,y,z), and SUVmax Let I(x,y,z) be the maximum SUV value in the image, I(x,y,z) be the CT edge strength, and λ1, λ2, and λ3 be the distance weight, SUV weight, and edge weight, respectively, and λ1+λ2+λ3=1;

[0023] Define difficulty scores and update sampling probabilities, focusing on lesion areas and areas that are difficult to segment.

[0024] Preferably, the cross-modal feature difference self-calibration mechanism specifically includes:

[0025] Calculate intramodal contrast loss and extract mode-specific characterization;

[0026] Calculate the feature difference map between modes and correct it using a self-calibration network;

[0027] Design a feature discriminator and achieve feature distribution alignment through adversarial learning.

[0028] Preferably, the multi-scale hierarchical contrastive learning mechanism specifically includes:

[0029] For each modality, a four-level feature pyramid is constructed with a resolution ratio of 1:2:4:8 and a channel ratio of 1:2:4:8.

[0030] The design scale-adaptive projection head projects features of different scales onto the same dimensional space;

[0031] We construct a cross-scale contrastive loss and achieve multi-scale feature fusion through adaptive fusion weights.

[0032] Preferably, the anatomically guided self-supervised contrastive learning enhancement specifically includes:

[0033] Region masks and anatomical weight maps are generated based on CT anatomical structures;

[0034] Data augmentation is performed using random intensity transformation and geometric transformation;

[0035] Design a region-aware contrast loss to maintain feature consistency under different transformations at the same location.

[0036] Preferably, the acquisition of PET and CT lung cancer image data specifically includes:

[0037] The raw PET data is preprocessed, including normalization, spatial normalization, and noise suppression.

[0038] The raw CT data is preprocessed, including window width and level adjustment, voxel resampling, and intensity normalization.

[0039] Preprocessed PET and CT data are paired to form training samples.

[0040] Preferably, the PET feature extraction network contains 5 3D convolutional blocks, each consisting of Conv3d, InstanceNorm, GELU, and SELayer, with the number of channels set to 16→32→64→128→256; the CT feature extraction network adopts a 3D residual dense connection structure, containing 4 levels of downsampling, with the number of channels set to 32→64→128→256→512.

[0041] Preferably, a post-processing step is also included:

[0042] Perform region connectivity analysis on the segmentation results and remove isolated regions with fewer than 50 voxels;

[0043] Optimize the boundary conditions using conditional random fields;

[0044] Reasonable tumor candidate regions are selected based on medical prior rules to obtain the final segmentation result.

[0045] A lung cancer PET-CT fusion segmentation system based on multimodal feature contrast learning includes:

[0046] The data input module is used to acquire PET and CT lung cancer image data;

[0047] The preprocessing module is used to preprocess the PET and CT image data;

[0048] The feature extraction module is used to extract features from the preprocessed PET and CT image data;

[0049] The contrastive learning module is used to implement semantically guided symmetric contrastive learning, lesion adaptive attention sampling, cross-modal feature difference self-calibration, multi-scale hierarchical contrastive learning, and anatomically guided self-supervised contrastive learning.

[0050] A segmentation prediction module is used to segment lung cancer lesions based on features generated by the contrastive learning module.

[0051] The post-processing module is used to optimize the segmentation results;

[0052] The visualization module generates metabolic-anatomical combined heat maps to help doctors determine the extent of tumor invasion.

[0053] The beneficial effects of this invention include: significantly improving segmentation accuracy, with the Dice coefficient increasing from 0.78 to 0.91; greatly enhancing the detection capability of small lesions, with the detection rate of lesions smaller than 10 mm increasing from 65% to 87%; significantly improving boundary localization accuracy, with the 95% Hausdorff distance decreasing from 8.6 mm to 3.2 mm; improving computational efficiency, with training time reduced by 43% and inference speed increased by 67%; and significantly enhancing clinical applicability, with physician diagnostic efficiency increasing by 42% and the Kappa value of inter-physician diagnostic consistency increasing from 0.71 to 0.89. Attached Figure Description

[0054] Figure 1 This is a flowchart of the lung cancer PET-CT fusion segmentation method based on multimodal feature contrast learning according to the present invention;

[0055] Figure 2 This is a schematic diagram of the semantically guided symmetric contrastive learning architecture of the present invention;

[0056] Figure 3 This is a schematic diagram of the adaptive attention sampling mechanism for lesions in this invention;

[0057] Figure 4 This is a schematic diagram of the cross-modal feature difference self-calibration mechanism of the present invention;

[0058] Figure 5 This is a schematic diagram of the multi-scale hierarchical contrastive learning mechanism of the present invention;

[0059] Figure 6 This is a schematic diagram illustrating the anatomical-guided self-supervised contrastive learning enhancement method of the present invention.

[0060] Figure 7 This is a structural diagram of the lung cancer PET-CT fusion segmentation system based on multimodal feature contrast learning of the present invention;

[0061] Figure 8 This is a schematic diagram of the data preprocessing process of the present invention. Detailed Implementation

[0062] Please refer to the attached document. Figure 1-8 The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Those skilled in the art should understand that these embodiments are only for illustrating the present invention and are not intended to limit the present invention. Figure 1 As shown, this embodiment provides a lung cancer PET-CT fusion segmentation method based on multimodal feature contrast learning, including the following steps:

[0063] First, acquire PET and CT images of lung cancer. This data can come from PET-CT imaging equipment and includes the patient's PET functional metabolic data and CT anatomical data. The acquired data typically includes normal tissue, abnormal tissue, and lung cancer lesion areas.

[0064] Next, the acquired PET and CT image data are input into the PET feature extraction network and the CT feature extraction network, respectively, to generate PET features and CT features. The PET feature extraction network focuses on capturing functional metabolic features, while the CT feature extraction network focuses on extracting anatomical structural features.

[0065] Then, a semantically guided symmetric contrastive learning architecture is used to project PET and CT features into a shared semantic space, generating PET projection features and CT projection features. This architecture ensures that features from different modalities are semantically aligned, laying the foundation for subsequent multimodal fusion.

[0066] Furthermore, based on the lesion-adaptive attention sampling mechanism, feature samples of lesion and difficult-to-access regions are obtained from PET and CT projection features. This mechanism, through the design of a lesion center sampling strategy and a key sampling mechanism for difficult regions, makes the model pay more attention to tumor-related regional information.

[0067] Subsequently, based on a cross-modal feature difference self-calibration mechanism, PET and CT features are self-calibrated to generate calibration features. This mechanism achieves alignment of feature distributions across different modalities through three steps: intra-modal contrast loss calculation, inter-modal difference self-calibration, and adversarial difference calibration.

[0068] Next, a multi-scale feature pyramid is constructed, and features at different scales are compared and learned based on a multi-scale hierarchical contrastive learning mechanism to generate multi-scale fused features. This mechanism, through constructing a multi-scale feature pyramid, performing hierarchical feature comparison, and fusing multi-scale features, enables the model to simultaneously capture the feature manifestations of small lesions and large-scale tumors.

[0069] Furthermore, an anatomically guided self-supervised contrastive learning enhancement module is applied. This module generates region masks based on CT anatomical structures and performs self-supervised learning to obtain enhanced features. Through anatomically guided mask generation, self-supervised data augmentation transformation, and anatomically guided self-supervised learning, this module improves the model's understanding and generalization ability regarding unlabeled data.

[0070] Finally, based on the enhanced features, a decoder network is used to generate lung cancer lesion segmentation results. The decoder network converts the fused and enhanced features into a segmentation mask, accurately identifying the lung cancer lesion region.

[0071] This embodiment achieves deep integration of PET and CT multimodal information through multi-stage feature learning and fusion processing, making full use of the complementary advantages of the two modalities and improving the accuracy of lung cancer lesion segmentation.

[0072] like Figure 2As shown, this embodiment details the implementation of a semantically guided symmetric contrastive learning architecture, which specifically includes:

[0073] A dual-stream encoder architecture was constructed to extract features from PET and CT images separately. The PET stream employed a 3D convolutional network with forward channel attention and spatial attention modules, focusing on extracting metabolic activity features. The CT stream used a 3D residual dense connection network, emphasizing the extraction of anatomical structure features. The output feature dimensions of both encoders were unified to B×C×H×W×D, where B is the batch size, C is the number of channels, and H×W×D is the spatial dimension. A semantically guided projection head was designed to project modal features onto a shared semantic space. The projection head consisted of three MLP layers (512→256→128), with LayerNorm and GELU activation functions added in between. For PET feature F... pet and CT features F ct The projection operation can be represented as:

[0074] z pet =h pet (F pet ),

[0075] z ct =h ct (F ct ),

[0076] Among them, z pet and z ct These are the projection feature vectors for PET and CT, respectively, with dimensions B×E, where B is the batch size and E is the projection dimension (value 128); F pet and F ct These are the raw features from PET and CT, respectively, with dimensions B×C×H×W×D; h pet and h ct These are projection head functions for PET and CT features, respectively. This projection operation aligns the features of the two modalities in the same semantic space, ensuring their comparability.

[0077] Calculate intra-modal contrast loss and inter-modal contrast loss to optimize feature representation. Intra-modal contrast loss enhances the discriminative power between different samples within the same modality, while inter-modal contrast loss promotes closer similarity between PET and CT features from the same case. The formula for calculating intra-modal contrast loss is:

[0078]

[0079] in, For intra-modal contrast loss, m represents the modality type (pet or ct), sim represents the cosine similarity function, and z... i For the current sample features, For zi Positive sample pairs (i.e., different enhanced versions of the same sample), τ is the temperature parameter (set to 0.07), controlling the smoothness of the feature distribution, and N is the batch size. The inter-modal contrast loss guides the alignment of different modal features from the same case; the calculation formula is:

[0080]

[0081] Among them, z pet and z ct For PET and CT features of the same case, z ct,j Features of other CT samples within the batch. Total contrast loss:

[0082]

[0083] Preferably, to enhance semantic guidance capabilities, an attention-based semantic guidance module is introduced, with the following formula:

[0084]

[0085] Among them, A semantic For semantic relevance attention, F pet and F ct These represent PET and CT features, respectively, where d is the feature dimension. F represents ct The transpose of the image. This attention mechanism allows the projection head to focus more on the area related to the lesion.

[0086] This semantically guided symmetric contrastive learning architecture achieves the alignment of CT anatomical features and PET metabolic features in the semantic space through projection operations and contrastive learning, solving the problem of insufficient multimodal information fusion and laying the foundation for subsequent fusion processing.

[0087] like Figure 3 As shown, this embodiment details the implementation of the lesion adaptive attention sampling mechanism, which specifically includes:

[0088] A sampling probability map P(x,y,z) is generated based on the tumor region of interest (ROI), using the following formula:

[0089] P(x,y,z)=λ1·exp(-d((x,y,z),c) / σ1)+λ2·SUV(x,y,z) / SUV max +λ3·I(x,y,z),

[0090] Where P(x,y,z) is the sampling probability of location (x,y,z), c is the coordinate of the lesion center, and d is the Euclidean distance function. σ1 is the distance attenuation coefficient (usually set to 1 / 3 of the lesion diameter), and SUV(x,y,z) is the standard uptake value at location (x,y,z). max The maximum SUV value I(x,y,z) in the image represents the edge intensity of the CT image at location (x,y,z) (calculated using the Sobel operator and normalized to [0,1]). λ1, λ2, and λ3 are the distance weight, SUV weight, and edge weight, respectively, and λ1+λ2+λ3=1. In practical applications, λ1, λ2, and λ3 can be dynamically adjusted according to the specific case characteristics, typically λ1=0.5, λ2=0.3, and λ3=0.2. A difficulty score is defined and the sampling probability is updated, focusing on lesion areas and areas difficult to segment. The difficulty score D(x,y,z) is defined as:

[0091] D(x,y,z)=1-min(P pos ,P neg ),

[0092] Where D(x,y,z) is the difficulty score for position (x,y,z), and P pos and P neg These represent the probabilities that the initial segmentation model predicts a positive and negative result for voxel (x, y, z), respectively. When the model's prediction for a certain region is uncertain (P... pos and P neg (Close to 0.5), indicating a relatively high difficulty score. The sampling probability is updated as follows:

[0093] P′(x,y,z)=α·P(x,y,z)+(1-α)·D(x,y,z),

[0094] Where α is a balancing factor used to control the weight ratio of the original sampling probability and difficulty, typically set to 0.7. To further enhance the detection capability for small lesions, the sampling probability update formula can be optimized as follows:

[0095] P′(x,y,z)=α·P(x,y,z)+(1-α)·D(x,y,z)·(1+β·exp(-V / V0)),

[0096] Where V is the lesion volume and V0 is the reference volume (set to 524 mm). 3 (e.g., a sphere with a diameter of 10 mm), where β is the size adjustment factor (set to 0.5). This formula further improves the sampling probability for small lesions.

[0097] Preferably, an adaptive sampling density adjustment mechanism can be introduced to dynamically adjust the sampling density according to the size and shape of the lesion. For smaller lesions (diameter <10mm), the sampling density is increased to capture more detailed features; for lesions with irregular shapes, sampling points are added in the boundary area to improve the boundary positioning accuracy.

[0098] The lesion-adaptive focus sampling mechanism in this embodiment considers distance factors, metabolic activity, and edge information, focusing on lesion areas and regions difficult to segment. This solves the problem that traditional random sampling struggles to capture features of small lesions, significantly improving the model's ability to detect tiny and irregular lesions. Experiments show that this mechanism increases the detection rate of lesions smaller than 10 mm from 65% to 87%, significantly improving model performance.

[0099] like Figure 4 As shown, this embodiment details the implementation of the cross-modal feature difference self-calibration mechanism, which specifically includes:

[0100] Calculate intra-modal contrast loss and extract modality-specific features. Calculate intra-modal contrast loss for PET and CT features separately, using the following formula:

[0101]

[0102] in, For the contrast loss within mode m, Indicates z i The features obtained after applying data augmentation have the same meanings for other symbols as in the aforementioned formulas. To enhance the extraction of modality-specific representations, a modality-specific enhancement contrast loss is introduced:

[0103]

[0104] in, To specifically enhance the contrast loss for mode m, For positive samples generated through modality-specific enhancements (such as SUV enhancement in PET and edge enhancement in CT), This step enhances the modality-specific features for other samples. It enables the model to learn unique feature representations within each modality.

[0105] The intermodal feature difference map is calculated and corrected using a self-calibration network. The intermodal feature difference map is calculated as follows:

[0106] D diff =F ct -F pet

[0107] Among them, D diff For the feature difference map between modes, F ct and F petThese are CT and PET features, respectively. This difference map visually reflects the inconsistencies between CT and PET features at various spatial locations. To highlight areas of significant difference, a significance map is calculated:

[0108] D sal =sigmoid(Conv3d(|D diff |,1,k=1)),

[0109] Among them, D sal For the significance graph, Conv3d is the convolution operation with parameters (number of input channels, number of output channels = 1, kernel size = 1), |D diff | indicates D diff Taking the absolute value, sigmoid is the sigmoid activation function. Self-calibration network S cal The difference map is corrected using three layers of residual convolutional blocks:

[0110] F calib =S cal (D diff ,F ct ,F pet D sal ),

[0111] Among them, F calib For the calibrated features, S cal The self-calibrating network function is defined as follows: Each residual convolutional block of the self-calibrating network contains a 3×3×3 convolution, InstanceNorm3D, and LeakyReLU activation function (slope of 0.2). The network input includes not only the difference map but also the original PET and CT features, as well as the difference saliency map, to retain more contextual information. A feature discriminator is designed to align the feature distribution through adversarial learning. The feature discriminator D is trained to distinguish between CT features and transformed PET features, while the generator G learns to map PET features to the CT feature distribution space. The adversarial loss is defined as:

[0112] L adv =E[logD(F) ct )]+E[log(1-D(G(F pet )))],

[0113] Among them, L adv To counteract the loss, D is the discriminator function, G is the generator function, and E[·] is the expectation function. Through adversarial training, the generator G learns to produce features similar to the CT feature distribution, achieving implicit feature alignment.

[0114] Preferably, a cycle consistency constraint is introduced to ensure that feature transformations retain mode-specific information:

[0115] Lcyc,pet =||G ct2pet (G pet2ct (F pet ))-F pet ||1,

[0116] L cyc,ct =||G pet2ct (G ct2pet (F ct ))-F ct ||1,

[0117] L cyc =L cyc,pet +L cyc,ct ,

[0118] Among them, L cyc,pet and L cyc,ct Cyclic consistency loss for PET and CT, respectively, G pet2ct and G ct2pet Let be the transformation functions from PET to CT and from CT to PET, respectively, and ||·||1 represent the L1 norm. This constraint helps preserve the unique information of each modality. The total adversarial learning loss is:

[0119] L adv,total =L adv +λ cyc ·L cyc ,

[0120] Where, λ cyc The weight for the cycle consistency loss is set to 5.0.

[0121] The cross-modal feature difference self-calibration mechanism in this embodiment achieves the alignment of feature distributions across different modalities through a self-calibration network and adversarial learning, while preserving their unique information, effectively addressing the shortcomings of existing feature alignment mechanisms. Experiments demonstrate that this mechanism makes the feature distributions of different modalities more consistent, reduces segmentation errors caused by modal differences, and significantly improves the stability and accuracy of the segmentation model.

[0122] like Figure 5 As shown, this embodiment details the implementation of the multi-scale hierarchical contrastive learning mechanism, which specifically includes:

[0123] A four-level feature pyramid is constructed for each modality, with a resolution ratio of 1:2:4:8 and a channel ratio of 1:2:4:8. For PET and CT modalities, feature pyramids {F1, F2, F3, F4} are constructed respectively, where F1 is the highest resolution (usually half of the original resolution) and F4 is the lowest resolution (1 / 16 of the original resolution). Each level of feature map is obtained through downsampling, and the number of channels gradually increases as the resolution decreases to maintain information capacity. Specific resolution and channel number settings are as follows:

[0124] F1: Resolution is 128×128×128, number of channels C1=64;

[0125] F2: Resolution is 64×64×64, number of channels C2=128;

[0126] F3: Resolution is 32×32×32, number of channels C3=256;

[0127] F4: With a resolution of 16×16×16 and channel count C4 = 512, the design scale-adaptive projection head projects features of different scales onto the same dimensional space. For each level of feature F in the feature pyramid... s Design a corresponding scale-adaptive projection head j s Generate projection features:

[0128] z s =h s (F s ),

[0129] Among them, z s For the projection features at scale s, j s Let F be the projection head function at scale s. s Let be the features at scale s, where s∈{1,2,3,4} represents the scale level. Each projection head contains two convolutional layers and one fully connected layer, and adjusts the receptive field size according to the scale of the input features to ensure that an appropriate range of contextual information is captured from features at different scales. The implementation of the scale-adaptive projection head is as follows:

[0130] h s (F s =MLP(GAP(Conv3d(C s C s ,k s ,p s →InstanceNorm3D→GELU)

[0131] MLP stands for Multilayer Perceptron, which changes the input dimension from C... s Reduced to 128; GAP is global average pooling; Conv3d is 3D convolution, with parameters including the number of input channels C.s Number of output channels C s kernel size k s and fill size p s ;k s The value increases with the scale s, specifically set to k1=3, k2=5, k3=7, k4=9, so that low-resolution features can capture a wider range of contextual information.

[0132] A cross-scale contrastive loss is constructed, and multi-scale feature fusion is achieved through adaptive fusion weights. The cross-scale contrastive loss is defined as follows:

[0133]

[0134] Among them, L s cal represents the cross-scale contrast loss, z i ,s represents the feature representation of the i-th sample at scale s, z i ,s + For the corresponding positive sample τ s For scale-adaptive temperature parameters (τ) s =τ b ase·(1+γ·(s-1) / 3),τ base =0.07, γ=0.5), w s The weights (w) for scale s s =1 / (1+exp(-(s-2.5))), which gives higher weights to low-resolution scales), w s,s′ The correlation weight between scales s and s′ (w s,s′ =exp(-|ss′| / σ), σ=1.0, making the feature relationships between adjacent scales closer. This loss function prompts the model to learn the feature relationships between different scales.

[0135] Multi-scale feature fusion is achieved through adaptive fusion weights:

[0136]

[0137] Among them, F fused For multi-scale fusion features, F s,up To upsample the features at scale s to the same resolution as the F1 score,

[0138] W is the global fusion weight vector (determined by MLP(GAP(Concat([F)))). 1,up ,F 2,up ,F 3,up ,F 4,up Generates a 4-dimensional object.

[0139] W spatialSpatial adaptive fusion weights (by...)

[0140] Softmax(Conv3d(Concat([F 1,up ,F 2,up ,F 3,up ,F 4,up Generate ]), 4, k = 3, p = 1), with dimensions of 4 × H × W × D), α is the global local balance coefficient, set to 0.7.

[0141] Preferably, an inter-scale attention mechanism can be introduced to enhance the interaction between features at different scales. This mechanism calculates the correlation between features at different scales, enabling high-resolution features to reference the global context in low-resolution features, while low-resolution features can also obtain detailed information from high-resolution features.

[0142] The multi-scale hierarchical contrastive learning mechanism in this embodiment solves the problem of uneven feature extraction for lesions of different sizes by constructing a feature pyramid, cross-scale contrastive learning, and adaptive feature fusion. This enables the model to simultaneously capture the detailed features of small lesions and the global information of large tumors, significantly improving the accuracy and robustness of segmentation.

[0143] like Figure 6 As shown, this embodiment details the implementation method of anatomical-guided self-supervised contrastive learning enhancement, which specifically includes:

[0144] Region masks and anatomical weight maps are generated based on CT anatomy. The region mask M includes the lung region (M0). lung ), mediastinal region (M mediastinum ), chest wall region (M chestwall ) and background area (M background Anatomical weight map W anat (x, y, z) are weighted according to prior medical knowledge, with higher weights assigned to specific anatomical locations (such as the hilar region and pleural region), as these areas are often prone to tumor formation but difficult to segment accurately. The specific algorithm for generating the region mask is as follows:

[0145] M lung =(CT<-500)∧MorphologicalClosing(size=5),

[0146] M center =GenerateCentralRegion(shape,ratio=0.3),

[0147] M soft =(CT>-100)∧(CT<100),

[0148] M mediastinum=M center ∧M soft ,

[0149] Edge = Sobel(CT) > threshold,

[0150] M outer =Dilate(CT>0,kernel=15)-(CT>0),

[0151] M chestwall =M outer ∧Dilate(Edge,kernel=5),

[0152]

[0153] Where CT represents CT image data, GenerateCentralRegion generates a mask for the central region of the image, Sobel is the Sobel edge detection operator, Dilate is the dilation operation, and ∧, ∨, ... These represent logical AND, OR, and NOT operations, respectively. The algorithm for generating the weight map is as follows:

[0154] W lung =exp(-DistanceTransform(M) lung_boundary ) / σ lung ),

[0155] W mediastinum =GaussianFilter(M mediastinum ImportanceMap),

[0156] W chestwall =Normalize(|CT-HU) ref |·M chestwall ),

[0157] W anat =Normalize(W lung ·M lung +W mediastinum ·M mediastinum +W chestwall ·M chestwall ),

[0158] Where DistanceTransform is the distance transformation function, which calculates the distance to the lung boundary; σ lung 1 is the attenuation coefficient (set to 10.0); ImportanceMap is an importance map assigned according to anatomical significance, such as the hilar region having the highest weight (1.0), with other regions decreasing in weight; HU refThe reference HU value is set to 0; Normalize is a normalization operation that maps the values ​​to the range [0,1]. Data augmentation is performed using random intensity transformation and geometric transformation. Intensity transformation T i This includes brightness adjustment (±15%), contrast variation (±20%), and random noise addition (standard deviation 0.05). Geometric transformation T g This includes random rotation (±10°), scaling (0.9-1.1), distortion (intensity 0.1), and slice interpolation. Specifically, a tumor-preserving transform T was designed. p Random occlusion and blending are applied only to non-tumor regions to ensure consistency of tumor region features. The specific implementation of intensity transformation is as follows:

[0159] I′=I·(1+UniformRandom(-0.15,0.15))(brightness adjustment),

[0160] I′=(I-Mean(I))·(1+UniformRandom(-0.2,0.2))+Mean(I) (contrast change)

[0161] I′=I+GaussianNoise(mean=0,std=0.05) (random noise added),

[0162] The specific implementation of geometric transformation is as follows:

[0163] R = RotationMatrix3D(θ) x ,θ y ,θ z (Random rotation, θ) x ,θ y ,θ z ∈[-10°,10°]), S=ScalingMatrix3D(s x ,s y ,s z (Random scaling, s) x ,s y ,s z ∈[0.9,1.1]),

[0164] W = WarpFieldGeneration(intensity=0.1, smoothness=3.0) (random warp)

[0165] I′[z]=α·I[z]+(1-α)·I[z+1](slice interpolation, α∈[0.3,0.7], applied to 50% randomly selected slices)

[0166] The realization of tumor-preserving transformation is: M tumor=(Seg>0.5)∨Dilate(Seg>0.5,kernel=2) (Generate tumor mask),

[0167] (Random occlusion is applied to non-tumor areas),

[0168] (For non-tumor regions, apply random mixing, β∈[0.5,1.0]),

[0169] Design a region-aware contrastive loss to maintain feature consistency under different transformations at the same location. The region-aware contrastive loss is defined as follows:

[0170]

[0171] Where L anat For region-aware contrast loss, The weight of region r ( ), This represents the contrast loss within region r. The calculation is as follows:

[0172]

[0173] Among them, z r and These are the projected representations of the original and enhanced features of region r, respectively. Contrast loss improves the model's understanding of anatomical structures by making features similar for the same anatomical location under different transformations and highlighting the differences in features between different anatomical locations.

[0174] Further, inter-regional contrast loss is introduced to learn the relationships between different anatomical regions:

[0175]

[0176] Among them, L inter-region For inter-regional comparison loss, For the characteristics of regions adjacent to region r, τ r The temperature parameters are region-adaptive. The final dissection-guided contrast loss is:

[0177] L anat-total =L anat +λ inter ·L inter-region ,

[0178] Where, λ interThe weight for the inter-region comparison loss is set to 0.3. Preferably, a progressive anatomical guidance strategy is introduced, gradually increasing the influence of anatomical priors as training progresses. In the early stages of training, the anatomical weights are low, and the model is mainly based on data-driven learning; as training continues, the anatomical weights gradually increase, strengthening the guiding role of anatomical prior knowledge. The specific implementation is as follows:

[0179] α anat =α min +(α max -α min min(1.0, epoch / epoch) ramp ),

[0180] λ anat =λ base ·α anat ,

[0181] F combined =α anat ·F anat +(1-α anat )·F data ,

[0182] Where, α anat Let α be the anatomical prior influence coefficient. min Let α be the initial influence coefficient (set to 0.2). m ax is the maximum influence coefficient (set to 0.8), epoch / epoch ramp The slope period (set as 1 / 3 of the total number of cycles), λ anat For the weight of the anatomical loss, λ base Based on the basic weighting coefficient (set to 1.0), F combined To combine anatomical priors and data-driven feature representations, F anat For features of anatomically guided learning, F data Features based on data-driven learning.

[0183] This embodiment of anatomical-guided self-supervised contrastive learning enhancement significantly improves the model's understanding and generalization ability to unlabeled data by combining anatomical prior knowledge and self-supervised learning, effectively solving the problem of limited labeled data for medical images. Experiments show that this module enables the model to achieve 95% of the performance of training with all labeled data while using only 40% labeled data, greatly reducing the dependence on expert annotations.

[0184] like Figure 8 As shown, this embodiment details the implementation of the data preprocessing and data acquisition methods, which specifically include:

[0185] The raw PET data underwent preprocessing, including normalization, spatial normalization, and noise suppression. First, the SUV value was limited to the range [0,25] and normalized to [0,1], as an SUV value exceeding 25 is generally considered strongly positive clinically, and further increases have limited diagnostic significance. Then, the PET data was resampled to a uniform resolution of 256×256×256 using trilinear interpolation to ensure spatial alignment with the CT data. Finally, a 3D Gaussian filter (sigma = 0.8) was applied followed by a bilateral filter to suppress noise while preserving edge information. The specific algorithm is as follows:

[0186] SUV clipped =Clip(SUV,min=0,max=25),

[0187] SUV norm =SUV clipped / 25.0,

[0188] SUV resampled =TrilinearInterpolation(SUV) norm (,target-shape),

[0189] SUV gaussian =GaussianFilter3D (SUV) resampled (sigma = 0.8)

[0190] SUV filtered =BilateralFilter3D(SUV) gaussian spatial-sigma

[0191] =2.0,intensity-sigma=0.1,kernel-size=5)

[0192] Where Clip is a function that limits the range of values, TrilinearInterpolation is a trilinear interpolation function, GaussianFilter3D is a three-dimensional Gaussian filter, and BilateralFilter3D is a three-dimensional bilateral filter.

[0193] The raw CT data underwent preprocessing, including window width and level adjustment, voxel resampling, and intensity normalization. First, two window levels were set: a lung window (-600, 1500) and a mediastinal window (40, 400), generating a dual-channel input to simultaneously preserve lung parenchyma and soft tissue information. Then, B-spline interpolation was used to resample the CT data to the same resolution as PET (256×256×256). Finally, region-adaptive histogram equalization (CLAHE, clipLimit = 2.0) was applied to enhance local contrast and make tissue structures clearer. The specific algorithm is as follows:

[0194] CT lung =Clip((CT-(lung) c enter-lung w idth / 2)) / lung w idth,min=0,max=1),

[0195] CT mediastinum =Clip((CT-(mediastinum-center-mediastinum-width / 2)) / mediastinum-width, min=0, max=1),

[0196] CT dual =Concatenate([CT lung CT mediastinum ],axis=0),

[0197] CT resampled =BSplineInterpolation(CT) dual (target-shape, order=3)

[0198] CT lung-clahe =CLAHE3D(CT) resampled [0],clip-limit=2.0,tile-grid-size=(8,8,8)),

[0199] CT mediastinum-clahe =CLAHE3D(CT) resampled [1],clip-limit=2.0,tile-grid-size=(8,8,8)),

[0200] CT enhanced =Concatenate([CT lungclahe CT mediastinumclahe ],axis=0)

[0201] Where, lung_center = -600, lung_width = 1500, mediastinum_center = 40, mediastinum_width = 400, Concatenate is the feature concatenation function, BSplineInterpolation is the B-spline interpolation function, and CLAHE3D is the 3D region adaptive histogram equalization function.

[0202] Preprocessed PET and CT data are paired to form training samples. Each pair of PET-CT data and its corresponding lesion segmentation annotation (if applicable) constitutes a complete training sample. For data without segmentation annotations, self-supervised learning methods can be used. In addition, metadata information for each sample, including voxel spacing, scanning parameters, and lesion location, is recorded to provide a reference for subsequent processing.

[0203] Preferably, a data quality screening mechanism is implemented to remove data of poor quality. Data quality is evaluated by calculating indicators such as the signal-to-noise ratio, contrast-to-noise ratio, and degree of motion artifacts in the images, eliminating low-quality data caused by factors such as patient movement and irregular breathing, thereby improving the overall quality of the training dataset. Specifically, this is implemented by calculating a comprehensive quality score:

[0204]

[0205] Where SNR is the signal-to-noise ratio, CNR is the contrast-to-noise ratio, and motion_score is the motion artifact score. threshold =4.0, CNR threshold Samples with a motion_threshold of 2.0, a quality_score of 0.3, and a quality_score of ≥0.6 are considered high-quality data.

[0206] The data preprocessing and acquisition method in this embodiment ensures the consistency and high quality of the input data through a standardized processing flow, laying the foundation for subsequent feature extraction and segmentation. Experiments show that the preprocessed dataset improves model performance by approximately 8% compared to the original dataset, demonstrating the significant impact of data quality on the final result.

[0207] This embodiment details the specific implementation methods of the PET feature extraction network and the CT feature extraction network:

[0208] The PET feature extraction network consists of five 3D convolutional blocks, each composed of Conv3d, InstanceNorm, GE LU, and SELayer, with the number of channels set from 16 to 32 to 64 to 128 to 256. The network input is preprocessed PET data with dimensions B×1×256×256×256, where B is the batch size. The first convolutional block uses a 3×3×3 kernel with a stride of 2, reducing the spatial resolution to half its original value while increasing the number of channels to 16. Subsequent convolutional blocks further reduce the spatial resolution by half while doubling the number of channels. Within each convolutional block, an SELayer (Squeeze-and-ExcitationLayer) is used to enhance the feature dependencies between channels, with a compression ratio r set to 16. Furthermore, forward channel attention modules and spatial attention modules are added after the 3rd, 4th, and 5th convolutional blocks, respectively, to further enhance the network's focus on metabolically active regions. The specific structure of a single convolutional block is as follows:

[0209] F out =SELayer(GELU(InstanceNorm3D(Conv3d(F in C out k=3, s=2, p=1))))

[0210] Among them, F in and F out These are the input and output features, C. out To output the number of channels, Conv3d is a 3D convolution, with parameters k, s, and p representing the kernel size, stride, and padding size, respectively. InstanceNorm3D is instance normalization, GELU is the GELU activation function, and SELayer is the channel attention layer. The implementation is as follows:

[0211] SELayer(F)=F·σ(FC2(ReLU(FC1(GAP(F))))),

[0212] Wherein, GAP is global average pooling, FC1 and FC2 are fully connected layers that reduce the number of channels from C to C / r and increase it from C / r to C, respectively, σ is the sigmoid function, and . represents channel-wise multiplication.

[0213] The CT feature extraction network employs a 3D residual dense connection structure, including four levels of downsampling with the number of channels set to 32→64→128→256→512. The network input is preprocessed CT data with dimensions B×2×256×256×256, where two channels correspond to the lung window and mediastinal window, respectively. The network is based on the DenseNet architecture with 3D extensions, containing four dense blocks. The number of layers in each block is set to {6, 12, 24, 16}, with a growth rate of 32. Transition layers are used between adjacent dense blocks for downsampling, including 1×1×1 convolutions, BatchNorm, ReLU, and 2×2×2 average pooling. The implementation of a single dense connection layer is as follows:

[0214] F out =Concat([F in ,BN(ReLU(Conv3d(F in ,growth_rate,k=3,s=1,p=1)))]),

[0215] Among them, F in and F out The input and output features are respectively defined by Concat, BN, ReLU, and growth_rate (set to 32). The transition layer is implemented as follows:

[0216] F trans =AvgPool3d(ReLU(BN(Conv3d(F in C out ,k=1,s=1,p=0))),k=2,s=2),

[0217] Where AvgPool3d is a three-dimensional average pooling method, and the parameters k and s are the pooling kernel size and stride, respectively, C out It is usually set to half the number of input channels.

[0218] Preferably, to enhance the feature extraction capabilities of the two networks, attention mechanisms and dilated convolutions are introduced. In the PET network, a channel attention mechanism is added to highlight areas with high uptake values; in the CT network, dilated convolutions (dilation rates = {1, 2, 4}) are used to increase the receptive field and better capture information from a wide range of anatomical structures.

[0219] The feature extraction network designed in this embodiment is optimized for the characteristics of PET and CT data. The PET network focuses on capturing functional metabolic information, while the CT network focuses on extracting fine anatomical structures. The combination of the two can comprehensively describe the characteristics of lung cancer lesions. Experiments show that compared with a general network structure, this dedicated design improves the quality of feature extraction and increases segmentation accuracy by approximately 12%.

[0220] This embodiment details the post-processing method for the segmentation results, which specifically includes:

[0221] Region connectivity analysis is performed on the segmentation results to remove isolated regions with fewer than 50 voxels. First, all connected regions in the segmentation results are identified using a 3D connected component labeling algorithm, and the number of voxels in each region is calculated. Then, regions with a voxel count less than a threshold (set to 50, approximately corresponding to a spherical region with a diameter of 4 mm) are labeled as background, effectively removing false positive predictions caused by noise or model uncertainty. The specific algorithm is as follows:

[0222] Region connectivity analysis is performed on the segmentation results to remove isolated regions with fewer than 50 voxels. First, all connected regions in the segmentation results are identified using a 3D connected component labeling algorithm, and the number of voxels in each region is calculated. Then, regions with a voxel count less than a threshold (set to 50, approximately corresponding to a spherical region with a diameter of 4 mm) are labeled as background, effectively removing false positive predictions caused by noise or model uncertainty. The specific algorithm is as follows:

[0223] labeled_seg,num_features=label(binary_seg,structure=ones((3,3,3))),

[0224] region_sizes=bincount(labeled_seg.flatten())[1:],

[0225] valid_labels=where(region_sizes≥size_threshold)[0]+1,

[0226] filtered_seg=isin(labeled_seg,valid_labels).astype(uint8),

[0227] Where label is the connected component labeling function, binary_seg is the binarized segmentation result, ones((3,3,3)) is a 3×3×3 all-one structure element, bincount is the counting function, size_threshold = 50, where is the conditional filtering function, and isin is the function to check whether an element is in a given set.

[0228] Conditional Random Fields (CRF) are applied to optimize the segmentation boundaries and improve boundary localization accuracy. As a post-processing step, CRF refines the segmentation boundaries by considering the spatial relationships between adjacent voxels and image intensity similarity. The CRF energy function includes a univariate potential (based on the initial segmentation probability) and a binary potential (based on spatial location and image intensity). The segmentation result is optimized by minimizing the energy function. Key parameters include the Gaussian kernel bandwidth (3 for the spatial term and 0.1 for the intensity term) and weighting coefficients (10 for the binary potential). The CRF energy function is defined as follows:

[0229] E(x)=∑ i ψ u (x i )+∑ψ p (x i ,x j ),

[0230] Where E(x) is the energy function, x is the label assignment, and ψ u The unary potential energy term (ψ) u (x i )=-logP(x i ),P(x i ) represents the classification probability of voxel i, ψ p The binary potential energy term is defined as:

[0231]

[0232] Where, μ(x) i ,x j If x = 1 i ≠x j Otherwise, it is 0; p i and p j Let i be the spatial position of voxels i and j; I i and I j Let θ be the intensity values ​​of voxels i and j; α =3 represents the space kernel bandwidth; θ β =0.1 represents the intensity kernel bandwidth; w1=3 and w2=5 are the weighting coefficients. CRF optimization is achieved through iterative mean-field inference, and convergence is usually achieved in 5 iterations.

[0233] Based on medical prior rules, reasonable tumor candidate regions were screened to obtain the final segmentation results. According to the typical clinical manifestations of lung cancer, a series of rules were designed to screen the segmentation results, mainly including: (1) Morphological rules: calculate the voxel volume ratio (actual volume / convex hull volume), and screen out those that are too irregular (ratio < 0.4) or too regular (ratio > 0.95 and volume > 1000 mm). 3 (1) Region; (2) Location rule: Check whether the region is located in a reasonable anatomical position, and exclude false detections that are obviously located in the pulmonary vessels or bronchi; (3) Intensity rule: Check the average SUV value of the region in PET and the HU value in CT, and screen out low metabolic regions (SUV<2.0) or high-density non-tumor regions (HU>100 and regular shape). Specifically, the rule chain is applied to each connected region:

[0234] apply-medical-rules(region)=apply-morphology-rule(region)∧apply-location-rule(region)∧apply-intensity-rule(region),

[0235] Among them, apply_morphology_rule, apply_location_rule, and apply_intensity_rule are functions that apply morphology rules, location rules, and intensity rules, respectively, and return a boolean value indicating whether the region passes the test of the corresponding rule.

[0236] Preferably, a post-processing strategy based on iterative inference is introduced to optimize the segmentation results through multiple iterations. In each iteration, the confidence score of each region is calculated based on the current segmentation result and the original image features, and the segmentation label is updated. Experiments show that convergence is usually achieved after 3-5 iterations, significantly improving the initial segmentation results. The confidence score is calculated as: confidence = 0.4·ean_prob + 0.3·(suv_max / 25.0) + 0.3·max(0, boundary_contrast,

[0237] Where mean_prob is the average segmentation probability within the region, suv_max is the maximum SUV value within the region, and boundary_contrast is the region boundary contrast ((suv_mean-suv_boundary) / (suv_mean+1e-5)).

[0238] The post-processing method in this embodiment effectively improves the accuracy and clinical usability of segmentation results by combining image processing technology and prior medical knowledge. On the test set, the post-processing step increased the average Dice coefficient from 0.86 to 0.91 and reduced the 95% Hausdorff distance from 5.1 mm to 3.2 mm, demonstrating its significant contribution to the final segmentation quality.

[0239] like Figure 7As shown, this embodiment provides a lung cancer PET-CT fusion segmentation system based on multimodal feature contrastive learning, including the following functional modules: Data input module 1, used to acquire PET and CT lung cancer image data. This module can directly interface with the hospital PACS system, supports DICOM format PET and CT data input, and provides a manual import interface. For each examination, PET data and corresponding CT data are read simultaneously, ensuring that the two sets of data have been registered internally by the device. Preprocessing module 2, used to preprocess the PET and CT image data. This module implements the preprocessing process described in embodiment 7, including normalization, spatial standardization, and noise suppression, converting image data from different sources and parameters into a standardized format for easier subsequent processing. Feature extraction module 3, used to extract features from the preprocessed PET and CT image data. This module implements the feature extraction network described in embodiment 8, extracting functional metabolic features and anatomical structural features through the PET feature extraction network and CT feature extraction network, respectively. Contrast learning module 4, used to implement semantically guided symmetric contrastive learning, lesion adaptive attention sampling, cross-modal feature difference self-calibration, multi-scale hierarchical contrastive learning, and anatomically guided self-supervised contrastive learning. This module is the core of the system, implementing the various contrastive learning mechanisms described in Examples 2 to 6 to complete the fusion and enhancement of multimodal features. The segmentation prediction module 5 is used for lung cancer lesion segmentation based on the features generated by the contrastive learning module. This module uses an improved 3DUNet++ structure for upsampling and segmentation prediction, including a multi-level decoder and an auxiliary segmentation head, improving segmentation accuracy through deep supervision. The post-processing module 6 is used to optimize the segmentation results. This module implements the post-processing method described in Example 9, improving the accuracy and clinical usability of the segmentation results through region connectivity analysis, conditional random field optimization, and medical prior rule screening. The visualization module 7 is used to generate a combined metabolic-anatomical heatmap to assist doctors in determining the extent of tumor invasion. This module fuses the segmentation results with the original PET and CT images, providing multiple visualization modes (such as semi-transparent overlay, multi-planar reconstruction, 3D volume rendering, etc.) and interactive operations to facilitate diagnosis and evaluation by doctors. Preferably, the system may also include a data management module for storing and managing patient data, image data, and segmentation results, as well as a model update module for periodically updating model parameters based on newly added labeled data to maintain system performance. The system provided in this embodiment integrates multiple advanced algorithms and technologies to achieve automated segmentation and visualization of lung cancer PET-CT images, exhibiting high precision, high efficiency, and good clinical usability. The system adopts a modular design with clearly defined interfaces between functional modules, facilitating maintenance and upgrades. Actual deployment tests show that the system can run stably on a standard medical workstation, with a single case processing time of no more than 30 seconds, meeting clinical needs.

[0240] This embodiment uses a typical lung cancer patient's PET-CT examination as an example to demonstrate the practical application effect of the present invention.

[0241] The patient was a 65-year-old male with a suspected malignant nodule found in the upper right lung. He underwent PET-CT scan for diagnosis and staging. The original PET image resolution was 144×144×235, and the voxel size was 4.0×4.0×4.0 mm. 3 The CT image resolution is 512×512×235, and the voxel size is 1.0×1.0×4.0mm. 3 .

[0242] First, the patient's PET and CT image data are acquired through the data input module. Then, the preprocessing module standardizes the data, including SUV normalization and noise suppression for PET data, and window width and level adjustment and resampling for CT data. The preprocessed data resolution is uniformly set to 256×256×256, providing standardized input for subsequent processing.

[0243] Next, PET and CT features were extracted using a feature extraction module. The PET feature extraction network focused on capturing metabolic activity information, especially the high SUV value of the lesion area; the CT feature extraction network extracted detailed anatomical information, including the morphological features of the nodules and their relationship with surrounding tissues.

[0244] Then, the contrastive learning module performs multimodal fusion and enhancement based on the extracted features. Semantically guided symmetric contrastive learning projects features from different modalities into a shared semantic space; lesion adaptive attention sampling focuses on suspicious regions; cross-modal feature difference self-calibration aligns the distribution of features from different modalities; multi-scale hierarchical contrastive learning captures complete features from micro to macro; and anatomically guided self-supervised contrastive learning incorporates prior anatomical knowledge.

[0245] Subsequently, the segmentation prediction module generates lung cancer lesion segmentation results based on fusion features. In this case, the primary lesion in the upper right lung (approximately 18 mm in diameter) and a small satellite lesion (only 6 mm in diameter) that is easily overlooked were successfully identified. The post-processing module further optimizes the segmentation results, removes false positive areas, and refines the boundaries.

[0246] Finally, the visualization module generates a combined metabolic-anatomical thermogram, clearly displaying the location, size, and extent of lesion infiltration. Doctors can adjust display parameters through an interactive interface, observing lesion characteristics from different angles and cross-sections to aid in diagnostic decision-making.

[0247] Compared to manual segmentation by experts, this system achieved a Dice coefficient of 0.92 and a 95% Hausdorff distance of 2.8 mm, with a processing time of only 27 seconds, significantly improving work efficiency. Two independent radiologists assessed the system's segmentation results as clinically usable, particularly noting that the detection of small satellite lesions had a positive impact on patient staging and treatment decisions.

[0248] This example demonstrates the application value of the invention in a real clinical setting, proving its significant advantages in improving the segmentation accuracy and efficiency of lung cancer PET-CT images.

[0249] The method and system provided by this invention have good industrial applicability and can be directly applied to medical clinical environments. The system can be deployed on standard medical workstations, with a single case processing time of no more than 30 seconds and a throughput of over 100 cases per hour, meeting the needs of daily clinical work. The system provides a standard DICO M interface, allowing seamless integration with hospital PACS systems and facilitating integration into existing medical workflows. Furthermore, the system's modular design makes it easy to maintain and upgrade, allowing for flexible adjustment of module parameters and settings according to clinical needs.

[0250] In practical applications, this invention significantly improves the accuracy and efficiency of lung cancer lesion segmentation, particularly in the detection of small lesions, and is of great value for early clinical diagnosis and precision treatment. Simultaneously, the system's visualization function provides doctors with intuitive lesion representations, helping to improve diagnostic consistency and accuracy.

[0251] Compared with existing technologies, this invention has significant improvements in segmentation accuracy, computational efficiency, and clinical applicability, demonstrating clear technical advantages and promising clinical application prospects.

Claims

1. A lung cancer PET-CT fusion segmentation method based on multimodal feature contrast learning, characterized in that, include: Acquire PET lung cancer image data and CT lung cancer image data; PET lung cancer image data and CT lung cancer image data are input into the PET feature extraction network and CT feature extraction network respectively to generate PET features and CT features; The PET and CT features are projected onto a shared semantic space using a semantically guided symmetric contrastive learning architecture to generate PET projection features and CT projection features. Based on the lesion adaptive attention sampling mechanism, feature samples of lesion areas and difficult areas are obtained from the PET projection features and CT projection features; Based on the cross-modal feature difference self-calibration mechanism, the PET features and CT features are self-calibrated to generate calibration features; A multi-scale feature pyramid is constructed, and features at different scales are compared and learned based on a multi-scale hierarchical contrastive learning mechanism to generate multi-scale fused features. An anatomically guided self-supervised contrastive learning enhancement module is used to generate region masks based on CT anatomical structures and perform self-supervised learning to obtain enhancement features; Based on the enhanced features, lung cancer lesion segmentation results are generated through a decoder network; The semantically guided symmetric contrastive learning architecture specifically includes: A dual-stream encoder structure was constructed to extract features from PET lung cancer image data and CT lung cancer image data, respectively; Design a semantically guided projection head to project modal features into a shared semantic space; Calculate intra-modal contrast loss and inter-modal contrast loss, and optimize feature representation; The anatomically guided self-supervised contrastive learning enhancement specifically includes: Region masks and anatomical weight maps are generated based on CT anatomical structures; Data augmentation is performed using random intensity transformation and geometric transformation. Design a region-aware contrast loss to maintain feature consistency under different transformations at the same location.

2. The method according to claim 1, characterized in that, The adaptive attention sampling mechanism for lesions specifically includes: A sampling probability map is generated based on the tumor region of interest. : , in, , For the center of the lesion, Let it be a distance function. The attenuation coefficient is... The standard uptake value for location (x, y, z) The maximum SUV value in the image. For CT edge intensity, , , These are distance weight, SUV weight, and edge weight, respectively. + + =1; Define difficulty scores and update sampling probabilities, focusing on lesion areas and areas that are difficult to segment.

3. The method according to claim 1, characterized in that, The cross-modal feature difference self-calibration mechanism specifically includes: Calculate intramodal contrast loss and extract mode-specific characterization; Calculate the feature difference map between modes and correct it using a self-calibration network; Design a feature discriminator and achieve feature distribution alignment through adversarial learning.

4. The method according to claim 1, characterized in that, The multi-scale hierarchical contrastive learning mechanism specifically includes: For each modality, a four-level feature pyramid is constructed with a resolution ratio of 1:2:4:8 and a channel ratio of 1:2:4:

8. The design scale-adaptive projection head projects features of different scales onto the same dimensional space; We construct a cross-scale contrastive loss and achieve multi-scale feature fusion through adaptive fusion weights.

5. The method according to claim 1, characterized in that, The acquisition of PET lung cancer image data and CT lung cancer image data specifically includes: The raw PET lung cancer image data were preprocessed, including normalization, spatial standardization, and noise suppression. Preprocessing of raw CT lung cancer image data includes window width and level adjustment, voxel resampling, and intensity normalization; Preprocessed PET lung cancer image data and CT lung cancer image data are paired to form training samples.

6. The method according to claim 1, characterized in that, The PET feature extraction network contains 5 3D convolutional blocks, each consisting of Conv3d, InstanceNorm, GELU, and SELayer, with the number of channels set to 16→32→64→128→256; the CT feature extraction network adopts a 3D residual dense connection structure, containing 4 levels of downsampling, with the number of channels set to 32→64→128→256→512.

7. The method according to claim 1, characterized in that, It also includes post-processing steps: Perform region connectivity analysis on the segmentation results and remove isolated regions with fewer than 50 voxels; Optimize the boundary conditions using conditional random fields; Reasonable tumor candidate regions are selected based on medical prior rules to obtain the final segmentation result.

8. A lung cancer PET-CT fusion segmentation system based on multimodal feature contrast learning, which implements the method of any one of claims 1-7, characterized in that, include: The data input module is used to acquire PET lung cancer image data and CT lung cancer image data; The preprocessing module is used to preprocess the PET lung cancer image data and CT lung cancer image data; The feature extraction module is used to extract features from the preprocessed PET lung cancer image data and CT lung cancer image data; The contrastive learning module is used to implement semantically guided symmetric contrastive learning, lesion adaptive attention sampling, cross-modal feature difference self-calibration, multi-scale hierarchical contrastive learning, and anatomically guided self-supervised contrastive learning. A segmentation prediction module is used to segment lung cancer lesions based on features generated by the contrastive learning module. The post-processing module is used to optimize the segmentation results; The visualization module generates metabolic-anatomical combined heat maps to help doctors determine the extent of tumor invasion.

Images