PET-CT lung cancer image segmentation method and system based on deep learning

Abstract

The present invention relates to the field of image segmentation technology, and in particular to a PET-CT lung cancer image segmentation method and system based on deep learning. The method comprises the following steps: obtaining PET lung cancer image data and CT lung cancer image data, performing multi-scale feature extraction on the PET lung cancer image data and the CT lung cancer image data respectively using a multi-scale feature encoder to obtain image feature data; performing image feature fusion based on a cross-modal attention mechanism according to the image feature data to obtain dual-modal image fusion data; performing interpretable attention module design according to the dual-modal image fusion data, and marking the attention intensity by color gradient to generate a pixel-by-pixel attention heat map. The present invention has significant advantages in improving the accuracy of lung cancer image segmentation and enhancing clinical interpretability through multi-modal fusion, multi-scale feature extraction and interpretable attention mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a PET-CT lung cancer image segmentation method and system based on deep learning. Background Art

[0002] PET uses positron tracers. When the nuclide decays, the positron collides with the free electron and annihilates, producing a pair of 511keV photons. By receiving these two photons through the detector, the location of the positron emission point can be determined and a tomographic tracer distribution image can be formed. Tissues or lesions with high metabolic rates show high metabolic bright signals on PET, and vice versa, low metabolic dark signals. CT uses the difference in the absorption of X-rays by human tissues, and obtains the image matrix through reconstruction calculation through the attenuation of X-rays after passing through the human body. CT has a high density resolution for tissues. PET / CT combines the functional imaging of PET with the anatomical information of CT to display the metabolism, function, blood flow, cell proliferation and receptor distribution of tissue cells at the molecular level. It uses the characteristics of high metabolism of malignant tumor cells and reflects the glucose utilization status in the body by injecting imaging agents such as 18F-FDG (fluorodeoxyglucose), thereby displaying the location, morphology, size and other information of the tumor in PET-CT imaging.

[0003] However, traditional PET-CT lung cancer image segmentation methods often have the following problems: Deep learning algorithms are known as black box algorithms. Due to the lack of interpretability, it is difficult to fully understand the factors that lead to poor segmentation performance. In clinical practice, deep learning algorithms may fail in unpredictable ways and fail to segment OARs (organs at risk) and GTVs (tumor volumes), which is dangerous in clinical practice. When processing X-rays, lesions that overlap with blind areas are not only difficult to detect, but also have low segmentation accuracy. Summary of the invention

[0004] Based on this, it is necessary for the present invention to provide a PET-CT lung cancer image segmentation method and system based on deep learning to solve at least one of the above technical problems.

[0005] To achieve the above purpose, a PET-CT lung cancer image segmentation method based on deep learning includes the following steps:

[0006] Step S1: acquiring PET lung cancer image data and CT lung cancer image data, and using a multi-scale feature encoder to perform multi-scale feature extraction on the PET lung cancer image data and the CT lung cancer image data to obtain image feature data;

[0007] Step S2: Perform image feature fusion based on the cross-modal attention mechanism according to the image feature data to obtain bimodal image fusion data; design an explainable attention module according to the bimodal image fusion data, and mark the attention intensity through color gradient to generate a pixel-by-pixel attention heat map;

[0008] Step S3: performing heat map-guided boundary correction processing on the pixel-by-pixel attention heat map based on blind spots, overlapping lesions, and false positive detection to obtain heat map correction feature data; constructing a multi-layer segmentation decoder based on the heat map correction feature data, and performing organ and tumor volume segmentation through the multi-layer segmentation decoder to obtain tumor segmentation image data;

[0009] Step S4: designing a mixed loss function including Dice loss and boundary loss according to the tumor segmentation image data, and dynamically adjusting the contribution ratio of the two types of losses through learnable weight coefficients to obtain mixed loss function data, wherein the Dice loss calculates the overlap of the segmentation area, and the boundary loss focuses on the accuracy of the segmentation boundary;

[0010] Step S5: Perform connected domain analysis on the tumor segmentation image data according to the mixed loss function data, remove artifact areas with an area smaller than a preset threshold, and obtain candidate segmentation optimization data under different loss function weight configurations; select the optimal segmentation image based on the segmentation performance of the candidate segmentation optimization data to obtain the optimal segmentation image data.

[0011] The present invention uses a multi-scale feature encoder to extract multi-scale features from PET and CT lung cancer image data, respectively capturing the functional metabolic features of PET and the anatomical structural features of CT, ensuring the integrity and diversity of image features. On this basis, the cross-modal attention mechanism is used to fuse the two modal features to generate dual-modal image fusion data, and the pixel-by-pixel attention heat map is generated through the interpretable attention module design to intuitively display the model's attention to the tumor area, thereby enhancing the credibility of the algorithm. Subsequently, the heat map is corrected based on the guided boundary of blind areas, overlapping lesions and false positive detection to solve the segmentation accuracy problem, and a multi-layer segmentation decoder is constructed to further optimize the segmentation results and achieve accurate segmentation of organs and tumor volumes. Based on the segmented image, a mixed loss function including Dice loss and boundary loss is designed, and the contribution ratio of the two losses is adjusted through dynamic learnable weights, which not only improves the overlap between the segmented area and the real tumor area, but also improves the accuracy of the segmentation boundary. Finally, the artifact area with an area less than the threshold is removed through connected domain analysis, and the performance of the candidate segmentation optimization data under different loss weight configurations is evaluated, and the optimal segmentation image data is automatically selected to ensure the accuracy and clinical practicality of the output results. This method comprehensively utilizes multimodal data characteristics, attention mechanism and segmentation optimization strategy to effectively improve the accuracy, robustness and interpretability of lung cancer image segmentation, providing high-quality support for clinical diagnosis and treatment.

[0012] The present invention also provides a PET-CT lung cancer image segmentation system based on deep learning, which is used to execute the above-mentioned PET-CT lung cancer image segmentation method based on deep learning. The PET-CT lung cancer image segmentation system based on deep learning includes:

[0013] A multi-scale feature extraction module is used to obtain PET lung cancer image data and CT lung cancer image data, and use a multi-scale feature encoder to perform multi-scale feature extraction on the PET lung cancer image data and the CT lung cancer image data to obtain image feature data;

[0014] The cross-modal attention fusion module is used to fuse image features based on the cross-modal attention mechanism according to the image feature data to obtain the bimodal image fusion data; the explainable attention module is designed according to the bimodal image fusion data, and the attention intensity is marked by the color gradient to generate a pixel-by-pixel attention heat map;

[0015] The heat map guided boundary correction module is used to perform heat map guided boundary correction processing on the pixel-by-pixel attention heat map based on blind spots, overlapping lesions and false positive detection to obtain heat map correction feature data; a multi-layer segmentation decoder is constructed based on the heat map correction feature data, and organs and tumor volumes are segmented through the multi-layer segmentation decoder to obtain tumor segmentation image data;

[0016] The hybrid loss optimization module is used to design a hybrid loss function including Dice loss and boundary loss according to the tumor segmentation image data, and dynamically adjust the contribution ratio of the two types of losses through learnable weight coefficients to obtain the hybrid loss function data, where Dice loss calculates the overlap of the segmentation area, and boundary loss focuses on the accuracy of the segmentation boundary;

[0017] The artifact removal and optimal segmentation selection module is used to perform connected domain analysis on the tumor segmentation image data according to the mixed loss function data, remove artifact areas with an area smaller than a preset threshold, and obtain candidate segmentation optimization data under different loss function weight configurations; and select the optimal segmentation image based on the segmentation performance of the candidate segmentation optimization data to obtain the optimal segmentation image data.

[0018] The present invention uses a multi-scale feature encoder to process PET and CT images respectively to extract image features at different scales. Since PET and CT images have different spatial resolutions and visual features, multi-level information of tumors can be captured through multi-scale extraction. Extracting multi-scale features can more comprehensively represent the information in the image, so that the model can recognize the morphology and changes of tumors at different scales. In view of the different characteristics of PET and CT images, their information can be effectively integrated to enhance the recognition ability of tumors. According to the extracted image feature data, the feature data of PET and CT images are fused using a cross-modal attention mechanism. The module focuses on important image areas through an attention mechanism and fuses the complementary information of the two modal images. Through the cross-modal attention mechanism, the most relevant parts of the image are automatically focused on and strengthened to reduce the interference of irrelevant information. The features of PET and CT images are effectively integrated to provide more comprehensive tumor information and improve the accuracy and robustness of detection. According to the dual-modal image fusion data, an interpretable attention module is designed to mark the attention intensity through a color gradient and generate a pixel-by-pixel attention heat map. This can help doctors understand the key areas of the model's attention and increase the transparency of diagnosis. The generated attention heat map provides a visual basis for model decision-making, helps explain the key areas of the model's attention, and increases the credibility of the model. Doctors can understand the location and characteristics of the tumor more intuitively based on the heat map, which assists in clinical decision-making. Based on the pixel-by-pixel attention heat map, the module detects and corrects the boundaries of blind areas, overlapping lesions, and false positive areas. In this way, the segmentation results are more accurate and the wrong segmentation areas are reduced. By guiding the boundary correction, the interference areas in the heat map, such as blind areas and false positives, are eliminated to ensure that the tumor boundaries are clearer. For the areas with overlapping lesions, the real boundaries of the tumor can be better distinguished through correction processing. Based on the heat map correction feature data, the module constructs a multi-layer segmentation decoder, gradually restores the image resolution, and performs organ and tumor volume segmentation to obtain tumor segmentation image data. Through the design of jump connections and decoder layers, low-level texture features can be retained and segmentation details can be improved. Through multi-layer decoders and jump connections, the detailed features of the tumor can be retained and the accuracy of the segmentation results can be improved. The image resolution is restored layer by layer, and the segmentation results are optimized through the decoder to provide high-quality tumor volume segmentation. According to the tumor segmentation image data, a mixed loss function of Dice loss and boundary loss is designed, and the contribution ratio of the two types of losses is dynamically adjusted through learnable weight coefficients. This module optimizes the accuracy of the segmentation boundary and the overlap of the tumor area. Through the mixed loss function, the overlap of the segmentation area and the boundary accuracy are balanced to improve the segmentation quality. The learnable loss weight allows the model to automatically adjust the loss contribution ratio according to the actual situation during the training process, thereby optimizing the segmentation results. This module performs connected domain analysis on the tumor segmentation image data, removes artifact areas with an area smaller than the preset threshold, and selects the optimal segmentation image based on the segmentation performance.Through these operations, it is possible to eliminate mis-segmentation and artifact areas to ensure the accuracy of the segmentation results. Small artifact areas are removed to reduce the interference of noise on the segmentation results. Through selection based on segmentation performance, the optimal tumor segmentation image is finally selected to improve the quality of detection results. These modules work together to ensure the accuracy and reliability of lung cancer tumor segmentation through multimodal information fusion, attention mechanism optimization, boundary correction and artifact removal. Each module helps to improve the quality of tumor segmentation, reduce mis-segmentation, and ultimately provide accurate tumor segmentation results to help doctors make better clinical diagnoses and decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments thereof made with reference to the following drawings:

[0020] Figure 1 It is a schematic diagram of the steps of the PET-CT lung cancer image segmentation method based on deep learning of the present invention;

[0021] Figure 2 for Figure 1 Detailed step flow diagram of step S1;

[0022] Figure 3 for Figure 1 Detailed step flow chart of step S2 in FIG. DETAILED DESCRIPTION

[0023] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0024] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0025] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0026] To achieve this, please refer to Figures 1 to 3 The present invention provides a PET-CT lung cancer image segmentation method based on deep learning, the method comprising the following steps:

[0027] Step S1: acquiring PET lung cancer image data and CT lung cancer image data, and using a multi-scale feature encoder to perform multi-scale feature extraction on the PET lung cancer image data and the CT lung cancer image data to obtain image feature data;

[0028] Step S2: Perform image feature fusion based on the cross-modal attention mechanism according to the image feature data to obtain bimodal image fusion data; design an explainable attention module according to the bimodal image fusion data, and mark the attention intensity through color gradient to generate a pixel-by-pixel attention heat map;

[0029] Step S3: performing heat map-guided boundary correction processing on the pixel-by-pixel attention heat map based on blind spots, overlapping lesions, and false positive detection to obtain heat map correction feature data; constructing a multi-layer segmentation decoder based on the heat map correction feature data, and performing organ and tumor volume segmentation through the multi-layer segmentation decoder to obtain tumor segmentation image data;

[0030] Step S4: designing a mixed loss function including Dice loss and boundary loss according to the tumor segmentation image data, and dynamically adjusting the contribution ratio of the two types of losses through learnable weight coefficients to obtain mixed loss function data, wherein the Dice loss calculates the overlap of the segmentation area, and the boundary loss focuses on the accuracy of the segmentation boundary;

[0031] Step S5: Perform connected domain analysis on the tumor segmentation image data according to the mixed loss function data, remove artifact areas with an area smaller than a preset threshold, and obtain candidate segmentation optimization data under different loss function weight configurations; select the optimal segmentation image based on the segmentation performance of the candidate segmentation optimization data to obtain the optimal segmentation image data.

[0032] In the embodiment of the present invention, reference Figure 1FIG. 1 is a schematic diagram of a step flow chart of a PET-CT lung cancer image segmentation method based on deep learning of the present invention. In this example, the PET-CT lung cancer image segmentation method based on deep learning includes the following steps:

[0033] Step S1: acquiring PET lung cancer image data and CT lung cancer image data, and using a multi-scale feature encoder to perform multi-scale feature extraction on the PET lung cancer image data and the CT lung cancer image data to obtain image feature data;

[0034] The embodiment of the present invention obtains PET lung cancer image data and CT lung cancer image data of a group of patients, and first pre-processes the two image data respectively, including normalization, denoising and slice adjustment. Then a multi-scale feature encoder based on a convolutional neural network (CNN) is used, which contains multiple convolution kernels of different scales, such as 3×3, 5×5 and 7×7 convolution layers, to capture local and global features in the image. Specifically, the PET image data generates a preliminary multi-scale feature map after a series of convolutions, batch normalization and ReLU activation functions. Similarly, the same processing is performed on the CT image data to generate the corresponding multi-scale feature map. In order to improve the representation ability of the features, the feature encoder also adds a deep separable convolution layer to reduce the number of parameters, and combines the residual structure to avoid the gradient disappearance problem. Finally, the extracted PET image feature data and CT image feature data are input into the subsequent fusion module.

[0035] Step S2: Perform image feature fusion based on the cross-modal attention mechanism according to the image feature data to obtain bimodal image fusion data; design an explainable attention module according to the bimodal image fusion data, and mark the attention intensity through color gradient to generate a pixel-by-pixel attention heat map;

[0036] The embodiment of the present invention inputs the image feature data extracted in step S1 into a feature fusion module based on a cross-modal attention mechanism, which uses a bilinear pooling technique to enhance the correlation between features of different modalities. In a specific implementation, a cross-modal self-attention layer is used to capture the dependencies between cross-modalities by calculating the attention weight matrix between the PET and CT feature maps. Subsequently, the fused bimodal features are input into an interpretable attention module. The module design adopts a joint mechanism of channel attention and spatial attention, and marks the attention intensity through a color gradient, for example, using a color gradient from blue to red to reflect the size of the attention weight, to generate a pixel-by-pixel attention heat map. The heat map shows a significant highlight area in the patient's lung tumor area, while the healthy area shows a lower weight distribution.

[0037] Step S3: performing heat map-guided boundary correction processing on the pixel-by-pixel attention heat map based on blind spots, overlapping lesions, and false positive detection to obtain heat map correction feature data; constructing a multi-layer segmentation decoder based on the heat map correction feature data, and performing organ and tumor volume segmentation through the multi-layer segmentation decoder to obtain tumor segmentation image data;

[0038] The embodiment of the present invention performs blind spot detection on the pixel-by-pixel attention heat map, and expands and repairs the blind spot using an algorithm based on density clustering by analyzing whether the area with low attention intensity covers important anatomical structures; for overlapping lesions, a morphological segmentation-based method is used to separate abnormal highlight areas; for false positive detection, an artifact area recognition algorithm is designed to remove areas with abnormal attention distribution in the heat map. The corrected heat map correction feature data is input into a multi-layer segmentation decoder, which uses a variant model of the U-Net architecture, including multiple jump connections and multi-scale feature fusion modules, and ultimately achieves accurate segmentation of lung organs and tumor volumes. Subsequent verification of the output tumor segmentation image data shows that the tumor boundary is clear and highly consistent with the actual morphology.

[0039] Step S4: designing a mixed loss function including Dice loss and boundary loss according to the tumor segmentation image data, and dynamically adjusting the contribution ratio of the two types of losses through learnable weight coefficients to obtain mixed loss function data, wherein the Dice loss calculates the overlap of the segmentation area, and the boundary loss focuses on the accuracy of the segmentation boundary;

[0040] The embodiment of the present invention constructs a hybrid loss function based on tumor segmentation image data, wherein the Dice loss function is expressed by the formula The overlap between the predicted segmentation area and the true segmentation area is calculated; the boundary loss uses the level set method to calculate the Euclidean distance between the segmentation boundary and the true boundary. The contribution ratio of the two loss functions is dynamically adjusted by the learnable weights. For example, the Dice loss weight is set to 0.6 and the boundary loss weight is set to 0.4 in the initial stage, and the weights are updated in real time according to the loss convergence during the training process. When this hybrid loss function optimizes the segmentation network, it significantly improves the model's ability to capture tumor morphology and the accuracy of boundary positioning.

[0041] Step S5: Perform connected domain analysis on the tumor segmentation image data according to the mixed loss function data, remove artifact areas with an area smaller than a preset threshold, and obtain candidate segmentation optimization data under different loss function weight configurations; select the optimal segmentation image based on the segmentation performance of the candidate segmentation optimization data to obtain the optimal segmentation image data.

[0042] The embodiment of the present invention performs connected domain analysis on the tumor segmentation image data optimized by the hybrid loss function, and screens out artifact areas less than 50 pixels by calculating the area of ​​each connected area. This threshold is set based on clinical experience to exclude most meaningless small artifact areas. Subsequently, candidate segmentation optimization data under multiple different loss function weight configurations are generated, and the segmentation effect is evaluated by model performance indicators (such as Dice coefficient, IoU, Hausdorff distance, etc.), and finally the segmentation image with the best performance is selected as the final result. For example, in a certain patient data, the segmentation result under the optimal configuration obtained a Dice coefficient of 0.92, showing excellent segmentation performance and meeting clinical diagnosis needs.

[0043] The present invention uses a multi-scale feature encoder to extract multi-scale features from PET and CT lung cancer image data, respectively capturing the functional metabolic features of PET and the anatomical structural features of CT, ensuring the integrity and diversity of image features. On this basis, the cross-modal attention mechanism is used to fuse the two modal features to generate dual-modal image fusion data, and the pixel-by-pixel attention heat map is generated through the interpretable attention module design to intuitively display the model's attention to the tumor area, thereby enhancing the credibility of the algorithm. Subsequently, the heat map is corrected based on the guided boundary of blind areas, overlapping lesions and false positive detection to solve the segmentation accuracy problem, and a multi-layer segmentation decoder is constructed to further optimize the segmentation results and achieve accurate segmentation of organs and tumor volumes. Based on the segmented image, a mixed loss function including Dice loss and boundary loss is designed, and the contribution ratio of the two losses is adjusted through dynamic learnable weights, which not only improves the overlap between the segmented area and the real tumor area, but also improves the accuracy of the segmentation boundary. Finally, the artifact area with an area less than the threshold is removed through connected domain analysis, and the performance of the candidate segmentation optimization data under different loss weight configurations is evaluated, and the optimal segmentation image data is automatically selected to ensure the accuracy and clinical practicality of the output results. This method comprehensively utilizes multimodal data characteristics, attention mechanism and segmentation optimization strategy to effectively improve the accuracy, robustness and interpretability of lung cancer image segmentation, providing high-quality support for clinical diagnosis and treatment.

[0044] Preferably, step S1 comprises the following steps:

[0045] Step S11: Acquire PET lung cancer image data and CT lung cancer image data;

[0046] Step S12: performing image grayscale standardization on the PET lung cancer image data and the CT lung cancer image data, and performing histogram equalization, so as to obtain PET lung cancer image cleaned data and CT lung cancer image cleaned data;

[0047] Step S13: using a multi-scale feature encoder to perform multi-scale feature extraction on the PET lung cancer image cleaned data and the CT lung cancer image cleaned data, respectively, so as to obtain PET image feature data and CT image feature data;

[0048] Step S14: performing residual connection feature fusion on the PET image feature data and the CT image feature data to obtain image feature data.

[0049] As an embodiment of the present invention, refer to Figure 2 As shown, Figure 1 Detailed step flow diagram of step S1 in the embodiment of the present invention, step S1 includes the following steps:

[0050] Step S11: Acquire PET lung cancer image data and CT lung cancer image data;

[0051] The embodiment of the present invention selects a set of clinical patients' lung cancer image data sets, including image data generated by PET scanning and image data generated by CT scanning. First, the original image data is read using the DICOM standard interface, and the necessary metadata (such as image resolution, pixel spacing, and scanner parameters) is extracted. For PET image data, ensure that the image unit is the standardized uptake value (SUV) for subsequent analysis; for CT image data, confirm that the density value is represented by the Hounsfield unit (HU). In addition, for missing slices or abnormal slices in the patient's image data, an interpolation method based on adjacent slices is used to complete the data, and the data is stored in a standardized format (such as NIfTI) for subsequent processing.

[0052] Step S12: performing image grayscale standardization on the PET lung cancer image data and the CT lung cancer image data, and performing histogram equalization, so as to obtain PET lung cancer image cleaned data and CT lung cancer image cleaned data;

[0053] The embodiment of the present invention performs grayscale value normalization processing on the acquired PET lung cancer image data and CT lung cancer image data, and uses a linear normalization method to map the grayscale value to the [0,1] interval, wherein the normalization formula is: , are the minimum and maximum grayscale values ​​of the image, respectively. Histogram equalization is then performed to enhance the image contrast. The adaptive histogram equalization (CLAHE) method is used to avoid artifacts caused by excessive concentration of highlight areas by setting the window size to 8×8 pixels and the clipping limit to 0.01. During the processing, the PET and CT data are operated independently, and the output image shows a more uniform grayscale distribution of the tumor area and surrounding tissues, and the PET image cleaning data and CT image cleaning data are obtained.

[0054] Step S13: using a multi-scale feature encoder to perform multi-scale feature extraction on the PET lung cancer image cleaned data and the CT lung cancer image cleaned data, respectively, so as to obtain PET image feature data and CT image feature data;

[0055] The embodiment of the present invention inputs the cleaned image data obtained in step S12 into a multi-scale feature encoder. The encoder is designed to include four main convolution blocks, each of which includes convolution kernels of different sizes (such as 3×3, 5×5, and 7×7) to extract features from different scales. In addition, each convolution block is connected to a batch normalization layer and a ReLU activation function to accelerate convergence and introduce nonlinear features. In the processing of PET image cleaned data, the feature encoder focuses on extracting the features of the metabolically active areas of the tumor; in the processing of CT image cleaned data, the encoder captures the density features and boundary information of the tumor. After multi-level convolution operations, the output PET image feature data and CT image feature data have high-dimensional spatial feature representations, respectively, covering the multi-scale structural information of the lesion area.

[0056] Step S14: performing residual connection feature fusion on the PET image feature data and the CT image feature data to obtain image feature data.

[0057] The embodiment of the present invention performs residual connection feature fusion on the PET image feature data and CT image feature data extracted in step S13. The specific implementation method is: the PET features and CT features are fused through element-by-element addition operation, and the two feature data are channel aligned before fusion to ensure the consistency of the dimensionality of the fusion result. Subsequently, the residual structure is introduced to superimpose the original PET features and CT features on the fused feature data to retain the independent information of each modality while improving the feature expression capability. During the feature fusion process, the number of channels of the fusion layer is set to 256, and the feature map size is set to 64×64 pixels. The image feature data finally generated integrates the metabolic and anatomical characteristics of PET and CT images, providing more robust input data for subsequent deep learning modules.

[0058] The present invention obtains PET and CT lung cancer image data, ensures the multimodality of the data source, and enables subsequent analysis to comprehensively utilize the metabolic function information of PET and the anatomical structure information of CT. In step S12, the original image data is grayscale normalized and histogram equalized, which solves the difference in grayscale distribution between different modal images, enhances the image contrast, and significantly improves the usability of the image and the accuracy of subsequent feature extraction. In step S13, a multi-scale feature encoder is used to extract features from the cleaned image data, and the key features of different spatial scales in the image are captured through multi-scale analysis, especially the metabolic information of PET and the fine anatomical structure information of CT, thereby generating PET image feature data and CT image feature data respectively. Finally, in step S14, the extracted PET and CT feature data are subjected to residual connection feature fusion, which can not only retain the integrity of the original features of each modality, but also fully combine the complementary advantages of the two modal features to generate more representative image feature data, providing a higher quality feature basis for subsequent analysis and segmentation. Through these steps, this method achieves full-process optimization from data acquisition and cleaning to feature extraction and fusion, enhances the model's ability to understand multimodal data, and lays the foundation for improving the accuracy and robustness of lung cancer image analysis.

[0059] Preferably, step S13 comprises the following steps:

[0060] The PET lung cancer image cleaning data is subjected to parallel extraction of multi-scale feature mapping based on functional metabolic features by using the first group of convolution kernels with different expansion rates, thereby obtaining PET image feature data, wherein the first group of convolution kernels with different expansion rates are 3×3 convolution kernels with an expansion rate of 1, 5×5 convolution kernels with an expansion rate of 2, and 7×7 convolution kernels with an expansion rate of 3;

[0061] The second group of convolution kernels with different expansion rates are used to perform parallel extraction of multi-scale feature mapping based on anatomical structure features on the CT lung cancer image cleaning data to obtain CT image feature data. The second group of convolution kernels with different expansion rates are 3×3 convolution kernels with a expansion rate of 1, 5×5 convolution kernels with a expansion rate of 2, and 7×7 convolution kernels with a expansion rate of 3.

[0062] The embodiment of the present invention uses the dilated convolution module in the deep convolutional network to extract multi-scale features from the PET lung cancer image cleaning data obtained in step S12. First, a convolution layer group with a parallel structure is constructed, including a 3×3 convolution kernel with a dilation rate of 1, a 5×5 convolution kernel with a dilation rate of 2, and a 7×7 convolution kernel with a dilation rate of 3. These convolution kernels can capture small-range, medium-range, and large-range metabolic activity features, respectively. The PET image cleaning data is used as input, and the convolution operation independently extracts features through the convolution kernel of each dilation rate, where the dilation rate expands the receptive field by inserting zero elements between the convolution kernels without increasing the computational cost. Through this operation, the 3×3 convolution kernel with a dilation rate of 1 can focus on the fine structure of the metabolic hotspot, the 5×5 convolution kernel with a dilation rate of 2 covers a larger metabolically active area, and the 7×7 convolution kernel with a dilation rate of 3 captures the overall metabolic features of the wide area. After the convolution operation, the feature map of each dilation rate is channel-joined to generate the final multi-scale PET image feature data. In the experiment, a single PET image with a size of 256×256 pixels and a convolution output feature map with a size of 64×64 pixels were used. The number of channels outputted at each dilation rate was 32, and the number of output channels after fusion was 96. For the CT lung cancer image cleaning data obtained in step S12, a dilated convolution strategy similar to that of PET data was adopted, but the optimization target was adjusted to focus on the extraction of anatomical structure features. First, a parallel convolution module was constructed, including a 3×3 convolution kernel with a dilation rate of 1, a 5×5 convolution kernel with a dilation rate of 2, and a 7×7 convolution kernel with a dilation rate of 3. These convolution kernels are responsible for extracting detailed boundary information, mesoscale anatomical features, and global tissue contour information, respectively. After the CT image cleaning data is input into the convolution module, convolution kernels with different dilation rates act on the input data through parallel paths. The 3×3 convolution kernel with a dilation rate of 1 highlights the sharp features of the lesion edge, the 5×5 convolution kernel with a dilation rate of 2 captures the density changes in the adjacent areas of the tumor, and the 7×7 convolution kernel with a dilation rate of 3 focuses on reflecting the morphological characteristics of the overall tissue structure. The output feature maps of all paths are merged in the channel dimension, and the channels are compressed by 1×1 convolution to obtain the final multi-scale CT image feature data. In the application scenario, for a CT image with a resolution of 512×512, the size of the output feature map of the convolution operation is 128×128 pixels, the number of output channels for each expansion rate is 64, and the number of channels of the final output feature map is 128.

[0063] The present invention can capture metabolic functional characteristics at different scales by applying a first group of convolution kernels with different expansion rates (the expansion rate of 3×3 is 1, the expansion rate of 5×5 is 2, and the expansion rate of 7×7 is 3) to the PET lung cancer image cleaning data. This design fully considers the spatial distribution differences of tumor metabolic characteristics in PET images, so that the convolution kernel with a small expansion rate focuses on local detail features, and the convolution kernel with a large expansion rate captures the global metabolic pattern, thereby generating multi-scale PET image feature data with comprehensive representativeness. Secondly, a second group of convolution kernels (same specifications) with different expansion rates are applied to the CT lung cancer image cleaning data to extract multi-scale feature mapping based on anatomical structure features. Tumors and tissue boundaries in CT images usually have complex anatomical details. Through this multi-expansion rate convolution kernel design, detailed edge features and rough morphological features can be effectively extracted, making the feature data more hierarchical and complete, and enhancing the expression of anatomical details. In addition, the introduction of dilated convolution expands the receptive field of the convolution kernel without increasing the amount of calculation, so that the model can capture local and global spatial information at the same time, which helps to overcome the problem of feature information loss caused by the receptive field limitation of traditional convolution. The multi-scale feature maps extracted in parallel further enhance the diversity and robustness of features by integrating information at multiple scales, providing a more discriminative data basis for subsequent feature fusion and image analysis. In summary, this multi-expansion rate convolution feature extraction strategy fully utilizes the modal characteristics of PET and CT images, which can not only highlight the metabolic functional characteristics of PET images, but also enhance the anatomical structural characteristics of CT images, thereby significantly improving the quality and expression ability of feature data and providing solid support for multimodal image analysis.

[0064] Preferably, step S2 comprises the following steps:

[0065] Step S21: calculating the similarity matrix between the two modal features in the image feature data by using cosine similarity, and constructing a feature correlation mapping, thereby obtaining cross-modal feature similarity mapping data;

[0066] Step S22: designing a learnable modality fusion weight matrix according to the cross-modality feature similarity mapping data, and adaptively adjusting the modality fusion weight matrix based on feature similarity through a cross-attention mechanism, thereby obtaining a modality fusion weight matrix;

[0067] Step S23: performing feature weighted fusion on the PET image feature data and the CT image feature data according to the modality fusion weight matrix, thereby obtaining dual-modality image fusion data;

[0068] Step S24: Design an explainable attention module based on the bimodal image integration data, mark the attention intensity with color gradient, and generate a pixel-by-pixel attention heat map.

[0069] As an embodiment of the present invention, refer to Figure 3 As shown, Figure 1 Detailed step flow diagram of step S2 in the embodiment of the present invention, step S2 includes the following steps:

[0070] Step S21: calculating the similarity matrix between the two modal features in the image feature data by using cosine similarity, and constructing a feature correlation mapping, thereby obtaining cross-modal feature similarity mapping data;

[0071] The PET image feature data and the CT image feature data obtained from step S14 in the embodiment of the present invention are respectively represented as shapes of The characteristic matrix of is the characteristic number, The two sets of feature matrices are standardized to eliminate the difference in feature dimensions, and the similarity matrix of cross-modal features is constructed by calculating the cosine similarity of the feature vectors. The formula is: ,in and are the i-th and j-th feature vectors of PET and CT features, respectively, is its similarity value. The matrix is ​​row-normalized to generate feature correlation mapping, and the mapping result directly represents the correlation between cross-modal features. , the size of the calculated similarity matrix is ​​256×256, and after normalization, it is output as the cross-modal feature similarity mapping data.

[0072] Step S22: designing a learnable modality fusion weight matrix according to the cross-modality feature similarity mapping data, and adaptively adjusting the modality fusion weight matrix based on feature similarity through a cross-attention mechanism, thereby obtaining a modality fusion weight matrix;

[0073] The embodiment of the present invention uses the cross-modal feature similarity mapping data generated in step S21 to initialize the modal fusion weight matrix as an N×N unit matrix, and constructs the weight matrix design formula by defining the learnable parameter W: , where S is the similarity mapping matrix, W is the learnable weight parameter, and softmax ensures that the matrix elements are normalized to weighted proportions. We further use the cross-attention mechanism to adaptively adjust the weight matrix, use the PET feature data as the query vector, and the CT feature data as the key and value, and update the fusion weights through the cross-dot product operation, so that the final output weight matrix highlights the feature pairs with high similarity. In the experiment, the initial weight matrix dimension is 256×256, and the cross-attention mechanism training is optimized through 5 rounds of iterations to finally generate the modality fusion weight matrix.

[0074] Step S23: performing feature weighted fusion on the PET image feature data and the CT image feature data according to the modality fusion weight matrix, thereby obtaining dual-modality image fusion data;

[0075] The embodiment of the present invention starts from the modality fusion weight matrix generated in step S22 and performs feature weighting processing on the PET image feature data and the CT image feature data. The formula is: ,in and are the PET and CT feature matrices respectively, is the fusion weight matrix. Through the matrix weighted sum operation, the two-modal features are mapped to a unified feature space to form a dual-modal image fusion data. In practical applications, for an input feature dimension of 256×128, the output fusion feature dimension is the same size, which can maintain efficient calculation and the integrity of semantic information at the same time.

[0076] Step S24: Design an explainable attention module based on the bimodal image integration data, mark the attention intensity with color gradient, and generate a pixel-by-pixel attention heat map.

[0077] The embodiment of the present invention utilizes the bimodal image generated in step S23 to integrate data and designs an interpretable attention module, which combines channel weighted attention and spatial attention to highlight the region of interest. First, the bimodal feature data is channel-weighted to generate an importance score for each channel; second, the importance information of the local pixel area is further extracted through the spatial attention module. The channel weight is multiplied by the spatial attention result to generate a pixel-by-pixel attention distribution map. The attention intensity is marked with a color gradient (such as a gradient from blue to red), where blue indicates a low attention area and red indicates a high attention area. The output pixel-by-pixel attention heat map is saved at a resolution of 256×256 pixels for subsequent boundary correction and segmentation optimization steps.

[0078] The present invention calculates the similarity matrix of the two modal features in the image feature data by cosine similarity, and constructs a cross-modal feature correlation mapping. This feature analysis method based on similarity can effectively quantify the correlation between PET and CT image features, reveal the complementary characteristics between modalities, and provide accurate correlation data support for subsequent fusion. The cross-modal feature similarity mapping data is used to design a learnable modality fusion weight matrix, and the weights are adaptively adjusted through a cross-attention mechanism. The cross-attention mechanism can dynamically optimize the modality weight distribution according to feature similarity, so that the fusion process can highlight important features while suppressing redundant information, thereby achieving more efficient and more accurate modality feature fusion. The modality fusion weight matrix performs feature weighted fusion on the PET and CT image feature data to generate dual-modality image fusion data. This weighted fusion strategy ensures the effective integration of PET metabolic features and CT anatomical features, retains modality specificity, and improves the expressive power of joint features, laying a solid foundation for the comprehensive utilization of multimodal information. An interpretable attention module is designed based on dual-modality image fusion data, and a pixel-by-pixel attention heat map is generated by marking the attention intensity with a color gradient. This visualization method intuitively shows the area and feature strength that the model focuses on, which not only enhances the interpretability of the model, but also provides a clear reference for medical staff, helping to improve the credibility and scientificity of the diagnosis. In summary, these steps, with cross-modal feature similarity as the core, combined with a dynamically adjusted modality fusion mechanism and interpretable visualization design, not only significantly improve the accuracy and reliability of dual-modal feature fusion, but also enhance the interpretability and application value of the results, providing an efficient and intelligent solution for PET and CT fusion diagnosis.

[0079] Preferably, step S21 includes the following steps:

[0080] Step S211: performing feature dimension alignment preprocessing on the image feature data, and determining the sameness of the space and channel dimensions of the two-modal feature data, thereby obtaining cross-modal feature alignment data;

[0081] In the embodiment of the present invention, the PET image feature data and the CT image feature data obtained in step S14 are respectively represented as and A tensor of , where H and W are spatial dimensions, , is the number of channels. Bilinear interpolation is used to adjust the spatial dimension and unify the two modal features to the same resolution. For example, Adjust to Then, a 1×1 convolution kernel is used to map the number of feature channels of the two modalities, so that and Alignment, for example, uniformly adjust to 128 channels. The shape of the feature data after alignment is 256×256×128, thus ensuring direct comparability between cross-modal features.

[0082] Step S212: calculating the cosine similarity between each feature vector in the PET image feature data and the corresponding feature vector in the CT image feature data according to the cross-modality feature alignment data, thereby constructing an initial feature similarity matrix;

[0083] In the embodiment of the present invention, in the aligned cross-modal feature tensor, the feature vector at each position is represented as a vector with a length of 128. The cosine similarity formula is used to calculate the similarity of the PET and CT feature vectors at the corresponding positions, and the formula is: ,in Indicates location After calculating each pair of feature vectors, an initial feature similarity matrix with a shape of 256×256 is generated, and the value range is between [-1,1].

[0084] Step S213: normalizing the initial feature similarity matrix and enhancing the matrix comparability, thereby obtaining a normalized feature similarity matrix;

[0085] In the embodiment of the present invention, the values ​​in the initial feature similarity matrix are adjusted to the interval [0, 1] by linear normalization, and the normalization formula is: ,in and are the minimum and maximum values ​​of the initial similarity matrix. At the same time, gamma correction is used to perform nonlinear adjustment on the similarity matrix, and the gamma value is set to 1.2 to enhance the contrast effect of high similarity areas. The normalized matrix size is kept at 256×256.

[0086] Step S214: performing spatial attention weight mapping on the normalized feature similarity matrix and highlighting feature regions with high correlation, thereby obtaining a feature similarity matrix;

[0087] The embodiment of the present invention uses a spatial attention module to perform weighted processing on the matrix based on the normalized feature similarity matrix. The specific steps include global average pooling and global maximum pooling of the matrix to obtain two spatial weight distributions, which are fused through a convolution layer with a convolution kernel size of 3×3 to generate the final spatial attention weight map. By multiplying the attention weights by the original matrix element by element, the feature areas with high correlation are highlighted. The size of the output feature similarity matrix is ​​maintained at 256×256.

[0088] Step S215: Enhance the feature similarity matrix through a multi-head attention mechanism, and capture cross-modal feature associations of different scales and channels, so as to obtain cross-modal feature association data;

[0089] The embodiment of the present invention further optimizes the feature similarity matrix using a multi-head attention mechanism, using 8 attention heads to capture the feature correlations between different scales and channels. In the specific implementation, each attention head performs self-attention calculation on the feature similarity matrix, and the formula is: )V, where denote query, key, and value matrices respectively, is the scaling factor to maintain stability. Finally, the output features of each head are concatenated and linearly transformed to obtain cross-modal feature association data, and the output size is 256×256.

[0090] Step S216: Generate a feature correlation map based on the cross-modal feature correlation data, and quantify the semantic and spatial similarities between the PET and CT image features, thereby obtaining cross-modal feature similarity mapping data.

[0091] The embodiment of the present invention extracts semantic similarity and spatial similarity features from the cross-modal feature association data generated by the multi-head attention mechanism, and calculates the degree of match between the PET and CT features at each position respectively. The semantic similarity score is defined as the channel-weighted sum of the cross-modal association matrix, and the spatial similarity score is the Euclidean distance of each feature vector. The two types of scores are converted into color gradient labels through linear mapping to generate a feature association map with a shape of 256×256. The final mapping result quantifies the strength of the correlation between the two modal features in different regions, providing guidance for subsequent modal fusion.

[0092] The present invention performs feature dimension alignment preprocessing on image feature data, unifying the representation of two-modal feature data in space and channel dimensions. This feature alignment operation effectively eliminates the structural differences between modalities, ensures the basic consistency of subsequent cross-modal feature analysis, and thus improves the accuracy and stability of cross-modal calculation. The initial feature similarity matrix is ​​constructed by calculating the cosine similarity between PET and CT feature vectors. This calculation method based on cosine similarity can accurately measure the relative relationship between the two modal features, highlight the angular similarity between features, and help to mine the intrinsic association between modalities. The initial feature similarity matrix is ​​normalized, which not only improves the comparability of the data range, but also enhances the recognition of the matrix when representing feature relationships. Normalization eliminates the influence of different modal feature value ranges, making similarity calculation more reliable and accurate. Through spatial attention weight mapping, highly correlated areas in the feature similarity matrix are highlighted, thereby strengthening the expression of key features. The introduction of spatial attention can focus on important feature areas related to diagnosis, reduce the interference of redundant information, and provide high-quality feature data for further analysis. The feature similarity matrix is ​​enhanced by using a multi-head attention mechanism, which effectively captures the correlation of cross-modal features at different scales and channels. The application of this multi-head attention mechanism not only expands the richness of feature representation, but also fully reveals the complex interactive relationship between the two modal features. The feature correlation map is generated based on the enhanced feature similarity matrix to quantify the semantic and spatial similarity between PET and CT image features. This feature correlation map not only realizes the precise correlation of features between modalities, but also provides intuitive and interpretable support for the joint analysis of dual-modal images. In summary, these steps effectively integrate PET metabolic features and CT anatomical features by aligning, calculating similarity and enhancing attention on cross-modal features, forming a comprehensive and accurate cross-modal feature similarity representation. This method significantly improves the reliability and scientificity of cross-modal image feature association, lays a solid foundation for multimodal medical image analysis, and provides richer feature support and higher diagnostic accuracy for disease diagnosis.

[0093] Preferably, step S24 comprises the following steps:

[0094] Step S241: Initialize the attention weight mapping layer, and integrate the bimodal image into the data input attention weight mapping layer for dimension reshaping, thereby obtaining attention mapping feature data;

[0095] The embodiment of the present invention uses a deep learning framework such as PyTorch or TensorFlow to construct a fully connected layer as the attention weight mapping layer and initialize its parameters. The weight initialization adopts the Xavier initialization method to ensure gradient stability. The input bimodal image is integrated into the data dimension (For example, ). In the weight mapping layer, the input data is passed through a shape of (For example, ) is used to reduce the number of channels to 64, and then the output data is adjusted to a dimension of (For example, ). The obtained attention map feature data retains the spatial distribution information and key feature weights.

[0096] Step S242: designing a softmax normalization layer according to the attention map feature data, and using the softmax normalization layer to normalize the attention weights of the attention map feature data, thereby obtaining normalized attention weight data, wherein the softmax normalization layer specifically maps the attention weights in the attention map feature data to the interval [0,1], and ensures that the sum of the attention weights is 1;

[0097] In the embodiment of the present invention, in the feature data, the weight vector at each pixel position is represented by a length of For example, a vector of 64). Design a softmax normalization layer to normalize the weight vector of each pixel position. The normalization formula is: ,in is the current weight value, ensuring that the sum of weights on all channels is 1. The normalization process is performed independently at all pixel positions, and the dimension of the output normalized attention weight data is (e.g., 256×256×64), and the weight values ​​are distributed between [0,1].

[0098] Step S243: generating a visually interpretable color mapping according to a preset color gradient mapping rule and normalized attention weight data, thereby obtaining a color-coded attention heat map;

[0099] The embodiment of the present invention converts the numerical value into color code based on the normalized attention weight data according to the preset color gradient mapping rules, for example, using the "jet" or "viridis" color scheme in Matplotlib. First, the weight vector at each pixel position is taken as the representative weight value to generate the color code; then the weight value is mapped to the RGB space through the color gradient rule, for example, a weight value of 0.5 corresponds to middle green, close to 1 corresponds to red, and close to 0 corresponds to blue. The size of the generated color code matrix is (For example, ), each pixel position is displayed as a color value, forming a preliminary attention heat map.

[0100] Step S244: Perform resolution restoration processing on the attention heat map through the deconvolution layer and superimpose the color gradient mapping to generate the final pixel-by-pixel attention heat map.

[0101] The embodiment of the present invention uses a deconvolution layer to perform upsampling and restoration processing on the initially generated heat map, and the specific setting is that the convolution kernel size is , with a step size of 2 and a padding of 1, thus increasing the heat map resolution from Restore to original size . Then, the color gradient map is superimposed again to ensure that the color mark of each pixel position is clear and enhance visual interpretability. The resolution of the final pixel-by-pixel attention heat map is , the attention strength of different regions can be clearly marked for further segmentation or feature analysis tasks.

[0102] The present invention can effectively extract key feature areas in the image by initializing the attention weight mapping layer and integrating the bimodal image into the data input layer for dimensional reshaping. The attention mapping layer can automatically identify the area with important information in the image and assign it a higher weight, thereby improving the accuracy of feature expression. Through dimensional reshaping, the model can better perform spatial processing on the input data, laying the foundation for subsequent feature fusion and attention enhancement. The attention weight of the attention mapping feature data is normalized using the softmax normalization layer. Softmax normalization ensures that the weight of each feature is in the interval [0, 1] and the sum of all attention weights is 1. This can further improve the stability of the model, make the distribution of attention more balanced and reasonable, and thus enhance the expressive power of the model. Normalization eliminates the influence of the numerical differences between different features, so that the model can focus more on the most important image area. Based on the preset color gradient mapping rules and the normalized attention weight data, a visually interpretable color map is generated. This color-coded attention heat map can provide clear visualization information for doctors or researchers, intuitively display the area of ​​attention of the model, and improve the transparency and interpretability of image analysis. Through color mapping, users can quickly identify the areas in the image that need to be focused on, which helps with accurate diagnosis. The resolution of the attention heat map is restored through the deconvolution layer, and the color gradient mapping is superimposed to generate the final pixel-by-pixel attention heat map. This process can restore the detailed information in the image and reflect the attention distribution to the pixel level of the original image, thereby providing more accurate positioning and analysis. The final pixel-by-pixel heat map shows the attention intensity of each pixel, providing a detailed reference for subsequent segmentation, recognition and diagnosis tasks. Overall, these steps effectively improve the interpretability and visualization of image features by building attention mechanisms, normalization processing, color mapping, and resolution restoration, and greatly enhance the transparency of image analysis. They can not only help professionals in the field of medical imaging better understand the decision-making process of the model, but also provide more valuable support for the diagnosis and treatment of diseases, thereby improving diagnostic accuracy and the practicality of clinical applications.

[0103] Preferably, step S3 comprises the following steps:

[0104] Step S31: identifying areas where the tumor boundary is unclear or blocked on the pixel-by-pixel attention heat map, thereby obtaining blind area detection data;

[0105] The embodiment of the present invention inputs the pixel-by-pixel attention heat map into the region growing algorithm, uses the initially segmented high-attention area as the seed point, gradually expands to the surrounding area, and detects the area where the growth stops or the attention drops significantly as the boundary. For the boundary area, calculate its gradient value and contrast. If the gradient amplitude is less than the set threshold (such as 0.05) and the contrast is lower than the set standard (such as 10%), it is marked as an unclear boundary area. At the same time, a convolutional neural network is used to detect the local occlusion of the attention heat map, which is specifically set to a feature detection network with a convolution kernel size of 3×3 and a step size of 1. Output blind spot detection data to mark the specific location of the unclear or occluded area.

[0106] Step S32: performing overlapping lesion area analysis on the pixel-by-pixel attention heat map, and locating tumor areas that may have multiple overlaps or blurs, thereby obtaining overlapping lesion area data;

[0107] The embodiment of the present invention performs connected domain analysis on multiple high-attention areas in the pixel-by-pixel attention heat map, extracts the boundaries of adjacent areas, and calculates the Euclidean distance and feature similarity between areas. If the distance is less than a set threshold (such as 5 pixels) and the feature similarity is higher than 90%, it is judged as an overlapping lesion area. In addition, the distribution of high-attention areas is fitted by Gaussian MixtureModel (GMM). If the distribution curve of the fitting result shows a multi-peak distribution, the fuzzy area is further decomposed. Overlapping lesion area data is generated, and the location and features of the areas where possible overlapping lesions are marked are output.

[0108] Step S33: performing false positive area detection on the pixel-by-pixel attention heat map to obtain false positive area data;

[0109] The embodiment of the present invention classifies possible false positive areas in the heat map through a random forest classifier, and the input features include attention intensity value, gradient change rate of the surrounding area, area of ​​the area, etc. During the training process, the model is trained using a data set containing real tumor and non-tumor areas. All attention areas are predicted by the classifier, and areas with a probability lower than a threshold (such as 0.7) are marked as false positive areas. At the same time, small isolated points are further cleaned up through morphological opening operations to avoid false detection. The output false positive area data contains the coordinates, area and classification confidence of the false positive area.

[0110] Step S34: performing guided boundary correction processing based on morphological operators according to the blind area detection data, the overlapping lesion area data and the false positive area data, so as to obtain thermal map correction feature data;

[0111] The embodiment of the present invention performs morphological closing operation on the marked areas in the blind spot detection data to fill small discontinuous areas, and the convolution kernel is set to a circular structural element of size 5×5; the watershed algorithm is used to segment the fuzzy boundary area for the overlapping lesion area data, and the segmentation line is determined by the regional gradient and attention value distribution; the morphological opening operation is performed on the false positive area data to remove artifact points, and the convolution kernel size is 3×3. After the three data processing results are superimposed, the corrected thermal map feature data is generated for further segmentation processing.

[0112] Step S35: constructing a multi-layer segmentation decoder according to the corrected characteristic data of the heat map, and performing organ and tumor volume segmentation through the multi-layer segmentation decoder, thereby obtaining a probabilistic segmentation map of the organ and tumor volume;

[0113] The embodiment of the present invention constructs a U-Net decoder containing 4 upsampling modules, where each layer of the decoder consists of a deconvolution layer, a ReLU activation layer, and a Batch Normalization layer, and the deconvolution kernel size is 4×4, the step size is 2, and the padding is 1. The corrected heat map feature data is input, and the spatial resolution is gradually restored through each layer of the decoder, and the intermediate features of the encoder are spliced ​​into the decoder in combination with the residual connection. The final output is a probability segmentation map of size 256×256×2, where each pixel corresponds to the segmentation probability of the organ or tumor volume.

[0114] Step S36: Select a segmentation threshold based on the probability segmentation map to obtain tumor segmentation image data.

[0115] The embodiment of the present invention analyzes the probability value of each pixel in the probability segmentation map and automatically calculates the segmentation threshold using the Otsu method. If the target application requires manual setting of the threshold, the threshold can be set to 0.5 to distinguish between the tumor area and the background area. Subsequently, the binarized tumor segmentation image data is generated by threshold segmentation, and the boundary is further repaired using morphological closing operations. The convolution kernel size is 3×3 to ensure the boundary continuity and accuracy of the segmentation result. The output tumor segmentation image data can be used for volume measurement of the tumor area and further clinical analysis.

[0116] The present invention obtains blind spot detection data by identifying areas where the tumor boundary is unclear or occluded on the pixel-by-pixel attention heat map. This process effectively identifies areas where information is lost due to unclear tumor boundaries or occlusion. Blind spot detection can help the model find out which areas may lack sufficient features or be mistakenly ignored, thereby providing a more accurate reference for subsequent correction and segmentation, avoiding misjudgment and omission. By performing overlapping lesion area analysis on the pixel-by-pixel attention heat map and locating tumor areas that may have multiple superpositions or blurs, overlapping lesion area data is obtained. This process can effectively discover the overlap of multiple tumor areas, especially in CT or PET images, where the overlap of tumors often leads to unclear boundaries. Through overlapping lesion area analysis, these fuzzy areas can be identified and processed in advance, thereby improving the accuracy of segmentation and preventing overlapping areas from causing incorrect tumor volume estimation. False positive area data is obtained by performing false positive area detection on the pixel-by-pixel attention heat map. False positive detection is an important link in processing medical images, which can identify areas that the model may mistakenly identify as tumors. Through this step, some misjudged areas can be effectively removed, irrelevant false signals can be avoided in tumor detection and segmentation, and the overall accuracy of the model can be improved. Through guided boundary correction processing based on morphological operators, the heat map is corrected according to the blind area detection data, overlapping lesion area data and false positive area data. Morphological operators can perform boundary correction in images, especially for areas where the tumor boundaries are unclear, overlapping or blocked. Morphological processing can smooth the edges and accurately delineate the actual boundaries of the tumor, thereby optimizing the heat map and ensuring that the data for subsequent segmentation processing is more accurate. By constructing a multi-layer segmentation decoder based on the corrected feature data of the heat map, and using the decoder to segment the organ and tumor volume, a probabilistic segmentation map of the organ and tumor volume is obtained. The multi-layer segmentation decoder can perform more detailed segmentation based on the data corrected by the heat map, accurately distinguish organs and tumors, and generate a probabilistic segmentation map for each area. This helps to clarify the spatial distribution of tumors in the image, optimize volume measurement results, and improve segmentation accuracy. By selecting the segmentation threshold based on the probabilistic segmentation map, the tumor segmentation image data is finally obtained. This processing further optimizes the segmentation effect of the tumor area by selecting the threshold for the probability value in the segmentation map. By selecting the appropriate threshold, irrelevant areas in the segmentation can be effectively eliminated, ensuring that the tumor area is accurately extracted and clearly distinguished from the surrounding tissue, thereby enhancing the accuracy and reliability of the final tumor segmentation. Overall, these steps optimize the tumor segmentation process and reduce errors and uncertainties by accurately processing blind areas, overlapping lesions, and false positive areas, combined with morphological operators to guide boundary correction. Ultimately, through precise multi-layer segmentation decoders and segmentation threshold selection, more accurate tumor segmentation image data is obtained, providing strong support for tumor diagnosis and treatment.

[0117] Preferably, step S35 includes the following steps:

[0118] A multi-layer segmentation decoder is built using a symmetric U-Net architecture. The heat map correction feature data is used as input, and the image resolution is gradually restored through the transposed convolutional layer of the multi-layer segmentation decoder. Skip connections are set in the multi-layer segmentation decoder to fuse low-level texture features and enhance the retention of segmentation details. A softmax activation function is applied to the last layer of the multi-layer segmentation decoder to output a probabilistic segmentation map of the organ and tumor volume.

[0119] The embodiment of the present invention first designs a symmetrical U-Net architecture, which consists of two parts: an encoder and a decoder, wherein the decoder consists of four upsampling modules. The heat map correction feature data (with a shape of 256×256×64) is used as input, and the features are gradually restored through the decoder. In each layer of the decoder, a transposed convolution layer is used for upsampling, and the transposed convolution kernel size is set to 4×4, the step size is 2, and the padding is 1, so that the spatial resolution is gradually improved while maintaining the consistency of the feature dimension. In each layer of the decoder, a jump connection is set to extract low-level texture features from the corresponding layer of the encoder, and fuse them with the current decoder features through feature concatenation (concat), so that the boundary texture information can be retained and the fineness of the segmentation can be improved. For example, the first layer of jump connection fuses the 64×64×128 features in the encoder with the upsampled features of the decoder, the second layer of jump connection fuses 128×128×64 features, and so on. The input heat map correction feature data is first passed to the bottom module of the decoder, and the resolution is gradually upsampled from 64×64 to 256×256 using the transposed convolution layer. Each transposed convolution layer is followed by a ReLU activation function and a Batch Normalization layer to ensure numerical stability and gradient flow. For example, the output feature size of the first layer of transposed convolution is 128×128×64, and the second layer output is 256×256×32. After each upsampling, the features are gradually restored, and low-level features are introduced through skip connections to ensure the accuracy of edge and local information. For each decoder module, while upsampling, feature splicing is performed by extracting features of the corresponding level from the encoder. The specific implementation method is: the encoder and decoder share the number of channels at each resolution layer. For example, the skip connection of the first layer obtains a 128×128×128 joint feature by splicing the 128×128×64 encoder feature to the corresponding decoder feature, and then compresses the feature through a 3×3 convolution kernel, and the number of output channels is restored to 64. Through this jump connection mechanism, the texture detail information lost in the deep features can be supplemented, thereby enhancing the edge detail retention of the segmentation. A 1×1 convolution layer is set at the last layer of the decoder to compress the number of channels to the number of categories (i.e., 2 channels, representing the segmentation probability of organs and tumors respectively). Next, the output 2-channel data is normalized by the softmax activation function to ensure that the sum of the segmentation probability of each pixel is 1. The output probability segmentation map has a size of 256×256×2, the first channel represents the probability of belonging to the organ region, and the second channel represents the probability of belonging to the tumor region.Finally, the probability segmentation map is verified through visualization tools. For example, in the simulated data, the heat map correction feature is set to the characteristic value of the actual tumor volume, and the segmentation result is compared with the actual annotation to calculate the Dice coefficient and IoU value to ensure that the segmentation accuracy meets clinical needs (Dice coefficient greater than 0.85, IoU value greater than 0.80).

[0120] The symmetrical U-Net architecture of the present invention has an encoder-decoder structure, where the encoder is responsible for extracting deep features of the image, and the decoder restores more detailed image information by gradually restoring the spatial resolution of the image. This architecture is very suitable for image segmentation tasks, especially in medical images, and can simultaneously consider the global context and detailed local features of the image, thereby improving segmentation accuracy. The transposed convolution layer learns how to restore low-resolution feature maps to higher spatial resolutions through back propagation, and gradually restores more detailed structural details through multiple layers of transposed convolution. For the segmentation of medical images, especially in tumor and organ segmentation, restoring resolution can effectively retain image details and help accurately segment tumor areas with blurred edges and organs with complex structures. The jump connection directly passes the low-level feature map (usually containing texture and detail information) to the high-level part of the decoder, thereby fusing the detail information during the upsampling process. This is particularly important for medical image segmentation, because the low-level features contain detailed information of the image (such as edges, textures, etc.), which are often lost in high-level features. Through the jump connection, the decoder can use the low-level features to retain more details, avoid the problems of blur and unclear boundaries, enhance the accuracy of tumor boundaries, and ensure accurate positioning and segmentation of tumors. The jump connection enables the multi-layer decoder to make full use of low-level detail features when restoring the image, avoiding the loss of key details in the resolution restoration process and ensuring the precision of the segmentation results. In particular, for complex structures such as tumors, the retention of segmentation details can improve the accuracy of their contours and the precise measurement of tumor volume. The Softmax activation function is applied in the last layer to convert the output of the network into a category probability distribution for each pixel. The Softmax function ensures that the segmentation result of each pixel in the output image corresponds to a clear category (such as tumor, normal tissue, etc.) and is accurately classified at the pixel level. This can effectively distinguish tumors from normal tissues, thereby generating a probabilistic segmentation map of the tumor volume. The Softmax normalization process ensures that the output probability distribution meets the normalization conditions, which helps to improve the stability and consistency of the segmentation results. In general, these steps improve the accuracy, detail retention and boundary clarity of tumor and organ segmentation by combining the powerful characteristics of symmetric U-Net, using transposed convolutional layers, jump connections and Softmax activation functions. By gradually restoring image resolution and fusing low-level detail information through a multi-layer decoder structure, this method can demonstrate greater robustness and accuracy in complex medical image segmentation tasks, especially tumor segmentation, and ultimately provide doctors with more accurate tumor volume assessment and diagnostic information.

[0121] Preferably, step S5 comprises the following steps:

[0122] Step S51: performing connected domain analysis on the tumor segmentation image data according to the mixed loss function data, extracting segmentation regions of different sizes and shapes, and identifying potential artifact regions, thereby obtaining artifact marker segmentation data;

[0123] The embodiment of the present invention inputs the tumor segmentation image data into the connected domain analysis module, and first uses the eight-connectivity algorithm to mark the connected regions in the segmented image, and each connected region is assigned a unique label value. Then, the confidence value of each connected region is calculated according to the mixed loss function data (for example, combining Dice loss and cross entropy loss) to identify potential artifact regions. Artifact regions usually appear as discrete small regions or irregular regions with low confidence, and these regions are marked by setting a confidence threshold (such as less than 0.3). The output artifact marking segmentation data contains the shape, size and artifact marking information of all connected regions in the segmented image.

[0124] Step S52: removing artifact regions whose areas are smaller than a preset threshold according to the artifact mark segmentation data, thereby obtaining candidate segmentation optimization data under different loss function weight configurations;

[0125] The embodiment of the present invention counts the area of ​​each connected region based on the artifact marker segmentation data and compares it with a preset threshold (such as 100 pixels). Artifact regions with an area smaller than the preset threshold are judged as noise and removed from the segmented image, while regions with larger areas are retained to form candidate segmentation optimization data. The specific operation of removing artifacts includes setting the pixel value of the region less than the threshold to the background value (0). The candidate segmentation optimization data finally output is a segmentation image with artifact interference removed, and the main connected regions that may belong to the tumor are retained.

[0126] Step S53: extracting morphological features based on roundness, boundary smoothness and convexity from the candidate segmentation optimization data, thereby obtaining morphological feature data;

[0127] The embodiment of the present invention extracts morphological features from each connected region in the candidate segmentation optimization data, specifically including three indicators: roundness, boundary smoothness, and convexity. The roundness is obtained by calculating the ratio of the area of ​​the region to the area of ​​the circumscribed circle, and the formula is: , which is used to evaluate the degree of closeness of the segmented area to a circle; the boundary smoothness is obtained by calculating the gradient change rate of the pixels at the boundary of the region, which is used to measure the smoothness of the boundary; the convexity is obtained by calculating the ratio of the area of ​​the region to the area of ​​its convex hull, which is used to reflect the degree of concavity of the region. These morphological features of each connected region are stored as morphological feature data for subsequent segmentation image performance optimization.

[0128] Step S54: performing optimal segmentation image selection based on segmentation performance on the morphological feature data to obtain optimal segmentation image data.

[0129] The embodiment of the present invention uses morphological feature data to score each candidate segmentation optimization data through a segmentation performance evaluation function, which combines the weights of the region's roundness, boundary smoothness, and convexity (such as a weight ratio of 0.4:0.3:0.3). First, the segmentation score is calculated based on the morphological features of each connected region, and the overall segmented image is comprehensively scored; then, the segmented image with the highest score is selected as the optimal segmented image data. In actual application scenarios, the quality reliability of the segmented image is ensured by verifying the degree of match between the segmented image and the true annotation (for example, the Dice coefficient is greater than 0.85), thereby outputting the optimal segmentation result for clinical diagnosis.

[0130] The present invention performs connected domain analysis on tumor segmentation image data through a mixed loss function, and can effectively identify and extract segmentation regions of different sizes and shapes. In medical images, artifacts (i.e., irrelevant errors or noise) may affect the accuracy of tumor regions. Through connected domain analysis, different regions can be distinguished, potential artifact regions can be identified, and these artifacts can be prevented from affecting the final segmentation results. This lays the foundation for subsequent artifact removal and segmentation optimization. According to the artifact marking segmentation data, the segmentation results are further optimized by removing artifact regions whose areas are less than a preset threshold. This step focuses on the tumor itself by excluding regions with small volumes that may be noise or artifacts. This method helps to eliminate the noise of mis-segmentation, making the segmentation results clearer and more accurate. Extracting morphological features such as roundness, boundary smoothness and convexity from candidate segmentation optimization data helps to evaluate the shape and quality of the segmentation region. These morphological features can quantify the geometric properties of the segmentation region, such as whether the tumor region is smooth and whether it has a reasonable shape. By extracting these features, the accuracy of the segmentation results can be further improved. Through morphological feature analysis based on segmentation performance, the morphological feature data is selected for optimal segmentation images. This step selects the optimal segmentation image by comprehensively considering morphological features and segmentation performance. This ensures that the final segmentation result not only conforms to the actual structure of the tumor in terms of morphology, but also reaches the optimal state in terms of performance (such as segmentation accuracy, sensitivity, etc.). Overall, these steps gradually remove interference factors in the segmentation results through connected domain analysis, artifact removal, morphological feature extraction and optimal segmentation selection, thereby improving the accuracy and detail retention of tumor segmentation. Each step helps to enhance the accuracy, stability and adaptability of the segmentation results to complex tumor morphology, ultimately achieving high-quality tumor segmentation and providing doctors with more reliable tumor detection and evaluation tools.

[0131] The present invention also provides a PET-CT lung cancer image segmentation system based on deep learning, which is used to execute the above-mentioned PET-CT lung cancer image segmentation method based on deep learning. The PET-CT lung cancer image segmentation system based on deep learning includes:

[0132] A multi-scale feature extraction module is used to obtain PET lung cancer image data and CT lung cancer image data, and use a multi-scale feature encoder to perform multi-scale feature extraction on the PET lung cancer image data and the CT lung cancer image data to obtain image feature data;

[0133] The cross-modal attention fusion module is used to fuse image features based on the cross-modal attention mechanism according to the image feature data to obtain the bimodal image fusion data; the explainable attention module is designed according to the bimodal image fusion data, and the attention intensity is marked by the color gradient to generate a pixel-by-pixel attention heat map;

[0134] The heat map guided boundary correction module is used to perform heat map guided boundary correction processing on the pixel-by-pixel attention heat map based on blind spots, overlapping lesions and false positive detection to obtain heat map correction feature data; a multi-layer segmentation decoder is constructed based on the heat map correction feature data, and organs and tumor volumes are segmented through the multi-layer segmentation decoder to obtain tumor segmentation image data;

[0135] The hybrid loss optimization module is used to design a hybrid loss function including Dice loss and boundary loss according to the tumor segmentation image data, and dynamically adjust the contribution ratio of the two types of losses through learnable weight coefficients to obtain the hybrid loss function data, where Dice loss calculates the overlap of the segmentation area, and boundary loss focuses on the accuracy of the segmentation boundary;

[0136] The artifact removal and optimal segmentation selection module is used to perform connected domain analysis on the tumor segmentation image data according to the mixed loss function data, remove artifact areas with an area smaller than a preset threshold, and obtain candidate segmentation optimization data under different loss function weight configurations; and select the optimal segmentation image based on the segmentation performance of the candidate segmentation optimization data to obtain the optimal segmentation image data.

[0137] The present invention uses a multi-scale feature encoder to process PET and CT images respectively to extract image features at different scales. Since PET and CT images have different spatial resolutions and visual features, multi-level information of tumors can be captured through multi-scale extraction. Extracting multi-scale features can more comprehensively represent the information in the image, so that the model can recognize the morphology and changes of tumors at different scales. In view of the different characteristics of PET and CT images, their information can be effectively integrated to enhance the recognition ability of tumors. According to the extracted image feature data, the feature data of PET and CT images are fused using a cross-modal attention mechanism. The module focuses on important image areas through an attention mechanism and fuses the complementary information of the two modal images. Through the cross-modal attention mechanism, the most relevant parts of the image are automatically focused on and strengthened to reduce the interference of irrelevant information. The features of PET and CT images are effectively integrated to provide more comprehensive tumor information and improve the accuracy and robustness of detection. According to the dual-modal image fusion data, an interpretable attention module is designed to mark the attention intensity through a color gradient and generate a pixel-by-pixel attention heat map. This can help doctors understand the key areas of the model's attention and increase the transparency of diagnosis. The generated attention heat map provides a visual basis for model decision-making, helps explain the key areas of the model's attention, and increases the credibility of the model. Doctors can understand the location and characteristics of the tumor more intuitively based on the heat map, which assists in clinical decision-making. Based on the pixel-by-pixel attention heat map, the module detects and corrects the boundaries of blind areas, overlapping lesions, and false positive areas. In this way, the segmentation results are more accurate and the wrong segmentation areas are reduced. By guiding the boundary correction, the interference areas in the heat map, such as blind areas and false positives, are eliminated to ensure that the tumor boundaries are clearer. For the areas with overlapping lesions, the real boundaries of the tumor can be better distinguished through correction processing. Based on the heat map correction feature data, the module constructs a multi-layer segmentation decoder, gradually restores the image resolution, and performs organ and tumor volume segmentation to obtain tumor segmentation image data. Through the design of jump connections and decoder layers, low-level texture features can be retained and segmentation details can be improved. Through multi-layer decoders and jump connections, the detailed features of the tumor can be retained and the accuracy of the segmentation results can be improved. The image resolution is restored layer by layer, and the segmentation results are optimized through the decoder to provide high-quality tumor volume segmentation. According to the tumor segmentation image data, a mixed loss function of Dice loss and boundary loss is designed, and the contribution ratio of the two types of losses is dynamically adjusted through learnable weight coefficients. This module optimizes the accuracy of the segmentation boundary and the overlap of the tumor area. Through the mixed loss function, the overlap of the segmentation area and the boundary accuracy are balanced to improve the segmentation quality. The learnable loss weight allows the model to automatically adjust the loss contribution ratio according to the actual situation during the training process, thereby optimizing the segmentation results. This module performs connected domain analysis on the tumor segmentation image data, removes artifact areas with an area smaller than the preset threshold, and selects the optimal segmentation image based on the segmentation performance.Through these operations, it is possible to eliminate mis-segmentation and artifact areas to ensure the accuracy of the segmentation results. Small artifact areas are removed to reduce the interference of noise on the segmentation results. Through selection based on segmentation performance, the optimal tumor segmentation image is finally selected to improve the quality of detection results. These modules work together to ensure the accuracy and reliability of lung cancer tumor segmentation through multimodal information fusion, attention mechanism optimization, boundary correction and artifact removal. Each module helps to improve the quality of tumor segmentation, reduce mis-segmentation, and ultimately provide accurate tumor segmentation results to help doctors make better clinical diagnoses and decisions.

[0138] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0139] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A PET-CT lung cancer image segmentation method based on deep learning, characterized in that: The following steps are involved: Step S1: acquiring PET lung cancer image data and CT lung cancer image data, and using a multi-scale feature encoder to perform multi-scale feature extraction on the PET lung cancer image data and the CT lung cancer image data to obtain image feature data; Step S2: Perform image feature fusion based on the cross-modal attention mechanism according to the image feature data to obtain bimodal image fusion data; design an explainable attention module according to the bimodal image fusion data, and mark the attention intensity through color gradient to generate a pixel-by-pixel attention heat map; Step S3: performing heat map-guided boundary correction processing on the pixel-by-pixel attention heat map based on blind spots, overlapping lesions, and false positive detection to obtain heat map correction feature data; constructing a multi-layer segmentation decoder based on the heat map correction feature data, and performing organ and tumor volume segmentation through the multi-layer segmentation decoder to obtain tumor segmentation image data; Step S4: designing a mixed loss function including Dice loss and boundary loss according to the tumor segmentation image data, and dynamically adjusting the contribution ratio of the two types of losses through learnable weight coefficients to obtain mixed loss function data, wherein the Dice loss calculates the overlap of the segmentation area, and the boundary loss focuses on the accuracy of the segmentation boundary; Step S5: performing connected domain analysis on the tumor segmentation image data according to the mixed loss function data, removing artifact areas whose areas are smaller than a preset threshold, and obtaining candidate segmentation optimization data under different loss function weight configurations; The optimal segmentation image is selected based on the segmentation performance for the candidate segmentation optimization data to obtain the optimal segmentation image data.

2. The deep learning-based PET-CT lung cancer image segmentation method according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: Acquire PET lung cancer image data and CT lung cancer image data; Step S12: performing image grayscale standardization on the PET lung cancer image data and the CT lung cancer image data, and performing histogram equalization, so as to obtain PET lung cancer image cleaned data and CT lung cancer image cleaned data; Step S13: using a multi-scale feature encoder to perform multi-scale feature extraction on the PET lung cancer image cleaned data and the CT lung cancer image cleaned data, respectively, so as to obtain PET image feature data and CT image feature data; Step S14: performing residual connection feature fusion on the PET image feature data and the CT image feature data to obtain image feature data.

3. The deep learning-based PET-CT lung cancer image segmentation method according to claim 2, characterized in that: Step S13 includes the following steps: The PET lung cancer image cleaning data is subjected to parallel extraction of multi-scale feature mapping based on functional metabolic features by using the first group of convolution kernels with different expansion rates, thereby obtaining PET image feature data, wherein the first group of convolution kernels with different expansion rates are 3×3 convolution kernels with an expansion rate of 1, 5×5 convolution kernels with an expansion rate of 2, and 7×7 convolution kernels with an expansion rate of 3; The second group of convolution kernels with different expansion rates are used to perform parallel extraction of multi-scale feature mapping based on anatomical structure features on the CT lung cancer image cleaning data to obtain CT image feature data. The second group of convolution kernels with different expansion rates are 3×3 convolution kernels with a expansion rate of 1, 5×5 convolution kernels with a expansion rate of 2, and 7×7 convolution kernels with a expansion rate of 3.

4. The deep learning-based PET-CT lung cancer image segmentation method according to claim 3, characterized in that: Step S2 includes the following steps: Step S21: calculating the similarity matrix between the two modal features in the image feature data by using cosine similarity, and constructing a feature correlation mapping, thereby obtaining cross-modal feature similarity mapping data; Step S22: designing a learnable modality fusion weight matrix according to the cross-modality feature similarity mapping data, and adaptively adjusting the modality fusion weight matrix based on feature similarity through a cross-attention mechanism, thereby obtaining a modality fusion weight matrix; Step S23: performing feature weighted fusion on the PET image feature data and the CT image feature data according to the modality fusion weight matrix, thereby obtaining dual-modality image fusion data; Step S24: Design an explainable attention module based on the bimodal image integration data, mark the attention intensity with color gradient, and generate a pixel-by-pixel attention heat map.

5. The deep learning-based PET-CT lung cancer image segmentation method according to claim 4, characterized in that: Step S21 includes the following steps: Step S211: performing feature dimension alignment preprocessing on the image feature data, and determining the sameness of the space and channel dimensions of the two-modal feature data, thereby obtaining cross-modal feature alignment data; Step S212: calculating the cosine similarity between each feature vector in the PET image feature data and the corresponding feature vector in the CT image feature data according to the cross-modality feature alignment data, thereby constructing an initial feature similarity matrix; Step S213: normalizing the initial feature similarity matrix and enhancing matrix comparability to obtain a normalized feature similarity matrix; Step S214: performing spatial attention weight mapping on the normalized feature similarity matrix and highlighting feature regions with high correlation, thereby obtaining a feature similarity matrix; Step S215: Enhance the feature similarity matrix through a multi-head attention mechanism, and capture cross-modal feature associations of different scales and channels, so as to obtain cross-modal feature association data; Step S216: Generate a feature correlation map based on the cross-modal feature correlation data, and quantify the semantic and spatial similarities between the PET and CT image features, thereby obtaining cross-modal feature similarity mapping data.

6. The deep learning-based PET-CT lung cancer image segmentation method according to claim 5, characterized in that: Step S24 includes the following steps: Step S241: Initialize the attention weight mapping layer, and integrate the bimodal image into the data input attention weight mapping layer for dimension reshaping, thereby obtaining attention mapping feature data; Step S242: designing a softmax normalization layer according to the attention map feature data, and using the softmax normalization layer to normalize the attention weights of the attention map feature data, thereby obtaining normalized attention weight data, wherein the softmax normalization layer specifically maps the attention weights in the attention map feature data to the interval [0,1], and ensures that the sum of the attention weights is 1; Step S243: generating a visually interpretable color mapping according to a preset color gradient mapping rule and normalized attention weight data, thereby obtaining a color-coded attention heat map; Step S244: Perform resolution restoration processing on the attention heat map through the deconvolution layer and superimpose the color gradient mapping to generate the final pixel-by-pixel attention heat map.

7. The deep learning-based PET-CT lung cancer image segmentation method according to claim 6, characterized in that: Step S3 includes the following steps: Step S31: identifying areas where the tumor boundary is unclear or blocked on the pixel-by-pixel attention heat map, thereby obtaining blind area detection data; Step S32: performing overlapping lesion area analysis on the pixel-by-pixel attention heat map, and locating tumor areas that may have multiple overlaps or blurs, thereby obtaining overlapping lesion area data; Step S33: performing false positive area detection on the pixel-by-pixel attention heat map to obtain false positive area data; Step S34: performing guided boundary correction processing based on morphological operators according to the blind area detection data, the overlapping lesion area data and the false positive area data, so as to obtain thermal map correction feature data; Step S35: constructing a multi-layer segmentation decoder according to the corrected characteristic data of the heat map, and performing organ and tumor volume segmentation through the multi-layer segmentation decoder, thereby obtaining a probabilistic segmentation map of the organ and tumor volume; Step S36: Select a segmentation threshold based on the probability segmentation map to obtain tumor segmentation image data.

8. The deep learning-based PET-CT lung cancer image segmentation method according to claim 7, characterized in that: Step S35 includes the following steps: A multi-layer segmentation decoder is built using a symmetric U-Net architecture. The heat map correction feature data is used as input, and the image resolution is gradually restored through the transposed convolutional layer of the multi-layer segmentation decoder. Skip connections are set in the multi-layer segmentation decoder to fuse low-level texture features and enhance the retention of segmentation details. A softmax activation function is applied to the last layer of the multi-layer segmentation decoder to output a probabilistic segmentation map of the organ and tumor volume.

9. The deep learning-based PET-CT lung cancer image segmentation method according to claim 8, characterized in that: Step S5 includes the following steps: Step S51: performing connected domain analysis on the tumor segmentation image data according to the mixed loss function data, extracting segmentation regions of different sizes and shapes, and identifying potential artifact regions, thereby obtaining artifact marker segmentation data; Step S52: removing artifact regions whose areas are smaller than a preset threshold according to the artifact mark segmentation data, thereby obtaining candidate segmentation optimization data under different loss function weight configurations; Step S53: extracting morphological features based on roundness, boundary smoothness and convexity from the candidate segmentation optimization data, thereby obtaining morphological feature data; Step S54: performing optimal segmentation image selection based on segmentation performance on the morphological feature data to obtain optimal segmentation image data.

10. A PET-CT lung cancer image segmentation system based on deep learning, characterized in that: Used to execute the deep learning-based PET-CT lung cancer image segmentation method according to claim 1, the deep learning-based PET-CT lung cancer image segmentation system comprises: A multi-scale feature extraction module is used to obtain PET lung cancer image data and CT lung cancer image data, and use a multi-scale feature encoder to perform multi-scale feature extraction on the PET lung cancer image data and the CT lung cancer image data to obtain image feature data; The cross-modal attention fusion module is used to fuse image features based on the cross-modal attention mechanism according to the image feature data to obtain the bimodal image fusion data; the explainable attention module is designed according to the bimodal image fusion data, and the attention intensity is marked by the color gradient to generate a pixel-by-pixel attention heat map; The heat map guided boundary correction module is used to perform heat map guided boundary correction processing on the pixel-by-pixel attention heat map based on blind spots, overlapping lesions and false positive detection to obtain heat map correction feature data; a multi-layer segmentation decoder is constructed based on the heat map correction feature data, and organs and tumor volumes are segmented through the multi-layer segmentation decoder to obtain tumor segmentation image data; The hybrid loss optimization module is used to design a hybrid loss function including Dice loss and boundary loss according to the tumor segmentation image data, and dynamically adjust the contribution ratio of the two types of losses through learnable weight coefficients to obtain the hybrid loss function data, where Dice loss calculates the overlap of the segmentation area, and boundary loss focuses on the accuracy of the segmentation boundary; The artifact removal and optimal segmentation selection module is used to perform connected domain analysis on the tumor segmentation image data according to the mixed loss function data, remove artifact areas with an area smaller than a preset threshold, and obtain candidate segmentation optimization data under different loss function weight configurations; and select the optimal segmentation image based on the segmentation performance of the candidate segmentation optimization data to obtain the optimal segmentation image data.

Images