Lung adenocarcinoma EGFR gene mutation detection system and method based on PET / CT deep learning

Abstract

The invention relates to the field of medical image analysis, in particular to a lung adenocarcinoma EGFR gene mutation detection system and method based on PET / CT deep learning, and the system comprises a data collection module which is used for obtaining PET / CT image data and clinical information of a lung adenocarcinoma patient; the image preprocessing module is connected with the data acquisition module and is used for carrying out standardization processing and ROI extraction on the PET / CT image data; the feature extraction module is connected with the image preprocessing module and used for extracting depth features, metabolic parameter features and CT sign features from the PET / CT image data; the multi-modal data fusion module is connected with the feature extraction module and used for fusing the extracted multi-modal features; and the prediction model module is connected with the multi-modal data fusion module and is used for predicting the lung adenocarcinoma EGFR gene mutation state and prognosis based on the fusion features, and the prediction accuracy is improved through multi-modal data fusion and deep learning technologies.

Description

Technical Field

[0001] The present invention relates to the field of medical image analysis, and particularly to a system and method for detecting EGFR gene mutations in lung adenocarcinoma based on PET / CT deep learning, which are used for non-invasive prediction of the epidermal growth factor receptor (EGFR) gene mutation status and prognosis assessment of lung adenocarcinoma patients. Background Art

[0002] Lung cancer is one of the most common malignant tumors globally, and lung adenocarcinoma is the most common histological type among non-small cell lung cancer (NSCLC), accounting for approximately 85% of NSCLC. Currently, the 5-year survival rate of lung adenocarcinoma is only about 17%, and the prognosis of patients is poor. The epidermal growth factor receptor (EGFR) gene mutation is an important biomarker for lung adenocarcinoma patients and has a decisive impact on the selection of treatment regimens. Advanced lung adenocarcinoma patients carrying EGFR gene mutations have significant therapeutic effects when treated with the targeted drug tyrosine kinase inhibitor (TKI), which can effectively extend the survival period of patients, while the effect on wild-type patients is minimal. Therefore, early determination of the EGFR gene mutation status is crucial for formulating individualized treatment regimens.

[0003] Currently, the gold standard for detecting EGFR gene mutations is to obtain specimens through histopathological biopsy for gene detection. However, this detection method is invasive and may cause serious complications such as pneumothorax or hemoptysis. In addition, advanced patients may be unable to obtain detection specimens due to poor health conditions or unwillingness to undergo invasive examinations. Although liquid biopsy (circulating tumor DNA detection) is another option, its accuracy is only about 65%, and it still needs to be improved. Therefore, there is an urgent clinical need for a non-invasive and accurate EGFR gene mutation detection method.

[0004] In recent years, 18F-fluorodeoxyglucose positron emission tomography ( 18 F-FDG PET / CT), as a non-invasive imaging method that combines tissue morphology and functional metabolism, has shown unique value in the diagnosis, staging, and prognosis assessment of lung adenocarcinoma. Existing studies have shown that there may be a certain correlation between PET / CT metabolic parameters and EGFR gene mutations in lung adenocarcinoma, but these parameters are often affected by factors such as scanning methods and acquisition equipment, and their stability is poor.

[0005] With the rapid development of artificial intelligence technology, deep learning has shown great potential in the field of medical image analysis. Existing studies have attempted to use CT image features to predict the EGFR gene mutation status of lung adenocarcinoma, but most studies are only based on single-modal data and are mostly single-center studies, with limited clinical application value. The multi-modal deep learning method based on PET / CT images has not been fully explored.

[0006] Therefore, developing an EGFR gene mutation detection and prognosis evaluation system based on PET / CT deep learning will help improve the diagnostic accuracy, provide decision-making support for personalized treatment of lung adenocarcinoma patients, and has important clinical application value. Summary of the Invention

[0007] The object of the present invention is to provide a lung adenocarcinoma EGFR gene mutation detection system and method based on PET / CT deep learning, to achieve non-invasive and accurate prediction of the EGFR gene mutation status of lung adenocarcinoma patients, and at the same time evaluate the prognosis of patients, providing decision-making support for clinical personalized precision treatment.

[0008] The present invention provides a lung adenocarcinoma EGFR gene mutation detection system based on PET / CT deep learning, which is characterized by including:

[0009] A data acquisition module, used to obtain PET / CT image data and clinical information of lung adenocarcinoma patients;

[0010] An image preprocessing module, connected to the data acquisition module, used to perform normalization processing and ROI extraction on the PET / CT image data;

[0011] A feature extraction module, connected to the image preprocessing module, used to extract deep features, metabolic parameter features, and CT sign features from the PET / CT image data;

[0012] A multi-modal data fusion module, connected to the feature extraction module, used to fuse the extracted multi-modal features;

[0013] A prediction model module, connected to the multi-modal data fusion module, used to predict the EGFR gene mutation status and prognosis of lung adenocarcinoma based on the fused features.

[0014] Preferably, the image preprocessing module includes:

[0015] A normalization unit, used to perform intensity normalization processing on the PET / CT image;

[0016] A noise elimination unit, used to perform noise reduction processing on the normalized image;

[0017] An ROI extraction unit, used to automatically locate and extract the tumor region;

[0018] A contrast enhancement unit, used to enhance the image contrast by applying the CLAHE algorithm;

[0019] A voxel resampling unit, used to resample the extracted ROI to a predefined unified size.

[0020] Preferably, the feature extraction module includes:

[0021] A CT encoder for extracting morphological features of CT images through a 3D convolutional neural network;

[0022] A PET encoder for extracting metabolic activity features of PET images through a 3D convolutional neural network;

[0023] A clinical feature encoder for processing patients' clinical data, CT sign features, and metabolic parameters;

[0024] A radiomics feature extractor for extracting statistical features from images;

[0025] A multi-scale feature pyramid extraction network for extracting tumor feature information at different scales

[0026] Both the CT encoder and the PET encoder adopt a residual connection structure.

[0027] Preferably, the multi-modal data fusion module includes at least one of the following fusion mechanisms:

[0028] An attention fusion unit for dynamically adjusting the weights of different modal features through an attention mechanism;

[0029] A bilinear fusion unit for capturing high-order interaction relationships between modalities through bilinear transformation;

[0030] A simple connection fusion unit for directly splicing and processing multi-modal features;

[0031] A heterogeneous attention adaptive fusion unit for dealing with the heterogeneity problem between multi-modal data;

[0032] The fusion unit adaptively selects the best fusion strategy according to different data quality and integrity degrees.

[0033] Preferably, the attention fusion unit includes:

[0034] A feature transformation layer for performing unified spatial mapping on each modal feature;

[0035] An attention calculation layer for calculating the attention weights of different modal features;

[0036] A weighted fusion layer for weighted fusion of different modal features according to the attention weights;

[0037] The attention calculation layer outputs an attention weight matrix for interpretability analysis.

[0038] Preferably, the prediction model module includes:

[0039] An EGFR mutation classifier for predicting the EGFR gene mutation status of lung adenocarcinoma based on fusion features, including mutation types and mutation sites;

[0040] A prognosis estimator for predicting the progression-free survival of patients after receiving EGFR-targeted therapy;

[0041] A confidence evaluation unit for quantitatively analyzing the uncertainty of the prediction results;

[0042] A calibration unit for ensuring the consistent performance of the prediction model on data from different institutions.

[0043] Preferably, it further includes:

[0044] A clinical decision support module, connected to the prediction model module, for:

[0045] Generating personalized treatment plan recommendations based on the prediction results;

[0046] Providing a visual interpretation of the prediction results;

[0047] Outputting the confidence analysis of the model prediction results;

[0048] Generating relevant clinical trial and literature information.

[0049] Preferably, it further includes:

[0050] A system monitoring and adaptive module for:

[0051] Monitoring the operating status of each component of the system;

[0052] Evaluating the performance of the model on new data and recording the deviations;

[0053] Automatically updating the model parameters according to the accumulated new data;

[0054] Generating a system operation report and outputting performance metrics.

[0055] Preferably, the prediction model module adopts the following evaluation methods:

[0056] ROC curve analysis for evaluating the classification performance of the model;

[0057] Decision curve analysis (DCA) for quantifying the clinical net benefit at different threshold probabilities;

[0058] Calibration curve analysis for evaluating the consistency between the predicted probability and the actual probability;

[0059] Multi-center external validation for evaluating the generalization ability of the model on data from different institutions.

[0060] A method for detecting EGFR gene mutations in lung adenocarcinoma based on PET / CT deep learning. Using the described system, it includes the following steps:

[0061] Obtain the PET / CT image data and clinical information of lung adenocarcinoma patients;

[0062] Perform normalization processing and ROI extraction on the PET / CT image data;

[0063] Extract deep features, metabolic parameter features, and CT sign features from the PET / CT image data;

[0064] Fuse the extracted multi-modal features and dynamically adjust the weights of different features;

[0065] Based on the fused features, predict the EGFR gene mutation status of lung adenocarcinoma through a deep learning model;

[0066] Based on the fused features, predict the progression-free survival period of patients after receiving EGFR targeted therapy;

[0067] Generate a visual interpretation of the prediction results and suggestions for personalized treatment plans;

[0068] Evaluate the model performance and dynamically update the model parameters according to new data.

[0069] The technical effects of the present invention are as follows: 1) It provides a new method for non-invasive detection of EGFR gene mutation status in lung adenocarcinoma, reducing patient pain and the risk of complications; 2) Through multi-modal data fusion and deep learning technology, it improves the prediction accuracy and provides a reliable basis for clinical decision-making; 3) Combining the prognostic evaluation function, it screens out the best-benefiting population for targeted therapy and improves the treatment response rate; 4) Adopting a multi-center verification strategy to ensure that the model has good generalization ability and clinical practical value; 5) Through an innovative multi-scale feature pyramid extraction network, it enhances the ability to capture tumor features at different scales; 6) Using a heterogeneous attention adaptive fusion mechanism, it effectively processes the heterogeneity between different modal data and improves the fusion efficiency. Description of the Drawings

[0070] Figure 1 is the overall architecture diagram of the EGFR gene mutation detection system for lung adenocarcinoma based on PET / CT deep learning of the present invention;

[0071] Figure 2 is the structural schematic diagram of the image preprocessing module of the present invention;

[0072] Figure 3 is the workflow diagram of the feature extraction module of the present invention;

[0073] Figure 4It is a schematic structural diagram of the multi-modal data fusion module of the present invention;

[0074] Figure 5 It is a detailed structural diagram of the attention fusion unit of the present invention;

[0075] Figure 6 It is a schematic structural diagram of the prediction model module of the present invention;

[0076] Figure 7 It is a schematic diagram of the structure and function of the clinical decision support module of the present invention;

[0077] Figure 8 It is a workflow diagram of the system monitoring and self-adaptation module of the present invention;

[0078] Figure 9 It is a flowchart of the method of the present invention;

[0079] Figure 10 It is a schematic diagram of the ROC curve and decision curve analysis results in the embodiment of the present invention;

[0080] Figure 11 It is a schematic structural diagram of the multi-scale feature pyramid extraction network of the present invention;

[0081] Figure 12 It is a working principle diagram of the heterogeneous attention self-adaptive fusion mechanism of the present invention. Detailed implementation manners

[0082] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

[0083] Embodiment 1

[0084] See Figure 1 , the lung adenocarcinoma EGFR gene mutation detection system based on PET / CT deep learning provided by the present invention includes a data acquisition module 1, an image preprocessing module 2, a feature extraction module 3, a multi-modal data fusion module 4, a prediction model module 5, a clinical decision support module 6, and a system monitoring and self-adaptation module 7.

[0085] The data acquisition module 1 is used to obtain the PET / CT image data and clinical information of lung adenocarcinoma patients. Specifically, the data acquisition module 1 can receive DICOM format image files from a PET / CT device, and at the same time collect the clinical data of the patient, including but not limited to basic information such as patient gender, age, smoking history, tumor location, clinical stage, and biochemical indexes such as serum CEA level. Preferably, the PET / CT examination is performed using a GE Discovery 710 PET / CT device in the United States. 18F-FDG is produced by a GE cyclotron with a radiochemical purity greater than 95%. All patients fast for more than 6 hours before the examination, with a fasting blood glucose not exceeding 8.3 mmol / L. 18F-FDG is intravenously injected at a dose of 3.70 - 5.55 MBq / kg body weight. After lying quietly for 1 hour, the patient urinates and then image acquisition is performed.

[0086] The image preprocessing module 2 is connected to the data acquisition module 1 and is used to perform normalization processing and ROI extraction on PET / CT image data. The image preprocessing module 2 converts the original DICOM format data into standardized three-dimensional volume data suitable for deep learning algorithms, and locates and extracts the tumor region through an automatic segmentation algorithm, providing high-quality data input for subsequent feature extraction.

[0087] The feature extraction module 3 is connected to the image preprocessing module 2 and is used to extract deep features, metabolic parameter features, and CT sign features from PET / CT image data. The feature extraction module 3 contains multiple encoders and feature extractors, which can comprehensively capture morphological features, functional metabolic features, and clinical feature information in the images, providing multi-dimensional feature representations for EGFR mutation prediction.

[0088] The multi-modal data fusion module 4 is connected to the feature extraction module 3 and is used to fuse the extracted multi-modal features. The multi-modal data fusion module 4 uses advanced methods such as attention mechanisms or bilinear transformations to effectively integrate feature information from different sources and dynamically adjust the weights of each modal feature to adapt to input data of different qualities and integrities.

[0089] The prediction model module 5 is connected to the multi-modal data fusion module 4 and is used to predict the EGFR gene mutation status and prognosis of lung adenocarcinoma based on the fused features. The prediction model module 5 can simultaneously achieve classification prediction of the EGFR gene mutation status and regression prediction of the progression-free survival of patients, providing comprehensive information support for clinical practice.

[0090] The clinical decision support module 6 is connected to the prediction model module 5 and is used to generate personalized treatment plan recommendations based on the prediction results and provide visual explanations. The system monitoring and adaptive module 7 is used to monitor the operating status of each component of the system, evaluate the model performance, and automatically update the model parameters according to the accumulated new data to ensure the continuous and effective operation of the system.

[0091] Example 2

[0092] As Figure 2 shown, the image preprocessing module 2 includes a normalization unit 21, a noise elimination unit 22, an ROI extraction unit 23, a contrast enhancement unit 24, and a voxel resampling unit 25.

[0093] The normalization unit 21 is used to perform intensity normalization on PET / CT images. For CT images, the normalization unit 21 sets the CT value window to [-1000, 400] Hounsfield units (HU), i.e., the lung window setting, and then performs linear normalization; for PET images, the normalization unit 21 sets the standard uptake value (SUV) range to [0, 15], and performs corresponding linear normalization. The normalization process can be expressed as:

[0094] ,

[0095] where, is the original image intensity value, and are the preset minimum and maximum values respectively (-1000 and 400 for CT images, 0 and 15 for PET images), is the image intensity value after normalization.

[0096] The noise cancellation unit 22 is used to perform noise reduction on the normalized images. For CT images, the noise cancellation unit 22 adopts the non-local means filtering algorithm, which can effectively suppress noise while preserving image details; for PET images, considering its inherent noise characteristics, the noise cancellation unit 22 adopts the Gaussian filtering algorithm with the parameter optimized to be σ = 0.8, which can smooth the image and reduce the influence of quantization noise.

[0097] The ROI extraction unit 23 is used to automatically locate and extract the tumor region. The present invention adopts a method combining the SUV threshold of PET images and morphological operations to automatically extract the tumor region. Specifically, the ROI extraction unit 23 first applies the SUV threshold of 2.5 (this threshold is determined based on clinical experience and literature reports and can effectively distinguish malignant tumors from normal tissues) to generate a preliminary tumor mask, and then applies morphological operations (including small region removal, dilation, and erosion operations) to optimize the tumor region. In a preferred embodiment, the ROI extraction unit 23 calculates the bounding box of the tumor mask and expands 5 voxels outside the bounding box to form the final ROI, ensuring complete inclusion of the tumor and its surrounding tissues.

[0098] The contrast enhancement unit 24 is used to enhance the image contrast by applying the CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm. The contrast enhancement unit 24 applies the CLAHE algorithm layer by layer to the three-dimensional volume data, with the parameter settings of block size 8×8 pixels and contrast limit threshold of 3.0. In an embodiment of the present invention, the contrast enhancement unit 24 can be selectively applied to CT images to enhance the visibility of tumor edges and internal structures.

[0099] The voxel resampling unit 25 is used to resample the extracted ROI to a predefined unified size. To ensure the input consistency of the deep learning model, the voxel resampling unit 25 resamples the ROI volume to a standard size of 64×64×64, while normalizing the voxel spacing to 1.0×1.0×1.0 mm3, and adopts the trilinear interpolation algorithm to maintain the image quality.

[0100] Embodiment 3

[0101] As Figure 3 shown, the feature extraction module 3 includes a CT encoder 31, a PET encoder 32, a clinical feature encoder 33, a radiomics feature extractor 34, and a multi-scale feature pyramid extraction network 35.

[0102] The CT encoder 31 adopts a 3D convolutional neural network structure and is used to extract morphological features from CT images. The CT encoder 31 contains four convolutional blocks, and each convolutional block consists of a convolutional layer, a batch normalization layer, a ReLU activation function, and a max pooling layer. Specifically, the first convolutional block uses 32 3×3×3 convolutional kernels, the second convolutional block uses 64 3×3×3 convolutional kernels, the third convolutional block uses 128 3×3×3 convolutional kernels, and the fourth convolutional block uses 256 3×3×3 convolutional kernels. To improve the model performance, the CT encoder 31 adopts a residual connection structure, enabling the deep network to more effectively learn feature representations. The output of the CT encoder 31 is a 256-dimensional feature vector, representing the deep representation of the CT image.

[0103] The structure of the PET encoder 32 is similar to that of the CT encoder 31, but the parameters are independently trained and are specifically used to capture the metabolic activity features in PET images. The four convolutional blocks of the PET encoder 32 maintain the same structure as those of the CT encoder 31, but capture the feature information of different modalities through independent training. The PET encoder 32 also adopts a residual connection structure to improve the feature extraction ability, and the output is a 256-dimensional feature vector.

[0104] The clinical feature encoder 33 is used to process the patient's clinical data, CT sign features, and metabolic parameters. The clinical feature encoder 33 contains three sub-networks: a clinical information processing network, a CT sign processing network, and a metabolic parameter processing network. The clinical information processing network maps 10-dimensional clinical features (including gender, age, smoking history, etc.) to a 32-dimensional hidden space; the CT sign processing network maps 5-dimensional CT sign features (including pleural indentation sign, spiculation sign, lobulation sign, vascular connection sign, and bronchial truncation sign) to a 16-dimensional hidden space; the metabolic parameter processing network maps 4-dimensional metabolic parameters (including SUVmax, SUVmean, MTV, TLG) to a 16-dimensional hidden space. The outputs of these three sub-networks are concatenated and fused, and finally a 64-dimensional comprehensive clinical feature representation is generated.

[0105] The radiomics feature extractor 34 is used to extract statistical features from images. The radiomics feature extractor 34 extracts various types of features, including first-order features (histogram statistics), shape features, and texture features (GLCM gray-level co-occurrence matrix, GLRLM gray-level run-length matrix, GLSZM gray-level size zone matrix, GLDM gray-level dependence matrix, and NGTDM normalized gray-level co-occurrence matrix). In an embodiment of the present invention, the radiomics feature extractor 34 extracts approximately 100-dimensional features from PET / CT images and performs feature selection through the LASSO (Least Absolute Shrinkage and Selection Operator) algorithm, finally retaining the most relevant feature subset.

[0106] The multi-scale feature pyramid extraction network 35 is an important innovation point of the present invention and is used to extract tumor feature information at different scales. As Figure 11 shown, the multi-scale feature pyramid extraction network 35 includes an image pyramid generation unit 351, a feature extraction unit 352, a scale fusion unit 353, and an attention enhancement unit 354.

[0107] The image pyramid generation unit 351 is used to create an image pyramid with different resolution levels. Specifically, the image pyramid generation unit 351 creates images of multiple scales from the original volume ROI (64×64×64) through continuous downsampling operations, forming an image pyramid containing 4 levels: the original size (64×64×64), 1 / 2 size (32×32×32), 1 / 4 size (16×16×16), and 1 / 8 size (8×8×8). The downsampling operation adopts a method combining a Gaussian kernel and downsampling, expressed as:

[0108] ,

[0109] where represents the -th level pyramid image, represents a Gaussian kernel with a standard deviation of (in this implementation , * represents the convolution operation, and Downsample represents the downsampling operation (stride is 2).

[0110] The feature extraction unit 352 is used to extract features from each level of the pyramid. The feature extraction unit 352 is equipped with independent feature extraction networks for each scale level, and these networks have similar structures but different parameters. For each level, the feature extraction unit 352 applies three consecutive 3D convolutional layers, each followed by batch normalization and the ReLU activation function. The convolutional kernel size is adaptively adjusted according to the scale level: a 3×3×3 convolutional kernel is used for the original scale, a 5×5×5 convolutional kernel is used for the 1 / 2 scale, a 7×7×7 convolutional kernel is used for the 1 / 4 scale, and a 9×9×9 convolutional kernel is used for the 1 / 8 scale. This design enables the network to capture an appropriate receptive field at different scales. The mathematical representation of the feature extraction unit 352 is:

[0111] ,

[0112] where represents the features extracted from the -level pyramid image, and represents the 3D convolutional network applied to this level.

[0113] The scale fusion unit 353 is used to integrate features from different scale levels. The scale fusion unit 353 first upsamples the low-resolution features to the highest resolution, then applies a 1×1×1 convolution to adjust the number of channels, and finally concatenates all scale features along the channel dimension. To achieve precise feature alignment, trilinear interpolation is used for upsampling. The scale fusion process can be expressed as:

[0114] ,

[0115] where , , , represent the features extracted from the four scale levels respectively, Up represents the upsampling operation, and Concat represents the feature concatenation operation.

[0116] The attention enhancement unit 354 is used to emphasize the most relevant multi-scale features. The attention enhancement unit 354 implements a channel attention mechanism. First, channel descriptors are obtained through global average pooling and max pooling, and then channel attention weights are generated through a shared multi-layer perceptron. The mathematical representation is:

[0117] ,

[0118] ,

[0119] ,

[0120] ,

[0121] Among them, GAP and GMP represent global average pooling and global max pooling respectively, and are the weights of the multi-layer perceptron, represents the Sigmoid activation function, represents element-wise multiplication, is the final multi-scale feature output.

[0122] The advantage of the multi-scale feature pyramid extraction network 35 is that it can capture both the global structure and local details of the tumor simultaneously. The small-scale level has a larger receptive field and can capture the overall shape of the tumor and its surrounding environment; the large-scale level retains more detailed information and can capture the fine structure inside the tumor. By fusing these multi-scale features, the system can more comprehensively understand the morphological and functional metabolic characteristics of the tumor, improving the accuracy of EGFR mutation prediction. In the test, after adding the multi-scale feature pyramid extraction network 35, the AUC value of the system increased from 0.86 to 0.89, and the accuracy increased from 81.2% to 84.7%, indicating that this innovation point significantly enhanced the system performance.

[0123] Each encoder and extractor in the feature extraction module 3 cooperate with each other to jointly construct a multi-dimensional feature space, providing rich feature representations for subsequent fusion and prediction tasks.

[0124] Example 4

[0125] As Figure 4 shown, the multi-modal data fusion module 4 includes an attention fusion unit 41, a bilinear fusion unit 42, a simple connection fusion unit 43, and a heterogeneous attention adaptive fusion unit 44, which can adaptively select the best fusion strategy according to the data quality and integrity degree.

[0126] The attention fusion unit 41 dynamically adjusts the weights of different modal features through an attention mechanism. As Figure 5 shown, the attention fusion unit 41 includes a feature transformation layer 411, an attention calculation layer 412, and a weighted fusion layer 413.

[0127] The feature transformation layer 411 is used to perform a unified spatial mapping on the features of each modality. The feature transformation layer 411 contains three linear transformation networks, which process the CT features, PET features, and clinical features respectively, keeping their respective feature dimensions unchanged, but performing non-linear transformations to enhance the feature expression ability.

[0128] The attention calculation layer 412 is used to calculate the attention weights of different modal features. The attention calculation layer 412 first concatenates the features of the three modalities, and then generates three normalized attention weight values through two layers of neural networks, representing the importance coefficients of the features of each modality. The attention weight calculation process can be expressed as:

[0129] ,

[0130] ,

[0131] ,

[0132] Among them, 、 and are CT features, PET features, and clinical features respectively, represents the feature concatenation operation, 、 、 、 are learnable parameters, is the three-dimensional attention weight vector, and the sum of the weights is ensured to be 1 through the softmax function.

[0133] The weighted fusion layer 413 is used to weightedly fuse different modality features according to the attention weights. The weighted fusion layer 413 multiplies each modality feature by the corresponding attention weight, then concatenates them and generates the fused feature through a fully connected layer. The fusion process can be expressed as:

[0134] ,

[0135] ,

[0136] ,

[0137] ,

[0138] Among them, 、 and are the attention weights, and are the parameters of the fully connected layer, is the final fused feature.

[0139] The bilinear fusion unit 42 is used to capture the high-order interaction relationships between modalities through bilinear transformation. Different from simple linear combinations, the bilinear fusion unit 42 can capture the product relationships between features and enhance the model's expressive ability. Specifically, the bilinear fusion unit 42 first reduces the dimensions of each modality feature through linear mapping, and then applies three bilinear transformations to fuse CT and PET features, PET and clinical features, and CT and clinical features respectively. Finally, these three bilinear features are concatenated and passed through a fully connected layer to generate the final fused feature. This method can effectively capture the complex mutual relationships between different modality features.

[0140] The simple concatenation fusion unit 43 serves as a baseline method, directly concatenating and processing multi-modal features. The simple concatenation fusion unit 43 directly concatenates the CT features, PET features, and clinical features, and then performs a non-linear transformation through a multi-layer perceptron to generate fused features. Although the structure is simple, this method can also achieve good results when the feature dimensions are balanced and the data quality is consistent.

[0141] The heterogeneous attention adaptive fusion unit 44 is another key innovation of the present invention, specifically designed to address the heterogeneity issue among multi-modal data. As Figure 12 shown, the heterogeneous attention adaptive fusion unit 44 includes a feature space mapping sub-unit 441, an inter-modal correlation modeling sub-unit 442, a heterogeneous feature collaborative enhancement sub-unit 443, and a dynamic fusion weight learning sub-unit 444.

[0142] The feature space mapping sub-unit 441 is used to map the features of different modalities into a shared feature space. Since the CT features, PET features, and clinical features are from different domains, there are significant differences in their statistical characteristics and distributions. The feature space mapping sub-unit 441 maps these heterogeneous features into a unified semantic space through a non-linear transformation network, facilitating subsequent fusion operations. The mathematical expression is:

[0143] ,

[0144] ,

[0145] ,

[0146] where 、 and are the feature mapping networks of each modality, each consisting of two fully connected layers, and the hidden layer uses the LeakyReLU activation function ( ). The mapped features 、 and have the same dimension (128 dimensions) and similar distribution characteristics.

[0147] The inter-modal correlation modeling sub-unit 442 is used to capture the interdependent relationships between the features of different modalities. The inter-modal correlation modeling sub-unit 442 implements a cross-attention mechanism that enables each modality feature to focus on relevant information in other modalities. Specifically, for each pair of modality features, a correlation matrix is calculated:

[0148] ,

[0149] where and is the mapping feature between modality i and modality j, and d is the feature dimension. is the correlation matrix. Then, based on the correlation matrix, the cross-attention feature is calculated:

[0150] ,

[0151] where, represents the cross-attention feature from one modality to another. In this way, the inter-modal correlation modeling unit 442 creates two cross-attention features for each modality. For example, the CT modality gets and .

[0152] The heterogeneous feature collaborative enhancer unit 443 is used to integrate the original feature and the cross-attention feature to generate an enhanced modality representation. For each modality, the heterogeneous feature collaborative enhancer unit 443 fuses the original feature and the cross-attention feature through a gating mechanism:

[0153] ,

[0154] ,

[0155] where, is the gating weight, represents the Sigmoid activation function, and are learnable parameters, represents element-wise multiplication is the enhanced modality feature. This gating mechanism can adaptively control the degree of information obtained from other modalities and effectively handle the complementary and redundant relationships between modalities.

[0156] The dynamic fusion weight learning sub-unit 444 is used to learn the dynamic fusion weights of different modalities. Different from the basic attention fusion, the dynamic fusion weight learning sub-unit 444 considers the data quality and task relevance and determines the final weights through two factors: the modality quality score and the task relevance score.

[0157] The modality quality score is calculated based on information entropy:

[0158] ,

[0159] where Entropy calculates the information entropy of the feature, is the multi-layer perceptron that maps the entropy value to the quality score.

[0160] The task relevance score is calculated by the similarity with the task-specific embedding:

[0161] ,

[0162] where cos represents the cosine similarity, is a task-specific learnable embedding vector.

[0163] The final fusion weights are obtained by comprehensively considering quality and relevance:

[0164] ,

[0165] The fused feature is calculated as:

[0166] ,

[0167] Through the above design, the heterogeneous attention adaptive fusion unit 44 can effectively handle the heterogeneity problem of multimodal data, and is particularly suitable for the situation where there are large differences in the quality of different modal data or partial missing. In clinical applications, when the quality of certain modal data is poor (such as motion artifacts in PET images) or certain clinical information is missing, this unit can adaptively reduce the weight of the corresponding modality and enhance the contribution of other high-quality modalities, thereby maintaining the stability of the system prediction performance. Experiments have shown that in the test cases with missing modalities (such as randomly masking 20% of the PET features), the system performance degradation of using the heterogeneous attention adaptive fusion unit 44 is only 3.5%, while the system performance degradation of using standard attention fusion reaches 8.2%, significantly enhancing the robustness of the system.

[0168] The multimodal data fusion module 4 can adaptively select the best fusion strategy according to the characteristics of the input data. For example, when the quality of a certain modality data is poor or missing, the heterogeneous attention adaptive fusion unit 44 will automatically adjust the strategy; when the quality of each modality data is balanced and complete, the bilinear fusion unit 42 can be selected to fully explore the high-order relationships between features.

[0169] Example 5

[0170] As Figure 6 shown, the prediction model module 5 includes an EGFR mutation classifier 51, a prognosis estimator 52, a confidence evaluation unit 53, and a calibration unit 54.

[0171] The EGFR mutation classifier 51 is used to predict the EGFR gene mutation status of lung adenocarcinoma based on fusion features, including mutation types and mutation sites. The EGFR mutation classifier 51 adopts a multi-layer perceptron structure, specifically including four fully connected layers, with the hidden layer dimensions being 256, 128, and 64 respectively, and the output layer being 2-dimensional, corresponding to two categories of mutant and wild type. A batch normalization layer, a ReLU activation function, and a Dropout layer (with Dropout rates of 0.5, 0.4, and 0.3, decreasing layer by layer) follow each hidden layer to prevent overfitting and improve generalization ability. The EGFR mutation classifier 51 is trained using the cross-entropy loss function, adopts the Adam optimizer, sets the initial learning rate to 0.001, and uses a learning rate decay strategy, decaying by 10% every 10 epochs.

[0172] The prognostic evaluator 52 is used to predict the progression-free survival period of patients after receiving EGFR targeted therapy. The prognostic evaluator 52 also adopts a multi-layer perceptron structure, but the output layer is a single neuron, representing the predicted number of progression-free survival days. The prognostic evaluator 52 is trained using the mean squared error loss function and adopts an early stopping strategy to avoid overfitting. It should be noted that the prognostic evaluator 52 shares the parameters of the first few layers with the EGFR mutation classifier 51, and this multi-task learning strategy can improve the overall performance of the model.

[0173] The confidence evaluation unit 53 is used to perform uncertainty quantification analysis on the prediction results. The confidence evaluation unit 53 adopts the Monte Carlo Dropout technique, keeps the Dropout layer activated during the test phase, performs multiple forward propagations to obtain multiple prediction results, and then calculates the mean and variance of these results. The mean is used as the final prediction result, and the variance reflects the uncertainty of the prediction. Specifically, the confidence evaluation unit 53 performs 50 forward propagations to obtain a set of predicted probability distributions, calculates the variance as the uncertainty measure, and provides a reliability reference for clinical decision-making.

[0174] The calibration unit 54 is used to ensure the consistent performance of the prediction model on data from different institutions. The calibration unit 54 adjusts the predicted probabilities using the temperature scaling method to make the probability distribution output by the model match the actually observed frequencies. The calibration process can be expressed as:

[0175] ,

[0176] where, is the logits output by the model, is the temperature parameter (optimized through the validation set), is the calibrated predicted probability. In the embodiment of the present invention, the temperature parameter is determined by minimizing the negative log-likelihood loss on the validation set, and the optimized value is 1.35.

[0177] The prediction model module 5 adopts a variety of evaluation methods, including ROC curve analysis, decision curve analysis (DCA), calibration curve analysis, and multi-center external validation, to comprehensively evaluate the model performance. ROC curve analysis calculates indicators such as AUC value, sensitivity, and specificity; decision curve analysis quantifies the clinical net benefit at different threshold probabilities; calibration curve analysis evaluates the consistency between the predicted probability and the actual probability; and multi-center external validation evaluates the generalization ability of the model on data from different institutions.

[0178] Example 6

[0179] As Figure 7 shown, the clinical decision support module 6 includes a treatment plan recommendation unit 61, a visualization explanation unit 62, a confidence analysis unit 63, and a literature information generation unit 64.

[0180] The treatment plan recommendation unit 61 generates personalized treatment plan recommendations based on the prediction results. The treatment plan recommendation unit 61 presets a variety of treatment plan templates, including EGFR-TKI treatment plans, chemotherapy plans, immunotherapy plans, etc., and combines the EGFR mutation prediction results, prognosis prediction results, and patient individual characteristics to automatically generate personalized treatment recommendations. For example, for patients predicted to be EGFR mutation positive and with good prognosis, the system will preferentially recommend the corresponding EGFR-TKI treatment plan, and at the same time provide drug selection, dosage suggestions, and possible adverse reaction prompts.

[0181] The visualization explanation unit 62 provides a visualization explanation of the prediction results. The visualization explanation unit 62 uses the Grad-CAM (Gradient-weighted Class Activation Mapping) technology to map the prediction results back to the original image space and display the image regions that contribute the most to the prediction results in the form of a heat map. At the same time, the visualization explanation unit 62 also displays the attention weight distribution of each modality feature to help clinicians understand the decision-making basis of the model.

[0182] The confidence analysis unit 63 outputs a confidence analysis of the model prediction results. The confidence analysis unit 63 generates an intuitive confidence display based on the uncertainty measure output by the confidence evaluation unit 53 of the prediction model module 5. Specifically, the confidence analysis unit 63 combines the predicted probability and the uncertainty measure to generate a three-level confidence rating (high, medium, low), and provides corresponding explanations. For example, when the predicted probability is greater than 0.8 and the uncertainty is less than 0.1, the system will mark it as a high-confidence prediction; when the predicted probability is between 0.6 and 0.8 or the uncertainty is greater than 0.2, the system will mark it as a medium-confidence prediction and recommend referring to other clinical indicators.

[0183] The literature information generation unit 64 generates relevant clinical trial and literature information. The literature information generation unit 64 accesses a preset literature database related to lung adenocarcinoma treatment and EGFR mutation, and automatically retrieves and recommends the most relevant clinical guidelines, research literature, and clinical trial information according to the prediction results and patient characteristics to help clinicians obtain the latest evidence-based medical evidence.

[0184] Each unit of the clinical decision support module 6 works collaboratively to transform the complex model prediction results into clinically understandable and actionable decision-making suggestions, enhancing the clinical practical value of the system.

[0185] Embodiment 7

[0186] As Figure 8 shown, the system monitoring and adaptive module 7 includes an operating status monitoring unit 71, a performance evaluation unit 72, a model update unit 73, and a report generation unit 74.

[0187] The operating status monitoring unit 71 is used to monitor the operating status of each component of the system. The operating status monitoring unit 71 continuously monitors the operating status of each module of the system, including computing resource usage, processing time, error logs, etc., to ensure the stable operation of the system. When an abnormal status is detected, the operating status monitoring unit 71 generates an alarm message and records detailed logs for maintenance personnel to troubleshoot problems.

[0188] The performance evaluation unit 72 is used to evaluate the performance of the model on new data and record the deviation. The performance evaluation unit 72 regularly evaluates the performance of the new data processed by the system, calculates indicators such as accuracy, sensitivity, and specificity, and compares them with the baseline performance. In addition, the performance evaluation unit 72 also analyzes the model performance on different subgroup data, such as patients of different ages, genders, and clinical stages, to discover potential model deviations.

[0189] The model update unit 73 is used to automatically update the model parameters according to the accumulated new data. The model update unit 73 adopts an incremental learning strategy. When enough new data is accumulated and verified, the model update process is automatically triggered. During the update process, the model update unit 73 retains the knowledge structure of the original model and only adjusts some parameters to adapt to the new data distribution, while paying attention to avoiding the problem of catastrophic forgetting. In an embodiment of the present invention, when the accumulated new data reaches 10% of the original training set (about 30 - 50 cases), the system starts the update process, and the updated model is verified by the validation set, and the performance improvement will be adopted only if it passes the verification.

[0190] The report generation unit 74 is used to generate a system operation report and output performance metrics. The report generation unit 74 regularly generates a system operation report, including the number of processed cases, various prediction distributions, trends in performance metric changes, etc., providing intuitive operation status monitoring for system administrators. The report forms include daily reports, weekly reports, and monthly reports, with different focuses for reports of different time scales, facilitating management decisions at different levels.

[0191] The system monitoring and adaptive module 7 ensures the long-term stable and effective operation of the system in the actual clinical environment through continuous monitoring and adaptive updates, which is an important guarantee for the practicality and reliability of the system of the present invention.

[0192] Example 8

[0193] As Figure 9 shown, the present invention also provides a method for detecting EGFR gene mutations in lung adenocarcinoma based on PET / CT deep learning, including the following steps:

[0194] Step S1: Obtain the PET / CT image data and clinical information of lung adenocarcinoma patients. The obtained data includes PET / CT DICOM format image files, as well as clinical information such as patient gender, age, smoking history, tumor location, clinical stage, and biochemical indicators such as serum CEA level.

[0195] Step S2: Perform normalization processing and ROI extraction on the PET / CT image data. This step includes: adjusting the window width and window level and intensity normalization for CT and PET images; applying an appropriate noise reduction algorithm to process the images; automatically extracting the tumor region based on the SUV threshold of PET; applying the CLAHE algorithm to enhance the image contrast when necessary; resampling the extracted ROI to a standard size.

[0196] Step S3: Extract deep features, metabolic parameter features, and CT sign features from the PET / CT image data. This step includes: using 3D CNN to extract deep features of CT images; using 3D CNN to extract deep features of PET images; extracting and encoding patient clinical information, CT signs, and metabolic parameters; extracting radiomics features of PET / CT images; using a multi-scale feature pyramid network to extract tumor features at different scales.

[0197] Step S4: Fuse the extracted multi-modal features and dynamically adjust the weights of different features. This step preferably adopts a heterogeneous attention adaptive fusion mechanism, which effectively processes the heterogeneity problem of multi-modal data through feature space mapping, inter-modal correlation modeling, heterogeneous feature collaborative enhancement, and dynamic fusion weight learning, generating a comprehensive fused feature representation.

[0198] Step S5: Based on the fused features, predict the EGFR gene mutation status of lung adenocarcinoma through a deep learning model. This step uses a multi-layer perceptron structure to map the fused features to a binary classification output (mutant type and wild type), and outputs the prediction probability.

[0199] Step S6: Based on the fused features, predict the progression-free survival period of the patient after receiving EGFR-targeted therapy. This step also uses a multi-layer perceptron structure, but the output is a continuous value, representing the predicted number of progression-free survival days.

[0200] Step S7: Generate a visual explanation of the prediction results and suggestions for personalized treatment plans. This step includes: generating a heat map to display the image regions that contribute the most to the prediction results; showing the attention weight distribution of each modality feature; generating personalized treatment suggestions according to the prediction results; providing a confidence analysis of the prediction results; recommending relevant clinical guidelines and research literature.

[0201] Step S8: Evaluate the model performance and dynamically update the model parameters according to new data. This step includes: regularly evaluating the performance of the model on new data; analyzing the prediction effects of different subgroups of patients; when enough new data has been accumulated, using incremental learning to update the model parameters; generating a system operation report.

[0202] Through the above steps, the method of the present invention realizes the non-invasive and accurate prediction of the EGFR gene mutation status of lung adenocarcinoma and the evaluation of the patient's prognosis, providing strong support for clinical personalized precision treatment.

[0203] Example 9

[0204] As Figure 10 shown, the system of the present invention uses a variety of evaluation methods to comprehensively verify the prediction performance, including ROC curve analysis, decision curve analysis (DCA), calibration curve analysis, and multi-center external verification.

[0205] ROC curve analysis is used to evaluate the classification performance of the model. The AUC value of the system of the present invention on the internal validation set reaches 0.89 (95% confidence interval: 0.85 - 0.93), the sensitivity is 84.7%, the specificity is 83.2%, the accuracy is 84.7%, the positive predictive value (PPV) is 81.8%, and the negative predictive value (NPV) is 86.3%, which is better than the clinical variable model (AUC = 0.73) and the single PET / CT feature model (AUC = 0.78). In particular, after adding the multi-scale feature pyramid extraction network and the heterogeneous attention adaptive fusion mechanism, the system performance is improved by about 3% compared with the baseline method, significantly enhancing the prediction accuracy.

[0206] Decision curve analysis (DCA) is used to quantify the clinical net benefit at different threshold probabilities. The results of the decision curve analysis show that within the clinically relevant threshold probability range (10% - 70%), the system of the present invention provides a positive net benefit, and is significantly superior to the baseline strategies of all treatments and all non - treatments. When the threshold probability is 30%, the net benefit of the system of the present invention reaches a maximum value of 0.32, indicating that at this decision threshold, the system can avoid approximately 32 unnecessary treatments or missed diagnoses for every 100 patients, with an increase of about 14% compared to the baseline method (net benefit 0.28).

[0207] Calibration curve analysis is used to evaluate the consistency between the predicted probability and the actual probability. The calibration curve analysis shows that the predicted probability of the system of the present invention is highly consistent with the actually observed frequency. The p - value of the Hosmer - Lemeshow test result is 0.89 (>0.05), indicating that the model is well - calibrated. The calibration slope is 0.96 and the calibration intercept is 0.02, close to the ideal slope of 1 and intercept of 0, and the calibration performance is superior to the baseline method (calibration slope 0.94, calibration intercept 0.03).

[0208] Multi - center external validation is used to evaluate the generalization ability of the model on data from different institutions. The system of the present invention was validated on independent datasets from two external medical institutions, with AUC values of 0.87 (95% confidence interval: 0.82 - 0.92) and 0.85 (95% confidence interval: 0.80 - 0.90) respectively, indicating that the model has good generalization ability. The performance difference between different centers is less than 2.5%, significantly superior to the baseline method (performance difference <5%), further demonstrating the value of the innovation of the present invention in improving the generalization ability of the model.

[0209] Example 10

[0210] The system of the present invention has been tested for application in the clinical environment of three medical institutions, and a total of more than 300 lung adenocarcinoma patient data have been analyzed. The following are specific application cases:

[0211] Case 1: A 64-year-old female patient with no smoking history, peripheral lung adenocarcinoma in the upper lobe of the right lung, clinical stage IIIA. Through the system of the present invention, the PET / CT images of this patient were analyzed, and the predicted probability of EGFR mutation was 91.2% (high confidence), and the predicted progression-free survival was 12.8 months. The system recommended giving priority to EGFR-TKI targeted therapy. The visual explanation showed that the marginal area and the heterogeneous internal area of the tumor contributed the most to the prediction. In the modal contribution analysis, the weight of CT features was 0.35, the weight of PET features was 0.42, and the weight of clinical features was 0.23, indicating that the PET metabolic features played a dominant role in the mutation prediction of this patient. Subsequent genetic testing confirmed that the patient had a deletion mutation in exon 19 of EGFR. After treatment with osimertinib, the actual progression-free survival was 13.2 months, which was similar to the system prediction.

[0212] Case 2: A 58-year-old male patient with a 30-year smoking history, central lung adenocarcinoma in the lower lobe of the left lung, clinical stage IVA. The system predicted the probability of EGFR mutation to be 19.8% (high confidence), and the predicted progression-free survival was 4.2 months. In the modal contribution analysis, the weight of clinical features (especially the smoking history) reached 0.48, which was significantly higher than the weight of CT features (0.32) and the weight of PET features (0.20). The system recommended giving priority to chemotherapy or immunotherapy. Subsequent genetic testing confirmed that the patient was EGFR wild-type. After treatment with pemetrexed combined with carboplatin chemotherapy and immunotherapy, a good therapeutic response was achieved.

[0213] Case 3: A 71-year-old female patient with no smoking history, peripheral lung adenocarcinoma in the middle lobe of the right lung, clinical stage IIB, and the CT showed a ground-glass nodule. The system predicted the probability of EGFR mutation to be 96.5% (high confidence), and the fine internal structure of the tumor captured by the multi-scale feature pyramid network contributed significantly to the prediction. Due to the good physical condition of the patient, the clinical decision was to perform surgical treatment first. Postoperative pathology and genetic testing confirmed lung adenocarcinoma with an L858R mutation in exon 21 of EGFR, and adjuvant targeted therapy was given, and the prognosis was good.

[0214] Case 4: A 67-year-old female patient with a mild smoking history, lung adenocarcinoma in the upper lobe of the left lung, clinical stage IIIB. The quality of the PET image was poor (with respiratory motion artifacts), and the quality of the CT image was good. Through the heterogeneous attention adaptive fusion mechanism, the system automatically reduced the weight of PET features (0.15), increased the weight of CT features (0.60) and the weight of clinical features (0.25), and still gave an accurate prediction: the probability of EGFR mutation was 75.6% (medium confidence). Subsequent genetic testing confirmed that the patient had an L858R mutation in exon 21 of EGFR, verifying the robustness of the system in dealing with unbalanced modal data.

[0215] These cases indicate that the system of the present invention can accurately predict the EGFR gene mutation status of lung adenocarcinoma and provide valuable reference for clinical decision-making. In particular, the newly added multi-scale feature pyramid extraction network and heterogeneous attention adaptive fusion mechanism significantly improve the performance of the system in processing complex tumor features and unbalanced modal data, enhancing the clinical practical value of the system.

[0216] The system of the present invention has good industrial practicability and can be directly applied to the clinical environment. The system uses the standardized medical imaging format (DICOM) as the input, which has good compatibility with existing medical information systems. The degree of automation of the system processing flow is high, and automated operations can be realized from data input to result output, reducing manual intervention. The output results of the system are intuitive and easy to understand, including prediction probability, confidence rating, visual explanation, and treatment suggestions, which are convenient for clinicians to understand and apply. In addition, the system has the ability of adaptive learning and can continuously optimize its performance through accumulated clinical data, having long-term application value.

[0217] In terms of technical implementation, the system can be deployed on a standard server or cloud platform, or integrated into the existing hospital information system. The front end of the system can be developed into a Web application or a stand-alone software, which is convenient for doctors to access and use on different terminal devices. The back end of the system adopts a modular design, and each functional module can be independently upgraded and maintained, facilitating subsequent function expansion and performance optimization.

[0218] The multi-scale feature pyramid extraction network and heterogeneous attention adaptive fusion mechanism of the present invention further enhance the system's ability to process different types of tumor features and multi-modal data, enabling the system to maintain stable and reliable performance in a complex clinical environment and improving the industrial application value of the system. In particular, the heterogeneous attention adaptive fusion mechanism enables the system to effectively handle the situation of missing or uneven-quality modalities, which is of great significance in actual clinical applications because clinical data is often not as complete and standardized as research data.

[0219] In summary, the present invention provides a lung adenocarcinoma EGFR gene mutation detection system and method based on PET / CT deep learning, which can accurately predict the EGFR gene mutation status and patient prognosis in a non-invasive manner, providing strong support for the personalized and precise treatment of lung adenocarcinoma patients, having important clinical application value and broad market prospects. Through the innovative multi-scale feature pyramid extraction network and heterogeneous attention adaptive fusion mechanism, the system performance is significantly improved, enhancing the adaptability and reliability in a complex clinical environment.

[0220] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A lung adenocarcinoma EGFR gene mutation detection system based on PET / CT deep learning, characterized in that, Comprising: A data acquisition module for obtaining PET / CT image data and clinical information of lung adenocarcinoma patients; An image preprocessing module connected to the data acquisition module for performing normalization processing and ROI extraction on the PET / CT image data; A feature extraction module connected to the image preprocessing module for extracting deep features, metabolic parameter features, and CT sign features from the PET / CT image data; A multi-modal data fusion module connected to the feature extraction module for fusing the extracted multi-modal features; A prediction model module connected to the multi-modal data fusion module for predicting the EGFR gene mutation status and prognosis of lung adenocarcinoma based on the fused features.

2. The system according to claim 1, wherein The image preprocessing module includes: A normalization unit for performing intensity normalization processing on the PET / CT images; A noise elimination unit for performing noise reduction processing on the normalized images; An ROI extraction unit for automatically locating and extracting the tumor region; A contrast enhancement unit for enhancing the image contrast using the CLAHE algorithm; A voxel resampling unit for resampling the extracted ROI to a predefined unified size.

3. The system according to claim 1, characterized in that The feature extraction module includes: A CT encoder for extracting CT image morphological features through a 3D convolutional neural network; A PET encoder for extracting PET image metabolic activity features through a 3D convolutional neural network; A clinical feature encoder for processing the patient's clinical data, CT sign features, and metabolic parameters; A radiomics feature extractor for extracting statistical features from the images; A multi-scale feature pyramid extraction network for extracting tumor feature information at different scales Both the CT encoder and the PET encoder adopt a residual connection structure.

4. The system according to claim 1, characterized in that, The multi-modal data fusion module includes at least one of the following fusion mechanisms: An attention fusion unit for dynamically adjusting the weights of different modal features through an attention mechanism; A bilinear fusion unit for capturing high-order interaction relationships between modalities through bilinear transformation; A simple connection fusion unit for directly splicing and processing multi-modal features; A heterogeneous attention adaptive fusion unit for handling the heterogeneity problem between multi-modal data; The fusion unit adaptively selects the best fusion strategy according to different data quality and integrity degrees.

5. The system according to claim 4, wherein The attention fusion unit includes: A feature transformation layer for performing unified spatial mapping on each modal feature; An attention calculation layer for calculating the attention weights of different modal features; A weighted fusion layer for weighted fusion of different modal features according to the attention weights; The attention calculation layer outputs an attention weight matrix for interpretability analysis.

6. The system according to claim 1, wherein The prediction model module includes: An EGFR mutation classifier for predicting the EGFR gene mutation status of lung adenocarcinoma based on the fused features, including mutation types and mutation sites; A prognosis estimator for predicting the progression-free survival period of patients after receiving EGFR-targeted therapy; A confidence evaluation unit for performing uncertainty quantification analysis on the prediction results; A calibration unit for ensuring the consistent performance of the prediction model on data from different institutions.

7. The system according to claim 1, wherein Also including: The clinical decision support module, connected to the prediction model module, is used for: Generating personalized treatment plan suggestions based on the prediction results; Providing a visual interpretation of the prediction results; Outputting a confidence analysis of the model prediction results; Generating relevant clinical trial and literature information.

8. The system according to claim 1, wherein It also includes: The system monitoring and adaptive module, used for: Monitoring the running status of each component of the system; Evaluating the performance of the model on new data and recording the deviations; Automatically updating the model parameters according to the accumulated new data; Generating a system operation report and outputting performance metrics.

9. The system according to claim 1, characterized in that, The following evaluation methods are adopted by the prediction model module: ROC curve analysis, used to evaluate the classification performance of the model; Decision curve analysis, used to quantify the clinical net benefit at different threshold probabilities; Calibration curve analysis, used to evaluate the consistency between the predicted probability and the actual probability; Multi-center external validation, used to evaluate the generalization ability of the model on data from different institutions.

10. A method for detecting EGFR gene mutations in lung adenocarcinoma based on PET / CT deep learning, using the system according to any one of claims 1-9, characterized in that It includes the following steps: Obtaining the PET / CT image data and clinical information of lung adenocarcinoma patients; Performing standardization processing and ROI extraction on the PET / CT image data; Extracting deep features, metabolic parameter features, and CT sign features from the PET / CT image data; Fusing the extracted multimodal features and dynamically adjusting the weights of different features; Predicting the EGFR gene mutation status of lung adenocarcinoma based on the fused features through a deep learning model; Predicting the progression-free survival period of patients after receiving EGFR targeted therapy based on the fused features; Generating a visual interpretation of the prediction results and personalized treatment plan suggestions; Evaluating the model performance and dynamically updating the model parameters according to new data.

Images