EGFR Gene Mutation Status Prediction Method, System, Device and Medium

Abstract

The present invention discloses a method, system, device and medium for predicting the EGFR gene mutation status, relating to the field of artificial intelligence technology, and aiming to solve the technical problem of low accuracy in predicting EGFR gene mutations in the prior art. The cross-section, coronal plane and sagittal plane of the CT image sample data are input into the feature extraction network module of the EGFR gene mutation prediction model, and the features of the three-plane diagrams within each instance are fused into an instance-level latent vector and used as the output; the latent vector together with the corresponding clinical information is used as the input of the encoder, and the encoder outputs a packet-level embedding and a packet-level gating. The packet-level embedding is used as the input of the classification module, and the classification module outputs a prediction result. The packet-level gating selects the prediction result and controls the output. The attention mechanism is used to help the model focus on and highlight the features that are particularly important for predicting EGFR gene mutations, improve the prediction accuracy and generalization ability, and thus effectively improve the accuracy of predicting EGFR gene mutations.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, relates to the prediction of gene mutations, and particularly relates to a method, system, device and medium for predicting the EGFR gene mutation status. Background Art

[0002] EGFR is the abbreviation of Epidermal Growth Factor Receptor, which plays an important role in cell proliferation, differentiation and survival. The EGFR gene mutation is closely related to the occurrence and development of various cancers. Therefore, accurate prediction of its mutation status is of great significance for the treatment and prognosis of cancers.

[0003] As an effective cancer screening tool, computed tomography (CT) can effectively reduce the cancer mortality rate by early detection, so it is widely used in clinical examinations. More and more researchers have proposed that CT images contain rich information, which may essentially reflect the intrinsic characteristics of the EGFR gene mutation status. Therefore, many researchers have tried to use image data mining methods (such as radiomics and deep learning) to achieve non-invasive EGFR identification. Baihua Zhang (Baihua Zhang et al., 2021) proposed using radiomics features and deep learning models to accurately identify the EGFR gene mutation status of lung adenocarcinoma patients based on non-invasive CT images. However, radiomics involves time-consuming image segmentation and inevitable feature selection, making it difficult to be applied in a clinical environment. In contrast, deep learning can significantly overcome the above disadvantages and outperform radiomics methods in the same task. Suo Wang (Suo Wang et al., 2019) proposed a deep learning model based on non-invasive computed tomography (CT) that can predict the EGFR gene mutation status in lung adenocarcinoma, providing important guidance for treatment decisions and having better prediction performance and non-invasiveness compared with traditional EGFR genotype identification methods.

[0004] The invention patent application with the application number 202210385274.1 discloses a method for predicting the EGFR status of whole pathological sections of lung cancer tissues based on a converter, which includes: 1. Obtaining a dataset of whole pathological sections of lung cancer tissues and performing preprocessing; 2. In the first stage, establishing and training a vision transformer network model capable of predicting the positivity and negativity of image patches; 3. Using the trained vision transformer network model capable of predicting the positivity and negativity of image patches to predict the positivity and negativity categories of image patches in the dataset, screening out negative image patches, and generating an EGFR mutation type dataset using positive image patches; 4. In the second stage, establishing and training a vision transformer network model capable of predicting the EGFR mutation type of image patches using the generated EGFR mutation type dataset; 5. Using the models trained in the first and second stages to complete the prediction of the EGFR status of the whole section. The established deep learning network includes 2 vision transformer ViT networks, and a vision transformer ViT composed of L encoders is constructed as the network in the first stage. Each encoder includes: two normalization layers, a multi-head attention mechanism layer, and a multi-layer perceptron. The vision transformer is used to perform feature learning on the whole pathological section image of lung cancer tissues. The vision transformer can perform dynamic adaptive modeling, capture local and global features of the image based on the attention mechanism, and improve the feature representation ability of the whole pathological section image of lung cancer tissues.

[0005] The invention patent application with the application number 202210583718.2 discloses a method for predicting EGFR gene mutation status based on an end-to-end three-dimensional convolutional neural network, and the steps are as follows: Step 1: Preprocess the original three-dimensional lung CT image, clear the content outside the lung area in the CT image, and scale the CT to a unified size through spline interpolation, that is, 112×112×90; Step 1a: Segment the lungs through the U-Net model and set the area outside the lung area to zero; Step 1b: Obtain the preprocessing result through spline interpolation, that is, an image with a size of 112×112×90 and located at the center of the field of view, ensuring that each slice in the X-axis, Y-axis, and Z-axis directions can cut the lungs and retain the detailed features of the lungs; Step 2: Construct a neural network structure: Adopt a densely connected multi-layer convolutional neural network to extract the overall features of the target and fuse multi-scale features; Step 2a: The proposed method is based on DenseBlock to build a four-layer densely connected convolutional neural network, connect the outputs of each layer in the channel dimension, and use it as the input of the next-layer convolution; Step 2b: During the connection between layers, a bottleneck module for inter-channel feature fusion and dimensionality reduction is introduced. Specifically, the feature distribution is improved to a normal distribution with a mean of 0 and a variance of 1 through batch normalization, and then through a rectified linear unit and a 1×1 convolution to achieve the effect of dimensionality reduction; Step 2c: Embed a multi-scale multi-dilated asymmetric dilated convolution module in the baseline model to capture local micro-features and pay attention to lung nodules in different directions, different sizes, and different angles in the lungs of lung cancer patients; Step 3: Train the neural network structure in an end-to-end manner, that is, use the three-dimensional lung CT image dataset to train the neural network structure with the cross-entropy loss function, use stochastic gradient descent (SGD) as the optimizer of the model, and the momentum is 0.9; During the training process, set the batch size to 6, set the number of iterations of the model to 300, and set all learning rates to 0.01; Define the EGFR mutant type as 1 and the EGFR wild type as 0, that is, the closer the model output result is to 1, the easier it is to be judged as the EGFR mutant type. Set the weight decay of the l2 regularization coefficient to 0.0004 to prevent overfitting; Step 4: Input the three-dimensional lung CT image data into the trained end-to-end three-dimensional convolutional neural network. The input is the CT image of the whole lung, and the output is the prediction result, that is, the predicted EGFR gene mutation status. The invention patent application proposes a deep learning model, that is, three-dimensional dense connectivity asymmetric convolution and multi-expansion density network, to non-invasively predict the EGFR mutation status of lung adenocarcinoma patients. At the same time, it also innovatively proposes a new perspective for studying the EGFR mutation status, that is, the information about the EGFR status exists in the complete two lungs, rather than just in the lung nodules. Since the deep learning model requires the dimensions of the input data to be consistent, the present invention proposes a method for dealing with the problem of inconsistent CT image dimensions.

[0006] There are the following challenges in using deep learning to mine CT image features: 1) Tumor heterogeneity: Tumor heterogeneity poses a challenge to the identification of EGFR types because the diagnostic results are limited by the sampling site, and the true situation (actually without labels) of the residual lesion site is unknown even if it has been marked, which limits the application of traditional supervised learning models; 2) Multi-task learning: The interaction effect between multiple supervised tasks in the multi-task learning setting makes it difficult to achieve a highly discriminative EGFR mutation status representation, resulting in the AUC value of previous EGFR classification models based on CT images varying in the range of 65% - 81%, showing the difficulty of feature learning; 3) 3D input limitation: Low signal-to-noise 3D input is not conducive to feature learning, and the combination of multi-instance learning and 3D input increases the computational and memory overhead, limiting the mini-batch training size and the convergence ability of the backpropagation neural network model; 4) Packet-level false negative results: Despite using multi-instance learning constraints, packet-level false negative results of gene detection still exist, so active learning needs to be introduced to improve the training efficiency.

[0007] Applying semi-supervised learning methods to EGFR prediction may face challenges such as the quality of unlabeled data and uneven class distribution. In this case, the model is prone to bias towards predicting the majority class or being interfered by label noise.

[0008] Therefore, when applying existing deep learning or semi-supervised learning methods to EGFR gene mutation prediction, the accuracy of EGFR gene mutation prediction is generally low. Summary of the Invention

[0009] The purpose of the present invention is to solve the technical problem of low accuracy in predicting EGFR gene mutations in the prior art, and provide a method, system, device and medium for predicting EGFR gene mutation status to more accurately predict EGFR gene mutation status.

[0010] The present invention specifically adopts the following technical solutions to achieve the above purpose:

[0011] A method for predicting EGFR gene mutation status includes the following steps:

[0012] Step S1, obtaining sample data;

[0013] Obtain CT image sample data and corresponding clinical information sample data, and label part of the CT image sample data to form labeled data;

[0014] Step S2, building a prediction model;

[0015] Build an EGFR gene mutation prediction model, and the EGFR gene mutation prediction model includes a feature extraction network module, an encoder, and a classification module;

[0016] The marked tumor region of interest in the CT image sample data is divided to obtain pseudo-3D instances in the cross-sectional, coronal, and sagittal planes. The pseudo-3D instances form a patient-level package and a package-level label. The CT image sample data is used as the input to the EGFR gene mutation prediction model in the form of a package. The pseudo-3D instances in the cross-sectional, coronal, and sagittal planes are input into the feature extraction network module. The feature extraction network module fuses the features of the three-view diagrams within each instance into an instance-level latent vector and outputs it. The latent vector, together with the corresponding clinical information, is used as the input to the encoder. The encoder outputs a package-level embedding and a package-level gating. The package-level embedding is used as the input to the classification module, and the classification module outputs a prediction result. The package-level gating selects and controls the output of the prediction result.

[0017] Step S3, training the prediction model;

[0018] The EGFR gene mutation prediction model is trained using the CT image sample data and the corresponding clinical information sample data obtained in Step S1 to obtain a mature EGFR gene mutation prediction model.

[0019] Step S4, real-time prediction;

[0020] Obtain real-time CT images and input the CT images into the EGFR gene mutation prediction model to obtain a prediction result.

[0021] Further, in Step S1, the CT image sample data is subjected to data augmentation processing. The labeled CT image sample data is subjected to weak augmentation processing, and the unlabeled CT image sample data is subjected to weak augmentation processing and strong augmentation processing.

[0022] Further, in Step S2, the encoder includes a first-level attention pooling layer, a second-level attention pooling layer, a multi-layer perceptron, a sample activation module, and an instance-level group integration module;

[0023] The instance-level latent vectors output by the feature extraction network module are successively passed through the first-level attention pooling layer and the multi-layer perceptron, and then the instance-level latent vectors belonging to the same package are fused into a package-level embedded feature vector.

[0024] The embedded feature vector generates a package-level embedding vector after passing through the second-level attention pooling layer.

[0025] The embedded feature vector is input into the sample activation module. The sample activation module clusters the embedded feature vector and generates and outputs an instance-level gating and a package-level gating according to the clustering result.

[0026] The embedded feature vector is input into the instance-level group integration module. The group convolution within the instance-level group integration module divides each embedded feature vector into several groups, and through the multi-layer perceptron arranged in parallel, the multi-layer perceptron outputs the prediction result;

[0027] The instance-level gating output by the sample activation module selects and controls the output of the prediction result output by the instance-level group integration module.

[0028] Furthermore, in step S3 when training the prediction model, the gating smoothing loss function is:

[0029]

[0030]

[0031] Among them, represents the batch size of the labeled data, represents the batch count, represents the number of classes, represents the class count, is the gating value of the sample, represents the weight of the loss of the represents the sample result after smoothing the label value of the class, represents the predicted value of the latent feature manifold of the sample, represents the feature center of the true class of the sample, represents the square of the two-norm, represents the label value of the sample in the , represents the label smoothing coefficient.

[0032] Furthermore, in step S3 when training the prediction model, the global threshold is estimated through the confidence of the EGFR gene mutation prediction model as the exponentially weighted moving average of the confidence; the global threshold is:

[0033]

[0034] Among them, represents the number of classes, represents the th global threshold, represents the th global threshold, represents the exponentially weighted moving average coefficient, represents the size of each batch of the input unlabeled data, Indicates batch counting, Indicates finding the maximum value in the predicted values, Indicates the predicted value vector for each category of the weakly augmented image;

[0035] The local threshold of the probability estimation for each category through the EGFR gene mutation prediction model, the local threshold is:

[0036]

[0037] where, Indicates the number of categories, Indicates the exponential moving average coefficient, Indicates the size of each batch of the input unlabeled data, Indicates batch counting, using Indicates the th predicted probability output of the model, where Indicates the th model's predicted probability for category , Indicates the th model's predicted probability for category , Indicates the prediction result of the model for the category of the weakly augmented data.

[0038] Furthermore, the loss function of the unlabeled data is:

[0039]

[0040] where, Indicates the size of each batch of the input unlabeled data, Indicates batch counting, Indicates the prediction result of the model for the weakly augmented data, Indicates the one-hot label of the prediction result, Indicates the prediction result of the model for the strongly augmented data, Indicates the label is , the predicted value is of the gated smoothing loss, Indicates finding the maximum value in the predicted values, Indicates the th global threshold.

[0041] Furthermore, an adaptive method is also used to adjust the fairness penalty objective, the adaptive fairness penalty is:

[0042]

[0043]

[0044]

[0045]

[0046]

[0047] Among them, represents the set element, represents the sum of elements in the set, represents the cross-entropy, represents the th prediction probability of the model, represents the th histogram distribution, represents the histogram distribution of the one-hot vector of the weak augmentation prediction, represents the one-hot label of the prediction result, represents the size of each batch of input unlabeled data, represents the batch count, represents finding the maximum value in the predicted values, represents the th global threshold, represents the prediction result of the model for the weak augmentation data, represents the prediction result of the model for the strong augmentation data, represents the exponential moving average coefficient; represents the expectation that the model's prediction for the strong augmentation data is effective this time, represents the histogram distribution of the number of times the model's prediction for the strong augmentation data is effective this time, represents the prediction probability output of the model for the sample data at the t-th time, represents the histogram distribution of the effectiveness of the prediction probability of the model for the sample data at the t-th time.

[0048] An EGFR gene mutation status prediction system, comprising:

[0049] A sample data acquisition module, configured to acquire CT image sample data and corresponding clinical information sample data, and label part of the CT image sample data to form labeled data;

[0050] A prediction model construction module, configured to construct an EGFR gene mutation prediction model, and the EGFR gene mutation prediction model includes a feature extraction network module, an encoder, and a classification module;

[0051] Divide the marked tumor region of interest in the CT image sample data to obtain pseudo-3D instances in the cross-sectional, coronal, and sagittal planes. The pseudo-3D instances form a patient-level package and a package-level label. The CT image sample data is used as the input of the EGFR gene mutation prediction model in the form of a package. The pseudo-3D instances in the cross-sectional, coronal, and sagittal planes are input into the feature extraction network module. The feature extraction network module fuses the features of the three-view diagrams within each instance into an instance-level latent vector and outputs it. The latent vector, together with the corresponding clinical information, is used as the input of the encoder. The encoder outputs a package-level embedding and a package-level gate. The package-level embedding is used as the input of the classification module, and the classification module outputs a prediction result. The package-level gate selects and controls the output of the prediction result.

[0052] A prediction model training module for training the EGFR gene mutation prediction model built by the prediction model building module using the CT image sample data and the corresponding clinical information sample data obtained by the sample data acquisition module to obtain a mature EGFR gene mutation prediction model.

[0053] A real-time prediction module for obtaining a real-time CT image and inputting the CT image into the EGFR gene mutation prediction model to obtain a prediction result.

[0054] A computer device includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of the above method.

[0055] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the above method.

[0056] The beneficial effects of the present invention are as follows:

[0057] 1. In the present invention, through multi-instance learning, the lesions in each CT image are regarded as a sample containing multiple instances, and the relationships between the instances are mined to locate the regions of EGFR gene mutations. At the same time, the attention mechanism is used to help the model focus on and highlight the features that are particularly important for EGFR gene mutation prediction. And semi-supervised learning can make full use of unlabeled data, improve data utilization rate, and through the combination of labeled and unlabeled data, let the model learn more useful information from unlabeled data, improve prediction accuracy and generalization ability, thereby effectively improving the accuracy of EGFR gene mutation prediction and effectively solving the technical problem of low accuracy of EGFR gene mutation prediction in the prior art.

[0058] 2. The main architecture of the model of the present invention is an attention mechanism network based on multi-instance learning, which can effectively extract and fuse instance-level features, thereby further improving the representation ability of the model and enhancing its overall performance.

[0059] 3. The present invention introduces a sample activation module, which uses the idea of active learning to select valuable samples to reduce the probability of misjudgment, thereby improving the accuracy and credibility of the model.

[0060] 4. The present invention introduces the latest semi-supervised learning algorithm based on adaptive threshold and adaptive fair penalty, which can make full use of a large amount of unlabeled data and effectively address problems such as data class imbalance.

[0061] 5. Through the prediction of EGFR mutations and their mutant subtype categories, the results of the present invention can serve as an important reference for targeted lung cancer treatment, helping doctors make diagnostic and treatment decisions, thereby improving the survival expectancy of patients and reducing the treatment difficulty and cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 is a schematic flowchart of the present invention;

[0063] Figure 2 is a schematic flowchart of data augmentation in the present invention;

[0064] Figure 3 is a schematic structural diagram of the EGFR gene mutation prediction model in the present invention;

[0065] Figure 4 is a schematic structural diagram of the encoder in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention.

[0067] Therefore, based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0068] Embodiment 1

[0069] This embodiment provides a method for predicting the EGFR gene mutation status, as Figure 1 shown, which specifically includes the following steps:

[0070] Step S1, obtaining sample data;

[0071] Obtain CT image sample data and corresponding clinical information sample data, and label part of the CT image sample data to form labeled data.

[0072] The sample data comes from West China Hospital of Sichuan University. The inclusion and exclusion criteria for the sample data are as follows:

[0073] Annotated dataset:

[0074] Inclusion criteria: 1) Patients who underwent thin-section chest CT (0.75–1.5 mm) scans before biopsy or surgical treatment; 2) Patients with a detailed pathological report diagnosing lung adenocarcinoma; 3) Patients with a detailed EGFR mutation test report.

[0075] Exclusion criteria: 1) Lack of key clinical data; 2) Gene testing not performed or failed due to poor tissue quality; 3) Chest CT not performed, or the lesions in the CT images are difficult to distinguish and label, such as adhesions at the hilum of the lung or causing atelectasis; 4) Unknown EGFR mutation status or exon 18 mutation, exon 20 insertion mutation, or multiple exon mutations simultaneously.

[0076] Unannotated dataset:

[0077] Inclusion criteria: 1) Patients who underwent thin-section chest CT (0.75–1.5 mm) scans before biopsy or surgical treatment; 2) Patients with a detailed pathological report diagnosing lung adenocarcinoma.

[0078] Exclusion criteria: 1) Lack of key clinical data; 2) Gene testing not performed or failed due to poor tissue quality; 3) Chest CT not performed, or the lesions in the CT images are difficult to distinguish and label, such as adhesions at the hilum of the lung or causing atelectasis.

[0079] Consider a group of patients, each patient having an annotated region of interest (ROI) of the tumor based on CT images. The region of interest (ROI) of the tumor is manually marked by experienced respiratory medicine experts and adjusted from the original lung window image to 48×224×224 pixels, maintaining the original central position. The details of the adjustment are as follows: If the original size of the region of interest (ROI) of the tumor is larger than 48×224×224 pixels, the excess part will be cropped; conversely, if the original size of the region of interest (ROI) of the tumor is smaller than 48×224×224 pixels, it will be filled with a reference value to standardize the size of the region. For the annotated region of interest (ROI) of the tumor, with an EGFR gene mutation label and a pathological invasiveness label (It should be reminded that the writing of is correct, and it is only distinguished from ; that is, represent two different labels respectively). For the unannotated region of interest (ROI) of the tumor, with a pathological invasiveness label . First, a standard 48×224×224 cube is obtained based on the region of interest (ROI) of the tumor. Each cube at the lesion level is divided into m equal parts, and according to their centroid points, image patches of the cross-sectional, coronal, and sagittal planes are obtained. Specifically, the cross-sectional, coronal, and sagittal plane image patches passing through the same centroid point form an instance of multi-instance learning, representing the corresponding three-dimensional information (referred to as a pseudo 3D instance). Finally, these pseudo 3D instances form a patient-level bag and have a bag-level label.

[0080] In step S1, data augmentation processing is performed on the CT image sample data, as Figure 2 shown. Weak augmentation is performed on the labeled CT image sample data, and weak augmentation and strong augmentation are performed on the unlabeled CT image sample data. For weak augmentation, standard flipping and translation augmentation strategies are adopted; specifically, for all data, the image is randomly flipped horizontally at a probability level of 50%, and the image is also randomly translated up to 12.5% in the vertical and horizontal directions. For strong augmentation, two methods based on AutoAugment are used, and then the Cutout technique is applied. This application proposes variants of AutoAugment that do not require prior learning of augmentation strategies using labeled data, such as RandAugment and CTAugment; RandAugment randomly extracts an amplitude value that controls the severity of all transformations from a predefined range, while CTAugment dynamically learns the amplitude of each transformation for each sample.

[0081] Step S2, build a prediction model;

[0082] Build an EGFR gene mutation prediction model as Figure 3 shown. The EGFR gene mutation prediction model includes a feature extraction network module, an encoder, and a classification module;

[0083] The marked region of interest of the tumor in the CT image sample data is divided to obtain pseudo 3D instances of the cross-sectional, coronal, and sagittal planes. The pseudo 3D instances form a patient-level bag and a bag-level label; the CT image sample data is used as the input of the EGFR gene mutation prediction model in the form of a bag. The pseudo 3D instances of the cross-sectional, coronal, and sagittal planes are input into the feature extraction network module. The feature extraction network module fuses the features of the three-plane diagrams within each instance into an instance-level latent vector and outputs it; the latent vector together with the corresponding clinical information is used as the input of the encoder. The encoder outputs a bag-level embedding and a bag-level gating. The bag-level embedding is used as the input of the classification module, and the classification module outputs a prediction result. The bag-level gating selects and controls the output of the prediction result.

[0084] The feature extraction network module can be a Siamese MobileNet-V2 network or other existing networks, as long as it can output latent vectors with fast response. The attention mechanism is applied to feature aggregation within an instance and feature fusion between instances. To process pseudo-3D instances, a learnable embedding is added after the block sequences in the cross-sectional, coronal, and sagittal planes. Specifically, the latent vectors of the corresponding blocks are output through a Siamese MobileNet-V2 network. Then, the features of the three-view diagrams within each instance are fused into instance-level latent vectors through an attention pooling layer. Next, the instance-level latent vectors belonging to the same package are fused into package-level embedded feature vectors using an attention pooling layer.

[0085] As Figure 4 shown, the encoder includes a first-level attention pooling layer, a second-level attention pooling layer, a multi-layer perceptron, a sample activation module, and an instance-level group integration module;

[0086] The instance-level latent vectors output by the feature extraction network module are successively passed through the first-level attention pooling layer and the multi-layer perceptron, and then the instance-level latent vectors belonging to the same package are fused into package-level embedded feature vectors;

[0087] The embedded feature vectors generate package-level embedded vectors after passing through the second-level attention pooling layer;

[0088] The embedded feature vectors are input into the sample activation module, and the sample activation module clusters the embedded feature vectors and generates and outputs instance-level gating and package-level gating according to the clustering results;

[0089] The embedded feature vectors are input into the instance-level group integration module. The group convolution within the instance-level group integration module divides each embedded feature vector into several groups, and through the multi-layer perceptron arranged in parallel, the multi-layer perceptron outputs the prediction results;

[0090] The instance-level gating output by the sample activation module selects and controls the output of the prediction results output by the instance-level group integration module.

[0091] The sample activation module is a method for online adjustment of the model during the training phase. Its goal is to evaluate the beneficial effect of instance-level latent vectors on the discriminative learning of EGFR mutation status and to coordinate the online training of the model by defining sample weights through an adaptive gating technique. To achieve this goal, the sample activation module uses the K-means clustering method to cluster all instances in each mini-batch, and takes the label of the majority of instances in the cluster as the cluster label. For each instance, if its cluster label is the same as the EGFR label of the package it belongs to, its gating value is recorded as 1; otherwise, it is recorded as 0. For each package, if the cluster labels of all instances within it are consistent with the EGFR label of the package, its gating value is set to 1; otherwise, it is set to 0. For positive subtype (EGFR exon 19 and 21 mutations) packages, their package-level gating values are set to 1 because there are almost no false positive packages according to the clinical situation. By using the gating value as the sample weight, it is controlled whether the model performs supervised learning on the input samples, thus realizing the online adjustment of the model training process.

[0092] The instance-level group integration module is a group integration module for multi-task learning, aiming to improve the performance of the model on multiple related tasks. Traditional multi-task learning methods usually learn multiple related tasks by sharing the underlying representation to improve the generalization ability of the model. However, this method ignores the complex relationships between tasks and may lead to inefficient multi-task learning. The instance-level group integration module provides additional regularization by introducing instance-level semantic structure encoding to improve the performance of the shared representation. It uses group convolution to divide each word embedding vector into several groups and uses a multi-head structure to calculate the loss functions of multiple tasks simultaneously, such as EGFR classification and pathology classification. Each group member has an independent head classifier, thus increasing the diversity of task learning and improving the performance of the model on multiple tasks. In this way, the group integration module can effectively improve the learning ability and generalization ability of the model for multiple related tasks.

[0093] Step S3: Train the prediction model;

[0094] Use the CT image sample data and the corresponding clinical information sample data obtained in Step S1 to train the EGFR gene mutation prediction model to obtain a mature EGFR gene mutation prediction model.

[0095] The gated smooth loss is a loss function used to reduce the impact of noisy labels and enhance feature discriminability. It draws on the idea of active learning and uses the gating value given by the sample activation module for online sample weighting during the forward propagation process to actively learn valuable samples. At the same time, the gated smooth loss applies a center constraint to the feature space, pushing the encoded features towards the cluster center to reduce intra-class differences. Finally, by combining the weighted cross-entropy loss function and the label smoothing algorithm, the impact of noisy labels is further reduced and the positive and negative samples are balanced.

[0096] Specifically, when training the prediction model, the gated smoothing loss function is as follows:

[0097]

[0098]

[0099] Among them, represents the batch size of the labeled data, represents the batch count, represents the number of classes, represents the class count, is the gated value of the sample, represents the weight of the loss of the class, represents the sample result after smoothing the label value of the class, represents the predicted value of the class, represents the latent feature manifold of the sample, represents the feature center of the true class of the sample, represents the square of the L2 norm, represents the label value of the i-th class of the sample, , represents the label smoothing coefficient.

[0100] The Self-adaptive Threshold automatically defines and adaptively adjusts the confidence threshold for each class by using the predictions of the model during training. Specifically, when training the prediction model, the global threshold is estimated as the exponential moving average of the confidence through the confidence of the EGFR gene mutation prediction model; the global threshold is as follows:

[0101]

[0102] Among them, represents the number of classes, represents the global threshold at the -th time, represents the global threshold at the -th time, represents the exponential moving average coefficient, represents the size of each batch of the input unlabeled data, represents the maximum value in the predicted values, represents the predicted value vector for each class of the weakly augmented image.

[0103] The local threshold of the probability estimation for each category by the EGFR gene mutation prediction model, which is also the EMA of the probability, the local threshold is:

[0104]

[0105] where represents the number of categories, represents the exponential moving average coefficient, represents the size of each batch of input unlabeled data, represents the batch count, using represents the -th prediction probability output of the model, where represents the -th prediction probability of the model for category , represents the -th prediction probability of the model for category , represents the prediction result of the model for the category of weakly augmented data.

[0106] The loss function of unlabeled data is:

[0107]

[0108] where represents the size of each batch of input unlabeled data, represents the batch count, represents the prediction result of the model for weakly augmented data, represents the one-hot label of the prediction result, represents the prediction result of the model for strongly augmented data, represents the gating smoothing loss with label and predicted value , represents finding the maximum value in the predicted values, represents the -th global threshold.

[0109] At the beginning of training, the threshold is low, accepting more potentially correct samples into training. As the model becomes more confident, the threshold will increase adaptively, filtering out potentially incorrect samples and reducing confirmation bias.

[0110] Self-adaptive Fairness uses the model prediction probability The EMA is used as an estimate of the expected value of the predicted distribution over unlabeled data. Considering that the pseudo-label distribution may not be uniform, an adaptive approach is considered to adjust the fairness penalty objective, which can encourage the model to make more diverse and fair predictions. Therefore, this embodiment also uses an adaptive approach to adjust the fairness penalty objective, and the adaptive fairness penalty is:

[0111]

[0112]

[0113]

[0114]

[0115]

[0116] where represents the set element, represents the sum of the elements in the set, represents the cross-entropy, represents the th prediction probability of the model, represents the th histogram distribution, represents the histogram distribution of the one-hot vector of the weakly augmented prediction, represents the one-hot label of the prediction result, represents the size of each batch of the input unlabeled data, represents the batch count, represents finding the maximum value in the predicted values, represents the th global threshold, represents the prediction result of the model on the weakly augmented data, represents the prediction result of the model on the strongly augmented data, represents the exponential moving average coefficient; represents the expectation that the model's prediction of the strongly augmented data is valid this time, represents the histogram distribution of the number of times the model's prediction of the strongly augmented data is valid this time, represents the prediction probability output of the model for the sample data at the t-th time, represents the histogram distribution of the prediction probability of the model for the sample data at the t-th time being valid.

[0117] Finally, the overall loss is . The instance-level loss includes the loss of the EGFR mutation status and its mutation subtypes of the predicted instance , the loss of predicting the pathological type and the loss of the corresponding unlabeled data 、 and 。The loss at the package level includes the loss of predicting the EGFR mutation status and its mutation subtypes of the package , the loss of unlabeled data and 。

[0118] Step S4, real-time prediction;

[0119] Obtain real-time CT images, and input the CT images into the EGFR gene mutation prediction model to obtain the prediction results.

[0120] Example 2

[0121] This example provides an EGFR gene mutation status prediction system, which specifically includes the following steps:

[0122] A sample data acquisition module, which is used to acquire CT image sample data and corresponding clinical information sample data, and label part of the CT image sample data to form labeled data.

[0123] The sample data comes from West China Hospital of Sichuan University. The inclusion and exclusion criteria for the sample data are as follows:

[0124] Labeled data set:

[0125] Inclusion criteria: 1), patients who underwent thin-section chest CT (0.75–1.5 mm) scans before biopsy or surgical treatment; 2), patients with a detailed pathological report diagnosing lung adenocarcinoma; 3) patients with a detailed EGFR mutation detection report.

[0126] Exclusion criteria: 1), lack of key clinical data; 2), no gene detection or failed the test due to poor tissue quality; 3), no chest CT examination, or the lesions in the CT images are difficult to distinguish and label, such as adhesion at the hilum of the lung or causing atelectasis; 4), unknown EGFR mutation status or exon 18 mutation, exon 20 insertion mutation, or multiple exon mutations at the same time.

[0127] Unlabeled data set:

[0128] Inclusion criteria: 1), patients who underwent thin-section chest CT (0.75–1.5 mm) scans before biopsy or surgical treatment; 2), patients with a detailed pathological report diagnosing lung adenocarcinoma.

[0129] Exclusion criteria: 1) Lack of key clinical data; 2) Failure to perform genetic testing or failure to pass the test due to poor tissue quality; 3) Failure to perform chest CT examination, or the lesions in the CT images are difficult to distinguish and label, such as adhesions at the hilum of the lung or causing atelectasis.

[0130] Consider a group of patients, each with a labeled region of interest (ROI) of the tumor based on CT images. The region of interest (ROI) of the tumor is manually marked by experienced respiratory physicians and adjusted from the original lung window image to 48×224×224 pixels, maintaining the original central position. The details of the adjustment are as follows: If the original size of the region of interest (ROI) of the tumor is larger than 48×224×224 pixels, the excess part will be cropped; on the contrary, if the original size of the region of interest (ROI) of the tumor is smaller than 48×224×224 pixels, it will be filled with a reference value to standardize the size of the region. For the labeled region of interest (ROI) of the tumor, with EGFR gene mutation label and pathological invasiveness label . For the unlabeled region of interest (ROI) of the tumor, with pathological invasiveness label . First, a standard 48×224×224 cube is obtained according to the region of interest (ROI) of the tumor. Each cube at the lesion level is divided into m equal parts, and according to their centroid points, image patches of the cross-sectional, coronal, and sagittal planes are obtained. Specifically, the cross-sectional, coronal, and sagittal plane image patches passing through the same centroid point form an instance of multi-instance learning, representing the corresponding three-dimensional information (referred to as a pseudo-3D instance). Finally, these pseudo-3D instances form a patient-level bag and have a bag-level label.

[0131] In the sample data acquisition module, data augmentation processing is performed on the CT image sample data. Weak augmentation processing is performed on the labeled CT image sample data, and weak augmentation processing and strong augmentation processing are performed on the unlabeled CT image sample data. For weak augmentation, standard flipping and translation augmentation strategies are adopted; specifically, for all data, the image is randomly flipped horizontally at a 50% probability level, and the image is also randomly translated up to 12.5% in the vertical and horizontal directions. For strong augmentation, two methods based on AutoAugment are used, and then the Cutout technique is applied. This application proposes variants of AutoAugment that do not require prior learning of augmentation strategies using labeled data, such as RandAugment and CTAugment; RandAugment randomly extracts an amplitude value that controls the severity of all transformations from a predefined range, while CTAugment dynamically learns the amplitude of each transformation for each sample.

[0132] The prediction model building module is used to build as Figure 3The EGFR gene mutation prediction model shown, the EGFR gene mutation prediction model includes a feature extraction network module, an encoder, and a classification module;

[0133] The tumor region of interest marked in the CT image sample data is divided to obtain pseudo-3D instances in the cross-section, coronal plane, and sagittal plane. The pseudo-3D instances form a patient-level package and a package-level label; the CT image sample data is used as the input of the EGFR gene mutation prediction model in the form of a package. The pseudo-3D instances in the cross-section, coronal plane, and sagittal plane are input into the feature extraction network module. The feature extraction network module fuses the features of the three-dimensional diagrams within each instance into an instance-level latent vector and outputs it; the latent vector together with the corresponding clinical information is used as the input of the encoder. The encoder outputs a package-level embedding and a package-level gating. The package-level embedding is used as the input of the classification module, and the classification module outputs a prediction result. The package-level gating selects the prediction result and controls the output.

[0134] The feature extraction network module can be a Siamese MobileNet-V2 network or an existing network. It only needs to be able to output a latent vector with fast response. The attention mechanism is applied to feature aggregation within instances and feature fusion between instances. To process pseudo-3D instances, a learnable embedding is added after the block sequences in the cross-section, coronal plane, and sagittal plane. Specifically, the latent vectors of the corresponding blocks are output through a Siamese MobileNet-V2 network. Then, the features of the three-dimensional diagrams within each instance are fused into an instance-level latent vector through an attention pooling layer. Next, the instance-level latent vectors belonging to the same package are fused into a package-level embedded feature vector using an attention pooling layer.

[0135] As shown in the figure, the encoder includes a first-level attention pooling layer, a second-level attention pooling layer, a multi-layer perceptron, a sample activation module, and an instance-level group integration module;

[0136] The instance-level latent vector output by the feature extraction network module is successively passed through the first-level attention pooling layer and the multi-layer perceptron, and the instance-level latent vectors belonging to the same package are fused into a package-level embedded feature vector;

[0137] The embedded feature vector generates a package-level embedding vector after passing through the second-level attention pooling layer;

[0138] The embedded feature vector is input into the sample activation module. The sample activation module clusters the embedded feature vector and generates and outputs an instance-level gating and a package-level gating according to the clustering result;

[0139] The embedded feature vector is input into the instance-level group integration module. The group convolution in the instance-level group integration module divides each embedded feature vector into several groups, and through the multi-layer perceptron arranged in parallel, the multi-layer perceptron outputs a prediction result;

[0140] The instance-level gating output by the sample activation module selects and controls the output of the prediction results output by the instance-level group integration module.

[0141] The sample activation module is a method for online adjustment of the model during the training phase. Its goal is to evaluate the beneficial effects of instance-level latent vectors on the discriminative learning of EGFR mutation status, and to coordinate the online training of the model by defining sample weights through adaptive gating technology. To achieve this goal, the sample activation module uses the K-means clustering method to cluster all instances in each mini-batch, and takes the label of the majority of instances in the class as the clustering label. For each instance, if its clustering label is the same as the EGFR label of the package it belongs to, its gating value is recorded as 1; otherwise, it is recorded as 0. For each package, if the clustering labels of all instances within it are consistent with the EGFR label of the package, its gating value is set to 1; otherwise, it is set to 0. For positive subtype (EGFR exon 19 and 21 mutations) packages, their package-level gating values are set to 1 because, according to clinical situations, there are almost no false positive packages. By using the gating value as the sample weight, it controls whether the model performs supervised learning on the input samples, thus achieving online adjustment of the model training process.

[0142] The instance-level group integration module is a group integration module for multi-task learning, aiming to improve the performance of the model on multiple related tasks. Traditional multi-task learning methods usually learn multiple related tasks by sharing the underlying representation to improve the generalization ability of the model. However, this method ignores the complex relationships between tasks and may lead to low efficiency in multi-task learning. The instance-level group integration module provides additional regularization by introducing instance-level semantic structure encoding to improve the performance of the shared representation. It uses group convolution to divide each word embedding vector into several groups and uses a multi-head structure to calculate the loss functions of multiple tasks simultaneously, such as EGFR classification and pathological classification. Each group member has an independent head classifier, thus increasing the diversity of task learning and improving the performance of the model on multiple tasks. In this way, the group integration module can effectively improve the learning ability and generalization ability of the model for multiple related tasks.

[0143] The prediction model training module is used to train the EGFR gene mutation prediction model using the CT image sample data and the corresponding clinical information sample data obtained by the sample data acquisition module, and obtain a mature EGFR gene mutation prediction model.

[0144] The gated smoothing loss is a loss function used to reduce the impact of noisy labels and enhance the discriminability of features. It draws on the idea of active learning and uses the gating values given by the sample activation module for online sample weighting during the forward propagation process to actively learn valuable samples. At the same time, the gated smoothing loss applies a center constraint to the feature space, pushing the encoded features towards the clustering center and reducing the within-class differences. Finally, by combining the weighted cross-entropy loss function and the label smoothing algorithm, the impact of noisy labels is further reduced and the positive and negative samples are balanced.

[0145] Specifically, when training the prediction model, the gated smoothing loss function is:

[0146]

[0147]

[0148] where, represents the batch size of the labeled data, represents the batch count, represents the number of classes, represents the class count, is the gating value of the sample, represents the weight of the loss of the class, represents the result after smoothing the label value of the sample for the class, represents the predicted value for the class, represents the latent feature manifold of the sample, represents the feature center of the true class of the sample, represents the square of the L2 norm, represents the label value of the sample for the i-th class, , represents the label smoothing coefficient.

[0149] The Self-adaptive Threshold automatically defines and adaptively adjusts the confidence threshold for each class by using the predictions of the model during training. Specifically, when training the prediction model, the global threshold is estimated as the exponential moving average of the confidence through the confidence of the EGFR gene mutation prediction model; the global threshold is:

[0150]

[0151] where, represents the number of classes, represents the global threshold at the representation of the The global threshold for the next time, represents the exponential moving average coefficient, represents the size of each batch of input unlabeled data, represents the batch count, represents finding the maximum value in the predicted values, represents the vector of predicted values for each category of the weakly augmented image.

[0152] The local threshold of the probability estimation for each category by the EGFR gene mutation prediction model, which is also the EMA of the probability, the local threshold is:

[0153]

[0154] Among them, represents the number of categories, represents the exponential moving average coefficient, represents the size of each batch of input unlabeled data, represents the batch count, using represents the th prediction probability output of the model, where represents the th prediction probability of the model for the category , represents the th prediction probability of the model for the category , represents the prediction result of the model for the category of the weakly augmented data.

[0155] The loss function of the unlabeled data is:

[0156]

[0157] Among them, represents the size of each batch of input unlabeled data, represents the batch count, represents the prediction result of the model for the weakly augmented data, represents the one-hot label of the prediction result, represents the prediction result of the model for the strongly augmented data, represents the label as , the predicted value is of the gated smoothing loss, represents finding the maximum value in the predicted values, represents the th global threshold.

[0158] At the beginning of training, the threshold is low, allowing more potentially correct samples to enter the training. As the model becomes more confident, the threshold adaptively increases, filtering out potentially incorrect samples and reducing confirmation bias.

[0159] Self-adaptive Fairness uses the EMA of the model's predicted probabilities as an estimate of the expected value of the prediction distribution on unlabeled data. Considering that the pseudo-label distribution may not be uniform, an adaptive approach is considered to adjust the fairness penalty objective, which can encourage the model to make more diverse and fair predictions. Therefore, this embodiment also uses an adaptive approach to adjust the fairness penalty objective, and the self-adaptive fairness penalty is:

[0160]

[0161]

[0162]

[0163]

[0164]

[0165] where represents the set element, represents the sum of the elements in the set, represents the cross-entropy, represents the th predicted probability of the model, represents the th histogram distribution, represents the histogram distribution of the one-hot vector of the weakly augmented prediction, represents the one-hot label of the prediction result, represents the size of each batch of the input unlabeled data, represents the batch count, represents finding the maximum value in the predicted values, represents the th global threshold, represents the prediction result of the model on the weakly augmented data, represents the prediction result of the model on the strongly augmented data, represents the exponential moving average coefficient; represents the expectation that the model's prediction on the strongly augmented data is valid this time, represents the histogram distribution of the number of times the model's prediction on the strongly augmented data is valid this time, Denotes the predicted probability output of the model for the sample data at the t-th time. Denotes the histogram distribution of the effective predicted probability of the model for the sample data at the t-th time.

[0166] Finally, the overall loss is . The instance-level loss Includes the loss of predicting the EGFR mutation status and its mutation subtypes of the instance , the loss of predicting the pathological type And the loss of the corresponding unlabeled data 、 And . The package-level loss Includes the loss of predicting the EGFR mutation status and its mutation subtypes of the package , the loss of unlabeled data And .

[0167] A real-time prediction module, configured to obtain real-time CT images, input the CT images into an EGFR gene mutation prediction model, and obtain a prediction result.

[0168] Example 3

[0169] A computer device includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of the EGFR gene mutation status prediction method.

[0170] Wherein, the computer device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The computer device can perform human-computer interaction with a user through a keyboard, a mouse, a remote control, a touchpad, a voice control device, etc.

[0171] The memory at least includes one type of readable storage medium, which includes flash memory, hard disk, multimedia card, card-type memory (such as SD or D-interface display memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disc, etc. In some embodiments, the memory may be an internal storage unit of the computer device, such as the hard disk or memory of the computer device. In other embodiments, the memory may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMediaCard (SMC), Secure Digital (SD) card, FlashCard, etc. equipped on the computer device. Of course, the memory may also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the memory is commonly used to store the operating system and various application software installed on the computer device, such as the program code of the EGFR gene mutation status prediction method. In addition, the memory can also be used to temporarily store various data that have been output or will be output.

[0172] In some embodiments, the processor may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor is generally used to control the overall operation of the computer device. In this embodiment, the processor is used to run the program code stored in the memory or process data, such as running the program code of the EGFR gene mutation status prediction method.

[0173] Embodiment 4

[0174] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to execute the steps of the EGFR gene mutation status prediction method.

[0175] Wherein, the computer-readable storage medium stores an interface display program, and the interface display program can be executed by at least one processor to cause the at least one processor to execute the steps of the EGFR gene mutation status prediction method as described above.

[0176] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described example methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented through hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server or network device, etc.) to execute the EGFR gene mutation status prediction method described in the embodiments of the present application.

Claims

1. A method for predicting EGFR gene mutation status, characterized in that: The steps include: Step S1, obtaining sample data; Acquire CT image sample data and corresponding clinical information sample data, and annotate part of the CT image sample data to form label data; Step S2, building a prediction model; Build an EGFR gene mutation prediction model, which includes a feature extraction network module, an encoder, and a classification module; The marked tumor interest regions in the CT image sample data are divided to obtain pseudo 3D instances of the cross section, coronal plane and sagittal plane. The pseudo 3D instances constitute a patient-level package and a package-level label. The CT image sample data is used as the input of the EGFR gene mutation prediction model in the form of a package. The pseudo 3D instances of the cross section, coronal plane and sagittal plane are input into the feature extraction network module. The feature extraction network module fuses the features of the three-shape map in each instance into an instance-level latent vector as output. The latent vector and the corresponding clinical information are used as the input of the encoder. The encoder outputs package-level embedding and package-level gating. The package-level embedding is used as the input of the classification module. The classification module outputs the prediction result. The package-level gating selects the prediction result and controls the output. Step S3, training the prediction model; The CT image sample data obtained in step S1 and the corresponding clinical information sample data are used to train the EGFR gene mutation prediction model to obtain a mature EGFR gene mutation prediction model; Step S4, real-time prediction; Real-time CT images are acquired and input into the EGFR gene mutation prediction model to obtain prediction results.

2. The method for predicting EGFR gene mutation status according to claim 1, characterized in that: In step S1, data enhancement processing is performed on the CT image sample data, weak enhancement processing is performed on the labeled CT image sample data, and weak enhancement processing and strong enhancement processing are performed on the unlabeled CT image sample data.

3. The method for predicting EGFR gene mutation status according to claim 1, characterized in that: In step S2, the encoder includes a primary attention pooling layer, a secondary attention pooling layer, a multi-layer perceptron, a sample activation module, and an instance-level group integration module; The instance-level latent vectors output by the feature extraction network module are sequentially passed through the first-level attention pooling layer and the multi-layer perceptron to fuse the instance-level latent vectors belonging to the same package into a package-level embedded feature vector; The embedded feature vector is passed through a secondary attention pooling layer to generate a packet-level embedding vector; The embedded feature vector is input into the sample activation module, which clusters the embedded feature vector and generates and outputs instance-level gating and packet-level gating according to the clustering results; The embedded feature vector is input into the instance-level group integration module. The group convolution in the instance-level group integration module divides each embedded feature vector into several groups, and passes through the multi-layer perceptrons arranged in parallel. The multi-layer perceptron outputs the prediction results. The instance-level gating of the sample activation module output selects and controls the output of the prediction results of the instance-level group integration module output.

4. The method for predicting EGFR gene mutation status according to claim 1, characterized in that: Step S3: When training the prediction model, the gated smoothing loss function for: in, represents the batch size of labeled data, Indicates batch count, represents the number of categories, represents the category count, is the gate value of the sample, Indicates The weight of the class loss, Representation sample The result after smoothing the label value of the category, Express The predicted value of the category, represents the potential feature manifold of the sample, Represents the feature center of the true category of the sample, represents the square of the two-norm, represents the label value of sample i category, , Represents the label smoothing coefficient.

5. The method for predicting EGFR gene mutation status according to claim 4, characterized in that: Step S3: When training the prediction model, the global threshold is estimated by the confidence of the EGFR gene mutation prediction model as the exponential moving average of the confidence; the global threshold for: in, represents the number of categories, Indicates The global threshold of Indicates The global threshold of represents the exponential moving average coefficient, Indicates the size of each batch of input unlabeled data, Indicates batch count, It means to find the maximum value among the predicted values. Represents the prediction value vector for each category of the weakly enhanced image; The local threshold is estimated by the EGFR gene mutation prediction model for the probability of each category. for: in, represents the number of categories, represents the exponential moving average coefficient, Indicates the size of each batch of input unlabeled data, Indicates batch counting, using Represents the model The predicted probability output is Indicates The model for the category The predicted probability of Indicates The model for the category The predicted probability of Indicates the model's weakly enhanced data The prediction results of the categories.

6. The method for predicting EGFR gene mutation status according to claim 5, characterized in that: Loss function for unlabeled data for: in, Indicates the size of each batch of input unlabeled data, Indicates batch count, Represents the prediction result of the model for weakly enhanced data, Indicates the unique hot label of the prediction result, Represents the prediction result of the model for strongly enhanced data, Indicates that the label is The predicted value is The gated smoothing loss, It means to find the maximum value among the predicted values. Indicates The global threshold of times.

7. The method for predicting EGFR gene mutation status according to claim 6, characterized in that: We also use an adaptive method to adjust the fairness penalty target, adaptive fairness penalty for: in, Represents a collection element. represents the sum of the elements in a collection, represents the cross entropy, Represents the model The predicted probability of Indicates The histogram distribution of times, The histogram distribution of one-hot vectors representing weakly enhanced predictions, Indicates the unique hot label of the prediction result, Indicates the size of each batch of input unlabeled data, Indicates batch count, It means to find the maximum value among the predicted values. Indicates The global threshold of Represents the prediction result of the model for weakly enhanced data, Represents the prediction result of the model for strongly enhanced data, represents the exponential moving average coefficient; It indicates the expectation that the model is effective for predicting strongly enhanced data this time. The histogram distribution represents the number of effective predictions of the model for the strongly enhanced data. It represents the predicted probability output of the model for the sample data for the tth time, The histogram distribution represents the model's prediction probability for the sample data at the tth time.

8. A system for predicting EGFR gene mutation status, characterized in that: include: The sample data acquisition module is used to acquire CT image sample data and corresponding clinical information sample data, and annotate part of the CT image sample data to form label data; A prediction model building module is used to build an EGFR gene mutation prediction model, which includes a feature extraction network module, an encoder, and a classification module; The marked tumor interest regions in the CT image sample data are divided to obtain pseudo 3D instances of the cross section, coronal plane and sagittal plane. The pseudo 3D instances constitute a patient-level package and a package-level label. The CT image sample data is used as the input of the EGFR gene mutation prediction model in the form of a package. The pseudo 3D instances of the cross section, coronal plane and sagittal plane are input into the feature extraction network module. The feature extraction network module fuses the features of the three-shape map in each instance into an instance-level latent vector as output. The latent vector and the corresponding clinical information are used as the input of the encoder. The encoder outputs package-level embedding and package-level gating. The package-level embedding is used as the input of the classification module. The classification module outputs the prediction result. The package-level gating selects the prediction result and controls the output. The prediction model training module is used to train the EGFR gene mutation prediction model built by the prediction model building module using the CT image sample data acquired by the sample data acquisition module and the corresponding clinical information sample data to obtain a mature EGFR gene mutation prediction model; The real-time prediction module is used to obtain real-time CT images and input the CT images into the EGFR gene mutation prediction model to obtain prediction results.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

Images