System and method for predicting brain metastasis of lung cancer cells

Abstract

The invention discloses a lung cancer cell brain metastasis prediction system and method, and belongs to the field of computer image processing.The lung cancer cell brain metastasis prediction method comprises the steps that a primary lung cancer CT image sequence, a same-period pathological examination result and a brain nuclear magnetic report are collected; preprocessing the primary lung cancer CT image sequence, and marking the preprocessed CT image sequence as a first image sequence; obtaining a prediction classification result according to a lung cancer brain metastasis sniping model BM-Sniper; the aggregation feature vector is analyzed and processed, linear classification is executed, a classification prediction result is obtained, and linear classification is obtained through a support vector machine model or a Bayesian classifier. Through primary lung cancer CT image sequence processing, feature extraction of a lung scanning image is carried out according to an image feature extraction mode to obtain an aggregation feature vector, and the aggregation feature vector is input into a classification module of a prediction model to output a classification result, so that whether a detected object is subjected to lung cancer brain metastasis is predicted; and an intermediate result with high accuracy is provided for auxiliary diagnosis of a doctor.

Description

Technical Field The present invention belongs to the field of computer image processing, and in particular relates to a system and method for predicting brain metastasis of lung cancer cells. Background Art Lung cancer is the leading cause of cancer-related death and one of the most common types of cancer in the world. Lung cancer is mainly divided into non-small cell lung cancer (NSCLC) and small cell lung cancer (SCLC). NSCLC can be further divided into lung adenocarcinoma, lung squamous cell carcinoma and large cell carcinoma. NSCLC accounts for about 85% of all lung cancer cases, while SCLC and other subtypes account for the remaining 15%.

[0003] Because there are a large number of anastomoses between the cerebral blood vessels and the vertebral arteries and venous arterial plexuses that supply the brain, lung cancer cells can metastasize directly through the heart and carotid arteries to the brain without filtering through the pulmonary capillaries. Therefore, the brain is the most common metastatic site of lung cancer, and brain metastases (BM) account for about 50% of distant metastases. The incidence of brain metastases in NSCLC is between 17% and 54%, and the estimated incidence is around 30%. Compared with NSCLC, brain metastases are more common in SCLC, and more than 50% of SCLC patients will develop intracranial metastases during the course of the disease.

[0004] How to use lung cancer CT images to predict and judge lung cancer brain metastasis and assist doctors in making accurate diagnoses has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of the above problems, the present invention proposes to collect primary lung cancer CT image sequences, concurrent pathological examination results and brain MRI reports; preprocess the primary lung cancer CT image sequence, and record the preprocessed CT image sequence as the first image sequence; obtain the prediction classification result according to the lung cancer brain metastasis sniper model BM-Sniper; analyze and process the aggregated feature vector to perform linear classification to obtain the classification prediction result, and the linear classification is obtained by the support vector machine model or the Bayesian classifier. Through the processing of the primary lung cancer CT image sequence, the feature extraction of the lung scan image is performed in the manner of image feature extraction to obtain the aggregated feature vector, and the aggregated feature vector is input into the classification module of the prediction model to output the classification result, so as to predict whether the object under test has lung cancer brain metastasis, and provide a highly accurate intermediate result for the doctor's auxiliary diagnosis.

[0006] The technical solution adopted by the present invention to solve the above technical problems is: A system for predicting brain metastasis of lung cancer cells, the system comprising: Image acquisition module, used to collect primary lung cancer CT image sequences, concurrent pathological examination results and brain MRI reports; An image preprocessing module, used for preprocessing the primary lung cancer CT image sequence, and recording the preprocessed CT image sequence as a first image sequence; A prediction and classification module, used to obtain prediction and classification results according to the lung cancer brain metastasis sniping model BM-Sniper; the lung cancer brain metastasis sniping model BM-Sniper includes an image feature extraction module and an image classification module; An image feature extraction module, comprising an image encoder, a position encoder, a deep learning encoder and a pooling layer, for extracting an aggregate feature vector from the first image sequence, wherein the aggregate feature vector represents a dynamic change between images in the first image sequence; The image classification module is used to analyze and process the aggregated feature vectors to perform linear classification and obtain classification prediction results.

[0007] Furthermore, the filtering method includes a median filter model: ,in is the image pixel in the primary lung cancer CT image sequence, The matrix of order, ,for L=2, is the image pixel in the filtered image sequence.

[0008] Furthermore, the image acquisition module is also used to extract the identification patient identifier and pathological markers of primary lung cancer from the pathology and magnetic resonance imaging (MRI) radiology text reports. If brain cancer cells are found in a lung cancer patient during a 2-year MRI follow-up examination, the lung cancer patient is considered to be positive for brain metastasis; otherwise, the patient is considered to be negative for brain metastasis.

[0009] Furthermore, the preprocessing includes: S21, analyzing the noise distribution of the CT image sequence, and selecting the optimal filtering method for filtering according to the noise distribution; S22, analyzing the resolution of the CT image sequence, if the resolutions are inconsistent, taking the resolution of the largest number of images with the same resolution in the CT image sequence as a benchmark, resampling other images so that the resolution of the CT image sequence is the same; S23, performing normalization processing on images in the CT image sequence with the same resolution.

[0010] Further, the extracting the aggregated feature vector includes: S31, inputting the first image sequence into an image encoder to obtain a first image feature matrix of the first image sequence; S32, inputting the first image sequence into a position encoder to obtain a first image sequence position matrix; S33, inputting the first image feature matrix and the first image sequence position matrix into a deep learning encoder to obtain a second image feature matrix; S34, input the second image feature matrix into the pooling layer to obtain an aggregated feature vector.

[0011] Furthermore, the prediction and classification module also includes: a BM-Sniper training module, which is used to train the BM-Sniper model: The training includes: S41. Set the pre-training self-supervised learning strategy and use the masked image modeling strategy to pre-train the visual Transformer. S42, setting a loss function that needs to be minimized by the model in the pre-training stage, wherein the loss function is a cross entropy loss; S43, set learning rate, batch size, weight decay, Adam optimizer, and initialize model weights; S44, the model receives the 50% blocked images in the training set and starts predicting the blocked parts; updates the model weights through back propagation and gradient descent until the model meets the preset conditions; and completes the pre-training process; S45. Select to freeze all layers of the pre-trained model to keep the feature representation learned during pre-training unchanged; set the softmax layer at the top of the model and connect the image classification module. S46. Use labeled data to train the model and fine-tune the model weights to minimize the loss function of the classification task to complete model training.

[0012] A method for predicting brain metastasis of lung cancer cells is applied to a system for predicting brain metastasis of lung cancer cells. The method comprises the following steps: S1. Collect CT image sequences of primary lung cancer, concurrent pathological examination results and brain MRI reports; S2, preprocessing the primary lung cancer CT image sequence, and recording the preprocessed CT image sequence as a first image sequence; The preprocessing includes, S21, analyzing the noise distribution of the CT image sequence, and selecting the optimal filtering method for filtering according to the noise distribution; S22, analyzing the resolution of the CT image sequence, if the resolutions are inconsistent, taking the resolution of the largest number of images with the same resolution in the CT image sequence as a benchmark, resampling other images so that the resolution of the CT image sequence is the same; S23, performing normalization processing on images in the CT image sequence with the same resolution; S3. Obtain prediction and classification results based on the lung cancer brain metastasis sniper model BM-Sniper; The lung cancer brain metastasis sniping model BM-Sniper includes an image feature extraction module and an image classification module; An image feature extraction module, comprising an image encoder, a position encoder, a deep learning encoder and a pooling layer, for extracting an aggregate feature vector from the first image sequence, wherein the aggregate feature vector represents a dynamic change between images in the first image sequence; The image classification module is used to analyze and process the aggregated feature vectors to perform linear classification to obtain classification prediction results, wherein the linear classification is obtained by a support vector machine model or a Bayesian classifier.

[0013] Extracting the aggregate feature vector includes: S31, inputting the first image sequence into an image encoder to obtain a first image feature matrix of the first image sequence; S32, inputting the first image sequence into a position encoder to obtain a first image sequence position matrix; S33, inputting the first image feature matrix and the first image sequence position matrix into a deep learning encoder to obtain a second image feature matrix; S34, input the second image feature matrix into the pooling layer to obtain an aggregated feature vector.

[0014] Furthermore, the prediction and classification module also includes: a BM-Sniper training module, which is used to train the BM-Sniper model: The training includes: S41. Set the pre-training self-supervised learning strategy and use the masked image modeling strategy to pre-train the visual Transformer. S42, setting a loss function that needs to be minimized by the model in the pre-training stage, wherein the loss function is a cross entropy loss; S43, set learning rate, batch size, weight decay, Adam optimizer, and initialize model weights; S44, the model receives the 50% blocked images in the training set and starts predicting the blocked parts; updates the model weights through back propagation and gradient descent until the model meets the preset conditions; and completes the pre-training process; S45. Select to freeze all layers of the pre-trained model to keep the feature representation learned during pre-training unchanged; set the softmax layer at the top of the model and connect the image classification module. S46. Use labeled data to train the model and fine-tune the model weights to minimize the loss function of the classification task to complete model training.

[0015] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a method for predicting brain metastasis of lung cancer cells when executing the computer program.

[0016] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for predicting brain metastasis of lung cancer cells.

[0017] An electronic device, comprising: one or more processors; a storage device for storing one or more programs, Wherein, when the one or more programs are executed by the one or more processors, The one or more processors execute the steps of the method for predicting brain metastasis of lung cancer cells.

[0018] The beneficial effects of the present invention are as follows: Medical image annotation is difficult and costly, and it is difficult to obtain large-scale labeled data. Self-supervised learning does not rely on data annotation. Training a self-supervised learning model based on ultra-large-scale lung CT data can give full play to the advantages of ultra-large-scale data and large models. By constructing a self-supervised pre-training model in the early stage, the dependence of downstream classification tasks on labeled data can be greatly reduced.

[0019] By processing the primary lung cancer CT image sequence, the feature extraction of the lung scan image is performed in accordance with the image feature extraction method to obtain an aggregated feature vector, which is then input into the classification module of the prediction model to output the classification result. This enables the prediction of whether the subject has brain metastasis from lung cancer, and provides doctors with highly accurate intermediate results for auxiliary diagnosis.

[0020] By considering multiple consecutive chest CT image sequences from a given patient, the transformer algorithm is used to predict events. The prediction for each patient is done by aggregating the patient's image sequences and imitating the radiologist's interpretation procedure. This solution avoids manual intervention in the preprocessing and preparation of input data, enabling the classification model BM-sniper to learn fewer unbiased image features, improve the accuracy of prediction, and facilitate doctor-assisted diagnosis.

[0021] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above description and other purposes, features and advantages of the present invention more obvious and easy to understand, the preferred embodiments are specifically cited and described in detail as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] By reading the detailed description of the preferred embodiment below, various other advantages and benefits will become clear to those of ordinary skill in the art. The accompanying drawings are only used for the purpose of illustrating the preferred embodiment and are not considered to be limitations of the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings.

[0023] Figure 1 This is the structural diagram of the lung cancer cell brain metastasis prediction system; Figure 2 Schematic diagram of the method for predicting brain metastasis of lung cancer cells; Figure 3 Figure 2 is a graph of the receiver operating characteristic curve of the BM-sniper model using lung scan images. DETAILED DESCRIPTION

[0024] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0025] In the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be connected, detachably connected, or integrated; it can be mechanically connected or electrically connected; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0026] Example 1 like Figure 1 The system for predicting brain metastasis of lung cancer cells shown in the figure comprises: Image acquisition module, used to collect primary lung cancer CT image sequences, concurrent pathological examination results and brain MRI reports; An image preprocessing module, used for preprocessing the primary lung cancer CT image sequence, and recording the preprocessed CT image sequence as a first image sequence; A prediction and classification module, used to obtain prediction and classification results according to the lung cancer brain metastasis sniping model BM-Sniper; the lung cancer brain metastasis sniping model BM-Sniper includes an image feature extraction module and an image classification module; An image feature extraction module, comprising an image encoder, a position encoder, a deep learning encoder and a pooling layer, for extracting an aggregate feature vector from the first image sequence, wherein the aggregate feature vector represents a dynamic change between images in the first image sequence; The image classification module is used to analyze and process the aggregated feature vectors to perform linear classification and obtain classification prediction results.

[0027] Furthermore, the filtering method includes a median filter model: ,in is the image pixel in the primary lung cancer CT image sequence, The matrix of order, ,for L=2, is the image pixel in the filtered image sequence.

[0028] Furthermore, the image acquisition module is also used to extract the identification patient identifier and pathological markers of primary lung cancer from the pathology and magnetic resonance imaging (MRI) radiology text reports. If brain cancer cells are found in a lung cancer patient during a 2-year MRI follow-up examination, the lung cancer patient is considered to be positive for brain metastasis; otherwise, the patient is considered to be negative for brain metastasis.

[0029] Furthermore, the preprocessing includes: S21, analyzing the noise distribution of the CT image sequence, and selecting the optimal filtering method for filtering according to the noise distribution; S22, analyzing the resolution of the CT image sequence, if the resolutions are inconsistent, taking the resolution of the largest number of images with the same resolution in the CT image sequence as a benchmark, resampling other images so that the resolution of the CT image sequence is the same; S23, performing normalization processing on images in the CT image sequence with the same resolution.

[0030] Further, the extracting the aggregated feature vector includes: S31, inputting the first image sequence into an image encoder to obtain a first image feature matrix of the first image sequence; S32, inputting the first image sequence into a position encoder to obtain a first image sequence position matrix; S33, inputting the first image feature matrix and the first image sequence position matrix into a deep learning encoder to obtain a second image feature matrix; S34, input the second image feature matrix into the pooling layer to obtain an aggregated feature vector.

[0031] In the embodiments of the present disclosure, by treating N lung scan images of the same object as a lung scan image group, as an image sequence, and using image feature extraction, rather than treating N lung scan images of the same object as a video and using video feature extraction, the number of data sets can be increased and the classification accuracy can be improved.

[0032] At the same time, in the image feature extraction module, the deep learning encoder can be an encoder of a transformer model and an improved model of the transformer.

[0033] For example, the first image sequence includes N lung scan images, where the feature of the i-th lung scan image is , where f is the conversion function of the image encoder to the input image. The characteristics of the object under test can be expressed as an n-row k-column matrix X, , where k is the feature dimension. Use the deep learning encoder to project the matrix X onto the same output matrix Z. The i-th row of the matrix Z is i is obtained through the self-attention function, taking into account other image features x i The self-attention function maps a query and a set of key-value pairs to an output space, where the query (Q), key (K), and value (V) come from the input matrix X.

[0034] Q (Query): Indicates the element or word currently being processed, retrieving information from other elements.

[0035] K (Key): represents the key of all elements, which are used to compare with the query to determine which values ​​are most relevant to the query.

[0036] V(Value): represents the value associated with the key. When the key is matched with the query, the associated value will be taken into account.

[0037] Specifically, a linear layer is applied: Q=F Q (X), K=F K (X) and V = F V (X).

[0038] F Q (X): A linear transformation that maps the input X to the query space.

[0039] F K (X): A linear transformation that maps the input X to the key space.

[0040] F V (X): A linear transformation that maps the input X to the value space.

[0041] Define the weight matrix; define three weight matrices W for Q, K, and V Q , WK , W V . These matrices will be used to transform the input X into its corresponding representation.

[0042] By linear transformation: Q=XW Q +b Q ; K=XW K +b K ; V=XW V +b V ; Where, b is the bias correction value; The output of the self-attention function is: , is the scaling factor; In order to enable the model to utilize the relative position of the lung scan images in the lung scan image set, the position P = [p1, p2, …, p n ] T Therefore, the input matrix X' of the model is the sum of the image matrix and the position matrix, that is, X'=X+P. The position matrix P is encoded as a randomly initialized trainable parameter.

[0043] Furthermore, the prediction and classification module further includes: a BM-Sniper training module for training the BM-Sniper model: the training includes: S41. Set the pre-training self-supervised learning strategy and use the masked image modeling strategy to pre-train the visual Transformer. S42, setting a loss function that needs to be minimized by the model in the pre-training stage, wherein the loss function is a cross entropy loss; S43, set learning rate, batch size, weight decay, Adam optimizer, and initialize model weights; S44, the model receives the 50% blocked images in the training set and starts predicting the blocked parts; updates the model weights through back propagation and gradient descent until the model meets the preset conditions; and completes the pre-training process; Among them, the self-supervised learning algorithm can use Masked Image Modeling (MIM) to learn the feature representation of lung scan images. MIM can be used for self-supervised visual pre-training and can well expand the capacity of the model. For each subject, 50% of the images can be masked and the unmasked images can be used to reconstruct the masked images. A 12-layer visual converter is used as a feature representation encoder. The model was trained for 100 epochs using the Adam optimizer + weight decay (AdamW) with a weight decay of 0.01 with an initial learning rate of 3e-4 and a batch size of 64. Cosine learning rate decay is used to adjust the learning rate at the end of each epoch.

[0044] Furthermore, the first initial feature extraction module is obtained by pre-training the second initial feature extraction module using a self-supervised learning algorithm. Since the self-supervised learning algorithm does not rely on any label value, it can find the relationship between samples by mining the intrinsic features of the data, learn the features of the lung scan image group, and classify the samples according to feature similarity. Therefore, it can overcome the technical problems of difficulty and high cost in labeling medical training data.

[0045] S45, select to freeze all layers of the pre-trained model to keep the feature representation learned during pre-training unchanged; set a softmax layer on the top of the model and connect the image classification module; S46. Use labeled data to train the model and fine-tune the model weights to minimize the loss function of the classification task to complete model training.

[0046] Example 2 like Figure 2 As shown, a method for predicting brain metastasis of lung cancer cells is applied to a system for predicting brain metastasis of lung cancer cells. The method comprises the following steps: S1. Collect CT image sequences of primary lung cancer, concurrent pathological examination results and brain MRI reports; S2, preprocessing the primary lung cancer CT image sequence, and recording the preprocessed CT image sequence as a first image sequence; S3. Obtain prediction and classification results based on the lung cancer brain metastasis sniper model BM-Sniper; BM-Sniper includes: an image feature extraction module, including an image encoder, a position encoder, a deep learning encoder and a pooling layer, for extracting an aggregate feature vector for a first image sequence, wherein the aggregate feature vector represents a dynamic change between images in the first image sequence; The image classification module is used to analyze and process the aggregated feature vectors to perform linear classification and obtain classification prediction results.

[0047] Furthermore, the aggregated feature vector is analyzed and processed to perform linear classification which is completed by the linear layer. After the predicted probability is obtained, the probability and type are input into the SVM model to obtain the final result.

[0048] Furthermore, the model of the lung cancer brain metastasis prediction method using lung scan images is BM-Sniper. The lung scan image group of the subject to be tested is input into the model, and the image encoder and position encoder in the feature extraction module generate a first image feature matrix, and the deep learning encoder projects the first image feature matrix into other vector spaces to obtain a second image feature matrix, and the second image feature matrix is ​​input into the pooling layer, average pooling is performed, and an aggregated feature vector is output, and then the aggregated feature vector is input into the classification module of the prediction model, and the classification result is output, which is the classification result of the lung cancer brain metastasis of the subject to be tested.

[0049] According to the embodiments of the present disclosure, the performance of the prediction model can be evaluated by calculating multiple indicators of the prediction model. For example, the performance of the model can be judged by the ROC curve and the area under the ROC curve, accuracy, sensitivity, specificity, positive predictive rate, negative predictive rate, and F1 score.

[0050] Figure 3 The receiver operating characteristic curve of the model of the method for predicting brain metastasis of lung cancer using lung scan images according to an embodiment of the present disclosure is schematically shown. Figure 3 As shown, part A is the ROC curve of the BM-Sniper model on the first self-made data set, and part B is the ROC curve of the model of this method on the second self-made data set.

[0051] According to an embodiment of the present disclosure, the clinical pathological data is desensitized to establish a data set, including a group of lung scan images of 8217 subjects, of which 3389 groups of lung scan images of the subjects indicate that the subjects have brain metastasis, and 4553 groups of lung scan images of the subjects indicate that the subjects do not have brain metastasis.

[0052] According to an embodiment of the present disclosure, the test set includes a group of lung scan images of 915 subjects selected from a self-made first data set, and a group of lung scan images of 295 subjects selected from a self-made second data set. Among them, the group of lung scan images of 334 subjects in the self-made first data set indicates that the subjects have brain metastasis, and the group of lung scan images of 581 subjects indicates that the subjects do not have brain metastasis; the group of lung scan images of 66 subjects in the self-made second data set indicates that the subjects have brain metastasis, and the group of lung scan images of 229 subjects indicates that the subjects do not have brain metastasis.

[0053] According to an embodiment of the present disclosure, the ROC curve is a line connecting points drawn under specific stimulation conditions, with the false alarm probability P (y / N) obtained by the test subject under different judgment criteria as the horizontal coordinate and the hit probability P (y / SN) as the vertical coordinate.

[0054] In the ROC curve (Receiver Operating Characteristic Curve), the area under the ROC curve (AUC) is the area under the ROC curve and the coordinate axis. The ROC curve is a tool used to evaluate the performance of a binary classification model. It shows the classification ability of the model by plotting the True Positive Rate (TPR) and the False Positive Rate (FPR) at different thresholds. The AUC value ranges from 0 to 1, and the larger the value, the better the classification performance of the model. Specifically, the AUC value measures the probability that the model will rank the positive sample before the negative sample when a positive sample and a negative sample are randomly selected. Therefore, the higher the AUC value, the stronger the model's ability to distinguish, and the more accurately it can distinguish between positive and negative samples. Since the ROC curve is usually above y=x, the AUC value ranges from 0.5 to 1. The closer the AUC is to 1.0, the higher the authenticity of the detection method. When it is equal to 0.5, the authenticity is the lowest and there is no application value.

[0055] Figure 3 In the figure, the horizontal axis is the false positive rate (FPR) and the vertical axis is the sensitivity (Sensitivity). The AUC of part A is 0.870 and the AUC of part B is 0.845. Therefore, it can be determined that the classification results of the prediction model of the lung cancer brain metastasis prediction method using lung scan images are more authentic.

[0056] According to the embodiments of the present disclosure, accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and F1 score are used to judge the performance of the BM-Sniper model, and the confidence interval is 95%, as shown in Table 1.

[0057]

[0058] According to the embodiments of the present disclosure, medical image annotation is difficult and costly, and it is difficult to obtain large-scale labeled data. Since self-supervised learning does not rely on data annotation, training a self-supervised learning model based on ultra-large-scale lung scan image data can give full play to the advantages of ultra-large-scale data and large models. By constructing a self-supervised pre-training model in the early stage, the dependence of downstream classification tasks on labeled data can be greatly reduced.

[0059] According to the embodiments of the present disclosure, conventional lung scan image interpretation requires the interpreter to consider the dynamic changes of continuous images. This is a labor-intensive and time-consuming process that requires the interpreter to have rich experience. The BM-Sniper model implements judgment through the model by considering continuous lung scan images from a given subject.

[0060] The beneficial effects of the present invention are as follows: Medical image annotation is difficult and costly, and it is difficult to obtain large-scale labeled data. Self-supervised learning does not rely on data annotation. Training a self-supervised learning model based on ultra-large-scale lung CT data can give full play to the advantages of ultra-large-scale data and large models. By constructing a self-supervised pre-training model in the early stage, the dependence of downstream classification tasks on labeled data can be greatly reduced.

[0061] By processing the primary lung cancer CT image sequence, the feature extraction of the lung scan image is performed in accordance with the image feature extraction method to obtain an aggregated feature vector, which is then input into the classification module of the prediction model to output the classification result. This enables the prediction of whether the subject has brain metastasis from lung cancer, and provides doctors with highly accurate intermediate results for auxiliary diagnosis.

[0062] By considering multiple consecutive chest CT image sequences from a given patient, the transformer algorithm is used to predict events. The prediction for each patient is done by aggregating the patient's image sequences and imitating the radiologist's interpretation procedure. This solution avoids manual intervention in the preprocessing and preparation of input data, enabling the classification model BM-sniper to learn fewer unbiased image features, improve the accuracy of prediction, and facilitate doctor-assisted diagnosis.

[0063] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the embodiments shown here.

Claims

1. A system for predicting brain metastasis of lung cancer cells, characterized by: The system includes: Image acquisition module, used to collect primary lung cancer CT image sequences, concurrent pathological examination results and brain MRI reports; An image preprocessing module, used for preprocessing the primary lung cancer CT image sequence, and recording the preprocessed CT image sequence as a first image sequence; A prediction and classification module, used to obtain prediction and classification results according to the lung cancer brain metastasis sniping model BM-Sniper; the lung cancer brain metastasis sniping model BM-Sniper includes an image feature extraction module and an image classification module; An image feature extraction module, comprising an image encoder, a position encoder, a deep learning encoder and a pooling layer, for extracting an aggregate feature vector from the first image sequence, wherein the aggregate feature vector represents a dynamic change between images in the first image sequence; The image classification module is used to analyze and process the aggregated feature vectors to perform linear classification and obtain classification prediction results.

2. The system for predicting brain metastasis of lung cancer cells according to claim 1, characterized in that: The filtering methods include median filter model: ,in is the image pixel in the primary lung cancer CT image sequence, The matrix of order, ,for L=2, is the image pixel in the filtered image sequence.

3. The system for predicting brain metastasis of lung cancer cells according to claim 1, characterized in that: The image acquisition module is also used to extract the identification patient identifier and pathological markers of primary lung cancer from the pathology and magnetic resonance imaging (MRI) radiology text reports. If a lung cancer patient is found to have brain cancer cells in a 2-year MRI follow-up examination, the lung cancer patient is considered to be positive for brain metastasis. Otherwise, the patient is considered to be negative for brain metastasis.

4. The system for predicting brain metastasis of lung cancer cells according to claim 1, characterized in that: The pre-processing comprises: S21, analyzing the noise distribution of the CT image sequence, and selecting the optimal filtering method for filtering according to the noise distribution; S22, analyzing the resolution of the CT image sequence, if the resolutions are inconsistent, taking the resolution of the largest number of images with the same resolution in the CT image sequence as a benchmark, resampling other images so that the resolution of the CT image sequence is the same; S23, performing normalization processing on images in the CT image sequence with the same resolution.

5. The system for predicting brain metastasis of lung cancer cells according to claim 1, characterized in that: The extracting of the aggregated feature vector comprises: S31, inputting the first image sequence into an image encoder to obtain a first image feature matrix of the first image sequence; S32, inputting the first image sequence into a position encoder to obtain a first image sequence position matrix; S33, inputting the first image feature matrix and the first image sequence position matrix into a deep learning encoder to obtain a second image feature matrix; S34, input the second image feature matrix into the pooling layer to obtain an aggregated feature vector.

6. The system for predicting brain metastasis of lung cancer cells according to claim 1, characterized in that: The prediction and classification module also includes: BM-Sniper training module, which is used to train the BM-Sniper model: The training includes: S41. Set the pre-training self-supervised learning strategy and use the masked image modeling strategy to pre-train the visual Transformer. S42, setting a loss function that needs to be minimized by the model in the pre-training stage, wherein the loss function is a cross entropy loss; S43, set learning rate, batch size, weight decay, Adam optimizer, and initialize model weights; S44, the model receives the 50% blocked images in the training set and starts predicting the blocked parts; updates the model weights through back propagation and gradient descent until the model meets the preset conditions; and completes the pre-training process; S45. Select to freeze all layers of the pre-trained model to keep the feature representation learned during pre-training unchanged; set the softmax layer at the top of the model and connect the image classification module. S46. Use labeled data to train the model and fine-tune the model weights to minimize the loss function of the classification task.

7. A method for predicting brain metastasis of lung cancer cells, applied to a system for predicting brain metastasis of lung cancer cells, characterized in that the method comprises the following steps: S1. Collect CT image sequences of primary lung cancer, concurrent pathological examination results and brain MRI reports; S2, preprocessing the primary lung cancer CT image sequence, and recording the preprocessed CT image sequence as a first image sequence; S3. Obtain prediction and classification results based on the lung cancer brain metastasis sniper model BM-Sniper; BM-Sniper includes: an image feature extraction module, including an image encoder, a position encoder, a deep learning encoder and a pooling layer, for extracting an aggregate feature vector for a first image sequence, wherein the aggregate feature vector represents a dynamic change between images in the first image sequence; The image classification module is used to analyze and process the aggregated feature vectors to perform linear classification and obtain classification prediction results.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method described in claim 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method described in claim 7 are implemented.

10. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, Wherein, when the one or more programs are executed by the one or more processors, The one or more processors execute the method according to claim 7.

Images