Dynamic lung cancer curative effect prediction method and system based on CT image change

Abstract

The invention discloses a CT image change-based lung cancer curative effect dynamic prediction method and system, and relates to the technical field of medical imaging. Clinical information, a baseline CT image and a follow-up CT image of a patient are acquired; performing feature extraction on the baseline CT image and the follow-up CT image to obtain baseline features and follow-up features; obtaining an enhanced CT dynamic feature matrix and a flat-scanning CT dynamic feature matrix based on the baseline features and the follow-up visit features; inputting the enhanced CT dynamic feature matrix, the plain-scan CT dynamic feature matrix and clinical information into a curative effect prediction model, outputting logic risk scores at different time points, and obtaining progress or death risk probabilities of the patient at different time points; according to the method, image feature changes in a baseline and follow-up visit are effectively represented through a dynamic feature matrix, correlation between the interior of the dynamic feature matrix and between different matrixes is identified through a Transform attention network to predict the curative effect, and the relationship between accumulated data at each follow-up visit time point and the curative effect is learned through a time-sharing training strategy.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and particularly to a dynamic prediction method and system for the efficacy of lung cancer based on CT image changes. Background Art

[0002] The statements in this part merely provide background technical information related to the present disclosure, and do not necessarily constitute prior art.

[0003] Lung cancer is the cancer with the highest incidence and mortality globally. Among them, non-small cell lung cancer (NSCLC) accounts for the main type, and about 75% of NSCLC is in the advanced stage at the time of the first diagnosis. Immunotherapy targeting programmed death protein-1 (PD-1) and programmed death ligand-1 (PD-L1) can effectively prolong the survival period of patients with advanced NSCLC. However, the overall effective rate of immunotherapy is still low, and the five-year survival rate is 16.6% - 29.3%. Dynamically predicting the efficacy of immunotherapy during the pre-treatment and follow-up after treatment is crucial for the formulation and timely adjustment of treatment plans for advanced NSCLC.

[0004] Currently, the immune-related response evaluation criteria for solid tumors (iRECIST) can only evaluate the treatment effect based on the change in tumor size, and it is difficult to distinguish the true progression and pseudo-progression of tumors in a timely manner, which may cause treatment delays. Recent studies have used artificial intelligence technology to predict the efficacy of immunotherapy based on baseline CT features. The inventor's previous research used CT radiomics features to quantify the spatial heterogeneity of tumors and analyzed the characteristic changes of advanced NSCLC during radiotherapy. The results showed that early characteristic changes could effectively predict the tumor status and patient survival time at the end of treatment, and over time, the difference in the cumulative characteristic change values between the high-risk and low-risk groups increased. In terms of immunotherapy, recent studies have also found that the efficacy prediction performance of the CT radiomics feature change value is better than that of the baseline features. Analyzing the dynamic changes of CT images before treatment and during follow-up is expected to reveal the potential causes of individual efficacy differences in patients with advanced NSCLC and provide a basis for formulating personalized treatment plans.

[0005] The changes in advanced NSCLC after immunotherapy are highly complex and vary from person to person. Some patients progress or even die after only one follow-up, while some patients respond well to treatment and may be followed up more than five times. At the same time, there are large differences in the follow-up intervals between patients. Existing research still has shortcomings in analyzing such complex dynamic data. On the one hand, dynamic follow-up data are not fully utilized, and the model is only constructed based on a certain time point during the follow-up, resulting in the loss of key information. On the other hand, only the prediction results at a certain time point can be output, and the prediction results cannot be dynamically updated during the follow-up process. How to effectively integrate tumor changes in follow-up data to construct an efficacy prediction model and update the prediction results based on new follow-up data is crucial for predicting the efficacy of immunotherapy. Summary of the invention

[0006] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a method and system for dynamic prediction of lung cancer efficacy based on CT image changes. The dynamic feature matrices of plain CT and enhanced CT are obtained based on the patient's baseline CT images and follow-up CT images. The self-attention and cross-attention of Transformer are used in the prediction model to respectively identify the correlation between the dynamic feature matrix and different matrices, and comprehensively output the efficacy prediction results.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: In a first aspect, the present invention provides a method for dynamically predicting the efficacy of lung cancer treatment based on CT image changes, comprising: Obtain the patient's clinical information, baseline CT images, and follow-up CT images; Perform feature extraction on the baseline CT image and the follow-up CT image to obtain baseline features and follow-up features; obtain an enhanced CT dynamic feature matrix and a plain scan CT dynamic feature matrix based on the baseline features and the follow-up features; Inputting the enhanced CT dynamic feature matrix, the plain scan CT dynamic feature matrix and clinical information into a pre-trained efficacy prediction model, outputting the logical risk scores at different time points, and then obtaining the patient's progression or death risk probability at different time points; The therapeutic effect prediction model includes a first self-attention module and a second self-attention module, a first cross-attention module, a first linear layer, a second cross-attention module, a feedforward neural network and a second linear layer which are connected in sequence and in parallel.

[0008] According to a further technical solution, the enhanced CT dynamic feature matrix and the plain scan CT dynamic feature matrix both include baseline characteristics, last follow-up characteristics, long-term change characteristics, short-term change characteristics, cumulative long-term change characteristics, cumulative short-term change characteristics, and change trend characteristics.

[0009] For a further technical solution, the non-contrast CT dynamic feature matrix and the contrast-enhanced CT dynamic feature matrix are respectively input into a first self-attention module and a second self-attention module for processing to obtain the correlation information within the dynamic feature matrix.

[0010] For a further technical solution, the correlation information within the dynamic feature matrix is calculated through self-attention in the first self-attention module and the second self-attention module, and the calculation method is as follows:

[0011] Where, represents self-attention, represents the non-contrast CT dynamic feature matrix or the contrast-enhanced CT dynamic feature matrix, represents the query matrix, represents the key matrix, represents the value matrix, represents the matrix transpose operation, and and represent hyperparameters updated during the training process, Performs a scale transformation on

[0012] For a further technical solution, the outputs of the first self-attention module and the second self-attention module are respectively input into a first cross-attention module for information combination to obtain the non-contrast-enhanced correlation information.

[0013] For a further technical solution, the non-contrast-enhanced correlation information is input into the second cross-attention module after being processed by a first linear layer, and the clinical information is input into the second cross-attention module after being processed by a third linear layer. The second cross-attention module combines the non-contrast-enhanced correlation information with the clinical information to obtain a joint information cross feature matrix, and based on the joint information cross feature matrix, the logical risk scores at different time points are obtained.

[0014] For a further technical solution, the first cross-attention module and the second cross-attention module perform cross-attention calculation on the input features, and the calculation method is as follows:

[0015] Where, represents cross-attention; in the first cross-attention module, represents the contrast-enhanced CT dynamic feature matrix, represents the non-contrast CT dynamic feature matrix; in the second cross-attention module, represents the non-contrast-enhanced correlation information, represents the clinical information; and ​Represents hyperparameters updated during training.

[0016] In a second aspect, the present invention provides a dynamic prediction system for the efficacy of lung cancer based on changes in CT images, including: A data acquisition module configured to: acquire the clinical information, baseline CT images, and follow-up CT images of a patient; A dynamic feature matrix generation module configured to: extract features from the baseline CT images and follow-up CT images to obtain baseline features and follow-up features; obtain an enhanced CT dynamic feature matrix and a plain scan CT dynamic feature matrix based on the baseline features and follow-up features; An efficacy prediction module configured to: input the enhanced CT dynamic feature matrix, the plain scan CT dynamic feature matrix, and the clinical information into a pre-trained efficacy prediction model, output the logical risk scores at different time points, and further obtain the progression or death risk probabilities of the patient at different time points; The efficacy prediction model includes a first self-attention module and a second self-attention module, a first cross-attention module, a first linear layer, a second cross-attention module, a feed-forward neural network, and a second linear layer that are connected in parallel in sequence.

[0017] In a third aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps in the method for dynamically predicting the efficacy of lung cancer based on changes in CT images as described in the first aspect.

[0018] In a fourth aspect, the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, it implements the steps in the method for dynamically predicting the efficacy of lung cancer based on changes in CT images as described in the first aspect.

[0019] The above one or more technical solutions have the following beneficial effects: The present invention effectively characterizes the image feature changes in the baseline and follow-up through the dynamic feature matrix, identifies the associations within the dynamic feature matrix and between different matrices through the Transformer attention network for efficacy prediction, and learns the relationship between the cumulative data at each follow-up time point and the efficacy through a time-sharing training strategy. This method can update patient data without changing the data dimension, and enable the model to predict the efficacy at any follow-up time point, thereby realizing the dynamic prediction of efficacy. This method is of great significance for the evaluation of the tumor state during the follow-up process and the adjustment of treatment strategies, and is expected to comprehensively improve the prognosis of patients.

[0020] The present invention proposes a dynamic feature matrix for quantifying feature changes during the baseline and follow-up processes, including baseline features, last follow-up features, long-term change features, short-term change features, cumulative long-term change features, cumulative short-term change features, and change trend features.

[0021] Moreover, in order to effectively combine the dynamic feature matrices of non-contrast CT and contrast-enhanced CT, the present invention uses the self-attention and cross-attention of Transformer to respectively identify the correlation relationships inside the dynamic feature matrix and between different matrices, comprehensively output the efficacy prediction results, and combine the time-sharing training strategy to enable the model to have prediction ability at different follow-up time points.

[0022] In the present invention, the dynamic feature matrix can not only effectively unify the data dimensions of patients in different follow-up situations, but also be dynamically updated according to the follow-up process without changing the matrix dimensions. Combining the dynamic feature matrix with the time-sharing training strategy to train the Transformer efficacy prediction model can integrate various feature change information during the baseline and follow-up processes and achieve dynamic prediction of efficacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings forming a part of this invention are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation of the present invention.

[0024] Figure 1 is a flowchart for generating a dynamic feature matrix in the method for dynamic prediction of lung cancer efficacy according to an embodiment of the present invention; Figure 2 is a schematic diagram of the method for calculating dynamic features in the dynamic feature matrix according to an embodiment of the present invention; Figure 3 is a structural diagram of the efficacy prediction model according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0026] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0027] In the case of no conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0028] Embodiment 1 This embodiment discloses a dynamic prediction method for the efficacy of lung cancer based on CT image changes. The method includes the following steps: S1: Obtain the clinical information, baseline CT images, and follow-up CT images of the patient; In this embodiment, 1200 patients with advanced NSCLC who received immunotherapy were included from three hospitals. The clinical information (clinical diagnosis and treatment information), baseline CT images, follow-up CT images, progression-free survival time (PFS), and overall survival time (OS) of the 1200 patients were retrospectively collected. The efficacy includes progression-free survival time (PFS) and overall survival time (OS).

[0029] The inclusion criteria were as follows: ① Age > 18 years old; ② Patients with advanced NSCLC, with a clinical stage of stage IIIB to VI (referring to the 8th edition of the lung cancer TNM staging standard); ③ Patients who had received immune checkpoint inhibitor therapy, including those treated with immune monotherapy, immune combination chemotherapy, and immune combination targeted therapy; ④ Having complete baseline clinical data before immunotherapy; ⑤ Undergoing chest CT examination within 1 month before immunotherapy (to obtain baseline CT images), and having two CT sequences of plain scan + contrast enhancement; ⑥ Undergoing chest CT examination approximately every 2 months after immunotherapy (to obtain follow-up CT images), and having two CT sequences of plain scan + contrast enhancement. The exclusion criteria were as follows: ① Patients with a history of more than one primary malignant tumor; ② Patients with a history of lung surgery; ③ Patients who could not be evaluated for treatment response, including those without follow-up data or with an inmeasurable primary lesion (lesion long diameter < 10 mm); ④ Poor quality of any CT image.

[0030] PFS is the time from the start of immunotherapy to tumor progression or death. iRECIST is used to determine whether the tumor has progressed. For patients who have neither experienced tumor progression nor died at the end of the follow-up, PFS is statistically analyzed based on the end of the follow-up time. OS is the time from the start of immunotherapy to death. For patients who are alive at the end of the follow-up, OS is statistically analyzed based on the end of the follow-up time.

[0031] S2: Extract features from the baseline CT images and follow-up CT images to obtain baseline features and follow-up features; obtain an enhanced CT dynamic feature matrix and a plain scan CT dynamic feature matrix based on the baseline features and follow-up features; In this embodiment, the PFS and OS of advanced NSCLC can be predicted through dynamic changes in CT, and during follow-up, the predicted values of disease progression and death risk can be updated in a timely manner according to tumor changes.

[0032] There are significant differences in the number of follow-ups and follow-up intervals among patients with advanced NSCLC. In order to effectively utilize the data of patients with different follow-up situations to construct a curative effect prediction model, the present invention proposes a dynamic feature matrix to quantify the image features and their changes at baseline and during multiple follow-ups.

[0033] As Figure 1 shown, feature extraction is performed on the plain CT sequence of the baseline CT image and the plain CT sequence of the follow-up CT image respectively to obtain the first baseline feature and the first follow-up feature. The first baseline feature and the first follow-up feature constitute the first multi-time point CT feature set, and a plain CT dynamic feature matrix is constructed based on this first multi-time point CT feature set. Feature extraction is performed on the enhanced CT sequence of the baseline CT image and the enhanced CT sequence of the follow-up CT image respectively to obtain the second baseline feature and the second follow-up feature. The second baseline feature and the second follow-up feature constitute the second multi-time point CT feature set, and an enhanced CT dynamic feature matrix is constructed based on this second multi-time point CT feature set. In this embodiment, the baseline feature and the follow-up feature are obtained by performing feature extraction on the tumor region marked by the doctor using the PyRadiomics tool. Specifically, the feature extraction method can also be flexibly selected according to the actual situation, and no specific limitation is made.

[0034] The following specifically describes the dynamic feature matrix: As Figure 2 shown, the dynamic feature matrix is denoted as (which can be the enhanced CT dynamic feature matrix or the plain CT dynamic feature matrix ), and is a row column matrix, is the number of features, is the number of feature types; in this embodiment, the dynamic feature matrix has 9 feature types, . Assuming that the baseline feature set is , and the follow-up feature sets are to , where is the total number of follow-ups, then the feature types of each row of the dynamic feature matrix are respectively: ① The first row, initial information, that is, the baseline feature: ; ② The second row, latest information, that is, the last follow-up feature: ; ③ The third row, long-term change feature, that is, the feature difference of the last follow-up relative to the baseline: ; ④Line 4, short-term change feature, i.e., the feature difference at the last follow-up relative to the previous follow-up: (When there is only one follow-up, it is ); ⑤Line 5, cumulative long-term change feature, i.e., the cumulative feature difference relative to the baseline during the follow-up, calculated as : (1) ⑥Line 6, cumulative short-term change feature, i.e., the cumulative feature difference relative to the previous follow-up during the follow-up , calculated as: (2) ⑦Lines 7 - 9, change trend feature, i.e., the feature values of each feature at all time points ( to ) are linearly fitted with the time points (days from the baseline), and the goodness of fit (R²), slope and intercept are used as the dynamic feature values of lines 7, 8, and 9 respectively.

[0035] Each column in the dynamic feature matrix represents a CT radiomics feature, which is extracted from the tumor region on the CT image using the PyRadiomics tool.

[0036] The baseline and follow-up data of each patient can be quantified into two dynamic feature matrices for non-contrast and contrast-enhanced scans. The features in lines 3 - 6 are the same when there is only one follow-up. The dynamic feature matrix can integrate various change information in the baseline and follow-up, unify the data dimensions of patients with different follow-up situations, and be updated according to the results of each follow-up without changing the matrix dimensions, facilitating subsequent modeling and analysis.

[0037] S3: Input the contrast-enhanced CT dynamic feature matrix, non-contrast CT dynamic feature matrix, and clinical information into a pre-trained efficacy prediction model to output the logical risk scores at different time points, and then obtain the progression or death risk probabilities of the patient at different time points.

[0038] The attention mechanism of Transformer is used to identify the correlation relationships between different feature types within the dynamic feature matrix and between different matrices for constructing the efficacy prediction model.

[0039] Such as Figure 3As shown, the efficacy prediction model includes a first self-attention module and a second self-attention module in parallel connected in sequence, a first cross-attention module, a first linear layer, a second cross-attention module, a feed-forward neural network, and a second linear layer. The second cross-attention module is also connected to a third linear layer.

[0040] The plain scan CT dynamic feature matrix is input into the first self-attention module for processing, and the enhanced CT dynamic feature matrix is input into the second self-attention module for processing. The processing processes of the first self-attention module and the second self-attention module for the dynamic feature matrix are the same. First, self-attention calculation is performed on the input dynamic feature matrix, and the self-attention calculation is used to calculate the correlation relationship between different rows (feature types) inside the dynamic feature matrix. The calculation method is: (3) Among them, represents self-attention, represents the plain scan CT dynamic feature matrix or the enhanced CT dynamic feature matrix, represents the query matrix, represents the key matrix, represents the value matrix, represents the matrix transpose operation, 、 and represent hyperparameters updated during the training process; is used to perform scale transformation to prevent the gradient of the Softmax function from vanishing when its value is too large.

[0041] The first cross-attention module calculates the correlation relationship between different dynamic feature matrices (plain scan CT dynamic feature matrix and enhanced CT dynamic feature matrix) to obtain plain scan-enhanced correlation information. The plain scan-enhanced correlation information is input into the second cross-attention module after being processed by the first linear layer. The obtained clinical information is input into the second cross-attention module after being processed by the third linear layer. The second cross-attention module calculates the correlation relationship between the plain scan-enhanced correlation information processed by the linear layer and the clinical information to obtain a joint information cross-feature matrix.

[0042] The cross-attention calculates the correlation relationship between different dynamic feature matrices (plain scan CT dynamic feature matrix and enhanced CT dynamic feature matrix). The calculation method is: (4) Among them, represents cross-attention; in the first cross-attention module, represents the enhanced CT dynamic feature matrix, represents the plain scan CT dynamic feature matrix; in the second cross-attention module, Indicates the plain scan-enhanced association information, Indicates clinical information; , Indicates the hyperparameters updated during training, and the proportion of the two attentions can be adjusted according to the training results.

[0043] In the present invention, a prediction model is trained for PFS and OS respectively. The dynamic feature matrix of plain scan CT and the dynamic feature matrix of enhanced CT respectively use the self-attention module to output the association information within their dynamic feature matrices, and the information is jointly output through the first cross-attention module to output the plain scan-enhanced association information ; Then, it is jointly combined with the clinical information through the second cross-attention module to obtain the joint information cross-feature matrix , After passing through the feed-forward neural network and the second linear layer, the logical risk scores at different time points are output. Using the survival function of the Cox proportional hazards model, the progression / death risk probabilities of patients at different time points are calculated based on the logical risk scores. The difference between the output risk probability and the true progression / survival situation is calculated through the negative log-likelihood loss function of the discrete-time Cox model, and the model is trained and optimized according to this difference. The goal of optimization is to reduce this difference, and the calculation method is: (5) Wherein, Represents the negative log-likelihood loss function, Represents the patient index, Represents the time interval index, Represents the individual output by the model At the th year of the progression / death risk probability, Represents the number of individuals who actually progressed / died at the th year, Represents the number of individuals who were actually at risk at the beginning of the th year. The present invention adopts a time-sharing training strategy to train the model, enabling the model to predict the curative effect according to the tumor changes at different follow-up time points, thereby possessing the ability of dynamic prediction. Specifically, the dynamic feature matrix is calculated for the cumulative data of each patient at different follow-up time points respectively, and each is used as a training sample. Using the curative effect of the patient as the prediction target to train the model, enabling the model to learn the relationship between the tumor changes and the curative effect at different follow-up time points, and obtaining a pre-trained curative effect prediction model. This method can, on the basis of expanding the training samples, enable the model to predict the curative effect according to the data at any follow-up time point, thereby realizing the dynamic update of the curative effect prediction results during the follow-up process.

[0044] After the model training is completed, the attention maps in the Transformer model are further visualized to explain the key features that the model focuses on when processing the dynamic feature matrix, so as to reveal the potential association between tumor dynamic changes and treatment response. Specifically, the visualization tool "Visualizer" is used to obtain the attention maps of each sample for each attention head in each Transformer encoder layer, and the softmax function is applied to each attention map for normalization. Subsequently, the results of all attention heads within each Transformer encoder layer are averaged to obtain one attention map for each layer; then, the attention maps of each group of samples (such as the OS>5-year group and the OS≤5-year group) are averaged for each layer to obtain the attention distribution map for each group and each layer. Further, by averaging the multi-layer attention maps and symmetrizing the attention directions, the attention network for each group is constructed to reflect the overall interaction relationship between different features.

[0045] To further quantify the degree of attention of the model to each feature, the average attention intensity index is introduced, which represents the average attention intensity between each feature and all other features, and is defined as the average of the attention intensities between a certain feature and all other features. Specifically, first, the multi-layer attention maps of each group are averaged to obtain a single average attention map, and it is normalized to the [0, 1] interval; subsequently, the elements of each row and column of this matrix are summed and averaged to represent the total attention intensity between the current feature and all other features; finally, this value is divided by the total number of features to obtain the average attention intensity value of this feature. This index is used to measure the degree of attention of the model to different features in efficacy prediction.

[0046] Finally, the visualization of the attention maps and the average attention intensity by Matplotlib helps to intuitively understand the key dynamic features relied on by the model in the efficacy prediction process, providing an explanatory basis for the mechanism study between tumor change patterns and treatment response.

[0047] Among 1200 samples of advanced NSCLC, the samples from the hospital with the largest sample size are divided into a training set, a test set, and an internal validation set according to the ratio of 7:1:2, and the samples from the remaining hospitals are used as an external validation set. The optimization and validation of all prediction models are carried out under the same data set division method. The data in the training set are used to optimize the model parameters, the data in the test set are used to select the best model hyperparameters, and the data in the internal and external validation sets are used to verify the performance of the model.

[0048] The changes in advanced NSCLC after immunotherapy are highly complex and vary among individuals. There are significant differences in the number and intervals of follow-up visits among different patients, making it difficult for traditional artificial intelligence models to effectively analyze the follow-up data. In addition, previous efficacy prediction models usually give diagnostic results based on data at a certain time point and cannot provide new prediction results according to the changes in tumors during the follow-up process. How to effectively integrate baseline and follow-up data and dynamically predict the efficacy during the follow-up is the key problem solved by the present invention.

[0049] Artificial intelligence algorithms provide effective methods for analyzing dynamically changing data and the correlation relationships between data. The inventors previously explored the feasibility of using the Transformer attention mechanism to identify the correlations between image features, and on this basis, proposed a modality interaction network that combines static and dynamic methods to fully exploit the intra-modal and inter-modal correlations, thereby effectively integrating quantitative multi-modal data such as imaging and pathology to improve the prediction performance of NSCLC immunotherapy efficacy. At the same time, a scheme based on multi-modal completion and knowledge distillation is proposed to solve the problem of constructing multi-modal models in the scenario of missing modalities. Therefore, the combination of effective image quantification methods and Transformer provides a new idea for analyzing the correlation relationships between CT data at different time points.

[0050] In summary, on the basis of the previous work, the present invention proposes to quantify the CT changes during the baseline and follow-up processes with a dynamic feature matrix, which is combined with the Transformer attention mechanism and a time-sharing training strategy to achieve dynamic prediction of the efficacy.

[0051] Example Two This example discloses a dynamic lung cancer efficacy prediction system based on CT image changes, including: A data acquisition module configured to: acquire the clinical information, baseline CT images, and follow-up CT images of a patient; A dynamic feature matrix generation module configured to: extract features from the baseline CT images and follow-up CT images to obtain baseline features and follow-up features; and obtain an enhanced CT dynamic feature matrix and a plain scan CT dynamic feature matrix based on the baseline features and follow-up features; An efficacy prediction module configured to: input the enhanced CT dynamic feature matrix, the plain scan CT dynamic feature matrix, and the clinical information into a pre-trained efficacy prediction model, output the logical risk scores at different time points, and further obtain the progression or death risk probabilities of the patient at different time points; The efficacy prediction model includes a first self-attention module and a second self-attention module, a first cross-attention module, a first linear layer, a second cross-attention module, a feed-forward neural network, and a second linear layer that are connected in sequence and in parallel.

[0052] Example Three The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the method in Embodiment 1 are implemented.

[0053] Embodiment 4 The purpose of this embodiment is to provide a computer-readable storage medium. A computer-readable storage medium stores a computer program, and when the program is executed by a processor, the steps of the method in Embodiment 1 are executed.

[0054] The steps involved in the devices in the above Embodiments 3 and 4 correspond to those in Method Embodiment 1. For specific implementation manners, reference may be made to the relevant description part of Embodiment 1. The term "computer-readable storage medium" should be understood to include a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0055] Those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computer device. Optionally, they can be implemented by program codes executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0056] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0057] Although the specific implementation manners of the present invention are described above in conjunction with the accompanying drawings, it is not a limitation to the protection scope of the present invention. Those skilled in the art should understand that based on the technical solutions of the present invention, various modifications or deformations that can be made without creative efforts by those skilled in the art are still within the protection scope of the present invention.

Claims

1. A dynamic prediction method for the efficacy of lung cancer based on changes in CT images, characterized in that Comprising: Obtaining the clinical information, baseline CT images, and follow-up CT images of a patient; Performing feature extraction on the baseline CT images and follow-up CT images to obtain baseline features and follow-up features; Obtaining an enhanced CT dynamic feature matrix and a non-enhanced CT dynamic feature matrix based on the baseline features and follow-up features; Inputting the enhanced CT dynamic feature matrix, non-enhanced CT dynamic feature matrix, and clinical information into a pre-trained efficacy prediction model, outputting the logical risk scores at different time points, and further obtaining the progression or death risk probabilities of the patient at different time points; The efficacy prediction model includes a first self-attention module and a second self-attention module in parallel connected in sequence, a first cross-attention module, a first linear layer, a second cross-attention module, a feed-forward neural network, and a second linear layer.

2. The dynamic prediction method for lung cancer treatment efficacy based on CT image changes according to claim 1, wherein Both the enhanced CT dynamic feature matrix and the non-enhanced CT dynamic feature matrix include baseline features, last follow-up features, long-term change features, short-term change features, cumulative long-term change features, cumulative short-term change features, and change trend features.

3. The dynamic prediction method for the efficacy of lung cancer based on CT image changes according to claim 1, wherein, Inputting the non-enhanced CT dynamic feature matrix and the enhanced CT dynamic feature matrix into the first self-attention module and the second self-attention module respectively for processing to obtain the correlation information within the dynamic feature matrix.

4. The dynamic prediction method for lung cancer treatment efficacy based on CT image changes according to claim 3, wherein In the first self-attention module and the second self-attention module, the correlation information within the dynamic feature matrix is calculated through self-attention, and the calculation method is: Among them, represents self-attention, represents the plain CT dynamic feature matrix or the enhanced CT dynamic feature matrix, represents the query matrix, represents the key matrix, represents the value matrix, represents the matrix transpose operation, 、 and represent hyperparameters updated during training, Performs a scale transformation on .

5. The dynamic prediction method for lung cancer treatment efficacy based on CT image changes according to claim 1, wherein The outputs of the first self-attention module and the second self-attention module are respectively input into the first cross-attention module for information combination to obtain non-enhanced-enhanced correlation information.

6. The dynamic prediction method for lung cancer treatment efficacy based on CT image changes as claimed in claim 5, wherein Inputting the non-enhanced-enhanced correlation information into the second cross-attention module after being processed by the first linear layer, inputting the clinical information into the second cross-attention module after being processed by the third linear layer, and the second cross-attention module combines the non-enhanced-enhanced correlation information with the clinical information to obtain a combined information cross-feature matrix, and obtaining the logical risk scores at different time points based on the combined information cross-feature matrix.

7. The dynamic prediction method for the efficacy of lung cancer based on CT image changes according to claim 6, characterized in that, The first cross-attention module and the second cross-attention module perform cross-attention calculation on the input features, and the calculation method is: Among them, represents cross-attention; in the first cross-attention module, represents the enhanced CT dynamic feature matrix, represents the plain scan CT dynamic feature matrix; in the second cross-attention module, represents the plain scan-enhanced correlation information, represents clinical information; and represent the hyperparameters updated during the training process.

8. A dynamic prediction system for the efficacy of lung cancer based on changes in CT images, characterized in that, Comprising: A data acquisition module configured to obtain the clinical information, baseline CT images, and follow-up CT images of a patient; A dynamic feature matrix generation module configured to perform feature extraction on the baseline CT images and follow-up CT images to obtain baseline features and follow-up features; Obtaining an enhanced CT dynamic feature matrix and a non-enhanced CT dynamic feature matrix based on the baseline features and follow-up features; An efficacy prediction module configured to input the enhanced CT dynamic feature matrix, non-enhanced CT dynamic feature matrix, and clinical information into a pre-trained efficacy prediction model, output the logical risk scores at different time points, and further obtain the progression or death risk probabilities of the patient at different time points; The efficacy prediction model includes a first self-attention module and a second self-attention module in parallel connected in sequence, a first cross-attention module, a first linear layer, a second cross-attention module, a feed-forward neural network, and a second linear layer.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps in the dynamic prediction method for lung cancer treatment efficacy based on CT image changes as described in any one of claims 1-7.

10. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the dynamic prediction method for lung cancer treatment efficacy based on CT image changes as described in any one of claims 1-7.

Images