Dynamic prediction method and system for lung cancer treatment efficacy based on CT image changes

Abstract

The present invention discloses a method and system for dynamically predicting the efficacy of lung cancer based on CT image changes, which relates to the field of medical imaging. The method comprises the following steps: obtaining clinical information, baseline CT images, and follow-up CT images of a patient; performing feature extraction on the baseline CT images and the follow-up CT images to obtain baseline features and follow-up features; obtaining an enhanced CT dynamic feature matrix and a plain scan CT dynamic feature matrix based on the baseline features and the follow-up features; inputting the enhanced CT dynamic feature matrix, the plain scan CT dynamic feature matrix, and clinical information into a efficacy prediction model, outputting logical risk scores at different time points, and obtaining the patient's progression or death risk probability at different time points; effectively characterizing changes in image features at baseline and follow-up through the dynamic feature matrix, identifying correlations within the dynamic feature matrix and between different matrices through a Transformer attention network for predicting efficacy, and learning the relationship between accumulated data at each follow-up time point and efficacy through a time-sharing training strategy.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and in particular to a method and system for dynamically predicting the therapeutic effect of lung cancer based on CT image changes. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] Lung cancer has the highest morbidity and mortality rates worldwide, with non-small cell lung cancer (NSCLC) being the predominant type. Approximately 75% of NSCLC cases are diagnosed at an advanced stage. Immunotherapy targeting programmed death-1 (PD-1) and programmed death-ligand 1 (PD-L1) can effectively prolong survival in patients with advanced NSCLC. However, the overall efficacy of immunotherapy remains low, with a five-year survival rate of 16.6%-29.3%. Dynamically predicting immunotherapy efficacy before and during follow-up is crucial for formulating and adjusting treatment plans for advanced NSCLC.

[0004] Currently, the Immune-related Response Evaluation Criteria in Solid Tumors (iRECIST) standard assesses treatment efficacy based solely on changes in tumor size, making it difficult to distinguish true tumor progression from pseudoprogression, potentially leading to treatment delays. Recent studies have employed artificial intelligence technology to predict immunotherapy efficacy based on baseline CT features. Previous studies by the inventors used CT radiomics features to quantify tumor spatial heterogeneity and analyzed changes in these features during radiotherapy for advanced NSCLC. The results showed that early feature changes can effectively predict tumor status and patient survival at the end of treatment, and that the difference in cumulative feature changes between high- and low-risk groups increases over time. In the context of immunotherapy, recent studies have similarly found that changes in CT radiomics features outperform baseline features in predicting efficacy. Analyzing dynamic changes in CT images before treatment and during follow-up is expected to reveal potential causes for individualized therapeutic differences in patients with advanced NSCLC, providing a basis for developing personalized treatment plans.

[0005] The changes in advanced NSCLC after immunotherapy are highly complex and vary from patient to patient. Some patients progress or even die after only one follow-up, while some patients respond well to treatment and may be followed up more than five times. At the same time, there are also large differences in the follow-up intervals between patients. Existing research still has shortcomings in analyzing such complex dynamic data. On the one hand, dynamic follow-up data are not fully utilized, and the model is only constructed based on a certain time point during the follow-up, resulting in the loss of key information. On the other hand, only the prediction results at a certain time point can be output, and the prediction results cannot be dynamically updated during the follow-up process. How to effectively integrate tumor changes in follow-up data to construct an efficacy prediction model and update the prediction results based on new follow-up data is crucial for predicting the efficacy of immunotherapy. Summary of the Invention

[0006] To overcome the shortcomings of the above-mentioned existing technologies, the present invention provides a method and system for dynamic prediction of lung cancer efficacy based on CT image changes. The dynamic feature matrices of plain CT and enhanced CT are obtained based on the patient's baseline CT images and follow-up CT images. The self-attention and cross-attention of Transformer are used in the prediction model to identify the correlation between the dynamic feature matrix and different matrices, respectively, and comprehensively output the efficacy prediction results.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0008] In a first aspect, the present invention provides a method for dynamically predicting the efficacy of lung cancer treatment based on CT image changes, comprising:

[0009] Obtain patients' clinical information, baseline CT images, and follow-up CT images;

[0010] Performing feature extraction on the baseline CT image and the follow-up CT image to obtain baseline features and follow-up features; obtaining an enhanced CT dynamic feature matrix and a plain scan CT dynamic feature matrix based on the baseline features and the follow-up features;

[0011] Inputting the enhanced CT dynamic feature matrix, the plain CT dynamic feature matrix and clinical information into a pre-trained efficacy prediction model, outputting the logistic risk scores at different time points, and then obtaining the patient's progression or death risk probability at different time points;

[0012] The efficacy prediction model includes a first self-attention module and a second self-attention module, a first cross-attention module, a first linear layer, a second cross-attention module, a feedforward neural network and a second linear layer that are connected in sequence.

[0013] According to a further technical solution, the enhanced CT dynamic feature matrix and the plain scan CT dynamic feature matrix both include baseline features, last follow-up features, long-term change features, short-term change features, cumulative long-term change features, cumulative short-term change features, and change trend features.

[0014] A further technical solution is to input the plain scan CT dynamic feature matrix and the enhanced CT dynamic feature matrix into the first self-attention module and the second self-attention module respectively for processing to obtain the correlation information in the dynamic feature matrix.

[0015] In a further technical solution, the first self-attention module and the second self-attention module calculate the correlation information in the dynamic feature matrix through self-attention, and the calculation method is:

[0016]

[0017] in, represents self-attention, represents the plain scan CT dynamic feature matrix or enhanced CT dynamic feature matrix, represents the query matrix, represents the bond matrix, represents the value matrix, represents the matrix transpose operation, 、 and represents the hyperparameters updated during training, right Perform scale transformation.

[0018] In a further technical solution, the outputs of the first self-attention module and the second self-attention module are respectively input into the first cross-attention module for information combination to obtain the plain scan-enhancement correlation information.

[0019] A further technical solution is to input the plain scan-enhancement correlation information into the second cross-attention module after being processed by the first linear layer, and input the clinical information into the second cross-attention module after being processed by the third linear layer. The second cross-attention module combines the plain scan-enhancement correlation information with the clinical information to obtain a joint information cross-feature matrix, and obtain the logical risk scores at different time points based on the joint information cross-feature matrix.

[0020] In a further technical solution, the first cross attention module and the second cross attention module perform cross attention calculation on the input features, and the calculation method is:

[0021]

[0022] in, represents cross attention; in the first cross attention module, represents the enhanced CT dynamic feature matrix, represents the plain scan CT dynamic feature matrix; in the second cross-attention module, represents the plain scan-enhancement correlation information, Indicates clinical information; 、 represents the hyperparameters that are updated during training.

[0023] In a second aspect, the present invention provides a dynamic prediction system for lung cancer treatment efficacy based on CT image changes, comprising:

[0024] A data acquisition module is configured to: acquire clinical information, baseline CT images, and follow-up CT images of the patient;

[0025] The dynamic feature matrix generation module is configured to: extract features from the baseline CT image and the follow-up CT image to obtain baseline features and follow-up features; and obtain an enhanced CT dynamic feature matrix and a plain scan CT dynamic feature matrix based on the baseline features and the follow-up features;

[0026] an efficacy prediction module configured to: input the enhanced CT dynamic feature matrix, the plain CT dynamic feature matrix, and clinical information into a pre-trained efficacy prediction model, output a logistic risk score at different time points, and thereby obtain the patient's risk probability of progression or death at different time points;

[0027] The efficacy prediction model includes a first self-attention module and a second self-attention module, a first cross-attention module, a first linear layer, a second cross-attention module, a feedforward neural network and a second linear layer that are connected in sequence.

[0028] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for dynamic prediction of lung cancer therapeutic effect based on CT image changes as described in the first aspect.

[0029] In a fourth aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for dynamic prediction of lung cancer therapeutic efficacy based on CT image changes as described in the first aspect are implemented.

[0030] One or more of the above technical solutions have the following beneficial effects:

[0031] The present invention effectively characterizes changes in image features at baseline and during follow-up using a dynamic feature matrix. The Transformer attention network identifies correlations within the dynamic feature matrix and between different matrices for therapeutic efficacy prediction. Furthermore, a time-sharing training strategy is used to learn the relationship between the cumulative data at each follow-up time point and the therapeutic efficacy. This method can update patient data without changing the data dimension and enables the model to predict therapeutic efficacy at any follow-up time point, thereby achieving dynamic prediction of therapeutic efficacy. This method is of great significance for evaluating tumor status and adjusting treatment strategies during follow-up, and is expected to comprehensively improve patient prognosis.

[0032] The present invention proposes a dynamic feature matrix for quantifying feature changes during baseline and follow-up, including baseline characteristics, last follow-up characteristics, long-term change characteristics, short-term change characteristics, cumulative long-term change characteristics, cumulative short-term change characteristics, and change trend characteristics.

[0033] In order to effectively combine the dynamic feature matrices of plain CT and enhanced CT, the present invention uses Transformer's self-attention and cross-attention to identify the correlation between the dynamic feature matrix and different matrices respectively, comprehensively outputs the efficacy prediction results, and combines the time-sharing training strategy to enable the model to have prediction capabilities at different follow-up time points.

[0034] The dynamic feature matrix in the present invention can not only effectively unify the data dimensions of patients with different follow-up conditions, but also be dynamically updated according to the follow-up process without changing the matrix dimensions. Combining the dynamic feature matrix with the time-sharing training strategy to train the Transformer efficacy prediction model can integrate various feature change information at baseline and during follow-up to achieve dynamic prediction of efficacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0036] Figure 1 This is a flow chart of generating a dynamic feature matrix in a method for dynamically predicting the efficacy of lung cancer according to an embodiment of the present invention;

[0037] Figure 2 Schematic diagram of a method for calculating dynamic features in a dynamic feature matrix according to an embodiment of the present invention;

[0038] Figure 3 It is a structural diagram of the efficacy prediction model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0040] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0041] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0042] Example 1

[0043] This embodiment discloses a method for dynamically predicting the efficacy of lung cancer treatment based on CT image changes, which includes the following steps:

[0044] S1: Obtain the patient's clinical information, baseline CT images, and follow-up CT images;

[0045] In this example, 1,200 patients with advanced NSCLC receiving immunotherapy were enrolled from three hospitals. Clinical information (clinical diagnosis and treatment information), baseline CT images, follow-up CT images, progression-free survival (PFS), and overall survival (OS) were retrospectively collected. Efficacy measures included progression-free survival (PFS) and overall survival (OS).

[0046] Inclusion criteria were: ① age > 18 years; ② patients with advanced NSCLC, clinical stage IIIB to VI (according to the 8th edition of the TNM staging system for lung cancer); ③ patients who had received prior immune checkpoint inhibitor therapy, including monotherapy, combined immunotherapy with chemotherapy, or combined targeted therapy; ④ patients with complete baseline clinical data before immunotherapy; ⑤ chest CT scan within 1 month before immunotherapy (baseline CT images) with both plain and contrast-enhanced CT sequences; ⑥ chest CT scan approximately every 2 months after immunotherapy (follow-up CT images) with both plain and contrast-enhanced CT sequences. Exclusion criteria were: ① patients with a history of more than one primary malignancy; ② patients with a history of lung surgery; ③ patients whose treatment response could not be assessed, including patients with no follow-up data or non-measurable primary lesions (lesion long diameter <10 mm); and ④ patients with poor image quality on any of the CT scans.

[0047] PFS is the time from the start of immunotherapy to tumor progression or death. Tumor progression is determined using iRECIST. For patients who had neither tumor progression nor death at the end of follow-up, PFS was calculated based on the follow-up cutoff. OS is the time from the start of immunotherapy to death. For patients who were alive at the end of follow-up, OS was calculated based on the follow-up cutoff.

[0048] S2: extracting features from the baseline CT image and the follow-up CT image to obtain baseline features and follow-up features; obtaining an enhanced CT dynamic feature matrix and a plain scan CT dynamic feature matrix based on the baseline features and the follow-up features;

[0049] In this embodiment, the PFS and OS of advanced NSCLC can be predicted by dynamic CT changes, and the predicted values ​​of disease progression and death risk can be updated in a timely manner according to tumor changes during follow-up.

[0050] There are large differences in the number and interval of follow-up visits among patients with advanced NSCLC. In order to effectively utilize the data of patients with different follow-up situations to construct an efficacy prediction model, the present invention proposes a dynamic feature matrix to quantify the image features and their changes at baseline and multiple follow-ups.

[0051] like Figure 1As shown, feature extraction is performed on the plain scan CT sequence of the baseline CT image and the plain scan CT sequence of the follow-up CT image, respectively, to obtain a first baseline feature and a first follow-up feature, which constitute a first multi-time point CT feature set, and a plain scan CT dynamic feature matrix is ​​constructed based on the first multi-time point CT feature set. Feature extraction is performed on the enhanced CT sequence of the baseline CT image and the enhanced CT sequence of the follow-up CT image, respectively, to obtain a second baseline feature and a second follow-up feature, which constitute a second multi-time point CT feature set, and an enhanced CT dynamic feature matrix is ​​constructed based on the second multi-time point CT feature set. In this embodiment, the baseline feature and the follow-up feature are obtained by extracting features from the tumor area marked by the doctor using the PyRadiomics tool. The feature extraction method can also be flexibly selected according to the actual situation and is not specifically limited.

[0052] The following is a detailed description of the dynamic feature matrix:

[0053] like Figure 2 As shown, the dynamic feature matrix is ​​recorded as (can be used to enhance CT dynamic feature matrix Or plain scan CT dynamic characteristic matrix ), is a OK A matrix of columns, is the number of features, is the number of feature types; in this embodiment, the dynamic feature matrix has 9 feature types, . Assume that the baseline feature set is , the follow-up feature set is to ,in is the total number of follow-up visits, then the feature types of each row of the dynamic feature matrix are:

[0054] ① Row 1, initial information, i.e. baseline characteristics: ;

[0055] ② The second row, the latest information, i.e. the characteristics of the last follow-up: ;

[0056] ③ The third row, long-term change characteristics, that is, the difference between the characteristics at the last follow-up and the baseline: ;

[0057] ④ The fourth row, short-term change characteristics, that is, the difference in characteristics between the last follow-up and the previous follow-up: (When only one follow-up is performed, for );

[0058] ⑤ The fifth row, cumulative long-term change characteristics, that is, the cumulative characteristic difference relative to the baseline during follow-up, is calculated as :

[0059] (1)

[0060] ⑥ The 6th row, cumulative short-term change characteristics, that is, the cumulative difference in characteristics during follow-up relative to the previous follow-up , calculated as:

[0061] (2)

[0062] ⑦ Rows 7-9, trend characteristics, that is, the characteristic values ​​of each feature at all time points ( to ) and time points (days from baseline) were linearly fitted, and the goodness of fit (R²), slope and intercept They are used as the dynamic eigenvalues ​​of lines 7, 8, and 9 respectively.

[0063] Each column in the dynamic feature matrix represents a CT radiomics feature, which is extracted from the tumor region on the CT image using the PyRadiomics tool.

[0064] Each patient's baseline and follow-up data can be quantified into two dynamic feature matrices, plain scan and enhanced scan, with features in rows 3-6 being the same across only one follow-up visit. The dynamic feature matrix integrates various baseline and follow-up change information, unifying the data dimensions for patients with different follow-up conditions. It can be updated based on each follow-up result without changing the matrix dimensions, facilitating subsequent modeling and analysis.

[0065] S3: Input the enhanced CT dynamic feature matrix, plain scan CT dynamic feature matrix and clinical information into a pre-trained efficacy prediction model, output the logical risk scores at different time points, and then obtain the patient's progression or death risk probability at different time points.

[0066] The Transformer's attention mechanism is used to identify the correlation between different feature types within the dynamic feature matrix and between different matrices, which is used to build a therapeutic effect prediction model.

[0067] like Figure 3 As shown, the efficacy prediction model includes a first self-attention module and a second self-attention module, a first cross-attention module, a first linear layer, a second cross-attention module, a feedforward neural network, and a second linear layer, which are connected in parallel in sequence. The second cross-attention module is also connected to a third linear layer.

[0068] The plain scan CT dynamic feature matrix is ​​input to the first self-attention module for processing, and the enhanced CT dynamic feature matrix is ​​input to the second self-attention module for processing. The first and second self-attention modules process the dynamic feature matrix in the same way. First, self-attention calculation is performed on the input dynamic feature matrix. The self-attention calculation calculates the correlation between different rows (feature types) within the dynamic feature matrix. The calculation method is:

[0069] (3)

[0070] in, represents self-attention, represents the plain scan CT dynamic feature matrix or enhanced CT dynamic feature matrix, represents the query matrix, represents the bond matrix, represents the value matrix, represents the matrix transpose operation, 、 and represents the hyperparameters updated during training; Used for Perform a scale transformation to prevent the gradient of the Softmax function from disappearing when its value is too large.

[0071] The first cross-attention module calculates the correlation between different dynamic feature matrices (plain CT dynamic feature matrix and enhanced CT dynamic feature matrix) to obtain plain-enhanced correlation information. This plain-enhanced correlation information is processed by the first linear layer and input into the second cross-attention module. The acquired clinical information is processed by the third linear layer and input into the second cross-attention module. The second cross-attention module calculates the correlation between the linear-processed plain-enhanced correlation information and the clinical information to obtain a joint information cross-feature matrix.

[0072] Cross attention is used to calculate the correlation between different dynamic feature matrices (plain scan CT dynamic feature matrix and enhanced CT dynamic feature matrix). The calculation method is:

[0073] (4)

[0074] in, represents cross attention; in the first cross attention module, represents the enhanced CT dynamic feature matrix, represents the plain scan CT dynamic feature matrix; in the second cross-attention module, represents the plain scan-enhancement correlation information, Indicates clinical information; 、 It represents the hyperparameter updated during the training process, and the proportion of the two attentions can be adjusted according to the training results.

[0075] The present invention trains a prediction model for PFS and OS respectively. The plain scan CT dynamic feature matrix and the enhanced CT dynamic feature matrix each use the self-attention module to output the correlation information in their dynamic feature matrices, and then combine the information through the first cross attention module to output the plain scan-enhanced correlation information. ; Then, the second cross attention module is combined with the clinical information to obtain the joint information cross feature matrix , After the feedforward neural network and the second linear layer, the logistic risk scores at different time points are output. The survival function of the Cox proportional hazards model is used to calculate the patient's progression / death risk probability at different time points based on the logistic risk score. The difference between the output risk probability and the actual progression / survival status is calculated using the negative log-likelihood loss function of the discrete-time Cox model. The model is trained and optimized based on this difference, with the goal of reducing this difference. The calculation method is:

[0076] (5)

[0077] in, represents the negative log-likelihood loss function, represents the patient index, represents the time interval index, Individuals representing model outputs In the Probability of progression / death risk within one year, Indicates in Number of individuals who actually progressed / died during the year, Indicates in The number of individuals actually in the risk set at the beginning of the year.

[0078] The present invention employs a time-sharing training strategy to train the model, enabling it to predict therapeutic efficacy based on tumor changes at different follow-up time points, thus acquiring dynamic prediction capabilities. Specifically, a dynamic feature matrix is ​​calculated for each patient's cumulative data at different follow-up time points, each serving as a training sample. The model is trained using the patient's therapeutic efficacy as the prediction target, allowing the model to learn the relationship between tumor changes and therapeutic efficacy at different follow-up time points, resulting in a pre-trained efficacy prediction model. This method, based on expanded training samples, enables the model to predict therapeutic efficacy based on data at any follow-up time point, thereby dynamically updating efficacy prediction results during the follow-up process.

[0079] After model training is complete, the attention maps in the Transformer model are further visualized to explain the key features that the model focuses on when processing the dynamic feature matrix, thereby revealing the potential association between dynamic changes in tumors and treatment response. Specifically, the visualization tool "Visualizer" is used to obtain the attention map of each attention head in each Transformer encoder layer for each sample, and the softmax function is applied to each attention map for normalization. Subsequently, the results of all attention heads in each Transformer encoder layer are averaged to obtain an attention map for each layer; then, the attention maps of each group of samples (such as the OS>5 years group and the OS≤5 years group) are averaged on each layer's attention map to obtain the attention distribution map of each group and layer. Furthermore, by averaging the multi-layer attention maps and symmetrizing the attention directions, an attention network for each group is constructed to reflect the overall interactive relationship between different features.

[0080] To further quantify the model's attention to each feature, we introduce the average attention strength metric, which represents the average of the attention strength between each feature and all other features. This metric is defined as the average of the attention strengths between a feature and all other features. Specifically, we first average the multi-layer attention maps for each group to obtain a single average attention map, which is normalized to the interval [0, 1]. Next, we sum and average the elements in each row and column of this matrix to represent the total attention strength between the current feature and all other features. Finally, we divide this value by the total number of features to obtain the average attention strength for that feature. This metric is used to measure the model's attention to different features in efficacy prediction.

[0081] Finally, visualizing the attention map and average attention intensity using Matplotlib helps us intuitively understand the key dynamic features that the model relies on in the efficacy prediction process, and provides an explanatory basis for the study of the mechanism between tumor change patterns and treatment responses.

[0082] Of the 1,200 advanced NSCLC samples, samples from the hospital with the largest sample size were divided into training, test, and internal validation sets in a 7:1:2 ratio. Samples from the remaining hospitals served as external validation sets. All prediction models were optimized and validated using this same dataset division. The training set data was used to optimize model parameters, the test set data was used to select optimal model hyperparameters, and the internal and external validation sets data were used to verify model performance.

[0083] The changes in advanced NSCLC following immunotherapy are highly complex and individual, and the number and intervals of follow-up visits vary significantly between patients. Traditional artificial intelligence models struggle to effectively analyze these data. Furthermore, previous efficacy prediction models typically provide diagnostic results based on data from a single point in time and are unable to generate new predictions based on tumor changes during follow-up. The key challenge addressed by this invention is how to effectively integrate baseline and follow-up data and dynamically predict efficacy during follow-up.

[0084] Artificial intelligence algorithms provide an effective method for analyzing dynamically changing data and the correlation between data. The inventors previously explored the feasibility of using the Transformer attention mechanism to identify the correlation between image features, and on this basis proposed a modal interaction network that combines static and dynamic, fully exploring the correlation within and between modalities, thereby effectively integrating quantitative multimodal data such as imaging and pathology, and improving the predictive performance of NSCLC immunotherapy efficacy. At the same time, a solution based on multimodal completion and knowledge distillation was proposed to solve the problem of constructing multimodal models in modality-missing scenarios. Therefore, the combination of effective image quantification methods and Transformer provides a new approach for analyzing the correlation between CT data at different time points.

[0085] In summary, based on the previous work, the present invention proposes a dynamic feature matrix to quantify CT changes at baseline and during follow-up, which is combined with the Transformer attention mechanism and time-sharing training strategy to achieve dynamic prediction of therapeutic efficacy.

[0086] Example 2

[0087] This embodiment discloses a dynamic prediction system for lung cancer treatment efficacy based on CT image changes, including:

[0088] A data acquisition module is configured to: acquire clinical information, baseline CT images, and follow-up CT images of the patient;

[0089] The dynamic feature matrix generation module is configured to: extract features from the baseline CT image and the follow-up CT image to obtain baseline features and follow-up features; and obtain an enhanced CT dynamic feature matrix and a plain scan CT dynamic feature matrix based on the baseline features and the follow-up features;

[0090] an efficacy prediction module configured to: input the enhanced CT dynamic feature matrix, the plain CT dynamic feature matrix, and clinical information into a pre-trained efficacy prediction model, output a logistic risk score at different time points, and thereby obtain the patient's risk probability of progression or death at different time points;

[0091] The efficacy prediction model includes a first self-attention module and a second self-attention module, a first cross-attention module, a first linear layer, a second cross-attention module, a feedforward neural network and a second linear layer that are connected in sequence.

[0092] Example 3

[0093] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of embodiment 1 when executing the program.

[0094] Example 4

[0095] The purpose of this embodiment is to provide a computer-readable storage medium, a computer-readable storage medium having a computer program stored thereon, which performs the steps of the method of embodiment 1 when executed by a processor.

[0096] The steps involved in the apparatuses of Examples 3 and 4 above correspond to those of Method Example 1. For detailed implementation, please refer to the relevant description of Example 1. The term "computer-readable storage medium" should be understood to mean a single medium or multiple media containing one or more instruction sets; it should also be understood to include any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and causing the processor to perform any of the methods of the present invention.

[0097] Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computer device. Alternatively, they can be implemented using program code executable by a computing device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0098] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

[0099] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A dynamic prediction method for lung cancer treatment efficacy based on CT image changes, characterized in that: include: Obtain patients' clinical information, baseline CT images, and follow-up CT images; Performing feature extraction on the baseline CT image and the follow-up CT image to obtain baseline features and follow-up features; Obtaining an enhanced CT dynamic feature matrix and a plain CT dynamic feature matrix based on the baseline characteristics and the follow-up characteristics; Inputting the enhanced CT dynamic feature matrix, the plain CT dynamic feature matrix and clinical information into a pre-trained efficacy prediction model, outputting the logistic risk scores at different time points, and then obtaining the patient's progression or death risk probability at different time points; The efficacy prediction model includes a first self-attention module and a second self-attention module, a first cross-attention module, a first linear layer, a second cross-attention module, a feedforward neural network, and a second linear layer, which are connected in parallel in sequence; A time-sharing training strategy is used to train the efficacy prediction model, so that the model predicts the efficacy based on tumor changes at different follow-up time points. Specifically, the dynamic feature matrix is ​​calculated for each patient's accumulated data at different follow-up time points, and each is used as a training sample. The model is trained with the patient's efficacy as the prediction target, so that the model learns the relationship between tumor changes and efficacy at different follow-up time points, and obtains a pre-trained efficacy prediction model. After the model training is completed, the attention map in the Transformer model is further visualized to explain the key features that the model focuses on when processing the dynamic feature matrix, revealing the potential correlation between tumor dynamic changes and treatment response.

2. The method for dynamic prediction of lung cancer therapeutic effect based on CT image changes according to claim 1, characterized in that: The enhanced CT dynamic feature matrix and the plain scan CT dynamic feature matrix both include baseline characteristics, last follow-up characteristics, long-term change characteristics, short-term change characteristics, cumulative long-term change characteristics, cumulative short-term change characteristics, and change trend characteristics.

3. The method for dynamic prediction of lung cancer treatment efficacy based on CT image changes according to claim 1, wherein: The plain scan CT dynamic feature matrix and the enhanced CT dynamic feature matrix are respectively input into the first self-attention module and the second self-attention module for processing to obtain the correlation information in the dynamic feature matrix.

4. The method for dynamic prediction of lung cancer treatment efficacy based on CT image changes according to claim 3, wherein: The first and second self-attention modules calculate the correlation information in the dynamic feature matrix through self-attention. The calculation method is: in, represents self-attention, represents the plain scan CT dynamic feature matrix or enhanced CT dynamic feature matrix, represents the query matrix, represents the bond matrix, represents the value matrix, represents the matrix transpose operation, 、 and represents the hyperparameters updated during training, right Perform scale transformation.

5. The method for dynamic prediction of lung cancer treatment efficacy based on CT image changes according to claim 1, wherein: The outputs of the first self-attention module and the second self-attention module are respectively input into the first cross-attention module for information combination to obtain the plain scan-enhancement correlation information.

6. The method for dynamic prediction of lung cancer therapeutic effect based on CT image changes according to claim 5, characterized in that: The plain scan-enhancement association information is processed by the first linear layer and input into the second cross-attention module. The clinical information is processed by the third linear layer and input into the second cross-attention module. The second cross-attention module combines the plain scan-enhancement association information with the clinical information to obtain a joint information cross-feature matrix. Based on the joint information cross-feature matrix, the logical risk scores at different time points are obtained.

7. The method for dynamic prediction of lung cancer therapeutic effect based on CT image changes according to claim 6, characterized in that: The first cross attention module and the second cross attention module perform cross attention calculation on the input features, and the calculation method is: in, represents cross attention; in the first cross attention module, represents the enhanced CT dynamic feature matrix, represents the plain scan CT dynamic feature matrix; in the second cross-attention module, represents the plain scan-enhancement correlation information, Indicates clinical information; 、 represents the hyperparameters that are updated during training.

8. A dynamic prediction system for lung cancer treatment efficacy based on CT image changes, characterized by: include: A data acquisition module is configured to: acquire clinical information, baseline CT images, and follow-up CT images of the patient; A dynamic feature matrix generation module is configured to: extract features from the baseline CT image and the follow-up CT image to obtain baseline features and follow-up features; Obtaining an enhanced CT dynamic feature matrix and a plain CT dynamic feature matrix based on the baseline characteristics and the follow-up characteristics; an efficacy prediction module configured to: input the enhanced CT dynamic feature matrix, the plain CT dynamic feature matrix, and clinical information into a pre-trained efficacy prediction model, output a logistic risk score at different time points, and thereby obtain the patient's risk probability of progression or death at different time points; The efficacy prediction model includes a first self-attention module and a second self-attention module, a first cross-attention module, a first linear layer, a second cross-attention module, a feedforward neural network, and a second linear layer, which are connected in parallel in sequence; A time-sharing training strategy is used to train the efficacy prediction model, so that the model predicts the efficacy based on tumor changes at different follow-up time points. Specifically, the dynamic feature matrix is ​​calculated for each patient's accumulated data at different follow-up time points, and each is used as a training sample. The model is trained with the patient's efficacy as the prediction target, so that the model learns the relationship between tumor changes and efficacy at different follow-up time points, and obtains a pre-trained efficacy prediction model. After the model training is completed, the attention map in the Transformer model is further visualized to explain the key features that the model focuses on when processing the dynamic feature matrix, revealing the potential correlation between tumor dynamic changes and treatment response.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method for dynamically predicting the therapeutic effect of lung cancer based on CT image changes as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method for dynamically predicting the therapeutic effect of lung cancer based on CT image changes are implemented as described in any one of claims 1 to 7.

Images