Solid tumor curative effect prediction method and system based on bimodal CT image

Abstract

The invention provides a solid tumor curative effect prediction method and system based on a bimodal CT image, and relates to the technical field of medical image analysis, and the method comprises the steps: obtaining a plain-scan CT image and an enhanced CT image of a solid tumor patient, and determining a tumor overlapping region through deformation registration; calculating an inter-modal correlation feature set based on the region, wherein the inter-modal correlation feature set comprises depth correlation features and difference features; the depth correlation features are used for quantifying correlation of channels of a bimodal CT depth feature map, and the difference features are used for quantifying relative variation of bimodal CT image omics features; and inputting the associated feature set into a prognosis prediction model to obtain a curative effect prediction result. According to the method, internal correlation information between the bimodal CT images is systematically excavated and utilized for the first time, the limitation that only simple feature fusion is carried out in an existing method is overcome, the structure-function relation of the tumor can be described more accurately, and therefore the accuracy and reliability of prediction of the curative effect of the solid tumor are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of medical image analysis technology, and in particular to a method and system for predicting the efficacy of treatment for solid tumors based on dual-modal CT images. Background Technology

[0002] In the clinical diagnosis and treatment of solid tumors, accurate prediction of treatment efficacy before treatment is crucial for optimizing treatment plans and improving patient prognosis. Many solid tumors, such as lung cancer, are often diagnosed at an advanced stage, making systemic therapy (such as chemotherapy, targeted therapy, and immunotherapy) the primary approach. However, patients exhibit significant heterogeneity in their response to the same treatment regimen, with only a subset achieving long-term clinical benefit. Therefore, accurately predicting treatment efficacy and identifying potential beneficiaries before treatment is of paramount importance for developing individualized strategies, optimizing the allocation of medical resources, and improving patient outcomes.

[0003] In existing technologies, using computed tomography (CT) images for non-invasive efficacy prediction is an important research direction. CT, as a routine clinical examination, can comprehensively and non-invasively acquire morphological and functional information about the entire tumor. Specifically, plain CT (nCE-CT) primarily reflects the inherent structure and density of the tumor, while contrast-enhanced CT (CE-CT), through contrast agent perfusion, can additionally reveal the hemodynamic characteristics and microenvironment heterogeneity of the tumor. Theoretically, combined analysis of plain and contrast-enhanced CT can yield more comprehensive tumor biological information.

[0004] However, most current CT image-based predictive studies typically employ only image features from a single modality, either plain or enhanced CT scans. Even those studies attempting to fuse bimodal information are mostly limited to simple feature stitching or feature averaging strategies. These methods essentially treat the two modalities as independent information sources, merely performing a "physical superposition" of information without delving into the deeper biological connections between them. Specifically, they ignore the potential inherent synergistic or antagonistic relationships between the tumor's basic structure as reflected in plain CT scans and the blood perfusion function revealed by enhanced CT scans. This lack of exploration into "intermodal correlation information" makes it difficult for models to capture the deeper "structure-function" unity within the tumor, thus limiting further improvements in predictive model performance. Summary of the Invention

[0005] To address the technical problem mentioned above, where existing bimodal CT image analysis methods fail to effectively quantify the intrinsic correlation between modalities, thus limiting prediction accuracy, this invention proposes a method and system for predicting the efficacy of treatment for solid tumors based on bimodal CT images. By calculating depth-related and differential features, this invention systematically mines and utilizes the deep correlation and differential information between plain CT and enhanced CT images for the first time, thereby achieving more accurate prediction of the efficacy of treatment for solid tumors.

[0006] To achieve the above objectives, a first aspect of the present invention provides a method for predicting the efficacy of treatment for solid tumors based on dual-modal CT imaging, comprising: Acquire plain CT images and enhanced CT images of patients with solid tumors; Deformation registration was performed on the plain CT images and enhanced CT images, and the overlapping area of ​​the tumor in the two images was determined based on the registration results; Based on the tumor overlap region, an intermodal correlation feature set is calculated, which includes depth correlation features and differential features. The depth correlation features are used to quantify the correlation between the depth feature maps extracted by the deep learning model of the plain CT image and the enhanced CT image in each channel. The differential features are used to quantify the relative change between the radiomics features of the plain CT image and the enhanced CT image in the same tumor region. The intermodal correlation feature set is input into a pre-trained prognostic prediction model to obtain the efficacy prediction results for patients with solid tumors.

[0007] Furthermore, deformation registration is performed on the plain CT images and enhanced CT images, and the overlapping area of ​​the tumor in the two images is determined based on the registration results, including: Using the plain CT image as a fixed reference image and the enhanced CT image as a floating image, spatial alignment is performed using an elastic registration algorithm. The tumor outline drawn on the enhanced CT image is mapped to the space of the plain CT image based on the spatial transformation field generated by registration. Calculate the Dice similarity coefficient between the mapped tumor contour and the original tumor contour drawn on the plain CT image; Tumor regions with a Dice similarity coefficient greater than or equal to a preset threshold are selected as the tumor overlapping regions.

[0008] Furthermore, the depth-related features are obtained through the following steps: Using a pre-trained convolutional neural network, plain CT feature maps and enhanced CT feature maps are extracted from the overlapping region of the tumor, respectively, with each feature map having C channels; Regarding the first One channel ( =1,2,…,C), and the first of the plain CT feature maps The channel feature vector is denoted as The first of the enhanced CT feature maps i The channel feature vector is denoted as ; Calculate the first one according to the following formula Spearman correlation coefficient of each channel :

[0009] in, M The length of the feature vector. It is a rank transformation function; The deep correlation feature is composed of the Spearman correlation coefficients of C channels: .

[0010] Furthermore, the difference features include a first difference feature and a second difference feature; The first difference feature is obtained through the following steps: K radiomics features were extracted from the tumor overlap region of the plain CT image and the enhanced CT image, respectively. The first K feature of the enhanced CT image was denoted as K. j The feature values ​​are The first plain CT image j The feature values ​​are ,in j =1,2,…,K; For each radiomics feature, the relative difference between its feature value in enhanced CT images and its feature value in plain CT images is calculated, and the j-th differential feature value is calculated according to the following formula. :

[0011] in, It is a positive real number close to 0; From K This constitutes the first distinguishing feature; The second differential feature is obtained through the following steps: subtraction processing is performed on the registered plain CT image and enhanced CT image to obtain a subtracted image, and radiomics features are extracted from the tumor overlap region of the subtracted image to form the second differential feature.

[0012] Furthermore, the prognostic prediction model is trained through the following steps: Obtain a training set containing plain CT images, enhanced CT images, and overall survival data of multiple patients with solid tumors; For each patient in the training set, calculate their intermodal correlation feature set; Using the intermodal correlation feature set as input features and total survival as the prediction target, a machine learning algorithm combined with the Cox proportional hazards loss function is used for training to obtain the prognostic prediction model.

[0013] Furthermore, the method also includes: The clinical characteristics of the patients with solid tumors are obtained, including any one or more of the following static indicators: age, sex, body mass index, albumin, white blood cell count, neutrophil percentage, absolute neutrophil count, lymphocyte percentage, hemoglobin, neuron-specific enolase, cancer antigen 125, and carcinoembryonic antigen. The clinical features are fused with the intermodal correlation feature set to form a comprehensive feature set; The comprehensive feature set is input into the pre-trained prognostic prediction model to obtain the efficacy prediction result.

[0014] A second aspect of the present invention provides a solid tumor treatment efficacy prediction system based on dual-modal CT imaging, comprising: The image acquisition module is used to acquire plain CT images and enhanced CT images of patients with solid tumors; The image registration module is used to perform deformation registration on the plain CT images and enhanced CT images, and to determine the tumor overlap area in the two images based on the registration results. The correlation feature calculation module is used to calculate the intermodal correlation feature set based on the tumor overlapping region. The intermodal correlation feature set includes depth correlation features and differential features. The depth correlation features are used to quantify the correlation of the depth feature maps extracted by the deep learning model from the plain CT image and the enhanced CT image in each channel. The differential features are used to quantify the relative change between the radiomics features of the plain CT image and the enhanced CT image in the same tumor region. The prognostic prediction module is used to input the intermodal correlation feature set into a pre-trained prognostic prediction model to obtain the efficacy prediction results for the patients with solid tumors.

[0015] A third aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps in the method for predicting the efficacy of solid tumors based on dual-modal CT images as described in the first aspect of the present invention.

[0016] A fourth aspect of the present invention provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the steps in the method for predicting the efficacy of solid tumor treatment based on dual-modal CT images as described in the first aspect of the present invention.

[0017] A fifth aspect of the present invention provides a computer program product comprising software code, wherein the program in the software code performs the steps of the method for predicting the efficacy of solid tumor treatment based on dual-modal CT images as described in the first aspect of the present invention.

[0018] Compared with existing technologies, the present invention provides a method and system for predicting the efficacy of treatment for solid tumors based on dual-modal CT imaging, which has the following beneficial effects: (1) This invention quantifies the spatial consistency of deep texture patterns in the same tumor region in plain and enhanced images by calculating depth-related features, thereby capturing the structural homogeneity and heterogeneity of the tumor. By calculating the first and second differential features, the relative changes in radiomics features before and after contrast agent perfusion are standardized and measured, thereby reflecting the hemodynamic characteristics of the tumor. These two features, from two novel and complementary dimensions of correlation and difference, achieve a deep characterization of the relationship between tumor structure and function, providing key information for prediction models that cannot be captured by traditional single-modality or simple fusion methods.

[0019] (2) This invention compensates for spatial deformation caused by patient breathing and displacement through an elastic registration algorithm, and uses the Dice similarity coefficient to perform quantitative quality screening of the registered tumor region. This process ensures that the pixels or voxels on which the subsequent calculation of depth-related features and differential features depend are strictly corresponding in anatomical position, effectively eliminating noise caused by technical misalignment and ensuring the biological authenticity of the subsequently extracted associated features.

[0020] (3) This invention integrates dual-modal CT imaging features and clinical features to construct a more comprehensive and robust individualized prognostic prediction model. The clinical features are formed by fusing imaging correlation features that reflect local tumor characteristics with baseline clinical indicators (such as serum albumin, blood routine, etc.) that reflect the patient's overall condition and bodily functions. This fusion mechanism enables the prediction model to not only rely on local information from imaging, but also integrate the patient's overall physiological and immune status, achieving complementary advantages between local and overall information, thereby improving the model's generalization ability and predictive robustness in complex real clinical scenarios. Attached Figure Description

[0021] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0022] Figure 1 This is a simplified flowchart of the method for predicting the efficacy of solid tumor treatment based on dual-modal CT imaging provided in Embodiment 1 of the present invention. Figure 2 This is a schematic diagram of the array standard provided in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the overall process of the solid tumor efficacy prediction method based on dual-modal CT images provided in Embodiment 1 of the present invention. Figure 4 This is a schematic diagram illustrating the verification and explanation of the model provided in Embodiment 1 of the present invention; Figure 4 In the figure, 'a' represents a bar chart: comparing the consistency index of the total survival predictions of the eight prediction models on the training set, internal validation set, and external validation set. Figure 4 In the diagram, 'b' represents the bee colony plot: it shows the contribution of each input feature to the prediction result of the integrated model. The scatter points represent the SHAP value (the degree of influence on the prediction result), and the colors represent the feature values ​​(red for high values ​​and blue for low values). Figure 5 This is a schematic diagram illustrating the hierarchical accuracy and predictive performance verification of the integrated model provided in Embodiment 1 of the present invention. Figure 5 In this context, 'a' represents the Kaplan-Meier survival curve of the training set. Figure 5 In this context, b represents the Kaplan-Meier survival curve of the internal validation set; Figure 5 In the figure, c represents the Kaplan-Meier survival analysis curve of the external validation set. The Kaplan-Meier survival analysis shows the difference in total survival between the high-risk group (red) and the low-risk group (green) in the training set, internal validation set, and external validation set after risk scoring stratification by the integrated model. The log-rank test p-value verifies the statistical significance of the difference in total survival between the two groups. Figure 5 In this context, d represents the time-dependent ROC curve of the training set. Figure 5 In this context, 'e' represents the time-dependent ROC curve of the internal validation set. Figure 5 f in the figure represents the time-dependent ROC curve of the external validation set. The time-dependent receiver operating characteristic (ROC) curve shows the prediction accuracy of the model for 1-year, 2-year, and 3-year overall survival (OS) on the training set, internal validation set, and external validation set. The corresponding area under the curve (AUC) value is marked in the figure to quantify the discriminative power of the model at each time point. Figure 6 This is an architecture diagram of the solid tumor efficacy prediction system based on dual-modal CT imaging provided in Embodiment 2 of the present invention. Detailed Implementation

[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0026] All data acquisition in this embodiment is carried out in accordance with laws and regulations and with user consent, and the data is used legally.

[0027] Terminology Explanation: 1. Dice Similarity Coefficient: This coefficient measures the similarity between two sets, ranging from 0 to 1. A higher value indicates a greater degree of overlap between the two regions. In this invention, it is used to evaluate the overlap quality of tumor regions after registration of plain CT and enhanced CT scans.

[0028] 2. Spearman's Rank Correlation Coefficient: A nonparametric statistic used to measure the monotonic correlation between two variables. It is obtained by calculating the Pearson correlation coefficient between the ranks after sorting the original data.

[0029] Example 1 like Figure 1 This embodiment provides a method for predicting the efficacy of treatment for solid tumors based on dual-modal CT imaging, including: Acquire plain CT images and enhanced CT images of patients with solid tumors; Deformation registration was performed on the plain CT images and enhanced CT images, and the overlapping area of ​​the tumor in the two images was determined based on the registration results; Based on the tumor overlap region, an intermodal correlation feature set is calculated, which includes depth correlation features and differential features. The depth correlation features are used to quantify the correlation between the depth feature maps extracted by the deep learning model of the plain CT image and the enhanced CT image in each channel. The differential features are used to quantify the relative change between the radiomics features of the plain CT image and the enhanced CT image in the same tumor region. The intermodal correlation feature set is input into a pre-trained prognostic prediction model to obtain the efficacy prediction results for patients with solid tumors.

[0030] This embodiment is based on a multicenter retrospective study. The study initially included baseline CT imaging data from 1113 patients with advanced non-small cell lung cancer who received immune checkpoint inhibitor therapy from three independent research centers. After image registration quality screening (Dice similarity coefficient ≥0.7), 892 patients were ultimately included to complete the study. The median age of the participants was 65 years (interquartile range 58–70), and 86.1% (768 patients) were male. In the final study cohort, 745 patients from Center 1 were randomly assigned to the training set (559 patients) and the internal validation set (186 patients), while two independent cohorts from Center 2 (97 patients) and Center 3 (50 patients) served as external validation set 1 and external validation set 2, respectively. The primary endpoint was overall survival, defined as the time from the start of immunotherapy to death from any cause.

[0031] Specifically, deformation registration is performed on the plain CT images and enhanced CT images, and the overlapping area of ​​the tumor in the two images is determined based on the registration results, including: Using the plain CT image as a fixed reference image (denoted as...) The enhanced CT image is used as a floating image (denoted as...). Image deformation registration was performed using the elastic registration algorithm from the Advanced Normalization Toolkit (ANTs). ANTs elastic registration completes the registration through three core steps: objective function construction, deformation field parameterization, and objective function optimization. First, an objective function is constructed that includes a similarity term and a regularization term to balance the grayscale matching degree between the floating image and the reference image and the physical rationality of the deformation field. The expression for the objective function is:

[0032] in, Indicates from arrive The deformation vector field of space, including The spatial displacement vector of each voxel; It is a similarity term measured using mutual information, used to evaluate the similarity after registration. and The grayscale similarity is calculated as follows:

[0033] in, and These represent univariate information entropy and bivariate joint information entropy, respectively. The higher the value, the better the grayscale matching between the two images; This is a regularization term used to constrain the smoothness of the deformation field, avoiding physically unreasonable local over-deformation. Its calculation method is as follows:

[0034] in, It is the image spatial domain. It is the Frobenius norm. yes gradient, yes divergence, and It is the elastic coefficient; These are the weighting coefficients that balance the similarity and regularization terms, and are adjusted during the registration process.

[0035] Secondly, B-spline basis functions are used for the deformation vector field. Parameterization is performed to ensure the smoothness and continuity of the deformation. The core of the B-spline basis function is to construct a smooth B-spline deformation vector field. This allows for fine-grained alignment of the floating image with the reference image, thus effectively handling non-rigid deformations. The B-spline model divides the floating image into a regular grid in space, with each grid node serving as a control point and possessing an adjustable displacement vector. The combination of all grids on the floating image forms the registered deformation vector field. Floating image based on B-spline deformation It can be represented as:

[0036] in, , and To enhance the B-spline basis functions in three directions on the CT scan, for The number of control points (i.e., the number of grids). , and These are the deformation parameters for the control points.

[0037] Finally, the optimal deformation field is solved through iterative optimization and applied to the floating image to achieve spatial alignment with the fixed image, ensuring accurate matching of the corresponding anatomical structures in the plain scan and enhanced CT images.

[0038] Before registration, all plain and enhanced CT images were resampled to 1×1×1 mm using B-spline interpolation. 3 isotropic voxels; The tumor outline drawn on the enhanced CT image is mapped to the space of the plain CT image based on the spatial transformation field generated by registration. Calculate the Dice similarity coefficient between the mapped tumor contour and the original tumor contour drawn on the plain CT image; Tumor regions with a Dice similarity coefficient greater than or equal to 0.7 are selected as the tumor overlapping regions for subsequent feature extraction.

[0039] Specifically, the depth-related features are obtained through the following steps: Using a pre-trained convolutional neural network, plain CT feature maps and enhanced CT feature maps are extracted from the overlapping tumor region, each feature map having C channels. Further, using a ResNet50 residual network pre-trained on ImageNet, plain CT feature maps and enhanced CT feature maps are extracted from the overlapping tumor region, each feature map having 64 channels. The feature map dimension is N×64×112×112, where N is the number of image layers in the overlapping tumor region. The feature maps from each phase are reconstructed and flattened into a 64×M matrix, where M=N×112×112. Regarding the first One channel ( =1,2,…,C), and the first of the plain CT feature maps The channel feature vector is denoted as The first of the enhanced CT feature maps The channel feature vector is denoted as ; Calculate the first one according to the following formula Spearman correlation coefficient of each channel :

[0040] in, M The length of the feature vector. It is a rank transformation function; The deep correlation feature is composed of the Spearman correlation coefficients of C channels: .

[0041] Specifically, this invention quantifies the difference information between two modalities by calculating two types of difference features: a first difference feature and a second difference feature. The difference features are obtained through the following steps: Using the open-source Python package PyRadiomics, radiomics features were extracted from plain CT and enhanced CT images within the tumor overlap region. The extracted features included morphological features, first-order statistical features, and texture features. The texture features were calculated based on the following five texture matrices: gray-level co-occurrence matrix, gray-level run-length matrix, gray-level size region matrix, gray-level dependency matrix, and neighborhood gray-level difference matrix. The feature extraction parameters were set as follows: gray-level discretization with a fixed bin width of 25 and gray-level normalization scaling to 2000. K radiomics features were extracted from the tumor overlap region of the plain CT image and the enhanced CT image, respectively. The first K feature of the enhanced CT image was denoted as K. j The feature values ​​are The first plain CT image j The feature values ​​are ,in j =1,2,…,K; For each radiomics feature, the relative difference between its feature value in enhanced CT images and its feature value in plain CT images is calculated, and the result is calculated using the following formula: j One differential eigenvalue :

[0042] in, It is a positive real number close to 0; From K This constitutes the first distinguishing feature; Subtraction processing is performed on the registered plain CT images and enhanced CT images to obtain subtracted images. Radiomic features are extracted from the tumor overlap region of the subtracted images (extraction parameters are set as follows: bin width 20, grayscale normalization scaling to 500) to form the second differential features.

[0043] Specifically, the prognostic prediction model is trained through the following steps: Obtain a training set containing plain CT images, enhanced CT images, and overall survival data of multiple patients with solid tumors; For each patient in the training set, calculate their intermodal correlation feature set; The screening of intermodal correlation features is carried out in four steps: retaining differential features that are robust to contour segmentation variation, and defining features with an inter-observer correlation coefficient > 0.75 as robust features; using univariate Cox regression analysis to initially screen features that are significantly associated with survival outcomes; applying Lasso-Cox regression analysis to solve the multicollinearity problem and achieve feature dimensionality reduction through regularization; and using the features screened by Lasso-Cox regression as a basis, using stepwise Cox regression analysis with the Akaike Information Criterion as the screening standard to further simplify the model and improve its simplicity.

[0044] Using the selected intermodal correlation feature set as input features and total survival as the prediction target, the prognostic prediction model is trained using an extreme gradient boosting algorithm combined with the Cox proportional hazards loss function. During training, hyperparameter tuning is performed using 5-fold cross-validation combined with grid search.

[0045] Specifically, the method further includes: The clinical characteristics of the patients with solid tumors are obtained, including any one or more of the following static indicators: age, sex, body mass index, albumin, white blood cell count, neutrophil percentage, absolute neutrophil count, lymphocyte percentage, hemoglobin, neuron-specific enolase, cancer antigen 125, and carcinoembryonic antigen. For missing values ​​in clinical features, linear imputation based on random forest was performed on both the training and validation sets; features with a missing value rate greater than 30% were excluded. Subsequently, a two-stage variable screening strategy was used to screen for independent prognostic factors: first, univariate Cox regression analysis was performed, and significant factors with P < 0.05 were included in multivariate Cox regression analysis. Variables with independent prognostic significance (P < 0.05) in the multivariate model were retained for subsequent model construction. The results of multivariate Cox regression analysis showed that albumin was an independent prognostic factor associated with overall survival (hazard ratio HR = 0.95, 95% confidence interval CI: 0.93–0.98, P < 0.001).

[0046] The clinical features are fused with the intermodal correlation feature set to form a comprehensive feature set; The comprehensive feature set is input into the pre-trained prognostic prediction model to obtain the efficacy prediction result.

[0047] In this embodiment, the model constructed based on deep correlation features demonstrated superior performance in overall survival prediction. In the training set, internal validation set, and two external validation sets, the consistency indices of this model were 0.71, 0.68, 0.62, and 0.66, respectively, significantly outperforming the single-modal model based on plain CT and enhanced CT, as well as the model based on differential features. Further integration of deep correlation features with the first differential feature further improved the predictive efficacy of the final integrated model, achieving consistency indices of 0.76, 0.71, 0.66, and 0.67 in the aforementioned datasets. This integrated model effectively stratified patients into high-risk and low-risk groups, with a significant difference in survival prognosis between the two groups (log-rank test p < 0.001). Furthermore, in the subgroup of patients with low expression of programmed death-ligand 1, the model still achieved stable risk stratification (hazard ratio HR = 3.26, 95% confidence interval: 1.81–5.85, p < 0.001).

[0048] In one specific embodiment, clinicopathological data of the study subjects were extracted from the electronic medical record system, prognostic information was obtained from the follow-up registration database, and baseline CT imaging data were compiled from the PACS system. This invention retrieved lung cancer diagnosed patients from January 2019 to July 2023 from Center 1's database and included those meeting the criteria... Figure 2 The research subjects included according to the inclusion criteria were randomly divided into a training set and an internal validation set at a ratio of 75% and 25%, respectively, for model construction and validation. Using the same inclusion and exclusion criteria, research subjects from centers 2 and 3 were selected as external validation set 1 and external validation set 2, respectively. For details of the overall design process of this invention, please refer to [link to full description]. Figure 3 The primary endpoint of the study was overall survival (OS), defined as the time from the start of immunotherapy to death from any cause; for participants still alive at the end of follow-up, their data were truncated at the last follow-up date.

[0049] (1) Collection of clinical characteristics Baseline clinical characteristics prior to the initiation of immunotherapy are divided into four categories: ① Demographic characteristics, including age, sex, body mass index, and smoking history; ② Pathological characteristics, including histological subtype and programmed death-ligand 1 (PD-L1) tumor proportion score; ③ Treatment-related characteristics, including clinical stage, number of treatment lines, treatment regimen, timing of radiotherapy, and radiotherapy site; ④ Laboratory indicators, including tumor markers and routine blood and biochemical indicators.

[0050] (2) CT image acquisition and image deformation registration Baseline CT images, including uncontrast CT (nCE-CT) and contrast-enhanced CT (CE-CT) sequences, were collected within one month prior to the initiation of immunotherapy. This invention selects the primary lung lesion as the target lesion; if multiple primary lung lesions are present, the largest lesion is selected. Senior radiologists used 3D Slicer software to segment the lesions from both uncontrast and contrast-enhanced CT scans layer by layer, ensuring complete delineation of the tumor region while excluding non-tumor tissues such as blood vessels, airways, atelectasis, and lymph nodes. To assess inter-observer consistency, 30 cases were randomly selected, and another senior radiologist performed segmentation according to the same criteria. The inter-observer correlation coefficient (ICC) was subsequently calculated.

[0051] To achieve spatial registration between plain CT and enhanced CT scans, the advanced normalization toolkit ANTs v0.6.1 (Python v3.10.18) was used for image deformation registration. Figure 3Before registration, all plain and enhanced CT images were resampled to 1×1×1 mm³ isotropic voxels using B-spline interpolation. An elastic registration algorithm was used for registration between the plain and enhanced CT images, with the plain CT image as the reference image and the enhanced CT image as the floating image. After registration, the enhanced CT image and its corresponding tumor region were spatially transformed according to the registered deformation vector field to match the coordinate system of the plain CT image. The Dice similarity coefficient (DSC) between the tumor region on the plain CT image and the tumor region mapped from the registered enhanced CT image was calculated; cases with DSC < 0.7 were excluded from subsequent analysis. Finally, the region of overlap between the two tumors and its bounding box were used as the region of interest to quantify the correlation-difference features between modalities.

[0052] (3) Quantification of intermodal correlation-difference features This invention quantifies the intermodal correlation between plain CT and enhanced CT by calculating three types of features. Figure 3 ): ① Depth correlation features, characterizing the multi-channel depth correlation of dual-modal CT images; ② First differential feature, representing the relative differences in radiomics features of dual-modal CT images; ③ Second differential feature, radiomics features extracted from subtraction images obtained from enhanced CT and plain CT.

[0053] Depth correlation features are used to quantify the correlation between depth feature maps extracted from plain CT and enhanced CT scans using a deep learning model across different channels. Depth correlation feature calculation: Plain CT and enhanced CT images are cropped using the bounding box of the tumor overlap region. The cropped images are then resized to 224×224 pixels and input into a ResNet50 network pre-trained on the ImageNet dataset. The feature map for the 64 channels after the first convolutional layer is extracted, with dimensions of 1×64×112×112. Assuming the tumor overlap region contains N layers, the feature map dimension for single-modality CT is N×64×112×112. First, the feature maps for each phase are reconstructed to 64×N×112×112, then flattened into a 64×M matrix (M=N×112×112). The flattened feature maps for plain CT and enhanced CT are denoted as X and Y, respectively, with dimensions of: Where C=64, For plain CT scan i M-dimensional feature vectors of each channel To enhance the M-dimensional feature vector of the i-th channel in CT ( i =1,2,…,64). Based on the M-dimensional feature vectors of plain and enhanced CT scans in each channel. and Calculate the Spearman correlation coefficient between the depth feature maps of the two respectively. The formula for calculating the depth correlation coefficient of the i-th channel is:

[0054] in, The rank transformation function sorts the M elements in the feature vector by size and assigns them ranks. Through the above calculations, Spearman correlation coefficients for 64 channels are obtained for each patient, forming a deep correlation feature set: The feature maps of 64 channels can specifically highlight different image details, providing support for the comprehensive quantification of depth correlation in dual-modal CT.

[0055] Differential features were used to quantify the relative changes in radiomics features between plain CT and enhanced CT scans within the same tumor region. Differential feature extraction: Radiomics features were extracted from overlapping tumor regions using the open-source Python package PyRadiomics v3.1.0. Feature extraction parameters for plain and enhanced CT scans were set as follows: fixed bin width of 25 for grayscale discretization and grayscale normalization scaling to 2000; feature extraction parameters for subtraction images were set as follows: bin width of 20 and grayscale normalization scaling to 500. Morphological features, first-order statistical features, and texture features were extracted from the original images and from images preprocessed using gradient, wavelet, Laplacian Transform (LoG), Local Binary Pattern (LBP2D), square, square root, and exponential algorithms. Texture features were calculated based on five texture matrices: Gray-Level Co-occurrence Matrix (GLCM), Gray-Level Run-Length Matrix (GLRLM), Gray-Level Size Region Matrix (GLSZM), Gray-Level Dependency Matrix (GLDM), and Neighborhood Gray-Level Difference Matrix (NGTDM). Let K be the number of features extracted from each modal CT. Then, the first differential feature for each patient is calculated as follows: ,in, , , The first j Enhanced CT features, plain CT features, first difference features ( j =1,2,…,K), To be a minimum value, in order to avoid The value of 0 indicates a calculation error, and the value is a positive real number close to 0. Subtraction is performed on the registered plain CT images and enhanced CT images to obtain subtracted images. Then, radiomics features are directly extracted from the tumor overlap region of the subtracted images, and these radiomics features are used as the second differential features.

[0056] (4) Model building, verification and interpretation Feature selection and model building were completed on the training set. The selection of intermodal correlation-difference features was carried out in four steps: ① Retaining differential features that are robust to contour segmentation variation, and defining features with ICC > 0.75 as robust features; ② Using univariate Cox regression analysis to initially screen features that are significantly related to survival outcomes; ③ Applying Lasso-Cox regression analysis to solve the multicollinearity problem and achieve feature dimensionality reduction through regularization; ④ Based on the features selected by Lasso-Cox regression, stepwise Cox regression analysis with Akaike Information Criterion as the screening standard was used to further simplify the model and improve its simplicity.

[0057] For clinical features, a linear imputation method based on random forest was used to imput missing values ​​in the training set, internal validation set, and external validation set. Features with a missing value rate >30% were not imputed and were directly excluded. A two-stage variable screening strategy was used to screen independent prognostic factors: first, univariate Cox regression analysis was performed, and significant factors with P < 0.05 were included in multivariate Cox regression analysis. Variables with independent prognostic significance (P < 0.05) in the multivariate model were retained for subsequent model construction.

[0058] Using the selected features as input, a series of models were constructed using the Extreme Gradient Boosting (XGBoost) algorithm combined with the Cox loss function: a single-modal baseline radiomics model (plain CT model, enhanced CT model), a feature fusion model (plain + enhanced CT model), intermodal correlation-difference models (first difference model, second difference model, correlation-difference feature model), and a clinical feature model. Subsequently, the features with the best prognostic value from the radiomics models were integrated with clinical variables to construct an integrated model (…). Figure 3 The model's performance was evaluated on both internal and external validation sets. Furthermore, SHAP analysis was used to visualize the contribution of each input variable to the model's predictions. Figure 3 ).

[0059] This invention employs a combination of 5-fold cross-validation and grid search in the training set for hyperparameter tuning, selecting the hyperparameter combination with the highest average consensus index (C-index) in the cross-validation as the optimal parameter.

[0060] (5) Statistics and survival analysis Using the median risk score from the training set as a fixed cutoff value, all patients were divided into high-risk and low-risk groups. This cutoff value was uniformly applied to all validation sets. Kaplan-Meier survival curves were plotted, and the log-rank test was used to compare survival differences between the different risk groups. The C-index was used to quantify the model's discriminative power for overall survival (OS). All analyses were performed using R (v4.4.2) and Python (v3.11.4), employing two-tailed tests. P < 0.05 was considered statistically significant.

[0061] (6) Baseline clinicopathological features This invention initially enrolled 1113 patients with advanced non-small cell lung cancer who underwent ICI treatment at multiple centers. After image registration quality screening (DSC ≥ 0.7), 892 patients were ultimately included to complete the study. The median age of the study participants was 65 years (interquartile range 58–70), and males accounted for 86.1% (768 cases). In the final study cohort, 745 patients from Center 1 were randomly assigned to the training set (559 cases) and the internal validation set (186 cases), while two independent cohorts from Center 2 (97 cases) and Center 3 (50 cases) served as external validation set 1 and external validation set 2, respectively.

[0062] (7) Model performance based on image features The performance of each image feature-based model is shown in Table 1. The baseline single-modal models (plain CT model and enhanced CT model) and their direct fusion models (plain CT + enhanced CT model) have limited discriminative ability: the consistency index (C-index) of the plain CT model in each dataset is 0.54~0.62, and the performance of the enhanced CT model is similar (C-index: 0.52~0.69); the fusion model that directly stitches together the features of two images only achieves a slight performance improvement, with a C-index of 0.54~0.65, indicating that the simple feature fusion strategy does not significantly enhance the predictive ability of the model compared with single-image analysis. This invention further explores whether the quantifiable correlation between paired plain and enhanced CT images can provide better predictive information. The results show that the models built based on differential features (first differential model and second differential model) have performance comparable to the baseline models mentioned above (C-index: 0.53~0.62). It is worth noting that the models built based on intermodal depth correlation features perform better, with a C-index of 0.62~0.71, and their performance is better than all single-modal models and differential feature models.

[0063]

[0064] (8) Efficacy of clinical feature models Univariate Cox regression analysis was performed on 18 clinical characteristics to identify variables associated with overall survival (OS), including age, sex, body mass index, albumin, white blood cell count, neutrophil percentage, absolute neutrophil count, lymphocyte percentage, hemoglobin, neuron-specific enolase, cancer antigen 125, and carcinoembryonic antigen. Multivariate Cox regression analysis showed that albumin was an independent prognostic factor associated with OS (hazard ratio HR = 0.95, 95% confidence interval CI: 0.93–0.98, P < 0.001). The clinical characteristic model constructed based on this independent predictor had a C-index of 0.60 for predicting OS in the internal validation set, and C-indexes of 0.56 and 0.62 in the two external validation sets, respectively.

[0065] (9) The effectiveness of the integrated model The final integrated model incorporates five variables, enabling comprehensive and comparative prognostic assessment. These include traditional baseline predictive indicators, namely single-modal CT radiomics scores (plain CT score and enhanced CT score) and albumin, as well as two newly proposed imaging predictive indicators: intermodal depth correlation and the first differential feature. Both of these imaging indicators are independent prognostic factors in multivariate Cox regression analysis (P<0.05). The second differential feature was not included in the model because it did not retain independent prognostic significance in multivariate analysis.

[0066] The ensemble model outperformed all individual models on the training set, internal validation set, and various external validation sets. Figure 4 (a) and Table 1). This invention uses a bee colony graph to visualize the contribution of each included variable in the integrated model, clarifies the ranking of feature importance, and presents the impact of feature value changes on the model output. Figure 4 (b) in the middle.

[0067] Subsequently, survival analysis and time-dependent ROC curves were used to further validate the stratification capability and predictive accuracy of the integrated model. Specific results are as follows: Figure 5 As shown.

[0068] Example 2 like Figure 6 As shown, this embodiment provides a solid tumor treatment efficacy prediction system based on dual-modal CT imaging, including: The image acquisition module is used to acquire plain CT images and enhanced CT images of patients with solid tumors; The image registration module is used to perform deformation registration on the plain CT images and enhanced CT images, and to determine the tumor overlap area in the two images based on the registration results. The correlation feature calculation module is used to calculate the intermodal correlation feature set based on the tumor overlapping region. The intermodal correlation feature set includes depth correlation features and differential features. The depth correlation features are used to quantify the correlation of the depth feature maps extracted by the deep learning model from the plain CT image and the enhanced CT image in each channel. The differential features are used to quantify the relative change between the radiomics features of the plain CT image and the enhanced CT image in the same tumor region. The prognostic prediction module is used to input the intermodal correlation feature set into a pre-trained prognostic prediction model to obtain the efficacy prediction results for the patients with solid tumors.

[0069] Example 3 Embodiment 3 of the present invention provides an electronic device.

[0070] An electronic device includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, it implements the steps in the method for predicting the efficacy of solid tumors based on dual-modal CT images as described in Embodiment 1 of the present invention.

[0071] The detailed steps are the same as those provided in Example 1 for predicting the efficacy of solid tumor treatment based on dual-modal CT images, and will not be repeated here.

[0072] Example 4 Embodiment 4 of the present invention provides a computer-readable storage medium.

[0073] A computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the method for predicting the efficacy of solid tumors based on dual-modal CT images as described in Embodiment 1 of the present invention.

[0074] The detailed steps are the same as those provided in Example 1 for predicting the efficacy of solid tumor treatment based on dual-modal CT images, and will not be repeated here.

[0075] Example 5 Embodiment 5 of the present invention provides a computer program product.

[0076] A computer program product includes software code, wherein the program in the software code performs the steps of the method for predicting the efficacy of solid tumors based on dual-modal CT images as described in Embodiment 1 of the present invention.

[0077] The detailed steps are the same as those provided in Example 1 for predicting the efficacy of solid tumor treatment based on dual-modal CT images, and will not be repeated here.

[0078] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0079] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0080] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0081] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0082] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. A method for predicting the efficacy of treatment for solid tumors based on dual-modal CT imaging, characterized in that, include: Acquire plain CT images and enhanced CT images of patients with solid tumors; Deformation registration was performed on the plain CT images and enhanced CT images, and the overlapping area of ​​the tumor in the two images was determined based on the registration results; Based on the tumor overlap region, an intermodal correlation feature set is calculated, which includes depth correlation features and differential features. The depth correlation features are used to quantify the correlation between the depth feature maps extracted by the deep learning model of the plain CT image and the enhanced CT image in each channel. The differential features are used to quantify the relative change between the radiomics features of the plain CT image and the enhanced CT image in the same tumor region. The intermodal correlation feature set is input into a pre-trained prognostic prediction model to obtain the efficacy prediction results for patients with solid tumors.

2. The method as described in claim 1, characterized in that, Deformation registration is performed on the plain CT images and enhanced CT images, and the overlapping area of ​​the tumor in the two images is determined based on the registration results, including: Using the plain CT image as a fixed reference image and the enhanced CT image as a floating image, spatial alignment is performed using an elastic registration algorithm. The tumor outline drawn on the enhanced CT image is mapped to the space of the plain CT image based on the spatial transformation field generated by registration. Calculate the Dice similarity coefficient between the mapped tumor contour and the original tumor contour drawn on the plain CT image; Tumor regions with a Dice similarity coefficient greater than or equal to a preset threshold are selected as the tumor overlapping regions.

3. The method as described in claim 1, characterized in that, The depth-related features are obtained through the following steps: Using a pre-trained convolutional neural network, plain CT feature maps and enhanced CT feature maps are extracted from the overlapping region of the tumor, respectively, with each feature map having C channels; Regarding the first One channel ( =1,2,…,C), and the first of the plain CT feature maps The channel feature vector is denoted as The first of the enhanced CT feature maps i The channel feature vector is denoted as ; Calculate the first one according to the following formula i Spearman correlation coefficient of each channel : in, M The length of the feature vector. It is a rank transformation function; The deep correlation feature is composed of the Spearman correlation coefficients of C channels: .

4. The method as described in claim 1, characterized in that, The difference features include a first difference feature and a second difference feature; The first difference feature is obtained through the following steps: K radiomics features were extracted from the tumor overlap region of the plain CT image and the enhanced CT image, respectively. The first K feature of the enhanced CT image was denoted as K. j The feature values ​​are The first plain CT image j The feature values ​​are ,in j =1,2,…,K; For each radiomics feature, the relative difference between its feature value in enhanced CT images and its feature value in plain CT images is calculated, and the j-th differential feature value is calculated according to the following formula. : in, It is a positive real number close to 0; From K This constitutes the first distinguishing feature; The second differential feature is obtained through the following steps: subtraction processing is performed on the registered plain CT image and enhanced CT image to obtain a subtracted image, and radiomics features are extracted from the tumor overlap region of the subtracted image to form the second differential feature.

5. The method as described in claim 1, characterized in that, The prognostic prediction model is trained through the following steps: Obtain a training set containing plain CT images, enhanced CT images, and overall survival data of multiple patients with solid tumors; For each patient in the training set, calculate their intermodal correlation feature set; Using the intermodal correlation feature set as input features and total survival as the prediction target, a machine learning algorithm combined with the Cox proportional hazards loss function is used for training to obtain the prognostic prediction model.

6. The method as described in claim 1, characterized in that, The method further includes: The clinical characteristics of the patients with solid tumors are obtained, including any one or more of the following static indicators: age, sex, body mass index, albumin, white blood cell count, neutrophil percentage, absolute neutrophil count, lymphocyte percentage, hemoglobin, neuron-specific enolase, cancer antigen 125, and carcinoembryonic antigen. The clinical features are fused with the intermodal correlation feature set to form a comprehensive feature set; The comprehensive feature set is input into the pre-trained prognostic prediction model to obtain the efficacy prediction result.

7. A system for predicting the treatment efficacy of solid tumors based on dual-modal CT imaging, characterized in that, include: The image acquisition module is used to acquire plain CT images and enhanced CT images of patients with solid tumors; The image registration module is used to perform deformation registration on the plain CT images and enhanced CT images, and to determine the tumor overlap area in the two images based on the registration results. The correlation feature calculation module is used to calculate the intermodal correlation feature set based on the tumor overlapping region. The intermodal correlation feature set includes depth correlation features and difference features. The depth correlation features are used to quantify the correlation between the depth feature maps extracted by the deep learning model of the plain CT image and the enhanced CT image in each channel. The differential features are used to quantify the relative changes in radiomic features between plain CT images and enhanced CT images in the same tumor region; The prognostic prediction module is used to input the intermodal correlation feature set into a pre-trained prognostic prediction model to obtain the efficacy prediction results for the patients with solid tumors.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for predicting the efficacy of solid tumors based on dual-modal CT images as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method for predicting the efficacy of solid tumors based on dual-modal CT images as described in any one of claims 1 to 6.

10. A computer program product, comprising software code, characterized in that, The program in the software code performs the steps of the method for predicting the efficacy of solid tumor treatment based on dual-modal CT images as described in any one of claims 1 to 6.

Images