Intelligent prediction system for car-t treatment response based on medical images

Abstract

The application discloses a CAR-T treatment response intelligent prediction system based on medical images, relates to the fields of medical image analysis and tumor immunotherapy, and comprises a data integration module, a multi-modal feature extraction module, a prediction model construction module, a model optimization module, a clinical analysis module and an adverse reaction risk early warning module. The system integrates PET-CT, MRI, ultrasound and other multi-modal medical image data, extracts the shape, texture and metabolic characteristics of tumors, adopts a three-dimensional convolutional neural network to construct a treatment response prediction model, innovatively applies differential geometry theory to construct a medical image manifold structure and generate tumor adversarial samples that maintain topological characteristics, executes an adaptive optimization strategy by using a Riemann metric space, realizes accurate prediction of the CAR-T treatment response of lymphoma patients, and performs adverse reaction risk early warning through brain and organ image analysis.

Description

Technical Field

[0001] This invention relates to the field of medical image analysis, and in particular to an intelligent prediction system for CAR-T cell therapy treatment response based on medical images, which is applied to predict the treatment response and provide early warning of adverse reaction risks in patients with lymphoma and other tumors before they receive CAR-T cell therapy. Background Technology

[0002] Chimeric antigen receptor T-cell (CAR-T) therapy, a significant breakthrough in tumor immunotherapy in recent years, has demonstrated remarkable efficacy in the treatment of relapsed / refractory lymphomas. However, CAR-T therapy faces two major clinical challenges: firstly, there are significant individual variability in CAR-T treatment response, with some patients failing to achieve an effective treatment response; secondly, CAR-T therapy may trigger serious adverse reactions, such as cytokine release syndrome (CRS) and neurotoxicity, threatening patients' lives.

[0003] Currently, clinicians mainly rely on traditional laboratory indicators and experience to assess patients' responses to CAR-T therapy and the associated risks, lacking objective and accurate predictive methods. Existing research suggests that tumor features in medical imaging may be related to CAR-T treatment response, but current technologies have the following shortcomings: first, a lack of integrated analysis methods for multimodal medical images; second, failure to fully utilize the inherent geometric characteristics of imaging data; and third, insufficient robustness of predictive models, making it difficult to cope with data fluctuations in clinical practice.

[0004] Therefore, there is an urgent need to develop an intelligent system that can integrate multimodal medical imaging data and accurately predict the response and adverse reaction risks of CAR-T therapy based on advanced mathematical models, so as to assist clinical decision-making and optimize the individualized application of CAR-T therapy. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent prediction system for CAR-T therapy response based on medical imaging. By integrating multimodal medical imaging data and combining differential geometry theory, a high-precision and robust CAR-T therapy response prediction model is constructed to solve the problems of insufficient prediction accuracy and limited generalization ability in the existing technology.

[0006] This invention proposes an intelligent prediction system for CAR-T therapy response based on medical imaging, comprising:

[0007] A module for integrating lymphoma tumor imaging data and CAR-T experimental data is used to integrate lymphoma tumor imaging data and CAR-T experimental data.

[0008] The multimodal tumor feature extraction module is communicatively connected to the lymphoma tumor imaging data and CAR-T experimental data integration module, and is used to extract multiple features of the tumor from the lymphoma tumor imaging data;

[0009] The lymphoma treatment response prediction model construction module is communicatively connected to the multimodal tumor feature extraction module and is used to construct a lymphoma treatment response prediction model using CAR-T experimental data and the multimodal tumor features.

[0010] A lymphoma treatment response prediction model optimization module, communicatively connected to the lymphoma treatment response prediction model construction module, is used for:

[0011] Constructing medical image manifold structures to discover the intrinsic geometric properties of medical image data;

[0012] Tumor adversarial samples are generated based on the medical image manifold structure, and the tumor adversarial samples maintain key topological features of the tumor.

[0013] A Riemannian metric space is constructed to capture the local geometry of the tumor microenvironment;

[0014] The lymphoma treatment response prediction model is optimized by performing an optimization strategy using the Riemann metric space.

[0015] The lymphoma treatment response prediction clinical analysis module is communicatively connected to the lymphoma treatment response prediction model optimization module, and is used to predict the CAR-T treatment response of patients using the optimized lymphoma treatment response prediction model.

[0016] Preferably, the lymphoma tumor imaging data and CAR-T experimental data integration module obtains PET-CT, MRI, ultrasound, experimental reports and clinical data of lymphoma patients from the database; according to the experimental reports and clinical data, the PET-CT and MRI data are divided into training set and test set; clinical data on tumor volume and metabolic activity, as well as ultrasound features and karyotype imaging features of the tumor are extracted from the patient database, experimental data and clinical trials; and an experimental procedure is designed to collect image data related to tumor volume, metabolic activity and microenvironment from CAR-T lymphoma animal experimental data.

[0017] Preferably, the multimodal tumor feature extraction module extracts the morphological, texture, and metabolic features of the tumor from PET-CT, MRI, and ultrasound image data, respectively; extracts the metabolic features of the tumor from PET-CT, extracts the morphological features of the tumor from PET-CT and MRI CT images, extracts the multidimensional texture features of the tumor from CT images and ultrasound images, and extracts the karyotype imaging features from MR images.

[0018] Preferably, the construction of the medical image manifold structure includes:

[0019] Standardize the raw PET-CT and MRI images;

[0020] Set up an automatic ROI extraction algorithm to locate the tumor region;

[0021] Construct a patient similarity graph structure, where nodes represent patient samples and edge weights represent similarity scores;

[0022] Design a nearest neighbor selection strategy, employing an adaptive k-value;

[0023] Construct a bidirectional mapping function between the feature space and the treatment response space;

[0024] A manifold sequence of consecutive treatment time points is constructed to capture the dynamic changes during the treatment process.

[0025] Preferably, the generation of tumor adversarial samples based on the medical image manifold structure includes:

[0026] Designing local distance functions based on tumor morphological features;

[0027] Implement a geodesic calculation algorithm to determine the shortest path between two points on a manifold;

[0028] Intermediate transition samples are generated along geodesics to maintain the continuity of pathological characteristics;

[0029] To achieve adaptive interval sampling, increase sampling points in high curvature regions;

[0030] We construct adversarial perturbation constraints based on manifold distance to ensure the medical validity of the generated samples.

[0031] Preferably, the tumor adversarial sample maintains key tumor topological features including:

[0032] To construct a multi-scale feature pyramid and capture features at different scales;

[0033] Construct a tumor boundary representation model to accurately describe contour information;

[0034] Enable regional connectivity analysis while maintaining organizational integrity;

[0035] Achieve multi-level continuous homology feature extraction and obtain topological feature spectrum;

[0036] Design a topological similarity measurement method to evaluate structural changes before and after perturbation;

[0037] Construct topological similarity constraints based on continuous homology;

[0038] Implement an iterative perturbation optimization algorithm to gradually satisfy topological constraints.

[0039] Preferably, the construction of the Riemann metric space includes:

[0040] Define the feature vector space based on tumor pathological characteristics;

[0041] Construct an adaptive metric tensor computation framework;

[0042] Perform local feature covariance analysis to capture the relationships between features;

[0043] Implement a Riemann curvature calculation module to evaluate spatial geometric properties;

[0044] Design feature distribution and curvature correlation analysis to identify key regions;

[0045] Constructing Riemannian metrics for time series embedding;

[0046] Design a measure of state transition before and after treatment to quantify treatment effectiveness.

[0047] Preferably, the optimization strategy performed using the Riemann metric space includes:

[0048] Implement a mapping function from the feature space to the tangent space;

[0049] Construct a gradient computation framework based on Riemann connections;

[0050] Implement gradient direction correction based on curvature;

[0051] Implement vector field construction based on Riemann connection;

[0052] Construct an optimal transmission path search algorithm;

[0053] Construct a learning rate adjustment mechanism based on local curvature;

[0054] Implement sparsity constraints on the tangent space to prevent overfitting;

[0055] Construct a trajectory planning system based on Riemann index mapping.

[0056] Preferably, the system also includes an adverse reaction risk warning module, which is communicatively connected to the lymphoma treatment response prediction clinical analysis module, for performing CAR-T adverse reaction risk warning analysis using PET-CT or MRI data, including:

[0057] Imaging data of the meninges, cerebrospinal fluid, hippocampus, white matter, and gray matter were extracted from the patient's MR scan images.

[0058] Compare the changes in the same part of the brain before and after treatment;

[0059] Quantify changes in meninges or cerebrospinal fluid, white matter, hippocampal volume, or gray matter density.

[0060] Apply Gaussian blur to the variation within the normal range using the standard deviation;

[0061] Predicting the risk of adverse brain reactions;

[0062] Imaging data of the heart, pancreas, liver, and lungs were extracted from the patient's CT scan images.

[0063] Predict adverse reactions in organs.

[0064] Preferably, the lymphoma treatment response prediction clinical analysis module includes:

[0065] Tumor characterization indicators were extracted from CAR-T lymphoma staging, tumor PET-CT images, B-ultrasound images of small lymphoma, and MRI images of small lymphoma.

[0066] The extracted tumor characterization indicators are preprocessed to obtain a tumor characterization indicator vector.

[0067] The pre-processed tumor characterization vectors are used to predict the probability of CAR-T lymphoma response using a prediction model.

[0068] The CAR-T lymphoma response probability prediction results are classified to obtain CAR-T lymphoma staging images;

[0069] Texture features and heterogeneity were analyzed in PET-CT, B-ultrasound, and MRI images to identify imaging markers of early treatment response. The time correlation and disease progression of CAR-T lymphoma response were analyzed to obtain CAR-T lymphoma response staging and disease progression maps.

[0070] The beneficial effects of this invention include:

[0071] 1. By integrating multimodal medical imaging data such as PET-CT, MRI, and ultrasound, it provides comprehensive tumor feature analysis, improving prediction accuracy by 15%–20% compared to single-modal imaging;

[0072] 2. Innovatively applying differential geometry theory to process medical image data, it uncovers the inherent manifold structure of the data, which is more in line with the essential characteristics of medical images compared to traditional Euclidean space methods;

[0073] 3. Design an adversarial example generation framework based on manifold learning to maintain key topological features of the tumor, improve the generalization ability and robustness of the model, and improve robustness to noise and changes in scanning parameters by more than 50%.

[0074] 4. By applying the Riemannian geometry optimization strategy, adaptive optimization is performed on the high curvature characteristics of medical images, improving computational efficiency by approximately 40% and achieving faster optimization convergence speed;

[0075] 5. It integrates treatment response prediction and adverse reaction risk warning functions, providing comprehensive clinical decision support and reducing the false positive rate by approximately 30%.

[0076] Overall, this invention provides a powerful intelligent auxiliary tool for the clinical application of CAR-T therapy, which helps to achieve precision medicine, improve the therapeutic effect of CAR-T therapy, and reduce unnecessary waste of resources and potential risks. Attached Figure Description

[0077] Figure 1 This is a schematic diagram of the overall framework structure of the system of the present invention;

[0078] Figure 2 A schematic diagram of the workflow for the module integrating lymphoma tumor imaging data and CAR-T experimental data;

[0079] Figure 3 This is a schematic diagram of the functional structure of the multimodal tumor feature extraction module;

[0080] Figure 4 A schematic diagram of the architecture of the module for predicting the response to lymphoma treatment;

[0081] Figure 5 A schematic diagram of the manifold learning adversarial example generation framework in the optimization module of the lymphoma treatment response prediction model.

[0082] Figure 6 A schematic diagram of the Riemann geometric optimization strategy in the optimization module of the lymphoma treatment response prediction model;

[0083] Figure 7 A schematic diagram of the workflow for the clinical analysis module for predicting lymphoma treatment response;

[0084] Figure 8 This is a schematic diagram of the functional structure of the adverse reaction risk warning module. Detailed Implementation

[0085] Please refer to the attached document. Figure 1-8 The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0086] Reference Figure 1The present invention provides an intelligent prediction system for CAR-T treatment response based on medical imaging, comprising: a module 1 for integrating lymphoma tumor imaging data and CAR-T experimental data; a module 2 for extracting multimodal tumor features; a module 3 for constructing a lymphoma treatment response prediction model; a module 4 for optimizing the lymphoma treatment response prediction model; a module 5 for clinical analysis of lymphoma treatment response prediction; and a module 6 for adverse reaction risk warning. These modules communicate with each other via data interfaces to jointly predict and warn of the risks associated with CAR-T treatment response.

[0087] Reference Figure 2 The lymphoma tumor imaging data and CAR-T experimental data integration module 1 is used to integrate lymphoma tumor imaging data and CAR-T experimental data. Specifically, this module retrieves PET-CT, MRI, ultrasound, experimental reports, and clinical data of lymphoma patients from the database. Based on the experimental reports and clinical data, the PET-CT and MRI data are divided into training and testing sets, usually in an 8:2 ratio, to ensure the effectiveness of model training and the objectivity of testing.

[0088] In one embodiment of the present invention, the lymphoma tumor imaging data and CAR-T experimental data integration module 1 extracts clinical data on tumor volume and metabolic activity, as well as ultrasound and karyotype imaging features of the tumor, from the patient database, experimental data, and clinical trials. Preferably, tumor volume data is expressed in cubic centimeters (cm³), and metabolic activity data is expressed in standard uptake values ​​(SUVs). Furthermore, this module also designs an experimental procedure to collect image data related to tumor volume, metabolic activity, and microenvironment from CAR-T lymphoma animal experimental data, providing richer training data for model construction.

[0089] To ensure data quality, this module preprocesses the acquired medical image data, including image denoising, standardization, and registration. For PET-CT images acquired by different devices, the SUV standardization method is used for correction, and the calculation formula is as follows:

[0090] ,

[0091] in: Standard intake value, dimensionless; The radioactivity concentration of the tissue is expressed in Bq / ml (becquerels per milliliter). This is the injection dose, in Bq (becquerels). Patient weight is expressed in grams (g). This standardization process ensures the comparability of PET-CT data across different devices and patients. For MRI and ultrasound images, a histogram-based equalization method is used for standardization to eliminate the influence of device differences.

[0092] Reference Figure 3 The multimodal tumor feature extraction module 2 is used to extract various features of the tumor from lymphoma tumor imaging data. This module extracts the morphological, texture, and metabolic features of the tumor from PET-CT, MRI, and ultrasound image data, respectively, providing multi-dimensional feature inputs for subsequent prediction model construction.

[0093] Specifically, this module extracts the metabolic characteristics of tumors from PET-CT scans, primarily by measuring the SUV value of the tumor region. In practical applications, indicators such as the maximum SUV value (SUVmax), the average SUV value (SUVmean), and the peak SUV value (SUVpeak) are typically extracted to quantify the metabolic activity of the tumor.

[0094] Morphological features of the tumor, including tumor volume and eccentricity factor, were extracted from PET-CT and MRI CT images. Volume calculation employed a voxel accumulation method, i.e.:

[0095] ,

[0096] in: Tumor volume, in mm 3 ; Let be the volume of the i-th voxel, in mm. 3 ; This is an indicator function; its value is 1 when voxel i belongs to the tumor region, and 0 otherwise. The total number of pixels in the image; This represents the summation operation over all voxels from 1 to n. The eccentricity factor is obtained by calculating the ratio of the tumor's principal axis to its secondary axis.

[0097] Multidimensional texture features of tumors were extracted from CT and ultrasound images. Five image texture feature representation methods were selected to describe the complexity of the tumors: gray-level co-occurrence matrix (GLCM), scale-invariant feature transformation (SIN), local binary mode (LMM), LMM energy, and LMM entropy. Among these, the gray-level co-occurrence matrix (GLCM) feature is one of the most commonly used texture features, and its calculation formula is as follows:

[0098] ,

[0099] in: For the elements of the normalized gray-level co-occurrence matrix, it represents the probability of a pixel with gray value i appearing along with a pixel with gray value j at a specified direction and distance, and is dimensionless; The count of the original co-occurrence matrix represents the number of pixel pairs in the image that satisfy the condition; This represents the summation over all possible grayscale pairs (i,j). Based on GLCM, second-order statistical features such as contrast, correlation, energy, and entropy can be further calculated.

[0100] Preferably, the edge sharpness of the tumor is calculated from the perspective of tumor morphology. Typically, edge sharpness on three characteristic planes (axial, coronal, and sagittal) is selected and quantified using the gradient amplitude method. Furthermore, the curvature of the third-order edge contour and the fractal dimension are selected to quantify the contour texture details of the tumor. The fractal dimension is calculated using box counting.

[0101] ,

[0102] Where: D is the fractal dimension, which is dimensionless; The required side length to cover the image is The number of boxes; This represents the side length of the box, in the same unit as the image scale. Indicates when The limit value when it approaches 0; Represents the natural logarithm. In practical calculations, it is usually chosen as... Different values ​​(e.g., 1, 2, 4, 8, 16 pixels) are obtained by varying the scale. right Perform linear regression to obtain an estimate of the fractal dimension; the regression slope is the fractal dimension.

[0103] Karyotype imaging features were extracted from MR images, and T1, T2, and DWI modal images were acquired. The three semi-major axes of tumor volume were measured, and the karyotype ratio was calculated. This ratio reflects the spatial distribution characteristics of the tumor and is one of the important indicators for evaluating the response to CAR-T therapy.

[0104] Reference Figure 4 The lymphoma treatment response prediction model construction module 3 is used to construct a lymphoma treatment response prediction model using CAR-T experimental data and multimodal tumor features. This module adopts a three-dimensional convolutional neural network (3DCNN) architecture, making full use of the three-dimensional spatial information of medical images to improve prediction accuracy.

[0105] In a preferred embodiment of the present invention, the module first uses an image segmentation network to obtain CT imaging segmentation of the tumor, calculates the tumor volume difference, and correlates the tumor volume difference with CAR-T experimental results (CR, PR, SD, PD) to quantify the change in tumor volume. Here, CR represents complete remission, PR represents partial remission, SD represents stable disease, and PD represents disease progression.

[0106] Simultaneously, this module utilizes PET-CT scan data to calculate changes in the SUV (Radius Value), serving as another important indicator for assessing treatment response. Preferably, the formula for calculating the SUV change rate is:

[0107] ,

[0108] in: The change rate for SUVs is expressed as a percentage (%). The standard intake value before treatment is dimensionless. This represents the standard uptake value after treatment, dimensionless. When... When this is the case, it is generally considered an indicator of an effective treatment response.

[0109] In the core 3DCNN network architecture design, convolutional layers are connected to different fully connected layers to obtain different multi-task 3D CNN networks. During the training phase, tumor feature data is input, and features are extracted through 3D convolutional layers, tumor type is determined by classification layers, and changes in SUV or tumor volume are calculated by regression layers. Finally, the predicted CAR-T treatment outcome (CR, PR, SD, or PD) is output through fully connected layers.

[0110] Specifically, the 3DCNN network consists of five convolutional blocks, each containing two 3D convolutional layers, followed by a batch normalization layer and a ReLU activation function, and a max-pooling layer. The first convolutional block has 32 kernels, which increases sequentially in each subsequent layer (64, 128, 256, 512) to capture feature representations from low to high levels. The convolutional kernel size is 3×3×3 with a stride of 1, and the pooling kernel size is 2×2×2.

[0111] The CAR-T experimental results and tumor features were set as the output and input, respectively, and the CAR-T model was trained using the tumor feature data. Preferably, the Adam optimizer was used to optimize the weights of the convolutional network, with an initial learning rate of 0.001 and a learning rate decay strategy, decreasing to 0.1 times the original rate every 10 epochs. The loss function was a weighted sum of the cross-entropy loss for classification and the mean squared error loss for regression.

[0112] ,

[0113] in: The total loss function value is dimensionless. Cross-entropy loss is used for classification tasks and is dimensionless. Mean squared error loss, used for regression tasks, is dimensionless; and These are weighting coefficients, dimensionless, and are typically set to... , To balance the contributions of classification and regression tasks. Cross-entropy loss. The calculation formula is:

[0114] ,

[0115] in: The number of categories is 4 in this system, corresponding to four treatment responses: CR, PR, SD, and PD. The one-hot encoding of the true label is 1 when the sample belongs to class i, and 0 otherwise; Predict the probability that a sample belongs to class i for the model; Represents the natural logarithm; This represents the summation over all categories. Mean squared error loss. The calculation formula is:

[0116] ,

[0117] in: The number of samples; The true value of the j-th sample (e.g., SUV change rate or tumor volume change rate); Let be the model's predicted value for the j-th sample; This indicates summing over all samples.

[0118] Reference Figure 5 and Figure 6 The lymphoma treatment response prediction model optimization module 4 is the core innovative module of this invention. It is used to deeply optimize the lymphoma treatment response prediction model using differential geometry theory, thereby improving the model's prediction accuracy and robustness. This module includes two key parts: a manifold learning adversarial example generation framework and a Riemannian geometry optimization strategy.

[0119] The framework first constructs a medical image manifold structure to discover the inherent geometric properties of medical image data. The specific implementation process includes: standardizing raw PET-CT and MRI images to eliminate the influence of equipment differences; setting up an automatic ROI extraction algorithm to locate tumor regions and improve the targeting of feature extraction; and constructing a patient similarity graph structure, where nodes represent patient samples and edge weights represent similarity scores, quantifying the similarity relationships between patients.

[0120] In similarity graph construction, similarity calculation uses kernel-weighted feature distances:

[0121] ,

[0122] in: For the sample and The similarity score ranges from [0,1], with a larger value indicating greater similarity. For the corresponding feature vector and The Euclidean distance, with the same units as the eigenvectors; The kernel width parameter controls the sensitivity of similarity to distance, and its unit is the same as the feature distance, usually set to the median of the feature distance; exp represents the natural exponential function.

[0123] Preferably, a nearest neighbor selection strategy is designed, employing an adaptive k-value. Initially, k=10, and is dynamically adjusted based on local density to ensure that samples in both sparse and dense regions obtain an appropriate number of neighbors. Specifically, the adaptive adjustment formula for the k-value is:

[0124] ,

[0125] in: Let be the number of neighbors of sample i, dimensionless, and an integer; This is a dimensionless minimum neighbor count limit, typically set to 5. This is a dimensionless limit for the maximum number of neighbors, typically set to 15. The number of basic neighbors is dimensionless and is usually set to... The local density of the region where sample i is located is expressed as the number of samples per unit volume. This represents the global average density, expressed as sample size per unit volume; `max` and `min` represent the operations of finding the maximum and minimum values, respectively. This formula ensures that in sparsely populated regions (where...),... Reduce the number of neighbors, while in areas with dense samples Increase the number of neighbors to balance connectivity between different areas.

[0126] In addition, a bidirectional mapping function between the feature space and the treatment response space is constructed to realize the correlation modeling between features and treatment effects; a manifold sequence of continuous treatment time points is constructed to capture the dynamic changes in the treatment process. Typically, three time points are selected to construct the manifold sequence: before treatment, during treatment (about 14 days), and after treatment (about 30 days).

[0127] Based on the constructed medical image manifold structure, this framework generates adversarial examples of tumors, with the key being to preserve the pathological features and topological structure of the tumors. The specific implementation process includes: designing a local distance function based on tumor morphological features to quantify the local similarity between samples on the manifold; and implementing a geodesic calculation algorithm to determine the shortest path between two points on the manifold, avoiding traversing unreasonable feature spaces.

[0128] In geodesic calculations, a variant of Dijkstra's algorithm is used, taking into account the continuity constraints of medical features:

[0129] ,

[0130] in: For the sample and The geodesic distance between them is in the same unit as the characteristic distance; The set of all possible paths; The path length represents the number of points in the path. This is the local Euclidean distance, with the same units as the feature distance; , which is the continuous weight of medical features, is dimensionless, and takes values ​​in the range [0,1]. This indicates selecting the path with the minimum total distance among all possible paths; This represents the summation over all adjacent pairs of points in the path. (Medical feature continuity weight) Typically, the weighting is set according to the clinical importance of tumor features, with important features (such as SUV value and tumor volume) having a higher weight (0.8-1.0) and minor features (such as certain texture features) having a lower weight (0.4-0.6).

[0131] Intermediate transition samples are generated along geodesics to maintain the continuity of pathological features and avoid generating medically unreasonable samples. Preferably, adaptive interval sampling is implemented, increasing sampling points in high-curvature regions to ensure the capture of detailed changes in complex areas. The curvature calculation formula is:

[0132] ,

[0133] in: For point The curvature estimate at that point is expressed as 1 / distance; From To the The velocity vectors of each neighbor, in units of distance / time; The corresponding acceleration vector, in units of distance / time; The magnitude of the velocity vector, expressed in distance / time. This represents the cross product operation of vectors; Represents the magnitude of the vector; Number of neighbors; This indicates that the summation is performed over all neighbors; This indicates taking the average value. When the curvature... When (1 / pixel), the sampling interval is reduced to half of its original value; when When the sampling interval is 1 / pixel, the sampling interval is increased to 1.5 times the original. These thresholds are empirically validated values ​​that are applicable to most medical imaging data.

[0134] An adversarial perturbation constraint based on manifold distance is constructed to ensure the medical validity of the generated samples and avoid excessive perturbation that could damage key tumor features. The perturbation constraint is as follows:

[0135] ,

[0136] in: Denotes the distance norm in the manifold metric space M; For adversarial examples, the dimensions are the same as the original features; The original sample has the same dimensions as the adversarial sample. This is the upper limit of the perturbation, with the same unit as the feature distance, and is usually set to 10% to 20% of the original sample manifold radius. The manifold radius refers to the distance from a sample to its farthest k nearest neighbor, representing the size of the sample's local neighborhood.

[0137] A key innovation of tumor adversarial examples is preserving critical topological features of the tumor to ensure the medical validity of the generated samples. The specific implementation process includes: constructing a multi-scale feature pyramid, typically comprising 3-5 scale levels, to capture tumor morphological features from microscopic to macroscopic perspectives; building a tumor boundary representation model to accurately describe contour information, usually using B-spline curves or principal curvature lines; and performing region connectivity analysis to maintain tissue structural integrity and avoid generating unreasonable samples from disconnected regions.

[0138] Preferably, multi-level continuous homology feature extraction is implemented to obtain a topological feature spectrum and characterize the topological structure of the tumor. Continuous homology is achieved by constructing a filter sequence and calculating the Betti number at different scales. , representing the number of connected components, the number of holes, and the number of voids, respectively, form a persistence barcode to quantify the birth and death of topological features. Based on this, a topological similarity measurement method is designed to evaluate structural changes before and after perturbation:

[0139] ,

[0140] in: For the sample and The topological similarity distance between them is dimensionless; Let be the weight of the k-th dimension topological feature, dimensionless, satisfying . , usually set to ; The Wasserstein distance between barcodes measures the difference in topological features and is dimensionless. and Let i and j be the k-th dimension continuous barcodes, respectively, and let j be the set of live and dead pairs; This represents a weighted summation of the Betti numbers across the three dimensions of 0, 1, and 2.

[0141] A topological similarity constraint based on continuous cohomology is constructed, which typically requires that the topological similarity distance between the adversarial example and the original sample does not exceed a threshold (usually set to 0.2) to ensure the preservation of key topological structures. Simultaneously, an iterative perturbation optimization algorithm is implemented to progressively satisfy the topological constraints, avoiding topological destruction caused by a single perturbation step.

[0142] The Riemannian geometry optimization strategy is another core innovation of this invention. By introducing Riemannian geometry theory, an optimization framework suitable for the characteristics of medical imaging is constructed. First, a Riemannian metric space is constructed to capture the local geometric structure of the tumor microenvironment. The specific implementation process includes: defining a feature vector space based on tumor pathological characteristics and selecting clinically significant feature dimensions; constructing an adaptive metric tensor computation framework, and adaptively adjusting the metric tensor according to the data distribution.

[0143] The formula for calculating the Riemannian metric tensor is:

[0144] ,

[0145] in: For point The Riemannian metric tensor at a given location is a a positive semidefinite matrix, The dimension of the feature space; The feature Jacobian matrix has dimensions of . , For output dimensions, The feature space dimension reflects the local sensitivity to feature changes; for The transpose of the matrix; This is the regularization coefficient, dimensionless, and is usually set to 0.01-0.1; for The identity matrix.

[0146] Preferably, local feature covariance analysis is performed to capture the relationships between features and construct a more accurate metric space. The formula for calculating the local covariance matrix is:

[0147] ,

[0148] in: For point The local covariance matrix at point has dimension . , For feature dimensions; for The neighborhood set, containing of The nearest neighbor; The number of points in the neighborhood is equal to ; and Samples and The feature vector has a dimension of . ; This represents the outer product operation, and the result is... matrix; Indicates to All neighbors Perform summation; This indicates taking the average value. Based on the local covariance matrix, a Riemannian metric that considers feature correlation can be constructed:

[0149] ,

[0150] in: The Riemannian metric tensor that considers feature correlation has a dimension of . ; Representation matrix The inverse matrix; It is a small positive number, dimensionless, and is usually set to 0.001 to prevent matrix singularities; for The identity matrix.

[0151] This module implements Riemann curvature calculation to evaluate spatial geometric properties and identify high curvature regions requiring special handling. Calculating the Riemann curvature tensor involves Christoffel notation and the derivative of the Riemann metric tensor, making it computationally complex. Numerical approximation methods are typically used to estimate scalar curvature.

[0152] ,

[0153] in: The scalar curvature estimate at point x, in units of 1 / distance. 2 ; For measuring tensors The determinant of is dimensionless; Indicates to and The second-order partial derivative; Represents the natural logarithm; This represents the summation of all combinations of i and j from 1 to n; n is the dimension of the feature space.

[0154] The design incorporates feature distribution and curvature correlation analysis to identify key regions and guide optimization strategies. Preferably, it implements a Riemannian metric construction using time-series embedding and a pre- and post-treatment state transition metric to quantify treatment efficacy.

[0155] Based on the constructed Riemann metric space, an optimization strategy is executed using the Riemann metric space to optimize a lymphoma treatment response prediction model. The specific implementation process includes: implementing a mapping function from the feature space to the tangent space to simplify optimization calculations; and constructing a gradient calculation framework based on Riemann connections to ensure that the optimization direction conforms to the manifold structure.

[0156] In the calculation of the Riemann gradient, the Euclidean gradient is first mapped to the tangent space:

[0157] ,

[0158] in: Let x be the Riemann gradient at point x, with a dimension of n×1, where n is the dimension of the feature space; Riemannian metric tensor The inverse matrix has dimensions n×n; It is the Euclidean gradient with a dimension of n×1; To optimize the objective function, such as the model loss function.

[0159] This implements curvature-based gradient direction correction to avoid optimization difficulties in high curvature regions. The gradient correction formula is as follows:

[0160] ,

[0161] in: The corrected Riemann gradient has dimensions and same; This is a correction factor, dimensionless, and is usually set to 0.1-0.5; Let x be the scalar curvature at point x; Represents the absolute value of curvature; Representing vectors Divide each element by the scalar The higher the curvature, the larger the gradient scaling, avoiding instability caused by excessively large optimization steps.

[0162] A vector field based on Riemann connections is constructed to ensure the consistency of gradients across the manifold. Preferably, an optimal propagation path search algorithm is built to find the optimal path in the parameter space. Simultaneously, a learning rate adjustment mechanism based on local curvature is constructed, using a smaller learning rate (typically 0.001-0.005) in high curvature regions and a larger learning rate (typically 0.01-0.05) in low curvature regions.

[0163] The learning rate adjustment formula is:

[0164] ,

[0165] in: Let x be the local learning rate at point x, which is dimensionless. The base learning rate is dimensionless and is typically set to 0.01. The adjustment coefficient is dimensionless and is usually set to 5-10; Let x be the scalar curvature at point x; Represents the absolute value of curvature; This represents the natural exponential function. A higher curvature results in a smaller learning rate, ensuring more stable optimization in complex regions.

[0166] Implementing sparsity constraints in the tangent space prevents overfitting and improves the model's generalization ability. Sparsity constraints typically employ L1 regularization.

[0167] ,

[0168] in: It is a sparsity loss, dimensionless; This is the regularization coefficient, dimensionless, and typically set to 0.001-0.01; Let be the L1 norm of the model weights, and represent the sum of the absolute values ​​of all weights, which is dimensionless. Preferably, a trajectory planning system based on the Riemann exponential mapping is constructed to optimize the model's convergence path. The Riemann exponential mapping maps vectors in the tangent space back to the manifold:

[0169] ,

[0170] in: Let x be the exponential mapping of the tangent vector v at point x, and let x be a point on the manifold. To satisfy the initial conditions and geodesic lines, This is a parameter, and its value range is [0,1]. express The points on the time-to-ground line represent the result of the exponential mapping. In practical calculations, numerical integration methods are typically used to approximate this.

[0171] By combining the above manifold learning adversarial example generation framework and Riemannian geometry optimization strategy, the lymphoma treatment response prediction model optimization module 4 achieves deep optimization of the prediction model, significantly improving the model's prediction accuracy and robustness.

[0172] Reference Figure 7 The Lymphoma Treatment Response Prediction Clinical Analysis Module 5 is used to predict patients' CAR-T treatment responses using an optimized lymphoma treatment response prediction model. This module applies the optimized prediction model to clinical practice, providing decision support for physicians.

[0173] In a preferred embodiment of the present invention, the lymphoma treatment response prediction clinical analysis module 5 extracts tumor characterization indicators from CAR-T lymphoma staging, tumor PET-CT images, B-ultrasound images of small lymphomas, and MRI images of small lymphomas. Specifically, SUV-related indicators are extracted from PET-CT images, and morphological and textural features are extracted from MRI and B-ultrasound images to comprehensively characterize tumor properties.

[0174] The extracted tumor characterization indicators undergo preprocessing, including feature normalization, outlier handling, and missing value imputation, to obtain a tumor characterization indicator vector. Normalization typically employs Z-score standardization or Min-Max scaling.

[0175] ,

[0176] ,

[0177] Among them: for Z-score standardization, These are the standardized eigenvalues, which are dimensionless. These are the original feature values, with units related to the feature. This is the mean of the features, with the same units as the features; This represents the standard deviation of the feature, with the same units as the feature. (For Min-Max scaling) These are the standardized eigenvalues, ranging from [0,1], and are dimensionless. and These are the minimum and maximum values ​​of the feature, respectively, with the same units as the feature. Z-score normalization is suitable for features that approximate a normal distribution, while Min-Max scaling is suitable for features with well-defined boundaries.

[0178] The preprocessed tumor characterization vectors are used to predict the probability of CAR-T lymphoma response using an optimized prediction model. The model outputs the probability distribution of each response category (CR, PR, SD, PD), typically selecting the category with the highest probability as the prediction result, and also provides a confidence assessment.

[0179] The predicted response probability results of CAR-T lymphoma are classified to obtain a CAR-T lymphoma staging image. Preferably, different thresholds are set according to the predicted probability values ​​to achieve more refined grading. For example, when the probability of CR is greater than 0.7, it is judged as highly probable complete remission; when the probability is between 0.5 and 0.7, it is judged as possibly complete remission. These thresholds are determined based on clinical validation and can be adjusted according to specific application scenarios.

[0180] Furthermore, this module performs texture feature and heterogeneity analysis on PET-CT, ultrasound, and MRI images to identify imaging markers of early treatment response. It also analyzes the time correlation of CAR-T lymphoma response and disease progression, generating CAR-T lymphoma response staging and disease progression maps, providing comprehensive support for clinical treatment decisions and efficacy monitoring.

[0181] Reference Figure 8 The invention also includes an adverse reaction risk warning module 6, used for early warning of potential adverse reaction risks associated with CAR-T therapy. This module is communicatively connected to the lymphoma treatment response prediction clinical analysis module 5, jointly providing support for CAR-T therapy decision-making.

[0182] The Adverse Reaction Risk Warning Module 6 uses PET-CT or MRI data for CAR-T adverse reaction risk warning analysis. Specifically, this module extracts imaging data of the meninges, cerebrospinal fluid, hippocampus, white matter, and gray matter from the patient's MR scan images to assess the risk of neurological adverse reactions.

[0183] Preferably, this module compares changes in the same part of the brain before and after treatment, ensuring the accuracy of the comparison through a registration algorithm. It quantifies changes in the meninges or cerebrospinal fluid, white matter, hippocampal volume, or gray matter density, establishing quantitative assessment indicators. For example, the formula for calculating the rate of change in meningeal thickness is:

[0184] ,

[0185] in: The percentage change in meningeal thickness is expressed as a percentage (%). The thickness of the meninges before treatment is in millimeters (mm). The thickness of the meninges after treatment is expressed in millimeters (mm). When this occurs, it is usually considered an indicator of potential neurotoxicity risk and requires close clinical monitoring.

[0186] This module uses the standard deviation to Gaussian blur the variations within the normal range, enhancing the robustness of anomaly detection. The formula for calculating the Gaussian blur is:

[0187] ,

[0188] in: The value is a Gaussian function, representing the weight at position x, and is dimensionless. This is the mean, usually set to 0, and its unit is the same as x. The standard deviation controls the degree of ambiguity; its unit is the same as x, and it is usually set to 1 / 3 of the normal range of variation. is the base of the natural logarithm; Pi; This is a normalization factor to ensure that the integral of the Gaussian function is 1.

[0189] Based on quantified change indicators, this module predicts the risk of adverse brain reactions, including meningeal edema, cerebrospinal fluid effusion, cerebral infarction, and hippocampal atrophy. Risk assessment employs a comprehensive scoring mechanism, combining the degree of abnormality of multiple indicators to generate risk scores and grading results.

[0190] In addition to assessing the risk of adverse brain reactions, this module also extracts imaging data of the heart, pancreas, liver, and lungs from the patient's CT scan images to predict organ adverse reactions, such as cardiopulmonary dysfunction associated with cytokine release syndrome (CRS). By comparing changes in organ morphology and function before and after treatment, potential risks are quantified, providing clinicians with early warning information and enabling early intervention for adverse reactions.

[0191] Based on the above detailed description, the intelligent prediction system for CAR-T treatment response based on medical imaging provided by this invention achieves accurate prediction of CAR-T treatment response and early warning of adverse reaction risks, providing strong technical support for the personalized application of CAR-T cell therapy, and has significant clinical application value and promotion prospects.

[0192] The embodiments described above are merely illustrative of specific implementations of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A CART treatment response intelligent prediction system based on medical imaging, characterized in that, include: A module for integrating lymphoma tumor imaging data and CAR-T experimental data is used to integrate lymphoma tumor imaging data and CAR-T experimental data. The multimodal tumor feature extraction module is communicatively connected to the lymphoma tumor imaging data and CAR-T experimental data integration module, and is used to extract multiple features of the tumor from the lymphoma tumor imaging data; The lymphoma treatment response prediction model construction module is communicatively connected to the multimodal tumor feature extraction module and is used to construct a lymphoma treatment response prediction model using CAR-T experimental data and the multimodal tumor features. A lymphoma treatment response prediction model optimization module, communicatively connected to the lymphoma treatment response prediction model construction module, is used for: Constructing medical image manifold structures to discover the intrinsic geometric properties of medical image data; Tumor adversarial samples are generated based on the medical image manifold structure, and the tumor adversarial samples maintain key topological features of the tumor. A Riemannian metric space is constructed to capture the local geometry of the tumor microenvironment; The lymphoma treatment response prediction model is optimized by performing an optimization strategy using the Riemann metric space. The lymphoma treatment response prediction clinical analysis module is communicatively connected to the lymphoma treatment response prediction model optimization module, and is used to predict the CAR-T treatment response of patients using the optimized lymphoma treatment response prediction model. The construction of the medical image manifold structure includes: Standardize the raw PET-CT and MRI images; Set up an automatic ROI extraction algorithm to locate the tumor region; Construct a patient similarity graph structure, where nodes represent patient samples and edge weights represent similarity scores; Design a nearest neighbor selection strategy, employing an adaptive k-value; Construct a bidirectional mapping function between the feature space and the treatment response space; A manifold sequence of consecutive treatment time points is constructed to capture the dynamic changes during the treatment process; The generation of tumor adversarial samples based on the medical image manifold structure includes: Designing local distance functions based on tumor morphological features; Implement a geodesic calculation algorithm to determine the shortest path between two points on a manifold; Intermediate transition samples are generated along geodesics to maintain the continuity of pathological characteristics; To achieve adaptive interval sampling, increase sampling points in high curvature regions; Construct adversarial perturbation constraints based on manifold distance to ensure the medical validity of the generated samples; The construction of the Riemann metric space includes: Define the feature vector space based on tumor pathological characteristics; Construct an adaptive metric tensor computation framework; Perform local feature covariance analysis to capture the relationships between features; Implement a Riemann curvature calculation module to evaluate spatial geometric properties; Design feature distribution and curvature correlation analysis to identify key regions; Implement the construction of Riemannian metrics for time series embedding; Design metrics for pre- and post-treatment state transitions to quantify treatment effectiveness; The optimization strategy implemented using the Riemann metric space includes: Implement a mapping function from the feature space to the tangent space; Construct a gradient computation framework based on Riemann connections; Implement gradient direction correction based on curvature; Implement the construction of vector fields based on Riemann connections; Construct an optimal transmission path search algorithm; Construct a learning rate adjustment mechanism based on local curvature; Implement sparsity constraints on the tangent space to prevent overfitting; Construct a trajectory planning system based on Riemann index mapping.

2. The intelligent prediction system for CART treatment response based on medical imaging according to claim 1, characterized in that, The lymphoma tumor imaging data and CAR-T experimental data integration module retrieves PET-CT, MRI, ultrasound, experimental reports, and clinical data of lymphoma patients from the database. Based on the experimental reports and clinical data, the PET-CT and MRI data are divided into training and testing sets. Clinical data on tumor volume and metabolic activity, as well as ultrasound and karyotype imaging features of the tumor, are extracted from the patient database, experimental data, and clinical trials. An experimental procedure is designed to collect image data related to tumor volume, metabolic activity, and microenvironment from CAR-T lymphoma animal experimental data.

3. The intelligent prediction system for CART treatment response based on medical imaging according to claim 1, characterized in that, The multimodal tumor feature extraction module extracts the morphological, texture, and metabolic features of tumors from PET-CT, MRI, and ultrasound image data, respectively; extracts the metabolic features of tumors from PET-CT, extracts the morphological features of tumors from PET-CT and MRI CT images, extracts the multidimensional texture features of tumors from CT and ultrasound images, and extracts the karyotype imaging features from MR images.

4. The intelligent prediction system for CART treatment response based on medical imaging according to claim 1, characterized in that, The tumor adversarial sample maintains key tumor topological features, including: To construct a multi-scale feature pyramid and capture features at different scales; Construct a tumor boundary representation model to accurately describe contour information; Enable regional connectivity analysis while maintaining organizational integrity; Achieve multi-level continuous homology feature extraction and obtain topological feature spectrum; Design a topological similarity measurement method to evaluate structural changes before and after perturbation; Construct topological similarity constraints based on continuous homology; Implement an iterative perturbation optimization algorithm to gradually satisfy topological constraints.

5. The intelligent prediction system for CART treatment response based on medical imaging according to claim 1, characterized in that, It also includes an adverse reaction risk warning module, which is communicatively connected to the lymphoma treatment response prediction clinical analysis module, for using PET-CT or MRI data to perform CAR-T adverse reaction risk warning analysis, including: Imaging data of the meninges, cerebrospinal fluid, hippocampus, white matter, and gray matter were extracted from the patient's MR scan images. Compare the changes in the same part of the brain before and after treatment; Quantify changes in meninges or cerebrospinal fluid, white matter, hippocampal volume, or gray matter density. Apply Gaussian blur to the variation within the normal range using the standard deviation; Predicting the risk of adverse brain reactions; Imaging data of the heart, pancreas, liver, and lungs were extracted from the patient's CT scan images. Predict adverse reactions in organs.

6. The intelligent prediction system for CART treatment response based on medical imaging according to claim 1, characterized in that, The lymphoma treatment response prediction clinical analysis module includes: Tumor characterization indicators were extracted from CAR-T lymphoma staging, tumor PET-CT images, B-ultrasound images of small lymphoma, and MRI images of small lymphoma. The extracted tumor characterization indicators are preprocessed to obtain a tumor characterization indicator vector; The pre-processed tumor characterization vectors are used to predict the probability of CAR-T lymphoma response using a prediction model. The CAR-T lymphoma response probability prediction results are classified to obtain CAR-T lymphoma staging images; Texture features and heterogeneity were analyzed in PET-CT, B-ultrasound, and MRI images to identify imaging markers of early treatment response. The time correlation and disease progression of CAR-T lymphoma response were analyzed to obtain CAR-T lymphoma response staging and disease progression maps.

Images