Tumor risk layering method and system based on multi-modal radiomics

Abstract

The invention discloses a tumor risk layering method and system based on multi-modal imaging omics, and is applied to the technical field of medical image processing, and the method specifically comprises the steps: delineating a region of interest of a tumor according to the clinical medical information of a patient; extracting radiomics features in the region of interest; screening the image omics characteristics to obtain updated image omics characteristics; inputting the updated radiomics features into a risk hierarchical prediction model for training to obtain a tumor risk prediction model; the non-invasive risk stratification prediction of tumor patients can be realized, good sensitivity and specificity can be achieved, and the high false negative rate existing in the current clinical technology is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a tumor risk stratification method and system based on multimodal imaging omics. Background Art

[0002] Tumor tissue biopsy technology is an invasive screening technology widely used in clinical practice to determine the risk level of cancer patients. This technology requires the patient to undergo local anesthesia and insert a guide needle with a core into the patient's lesion to obtain potential tumor tissue samples for further testing. However, due to the high heterogeneity of tumor tissue and the low resolution of imaging equipment, this technology not only brings about a false negative rate of about 20%-30%, but also brings the risk of hemorrhagic shock and wound infection to patients.

[0003] In view of the shortcomings of existing tissue biopsy technology, the present invention proposes a tumor risk stratification method based on multimodal imaging genomics to solve the high false negative rate of existing technologies in tumor risk stratification and the potential risks brought by invasive screening; the method proposed in the present invention adopts the multimodal high-throughput imaging genomics characteristics of cancer patients and constructs a risk stratification prediction model based on multimodal fusion imaging genomics characteristics, which improves the sensitivity and specificity of tumor risk stratification while avoiding the potential burden of invasive screening on patients. Summary of the invention

[0004] The purpose of this application is to provide a tumor risk stratification method and system based on multimodal imaging genomics, aiming to solve the problems of high false negative rate in tumor risk stratification in existing technologies and potential risks brought by invasive screening.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] The present application provides a tumor risk stratification method based on multimodal imaging omics, including:

[0007] Outline the region of interest of the tumor based on the patient's clinical medical information;

[0008] Extract radiomic features in the region of interest;

[0009] After screening the radiomics features, updated radiomics features are obtained;

[0010] The updated radiomics features are input into the risk stratification prediction model for training to obtain a tumor risk prediction model.

[0011] Furthermore, in the step of outlining the region of interest of the tumor according to the patient's clinical medical information, the clinical medical information includes clinical medical images and corresponding tumor risk stratification labels, the clinical medical images are acquired by scanning different medical imaging devices, the medical imaging devices include but are not limited to CT, MR and PET, and the modalities of the clinical medical images are no less than 2 different categories; the tumor risk stratification label adopts the corresponding tumor risk stratification label information based on the original pathological data of the patient before radiotherapy, and constructs a training database.

[0012] Further, in the step of extracting radiomics features in the region of interest, the radiomics features include shape-based features, statistical histogram-based features, texture features, model-based features, and mathematical transformation-based features;

[0013] The 3D shape-based features are defined as:

[0014] Sphericity:

[0015] Non-spherical proportions:

[0016] Where V is in mm 3 The grid volume in units of mm. 2 The surface area of ​​the grid per unit;

[0017] The 2D shape-based features are defined as:

[0018] Sphericity:

[0019] Non-spherical proportions:

[0020] Where A is in mm 2 is the surface area of ​​the grid in units of mm, P is the perimeter of the grid in units of mm;

[0021] The features based on statistical histogram are:

[0022] entropy:

[0023] Skewness:

[0024] Uniformity:

[0025] The total number of pixels in the region of interest is N p , X is a set of voxels contained in the region of interest, P(i) is a discrete first-order grayscale histogram, p(i) is a normalized first-order grayscale histogram, N gis the number of non-zero window widths, ε is an arbitrary positive number equal to 2.2×10 -16 .

[0026] Furthermore, the step of obtaining updated radiomics features after screening the radiomics features includes the following steps:

[0027] Eliminating the features of the radiomics features that have redundant information, and screening out the radiomics features that are most conducive to tumor risk stratification prediction;

[0028] The Spearman correlation coefficient was used to evaluate the new features after screening;

[0029] Feature recursive elimination was used to reduce the number of radiomics features after coefficient evaluation.

[0030] Furthermore, in the step of eliminating the features of the radiomics features with redundant information and screening out the radiomics features that are most conducive to tumor risk stratification prediction, the following steps are specifically included:

[0031] The eigenvalues ​​in each feature are standardized by the standardization function. The formula is:

[0032]

[0033] Where μ is the mean of the eigenvalues ​​in the radiomics feature, σ is the standard deviation of the eigenvalues, and x and x new are the values ​​before and after standardization, respectively; the imaging genomics features are converted into new features of a standard normal distribution with a mean of 0 and a standard deviation of 1 through the formula.

[0034] Furthermore, in the step of evaluating the Spearman correlation coefficient of the screened new features, the following steps are specifically included:

[0035] Calculate the Spearman correlation coefficient between any two sets of radiomics features:

[0036]

[0037] d i =R i -S i ,

[0038] Where R i and S i are the observed values ​​of any sample in the two groups of imaging features, and are the average numbers of the two groups of influencing omics features in the sample set, and N is the number of tumor patients in the risk stratification prediction model; if any group of features has a correlation coefficient greater than 0.75, the feature with a lower average correlation with the other N-2 features is retained, and the other imaging omics feature is discarded at the same time.

[0039] Furthermore, in the step of reducing the number of radiomics features by recursive feature elimination for the radiomics features after coefficient evaluation, the following steps are specifically included:

[0040] Feature recursion was used to eliminate the number of radiomics features. In the 10-fold cross validation, the linear regression model was selected as the scoring model. According to the results of the scoring model, the features with the highest number of radiomics features corresponding to the highest score were selected and input into the risk stratification prediction model for training.

[0041] Furthermore, the step of inputting the updated radiomics features into the risk stratification prediction model for training to obtain a tumor risk prediction model includes the following steps:

[0042] The minimum absolute value shrinkage and selection operator are used as examples of the risk stratification prediction model to be trained. The selection operator optimizes the loss function by iteratively determining the penalty factor. The formula of the loss function is:

[0043]

[0044] Where m is the number of samples, w and b are the coefficients corresponding to the imaging omics features in the model and the intercept of the model, respectively. and i are the true label of the model and the predicted value of the model respectively, n is the number of iterative calculations; when the loss function J is the smallest, the corresponding λ is the parameter of the model.

[0045] Furthermore, in the training of the risk stratification prediction model, a set of radiomics features X = {x1, x2, x3, ..., x n}, and the corresponding weight coefficient set W = {w1,w2,w3,......,w n}, w≠0; the optimal tumor risk prediction model F(X) is obtained by using the parameter λ, the radiomics feature set X, the weight coefficient set W and the threshold segmentation point α:

[0046]

[0047] If the output of F(X) is 1, it means the predicted result is a high-risk patient, and if the output is 0, it means the predicted result is not a high-risk patient.

[0048] The present application also provides a tumor risk stratification system based on multimodal imaging omics, including:

[0049] Outlining module: outlines the region of interest of the tumor based on the patient's clinical medical information;

[0050] Extraction module: extracts radiomic features in the region of interest;

[0051] Screening module: Screening the radiomics features to obtain updated radiomics features;

[0052] Training module: inputting the updated imaging genomics features into the risk stratification prediction model for training to obtain a tumor risk prediction model.

[0053] The present application provides a device, which includes a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing a tumor risk stratification method based on multimodal imaging genomics; the processor is used to execute the program instructions stored in the memory to implement multimodal imaging genomics tumor risk stratification.

[0054] The present application provides a storage medium storing program instructions executable by a processor, wherein the program instructions are used to execute a tumor risk stratification method based on multimodal imaging omics.

[0055] The present application provides a tumor risk stratification method and system based on multimodal imaging omics, which has the following beneficial effects:

[0056] (1) This application uses multimodal medical images to extract radiomic features in the region of interest by outlining the region of interest of the tumor based on the patient's clinical medical information; based on the radiomic features defined by the Imaging Biomarker Standardization Initiative, a highly robust tumor risk stratification prediction model is invented;

[0057] (2) By evaluating the correlation coefficients between omics features and the feature recursive elimination method, a fast, stable and efficient tumor risk imaging feature screening scheme is provided to screen out the imaging features that are most conducive to tumor risk stratification prediction, ensuring the stability and reliability of the model selection features in specific diseases, and having high clinical application value;

[0058] (3) Based on high-throughput imaging genomics features, a risk stratification prediction model based on multimodal fusion features was constructed, which not only improved the sensitivity and specificity of prostate cancer risk stratification, but also avoided the potential burden of invasive screening on patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a schematic diagram of a process of a tumor risk stratification method based on multimodal imaging omics in Example 1 of the present application;

[0060] Figure 2 This is a structural block diagram of obtaining a tumor medical image in Example 1 of the present application;

[0061] Figure 3 This is a structural diagram of the tumor risk stratification label of Example 1 of the present application;

[0062] Figure 4 A scheme for delineating the region of interest of Example 1 of the present application;

[0063] Figure 5 This is a confusion matrix diagram of the risk stratification prediction model in Example 1 of the present application in the training set;

[0064] Figure 6 This is a confusion matrix diagram of the risk stratification prediction model in Example 1 of the present application in the test set;

[0065] Figure 7 The subject characteristic curve of the risk stratification prediction model in Example 1 of the present application on the training set;

[0066] Figure 8 The subject characteristic curve of the risk stratification prediction model in Example 1 of the present application on the test set;

[0067] Fig. 9 This is a schematic diagram of the structure of a tumor risk stratification system based on multimodal imaging omics in Example 2 of the present application;

[0068] Fig.10 A schematic diagram of the device structure provided in Example 3 of the present invention;

[0069] Fig.11 A schematic diagram of the storage medium structure provided in Example 4 of the present invention. DETAILED DESCRIPTION

[0070] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0071] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0072] Example 1

[0073] See also Figure 1, which is a flow chart of a tumor risk stratification method based on multimodal imaging genomics proposed in Example 1 of the present application; it specifically includes the following steps S110 to S140, and the implementation method of each step is described in detail below.

[0074] S110: Outline the region of interest of the tumor according to the patient's clinical medical information.

[0075] In this embodiment, the clinical medical information includes clinical medical images and corresponding tumor risk stratification label information.

[0076] See also Figure 2 , which is a structural block diagram of obtaining tumor medical images provided in an embodiment; the clinical medical images are obtained by scanning different medical imaging devices, and the medical imaging devices include but are not limited to CT, MR and PET, and the modalities of the clinical medical images are no less than 2 different categories.

[0077] See also Figure 3 , which is a structural block diagram of a tumor risk stratification label; the tumor risk stratification label can use the corresponding tumor risk stratification label information based on the original pathological data of the patient before radiotherapy, and construct a training database.

[0078] See also Figure 4 , delineate a plan for the region of interest; the region of interest involved in the present invention is usually larger than or equal to the entire tumor lesion area of ​​the patient. Figure 4 a. The radiologist determines the actual distribution area of ​​the tumor (GTV, Gross Target Volume) based on the patient's medical image based on his / her observations. Then, based on clinical knowledge and experience, the possible invasion range of the tumor is considered, and based on the GTV, the tumor is diffused in the same direction for a certain distance to outline the potential invasion area of ​​the tumor: the clinical target volume (CTV);

[0079] like Figure 4 b. According to different isotropic distances, more different regions of interest can be obtained; for example, based on CTV, taking into account the position deviation factors such as the respiratory movement of tumor patients during radiotherapy, the planning target volume (PTV) for tumor radiotherapy can be obtained by expanding it in the same direction by 5mm.

[0080] It is understandable that since the original pathological data of each patient before radiotherapy is different, the corresponding tumor risk stratification standards (different tumor patients have different risk stratification standards) are used to assign risk labels to each patient and build a training database.

[0081] S120: Extracting radiomics features in the region of interest.

[0082] In this embodiment, the radiomics features include shape-based features, statistical histogram-based features, texture features, model-based features, and mathematical transformation-based features; specifically:

[0083] Shape-based features describe the geometric characteristics of the region of interest (ROI); statistical features include features based on statistical histograms and texture features. Among the statistical histogram features, there are grayscale mean, maximum, minimum, variance and percentile, skewness, kurtosis, entropy and energy based on single-pixel 2D or voxel 3D analysis; texture-based features include: absolute gradient, which reflects the degree of grayscale intensity fluctuation in the image; grayscale co-occurrence matrix, which can capture the spatial relationship between pixel pairs or voxel pairs with predefined grayscale intensities; grayscale run matrix, which provides information about the distribution of continuous pixels with the same grayscale level in one or more directions in 2D or 3D space, grayscale size area matrix, neighborhood grayscale difference matrix and grayscale dependency matrix; the present invention uses at least two of the above features to participate in the construction of risk prediction models; the following are the specific definitions of imaging omics features commonly used in the construction of tumor risk prediction models in each category:

[0084] The 3D shape-based features are defined as:

[0085] Sphericity:

[0086] Non-spherical proportions:

[0087] Where V is in mm 3 The grid volume in units of mm. 2 The surface area of ​​the grid per unit.

[0088] The 2D shape-based features are defined as:

[0089] Sphericity:

[0090] Non-spherical proportions:

[0091] Where A is in mm 2 is the surface area of ​​the grid in units of mm, and P is the perimeter of the grid in mm.

[0092] The feature definition based on statistical histogram is:

[0093] entropy:

[0094] Skewness:

[0095] Uniformity:

[0096] The total number of pixels in the region of interest is N p , X is a set of voxels contained in the region of interest, P(i) is a discrete first-order grayscale histogram, p(i) is a normalized first-order grayscale histogram, N g is the number of non-zero window widths, ε is an arbitrary positive number equal to 2.2×10 -16 .

[0097] Based on GLCM texture feature definition:

[0098] Cluster shadows:

[0099] Contrast ratio:

[0100] Differential Entropy:

[0101] Joint Entropy:

[0102] First homogeneity:

[0103] Second homogeneity:

[0104] Contrast moment:

[0105] where ε is an arbitrarily small positive number (≈2.2×10 -16 ), P(i,j) is the co-occurrence matrix in any case, p(i,j) is the normalized co-occurrence matrix in any case, N g is the number of discrete pixel intensity levels, is the marginal row probability, is the probability of the marginal column, μ x and μ y is the average grayscale intensity of the image, which can be defined as p x and p y , σ x Indicates p x The standard deviation, σ y Indicates p y The standard deviation of Where i+j=k, and k=2,3,....,2N g ; Where |ij|=k, and k=0,1,....,N g ; As p x The entropy of As p y The entropy of as the entropy of p(i,j).

[0106] Based on GLSZM, the texture feature is defined as:

[0107] Short-term focus:

[0108] Long-term focus:

[0109] Non-uniformity of run length:

[0110] Run Percentage:

[0111] Run variance:

[0112] Run-length entropy:

[0113] Low grayscale runs emphasize:

[0114] High grayscale runs emphasize:

[0115] Short-term low grayscale emphasis:

[0116] Short-term high grayscale emphasis:

[0117] Where N g is the number of discrete grayscale intensity values ​​in the image, N r is the number of discrete runs, N p is the number of voxels in the image, N r (θ) is the number of runs along the angle in the image, and is equal to P(i,j|θ) is the run matrix θ in any direction, and p(i,j|θ) is the normalized grayscale run matrix, defined as At the same time

[0118] Based on NGTDM, the texture feature is defined as:

[0119] Roughness:

[0120] Contrast ratio:

[0121] Complexity:

[0122] Neighboring rate of change:

[0123] where n i is the number of voxels in the grayscale level. N v,p is the total number of voxels, p i is the grayscale probability, and its value is equal to s i is the sum of the absolute grayscale differences, N g is the number of discrete gray levels, N g,p is the number of gray levels.

[0124] Based on GLDM, the texture feature is defined as:

[0125] Low dependency emphasizes:

[0126] High dependence emphasizes:

[0127] Grayscale non-uniformity:

[0128] Dependency entropy:

[0129] Where N g is the number of discrete grayscale intensity values ​​in the image, N d is the discrete dependent size in the image, N z is the number of dependent spaces, and its value is equal to Where P(i,j) is the discrete dependency matrix and p(i,j) is the standardized discrete dependency matrix.

[0130] It can be understood that based on the above definition of imaging omics features, this application extracts all high-throughput imaging omics features from ROI for the subsequent construction of tumor risk stratification model.

[0131] S130: Screening the radiomics features to obtain updated radiomics features.

[0132] In this embodiment, the updated radiomics features are obtained after screening the radiomics features, which specifically includes the following steps S131 to S133. The implementation method of each step is described in detail below.

[0133] S131: Eliminate the features of the radiomics features that have redundant information, and screen out the radiomics features that are most conducive to tumor risk stratification prediction; specifically, the following steps are included:

[0134] Since there are significant differences in the values ​​of each feature, such as the original values ​​of some radiomics features are large, while the data of other radiomics features are small, the characteristic values ​​in each feature are standardized through the standardization function, and the formula is:

[0135]

[0136] Where μ is the mean of the eigenvalues ​​in the radiomics feature, σ is the standard deviation of the eigenvalues, and x and x new are the values ​​before and after standardization, respectively; the imaging genomics features are converted into new features of a standard normal distribution with a mean of 0 and a standard deviation of 1 through the formula.

[0137] S132: Evaluate the Spearman correlation coefficient of the new features after screening; specifically, the following steps are included:

[0138] Since the new features may contain a lot of redundant information that has not been removed, this application needs to extract the most representative features to provide key information about tumor risk stratification. Therefore, it is also necessary to evaluate the Spearman correlation coefficient of the new features and calculate the Spearman correlation coefficient between any two groups of imaging omics features:

[0139]

[0140] d i =R i -S i ,

[0141] Where R i and S i are the observed values ​​of any sample in the two groups of imaging features, and are the average numbers of the two groups of influencing omics features in the sample set, and N is the number of tumor patients in the risk stratification prediction model; if any group of features has a correlation coefficient greater than 0.75, the feature with a lower average correlation with the other N-2 features is retained, and the other imaging omics feature is discarded at the same time.

[0142] S133: Reducing the number of radiomics features by recursive feature elimination for the radiomics features after coefficient evaluation; specifically comprising the following steps:

[0143] The Spearman coefficient cannot eliminate all redundant information, so feature recursion is further used to eliminate the number of imaging genomics features. In the 10-fold cross-validation, the linear regression model is selected as the scoring model. According to the results of the scoring model, the features with the highest number of imaging genomics features corresponding to the highest score are selected as input into the risk stratification prediction model for training; the regression model LASSO is used as the risk stratification prediction classification strategy, but there are many classification strategies for this method, such as random forest, support vector machine, logistic regression, naive Bayes and deep learning, etc. The technology that uses such supervised classification strategies to achieve tumor risk stratification is suitable for application.

[0144] S140: Inputting the updated radiomics features into a risk stratification prediction model for training to obtain a tumor risk prediction model.

[0145] In this embodiment, after obtaining the radiomics features for training, the training of the risk stratification prediction model can be started; the least absolute shrinkage and selection operator (LASSO) is used as an example of the risk stratification prediction model to be trained, and the method adopts a supervised learning classification strategy to achieve tumor risk stratification prediction; in this example, the selection operator optimizes the loss function by iteratively determining the penalty factor to achieve a more streamlined and reliable model, and the formula of the loss function is:

[0146]

[0147] Where m is the number of samples, w and b are the coefficients corresponding to the imaging omics features in the model and the intercept of the model, respectively. and i are the true label of the model and the predicted value of the model respectively, n is the number of iterative calculations; when the loss function J is the smallest, the corresponding λ is the parameter of the model.

[0148] By training the risk stratification prediction model, we obtain the radiomics feature set X = {x1, x2, x3, ..., x n}, and the corresponding weight coefficient set W = {w1,w2,w3,......,w n}, w≠0; comprehensively consider the best specificity, sensitivity and accuracy, and determine the best threshold segmentation point α of the model; finally, through the parameter λ, the radiomics feature set X, the weight coefficient set W and the threshold segmentation point α, the best tumor risk prediction model F(X) is obtained:

[0149]

[0150] If the output of F(X) is 1, it means the predicted result is a high-risk patient, and if the output is 0, it means the predicted result is not a high-risk patient.

[0151] See also Figure 5 , which is the confusion matrix diagram of the risk stratification prediction model in the training set; it shows the performance of the three risk prediction models on the training set, from the upper left corner to the lower right corner are the CT-based model, the MRI-based model and the multimodal model, among which the MRI-based model and the multimodal model have only one wrong identification in the identification of high-risk and non-high-risk patients.

[0152] See also Figure 6, which is the confusion matrix diagram of the risk stratification prediction model in the test set; it shows the performance of the three models in the test set, among which the CT-based model had 4 misclassifications among 15 test samples, while the CT and MRI-based models had only 3 misidentifications.

[0153] See also Figure 7 , which is the receiver operating characteristic curve of the risk stratification prediction model on the training set; it shows that each risk stratification prediction model can achieve a good AUC value on the training set. Among them, the multimodal model achieved the maximum AUC of 0.984.

[0154] See also Figure 8 , which is the receiver operating characteristic curve of the risk stratification prediction model on the test set; in the test set results, although the multimodal model can still achieve an AUC of 0.911, the AUC of the MRI-based risk prediction model can still reach 0.821.

[0155] In another embodiment, for a tumor patient with unknown risk, first, a multimodal medical image of the patient's tumor is acquired by scanning with a medical imaging device; secondly, the region of interest ROI of the tumor is outlined, and then the same features as those in the feature set X are extracted from the ROI, and these multimodal imaging features are input into the tumor risk prediction model F(X); if F(X) is greater than or equal to the optimal threshold α, the patient will be predicted as a high-risk patient by the tumor risk stratification model; otherwise, the patient will be predicted as a non-high-risk patient; based on this, the clinician can carry out the next step of treatment for the patient according to the predicted results of the tumor risk stratification.

[0156] In summary, the present application, through Example 1, outlines the region of interest of the tumor according to the patient's clinical medical information, and extracts the imaging genomics features in the region of interest, screens the imaging genomics features and inputs them into the risk stratification prediction model for training, to obtain the tumor risk prediction model, and finally outputs the patient's tumor risk stratification results; the present application, through this method, can not only realize non-invasive risk stratification prediction of tumor patients, but also achieve better sensitivity and specificity, and reduce the high false negative rate existing in current clinical technology.

[0157] Example 2

[0158] See also Fig. 9 , is a schematic diagram of the structure of a tumor risk stratification system based on multimodal imaging omics proposed in Example 2 of the present application; the specific contents include:

[0159] Outlining module: outlines the region of interest of the tumor based on the patient's clinical medical information;

[0160] Extraction module: extracts radiomic features in the region of interest;

[0161] Screening module: Screening the radiomics features to obtain updated radiomics features;

[0162] Training module: inputting the updated imaging genomics features into the risk stratification prediction model for training to obtain a tumor risk prediction model.

[0163] In this embodiment, the above system was used to perform risk stratification tests using clinical CT images of 50 prostate cancer patients and magnetic resonance images acquired on 1.5T MR. A total of 1037 imaging genomics features were extracted from the region of interest ROI of each category, including 93 original features, which mainly met the definition of relevant imaging genomics features in the imaging biomarker standardization initiative, and also included 944 features converted in the wavelet and Gaussian Laplace domains. Clinicians divided the risk level labeling information of prostate cancer patients based on the D'Amico standard, and the test results confirmed the feasibility of the present invention.

[0164] For example, the following table records in detail the performance of three risk stratification prediction models developed based on prostate tumor imaging omics on the training set and test set. In the training set, the multimodal omics prediction model achieved the best risk stratification prediction performance, with a specificity and sensitivity of 94.44% and 94.12%, respectively, and a positive prediction probability and a negative prediction probability of 94.10% and 94.40%, respectively. The second is the model based on MRI imaging omics features. Although the specificity and sensitivity are similar to those of the multimodal model, its positive prediction rate and negative prediction rate are 88.90% and 94.10%, respectively, which is better than the CT-based risk stratification prediction model, but weaker than the multimodal risk prediction model. The results on the independent sample test set showed that the best sensitivity (85.71%) and negative predictive rate (85.70%) could be obtained on the multimodal model, the best specificity (87.50%) and the best positive predictive rate (83.3%) could be obtained on the MRI-based risk stratification prediction model, and the specificity, sensitivity, positive predictive rate and negative predictive rate of the CT-based risk stratification prediction model were 75%, 71.43%, 71.40% and 75% respectively.

[0165]

[0166] Example 3

[0167] See also Fig.10 , is a schematic diagram of the device structure of an embodiment of the present application. The device 50 includes a processor 51 and a memory 52 coupled to the processor 51.

[0168] The memory 52 stores program instructions for implementing the above-mentioned three-network information fusion and retrieval system.

[0169] The processor 51 is used to execute the program instructions stored in the memory 52 to realize the fusion and retrieval of the three-network information.

[0170] The processor 51 may also be referred to as a CPU (Central Processing Unit).

[0171] The processor 51 may be an integrated circuit chip with signal processing capabilities. The processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0172] Example 4

[0173] See also Fig.11 , which is a schematic diagram of the structure of the storage medium of the embodiment of the present application. The storage medium of the embodiment of the present application stores a program file 61 that can implement all the above methods, wherein the program file 61 can be stored in the above storage medium in the form of a software product, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a computer, a server, a mobile phone, a tablet and other devices.

[0174] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0175] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

[0176] Although the embodiments of the present application have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the appended claims and their equivalents.

[0177] Of course, the present invention may have many other implementations. Based on this implementation, other implementations obtained by ordinary technicians in this field without any creative work are all within the scope of protection of the present invention.

Claims

1. A tumor risk stratification method based on multimodal imaging omics, characterized in that: include: Outline the region of interest of the tumor based on the patient's clinical medical information; Extract radiomic features in the region of interest; After screening the radiomics features, updated radiomics features are obtained; The updated radiomics features are input into the risk stratification prediction model for training to obtain a tumor risk prediction model.

2. A tumor risk stratification method based on multimodal imaging omics according to claim 1, characterized in that: In the step of outlining the region of interest of the tumor according to the patient's clinical medical information, the clinical medical information includes clinical medical images and corresponding tumor risk stratification labels, the clinical medical images are acquired by scanning different medical imaging devices, the medical imaging devices include but are not limited to CT, MR and PET, and the modalities of the clinical medical images are not less than 2 different categories; the tumor risk stratification label is based on the original pathological data of the patient before radiotherapy, uses the corresponding tumor risk stratification label information, and constructs a training database.

3. The method for tumor risk stratification based on multimodal imaging omics according to claim 1, characterized in that: In the step of extracting radiomics features in the region of interest, the radiomics features include shape-based features, statistical histogram-based features, texture features, model-based features, and mathematical transformation-based features; The 3D shape-based features are defined as: Sphericity: Non-spherical proportions: Where V is in mm 3 The grid volume in units of mm. 2 The surface area of ​​the grid per unit; The 2D shape-based features are defined as: Sphericity: Non-spherical proportions: Where A is in mm 2 is the surface area of ​​the grid in units of mm, P is the perimeter of the grid in units of mm; The features based on statistical histogram are: entropy: Skewness: Uniformity: The total number of pixels in the region of interest is N p , X is a set of voxels contained in the region of interest, P(i) is a discrete first-order grayscale histogram, p(i) is a normalized first-order grayscale histogram, N g is the number of non-zero window widths, ε is an arbitrary positive number equal to 2.2×10 -16 .

4. The method for tumor risk stratification based on multimodal imaging omics according to claim 1, characterized in that: The step of obtaining updated radiomics features after screening the radiomics features includes the following steps: Eliminating the features of the radiomics features that have redundant information, and screening out the radiomics features that are most conducive to tumor risk stratification prediction; The Spearman correlation coefficient was used to evaluate the new features after screening; Feature recursive elimination was used to reduce the number of radiomics features after coefficient evaluation.

5. The method for tumor risk stratification based on multimodal imaging omics according to claim 4, characterized in that: The step of eliminating the features of the radiomics features with redundant information and screening out the radiomics features that are most conducive to tumor risk stratification prediction specifically includes the following steps: The eigenvalues ​​in each feature are standardized by the standardization function. The formula is: Where μ is the mean of the eigenvalues ​​in the radiomics feature, σ is the standard deviation of the eigenvalues, and x and x new are the values ​​before and after standardization, respectively; the imaging genomics features are converted into new features of a standard normal distribution with a mean of 0 and a standard deviation of 1 through the formula.

6. A tumor risk stratification method based on multimodal imaging omics according to claim 4, characterized in that: The step of evaluating the Spearman correlation coefficient of the screened new features specifically includes the following steps: Calculate the Spearman correlation coefficient between any two sets of radiomics features: d i =R i -S i , Where R i and S i are the observed values ​​of any sample in the two groups of imaging features, and are the average numbers of the two groups of influencing omics features in the sample set, and N is the number of tumor patients in the risk stratification prediction model; if any group of features has a correlation coefficient greater than 0.75, the feature with a lower average correlation with the other N-2 features is retained, and the other imaging omics feature is discarded at the same time.

7. The method for tumor risk stratification based on multimodal imaging omics according to claim 4, characterized in that: In the step of reducing the number of radiomics features by recursive feature elimination on the radiomics features after coefficient evaluation, the following steps are specifically included: Feature recursion was used to eliminate the number of radiomics features. In the 10-fold cross validation, the linear regression model was selected as the scoring model. According to the results of the scoring model, the features with the highest number of radiomics features corresponding to the highest score were selected and input into the risk stratification prediction model for training.

8. The method for tumor risk stratification based on multimodal imaging omics according to claim 1, characterized in that: The step of inputting the updated radiomics features into the risk stratification prediction model for training to obtain a tumor risk prediction model comprises the following steps: The minimum absolute value shrinkage and selection operator are used as examples of the risk stratification prediction model to be trained. The selection operator optimizes the loss function by iteratively determining the penalty factor. The formula of the loss function is: Where m is the number of samples, w and b are the coefficients corresponding to the imaging omics features in the model and the intercept of the model, respectively. and i are the true label of the model and the predicted value of the model respectively, n is the number of iterative calculations; when the loss function J is the smallest, the corresponding λ is the parameter of the model.

9. A tumor risk stratification method based on multimodal imaging omics according to claim 8, characterized in that: In the training of the risk stratification prediction model, a set of radiomics features X = {x1, x2, x3, ..., x n }, and the corresponding weight coefficient set W = {w1,w2,w3,......,w n }, w≠0; through the parameter λ, the radiomics feature set C, the weight coefficient set W and the threshold segmentation point α, the optimal tumor risk prediction model F(X) is obtained: If the output of F(X) is 1, it means the predicted result is a high-risk patient, and if the output is 0, it means the predicted result is not a high-risk patient.

10. A system for a tumor risk stratification method based on multimodal imaging omics according to claim 1, characterized in that: include: Outlining module: outlines the region of interest of the tumor based on the patient's clinical medical information; Extraction module: extracts radiomic features in the region of interest; Screening module: Screening the radiomics features to obtain updated radiomics features; Training module: inputting the updated imaging genomics features into the risk stratification prediction model for training to obtain a tumor risk prediction model.

11. A device, characterized in that: The device includes a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing a tumor risk stratification method based on multimodal imaging genomics as described in any one of claims 1-9; and the processor is used to execute the program instructions stored in the memory to implement multimodal imaging genomics tumor risk stratification.

12. A storage medium, characterized in that: Program instructions executable by a processor are stored, and the program instructions are used to execute a tumor risk stratification method based on multimodal imaging omics as described in any one of claims 1 to 9.

Images