Multi-modal osteosarcoma data processing method and system based on artificial intelligence

Abstract

A multi-modal osteosarcoma data processing method and system based on artificial intelligence, the method comprising: obtaining image omics data, clinical data and recurrence condition data of a sample object, the image omics data comprising a T1 weighted image and a T2 weighted image, the field of view of the T1 weighted image being consistent with the field of view of the T2 weighted image on a corresponding plane, and the field of view of the T2 weighted image being consistent with the field of view of the T1 weighted image; the clinical data contains epidemiological data and tumor-related data, and the recurrence condition data indicates whether osteosarcoma of the sample object relapses or not; image omics characteristics of the image omics data are extracted; according to the recurrence condition data and various artificial intelligence classification methods, processing the image omics characteristics of the sample object and the clinical data of the sample object, and obtaining an image prediction model corresponding to the image omics characteristics and a clinical prediction model corresponding to the clinical data; integrating the image prediction model and the clinical prediction model to obtain a multi-modal processing model; and predicting the osteosarcoma recurrence risk of the target object according to the image prediction model, the clinical prediction model and the multi-modal processing model.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and particularly to a multi-modal osteosarcoma data processing method and system based on artificial intelligence. Background Art

[0002] Osteosarcoma is the most common primary malignant bone tumor, which occurs frequently in adolescents. Patients often show swelling, pain, and bone destruction in the limbs and joints. The current treatment options for osteosarcoma include extensive surgical resection and neoadjuvant chemotherapy, and the 5-year survival rate has increased to 65%. However, a considerable number of patients still cannot obtain long-term benefits from the existing treatment options. These patients often experience local recurrence and lung metastasis. Recurrence significantly reduces the prognosis of patients. Even for patients with locally recurrent osteosarcoma who receive standard treatment, the median recurrence-free survival (RFS) is only 1.6 - 1.8 years. It is worth noting that more than 30% of osteosarcoma patients experience recurrence within the first year after surgery, a phenomenon known as early recurrence (ER), which accounts for 60% of all recurrent cases. Although tumor necrosis induced by neoadjuvant chemotherapy can be used as a partial predictor of early recurrence, it can only be evaluated through postoperative pathological analysis, thus having certain limitations in terms of convenience and accuracy. Tumor characteristics such as tumor size, pathological fracture, and genomic changes also cannot comprehensively capture the heterogeneity of primary osteosarcoma, further limiting its clinical application. This highlights the urgent need to develop new biomarkers for predicting the risk of early recurrence, real-time monitoring of treatment effects, and guiding individualized treatment strategies for osteosarcoma patients.

[0003] Artificial intelligence radiomics is an emerging non-invasive diagnostic method that extracts high-dimensional quantitative data from medical images by applying artificial intelligence to explore the associations between imaging features and lesion features, treatment responses, and prognoses, and has been studied in tumors. Currently, some studies have begun to explore the potential of artificial intelligence radiomics in predicting the response of osteosarcoma to neoadjuvant chemotherapy. However, there is still no method or system for predicting early recurrence of osteosarcoma using artificial intelligence radiomics. In addition, there is still no multi-modal artificial intelligence machine learning system based on the combination of radiomics and other clinical or pathological features to evaluate the risk of early recurrence of osteosarcoma. Therefore, the existing systems for evaluating the risk of early metastasis of osteosarcoma still require invasive operations, are inconvenient to use, have poor specificity, low sensitivity, incomplete evaluation, low accuracy, and high costs. Summary of the Invention

[0004] To this end, this application discloses the following technical solutions:

[0005] The first aspect of this application provides a multi-modal osteosarcoma data processing method based on artificial intelligence, including:

[0006] Model construction stage:

[0007] Obtain radiomics data, clinical data, and recurrence status data of multiple sample objects. The radiomics data includes T1-weighted images and T2-weighted images. The field of view of the T1-weighted images is the same as that of the T2-weighted images on the corresponding plane. The clinical data includes epidemiological data and tumor-related data. The recurrence status data indicates whether the osteosarcoma of the corresponding sample object recurs;

[0008] Extract the radiomics features of the radiomics data of the sample object;

[0009] According to the recurrence status data and multiple artificial intelligence classification methods, respectively process the radiomics features of the sample object and the clinical data of the sample object to obtain an image prediction model corresponding to the radiomics features and a clinical prediction model corresponding to the clinical data;

[0010] Integrate the image prediction model and the clinical prediction model to obtain a multimodal processing model;

[0011] Data processing stage:

[0012] Obtain the radiomics data and clinical data of the target object;

[0013] According to the image prediction model, the clinical prediction model, and the multimodal processing model, process the radiomics data and clinical data of the target object to obtain the processing result of the target object, and the processing result characterizes the recurrence risk of the osteosarcoma of the target object.

[0014] Optionally, the extracting the radiomics features of the radiomics data of the sample object includes:

[0015] Perform image segmentation processing on the radiomics data of the sample object to obtain the radiomics data of the lesion area in the radiomics data of the sample object;

[0016] Extract the first-order features, texture features, and shape features of the radiomics data of the lesion area, and perform feature transformation processing on the first-order features and texture features of the radiomics data of the lesion area to obtain the transformed features of the radiomics data of the lesion area;

[0017] Based on the intraclass correlation coefficient, screen the first-order features, texture features, shape features, and transformed features of the radiomics data of the lesion area to obtain the radiomics features of the radiomics data of the sample object.

[0018] Optionally, according to the recurrence status data and multiple artificial intelligence classification methods, process the radiomics features of the sample object to obtain an image prediction model corresponding to the radiomics features, including:

[0019] Process the radiomics features of the sample object according to the recurrence situation data and the random forest method to obtain a first imaging model corresponding to the radiomics features;

[0020] Process the radiomics features of the sample object according to the recurrence situation data and the support vector machine to obtain a second imaging model corresponding to the radiomics features;

[0021] Process the radiomics features of the sample object according to the recurrence situation data and the logistic regression method to obtain a third imaging model corresponding to the radiomics features;

[0022] Process the radiomics features of the sample object according to the recurrence situation data and the decision tree to obtain a fourth imaging model corresponding to the radiomics features;

[0023] Process the radiomics features of the sample object according to the recurrence situation data and the gradient boosting tree to obtain a fifth imaging model corresponding to the radiomics features;

[0024] Fuse the first imaging model, the second imaging model, the third imaging model, the fourth imaging model, and the fifth imaging model to obtain an imaging prediction model corresponding to the radiomics features.

[0025] Optionally, process the clinical data of the sample object according to the recurrence situation data and various artificial intelligence classification methods to obtain a clinical prediction model corresponding to the clinical data, including:

[0026] Screen the clinical data of the sample object based on the log-rank test algorithm and the univariate analysis algorithm to obtain the first screened clinical data;

[0027] Screen the first screened clinical data based on the multivariate analysis algorithm to obtain the second screened clinical data;

[0028] Process the second screened clinical data according to the recurrence situation data and the random forest method to obtain a first clinical model corresponding to the clinical data;

[0029] Process the second screened clinical data according to the recurrence situation data and the support vector machine to obtain a second clinical model corresponding to the clinical data;

[0030] Process the second screened clinical data according to the recurrence situation data and the logistic regression method to obtain a third clinical model corresponding to the clinical data;

[0031] Process the second screened clinical data according to the recurrence situation data and the decision tree to obtain a fourth clinical model corresponding to the clinical data;

[0032] Processing the second screened clinical data according to the recurrence situation data and the gradient boosting tree to obtain a fifth clinical model corresponding to the clinical data;

[0033] Fusing the first clinical model, the second clinical model, the third clinical model, the fourth clinical model, and the fifth clinical model to obtain a clinical prediction model corresponding to the clinical data.

[0034] Optionally, integrating the image prediction model and the clinical prediction model to obtain a multimodal processing model, including at least one of the following:

[0035] Connecting the image prediction model and the clinical prediction model in series to obtain a multimodal processing model, wherein the input of the clinical prediction model includes the output of the image prediction model, and the output of the clinical prediction model is used as the output of the multimodal processing model;

[0036] Connecting the image prediction model and the clinical prediction model in parallel to obtain a multimodal processing model, wherein the output of the multimodal processing model is determined according to the output of the clinical prediction model and the output of the image prediction model.

[0037] The second aspect of the present application provides an artificial intelligence-based multimodal osteosarcoma data processing system, including:

[0038] An acquisition module, configured to acquire radiomics data, clinical data, and recurrence situation data of a plurality of sample objects, where the radiomics data includes T1-weighted images and T2-weighted images, the field of view of the T1-weighted images is the same as that of the T2-weighted images on the corresponding plane, the clinical data includes epidemiological data and tumor-related data, and the recurrence situation data indicates whether the osteosarcoma of the corresponding sample object recurs;

[0039] An extraction module, configured to extract radiomics features of the radiomics data of the sample object;

[0040] A construction module, configured to process the radiomics features of the sample object and the clinical data of the sample object according to the recurrence situation data and various artificial intelligence classification methods to obtain an image prediction model corresponding to the radiomics features and a clinical prediction model corresponding to the clinical data;

[0041] An integration module, configured to integrate the image prediction model and the clinical prediction model to obtain a multimodal processing model;

[0042] The acquisition module is configured to acquire radiomics data and clinical data of a target object;

[0043] A prediction module, configured to process the radiomics data and clinical data of the target object according to the imaging prediction model, the clinical prediction model, and the multimodal processing model, to obtain a processing result of the target object, where the processing result represents the osteosarcoma recurrence risk of the target object.

[0044] Optionally, when the extraction module extracts the radiomics features of the radiomics data of the sample object, it is configured to:

[0045] Perform image segmentation processing on the radiomics data of the sample object to obtain the radiomics data of the lesion area in the radiomics data of the sample object;

[0046] Extract the first-order features, texture features, and shape features of the radiomics data of the lesion area, and perform feature transformation processing on the first-order features and texture features of the radiomics data of the lesion area to obtain the transformed features of the radiomics data of the lesion area;

[0047] Based on the intraclass correlation coefficient, screen the first-order features, texture features, shape features, and transformed features of the radiomics data of the lesion area to obtain the radiomics features of the radiomics data of the sample object.

[0048] Optionally, when the construction module processes the radiomics features of the sample object according to the recurrence situation data and multiple artificial intelligence classification methods to obtain an imaging prediction model corresponding to the radiomics features, it is configured to:

[0049] Process the radiomics features of the sample object according to the recurrence situation data and the random forest method to obtain a first imaging model corresponding to the radiomics features;

[0050] Process the radiomics features of the sample object according to the recurrence situation data and the support vector machine to obtain a second imaging model corresponding to the radiomics features;

[0051] Process the radiomics features of the sample object according to the recurrence situation data and the logistic regression method to obtain a third imaging model corresponding to the radiomics features;

[0052] Process the radiomics features of the sample object according to the recurrence situation data and the decision tree to obtain a fourth imaging model corresponding to the radiomics features;

[0053] Process the radiomics features of the sample object according to the recurrence situation data and the gradient boosting tree to obtain a fifth imaging model corresponding to the radiomics features;

[0054] Fuse the first imaging model, the second imaging model, the third imaging model, the fourth imaging model, and the fifth imaging model to obtain an imaging prediction model corresponding to the radiomics features.

[0055] Optionally, when the construction module processes the clinical data of the sample object according to the recurrence situation data and multiple artificial intelligence classification methods to obtain a clinical prediction model corresponding to the clinical data, it is used for:

[0056] Screen the clinical data of the sample object based on the log-rank test algorithm and the univariate analysis algorithm to obtain the first screened clinical data;

[0057] Screen the first screened clinical data based on the multivariate analysis algorithm to obtain the second screened clinical data;

[0058] Process the second screened clinical data according to the recurrence situation data and the random forest method to obtain a first clinical model corresponding to the clinical data;

[0059] Process the second screened clinical data according to the recurrence situation data and the support vector machine to obtain a second clinical model corresponding to the clinical data;

[0060] Process the second screened clinical data according to the recurrence situation data and the logistic regression method to obtain a third clinical model corresponding to the clinical data;

[0061] Process the second screened clinical data according to the recurrence situation data and the decision tree to obtain a fourth clinical model corresponding to the clinical data;

[0062] Process the second screened clinical data according to the recurrence situation data and the gradient boosting tree to obtain a fifth clinical model corresponding to the clinical data;

[0063] Fuse the first clinical model, the second clinical model, the third clinical model, the fourth clinical model, and the fifth clinical model to obtain a clinical prediction model corresponding to the clinical data.

[0064] Optionally, when the integration module integrates the imaging prediction model and the clinical prediction model to obtain a multimodal processing model, it is used for at least one of the following:

[0065] Connect the imaging prediction model and the clinical prediction model in series to obtain a multimodal processing model, where the input of the clinical prediction model includes the output of the imaging prediction model, and the output of the clinical prediction model is used as the output of the multimodal processing model;

[0066] Connect the image prediction model and the clinical prediction model in parallel to obtain a multi-modal processing model, where the output of the multi-modal processing model is determined according to the output of the clinical prediction model and the output of the image prediction model.

[0067] This solution integrates radiomics data and clinical data, extracts features reflecting the pathological conditions of the subject from these data, constructs an artificial intelligence-based multi-modal processing model based on the extracted features, and then uses this multi-modal processing model to predict the early recurrence risk of osteosarcoma patients. This solution requires no invasive operations, is simple and convenient, can identify patients at high recurrence risk, provide them with individualized treatment plans, and thus improve the treatment effect of patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0069] Figure 1 is a flowchart of a multi-modal osteosarcoma data processing method based on artificial intelligence provided by an embodiment of the present application;

[0070] Figure 2 is a flowchart of a method for obtaining radiomics features provided by an embodiment of the present application;

[0071] Figure 3 is a schematic diagram of a verification result provided by an embodiment of the present application;

[0072] Figure 4 is a schematic diagram of another verification result provided by an embodiment of the present application;

[0073] Figure 5 is a schematic diagram of the structure of a multi-modal osteosarcoma data processing system based on artificial intelligence provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0074] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0075] This embodiment provides a multi-modal osteosarcoma data processing method based on artificial intelligence. Please refer to Figure 1, the method may include the following steps.

[0076] S101, obtaining radiomics data, clinical data, and recurrence data of multiple sample objects. The radiomics data includes T1-weighted images and T2-weighted images, and the field of view of the T1-weighted images is consistent with that of the T2-weighted images on the corresponding plane. The clinical data includes epidemiological data and tumor-related data, and the recurrence data indicates whether the osteosarcoma of the corresponding sample object recurs.

[0077] S102, extracting radiomics features of the radiomics data of the sample object.

[0078] S103, respectively processing the radiomics features of the sample object and the clinical data of the sample object according to the recurrence data and multiple artificial intelligence classification methods, to obtain an imaging prediction model corresponding to the radiomics features and a clinical prediction model corresponding to the clinical data.

[0079] S104, integrating the imaging prediction model and the clinical prediction model to obtain a multimodal processing model.

[0080] Steps S101 to S104 can be regarded as the model construction stage of the processing method of this embodiment. The model construction stage can be executed only once, that is, after obtaining the multimodal processing model, the above model construction stage can no longer be executed, and the subsequent data processing stage can be directly executed based on the multimodal processing model.

[0081] S105, obtaining the radiomics data and clinical data of the target object.

[0082] S106, processing the radiomics data and clinical data of the target object according to the imaging prediction model, the clinical prediction model, and the multimodal processing model, to obtain the processing result of the target object, and the processing result characterizes the recurrence risk of the osteosarcoma of the target object.

[0083] Steps S105 and S106 can be regarded as the data processing stage of the processing method of this embodiment.

[0084] The beneficial effects of this embodiment are as follows:

[0085] This solution integrates radiomics data and clinical data, extracts features reflecting the pathological conditions of the object from these data, and constructs an artificial intelligence-based multimodal processing model based on the extracted features, and then uses the multimodal processing model to predict the early recurrence risk of osteosarcoma patients. This solution does not require invasive operations, is simple and convenient, can identify patients with high recurrence risk, and provide personalized treatment plans for them, thereby improving the treatment effect of patients.

[0086] In step S101, the radiomics data and clinical data of the sample object can be obtained in the following manner.

[0087] Collect the imaging data and clinical data of osteosarcoma patients from multiple medical centers. Each osteosarcoma patient can be regarded as a sample object. The imaging data of the osteosarcoma patient is used as the radiomics data of the sample object, and the clinical data of the osteosarcoma patient is used as the clinical data of the sample object. Optionally, the data of osteosarcoma patients treated with neoadjuvant chemotherapy can be collected as the data of the above sample objects.

[0088] The imaging data of a sample object may include non-enhanced magnetic resonance imaging (MRI) of the tumor site of the sample object. For any sample object, when obtaining its imaging data, axial, coronal, and sagittal fat-suppressed T2-weighted (FST2W) images can be scanned and acquired as the T2-weighted images in S101; at the same time, coronal T1-weighted (T1W) images of parts such as the shoulder or sagittal T1W images of parts such as the knee are scanned as the T1-weighted images in S101. The field of view (FOV) of the T1-weighted image is kept consistent with the T2-weighted image on the corresponding plane during the image acquisition process. The slice thickness of all images can be standardized to 3.0 millimeters (mm).

[0089] The epidemiological data of the sample object may include data such as gender, age, height, weight, body mass index (BMI), and previous smoking history. The tumor-related data may include data such as tumor size, distribution site, pathological results, number of tumors, and synchronous chemotherapy information.

[0090] For each sample object with osteosarcoma, after treating the osteosarcoma of the sample object, it can be monitored whether the sample object relapses within a certain period (such as within one year or within two years) to obtain the relapse situation data of the sample object.

[0091] In step S102, please refer to Figure 2 , the radiomics data of each sample object can be processed in the following manner to obtain the radiomics features of the radiomics data of each sample object:

[0092] S201, perform image segmentation processing on the radiomics data of the sample object to obtain the radiomics data of the lesion area in the radiomics data of the sample object.

[0093] S202, extract the first-order features, texture features, and shape features of the radiomics data of the lesion area, and perform feature transformation processing on the first-order features and texture features of the radiomics data of the lesion area to obtain the transformed features of the radiomics data of the lesion area.

[0094] S203, based on the intraclass correlation coefficient, screen the first-order features, texture features, shape features, and transformed features of the radiomics data of the lesion area to obtain the radiomics features of the radiomics data of the sample object.

[0095] In step S201, the radiomics data can be transmitted to any workstation with image segmentation function, and the research software running on the workstation that can segment lesions in MRI images is used to process the MRI images included in the radiomics data, so as to determine the lesion area (which can also be called the region of interest, ROI) in the radiomics data, and obtain the radiomics data located within the lesion area as the radiomics data of the lesion area.

[0096] When executing S201, the research software for segmentation can be configured to ensure the consistency of the lesion areas segmented in the T1-weighted image and the T2-weighted image.

[0097] In step 202, the radiomics data of the lesion area extracted in S101 can be processed based on any existing feature extraction software library (such as the PyRadiomics library) to obtain the initial features of the radiomics data of the lesion area.

[0098] Among them, the initial features can include first-order features, texture features, and shape features. Exemplarily, in S202, 18 first-order features, 75 texture features, and 17 shape features of the radiomics data of the lesion area can be extracted.

[0099] In S202, feature transformation processing can also be performed on the above initial features. The feature transformation processing can be implemented based on any one or more mathematical transformation methods. For example, any one or more of the mathematical transformation methods such as square, square root, logarithm, exponent, and wavelet transformation can be applied to process the above initial features to obtain transformed features.

[0100] As some examples, after performing feature transformation processing on 18 first-order features, 75 texture features, and 17 shape features, 2452 transformed features can be obtained, among which 1226 transformed features are obtained by processing the radiomics data of the lesion area of the T1-weighted image, and 1226 transformed features are obtained by processing the radiomics data of the lesion area of the T2-weighted image.

[0101] In S203, the intraclass correlation coefficient (ICC) of each of the first-order features, texture features, shape features, and transformed features can be calculated, and then the features with an intraclass correlation coefficient less than the intraclass correlation coefficient threshold are deleted, and only the features with an intraclass correlation coefficient greater than or equal to the intraclass correlation coefficient threshold are retained. The intraclass correlation coefficient threshold can be set as needed. Exemplarily, it can be set to 0.8. The calculation method of the intraclass correlation coefficient can refer to the relevant existing technologies and will not be elaborated here.

[0102] After obtaining the features with the intra-class correlation coefficient greater than or equal to the intra-class correlation coefficient threshold, the least absolute shrinkage and selection operator (LASSO) method in the "glmnet" package of R language can be used to further screen these features with the intra-class correlation coefficient greater than or equal to the intra-class correlation coefficient threshold to reduce the redundancy and collinearity between features.

[0103] Then, the cross-validation of the screened features can be performed through the "sklearn" package in Python to determine the optimal number of features K. Then, the features screened by the "glmnet" package are sorted in descending order according to the ICC, and only the top K features are retained. Finally, these K features retained are used as the radiomics features of S203.

[0104] As an example, the radiomics features obtained in this embodiment may include 16 features that are most significant for the recurrence situation among the foregoing 18 first-order features, 75 texture features, 17 shape features, and 2452 transformation features.

[0105] In some embodiments, only the transformation features in S203 can be screened. For example, only the foregoing 2452 transformation features are screened, and the first-order features, texture features, and shape features obtained in S202 are not screened. Correspondingly, the radiomics features obtained after screening may not include first-order features, texture features, and shape features.

[0106] In step S103, on the one hand, the radiomics features of the sample object can be processed according to the recurrence situation data and various artificial intelligence classification methods to obtain an image prediction model corresponding to the radiomics features.

[0107] On the other hand, the clinical data of the sample object can be processed according to the recurrence situation data and various artificial intelligence classification methods to obtain a clinical prediction model corresponding to the clinical data.

[0108] Optionally, the various artificial intelligence classification methods may include random forest (RF), support vector machine (SVM), logistic regression (LR), decision tree (DT), and gradient boosting tree (GBT).

[0109] Processing the radiomics features of the sample object according to the recurrence situation data and various artificial intelligence classification methods to obtain an image prediction model corresponding to the radiomics features may include:

[0110] Processing the radiomics features of the sample object according to the recurrence situation data and the random forest method to obtain a first image model corresponding to the radiomics features;

[0111] Processing the radiomics features of the sample object according to the recurrence situation data and the support vector machine to obtain a second image model corresponding to the radiomics features;

[0112] Process the radiomics features of the sample objects according to the recurrence data and the logistic regression method to obtain a third imaging model corresponding to the radiomics features;

[0113] Process the radiomics features of the sample objects according to the recurrence data and the decision tree to obtain a fourth imaging model corresponding to the radiomics features;

[0114] Process the radiomics features of the sample objects according to the recurrence data and the gradient boosting tree to obtain a fifth imaging model corresponding to the radiomics features;

[0115] Fuse the first imaging model, the second imaging model, the third imaging model, the fourth imaging model and the fifth imaging model to obtain an imaging prediction model corresponding to the radiomics features.

[0116] Before obtaining the model, the data corresponding to multiple sample objects can be divided into a training group and a test group according to a ratio of 2:1.

[0117] Exemplarily, assuming that the radiomics features and recurrence data of 300 sample objects are obtained according to the foregoing method, the radiomics features and their recurrence data of 200 sample objects can be used as the training group, and the radiomics features and their recurrence data of the remaining 100 sample objects can be used as the test group. Then, the radiomics features in the training group can be processed according to the above method to obtain the corresponding imaging model.

[0118] For the specific process of obtaining the first imaging model by processing the radiomics features of the sample objects according to the recurrence data and the random forest method, reference can be made to the relevant descriptions of the random forest method in the prior art.

[0119] For the specific process of obtaining the second imaging model by processing the radiomics features of the sample objects according to the recurrence data and the support vector machine, reference can be made to the relevant descriptions of the support vector machine in the prior art.

[0120] For the specific process of obtaining the third imaging model by processing the radiomics features of the sample objects according to the recurrence data and the logistic regression method, reference can be made to the relevant descriptions of the logistic regression method in the prior art.

[0121] For the specific process of obtaining the fourth imaging model by processing the radiomics features of the sample objects according to the recurrence data and the decision tree, reference can be made to the relevant descriptions of the decision tree in the prior art.

[0122] For the specific process of obtaining the fifth imaging model by processing the radiomics features of the sample objects according to the recurrence data and the gradient boosting tree, reference can be made to the relevant descriptions of the gradient boosting tree in the prior art.

[0123] In the process of obtaining the above imaging models, the recurrence data of the sample objects can be used as the classification labels of the sample objects, and the sample objects can be divided into two categories: recurrent samples and non-recurrent samples. The radiomics features of the sample objects can be used as the input data for the above-mentioned various artificial intelligence classification methods. Each imaging model obtained by the above method can receive the radiomics features of any object as input and output the classification result of the object, and this classification result represents whether the osteosarcoma of the object will recur after treatment.

[0124] After obtaining the above first to fifth imaging models, the five models can be fused in any of the following ways to obtain an imaging prediction model (denoted as R-model).

[0125] In the first fusion method, the accuracies of the first to fifth imaging models are evaluated based on the test group, and then the model with the highest accuracy is selected as the imaging prediction model.

[0126] Taking the first imaging model as an example, the way to evaluate the accuracy of the first imaging model can be to process the radiomics features of each sample object in the test group with the first imaging model to obtain the first classification result, compare the first classification result with the recurrence data of each sample object in the test group, count the number of consistent results between the first classification result and the recurrence data, and divide the statistical result by the total number of sample objects in the test group. The obtained ratio is used as the accuracy of the first imaging model. The evaluation methods for the accuracies of other imaging models are the same.

[0127] In the second fusion method, the accuracies of the first to fifth imaging models are evaluated based on the test group, and the accuracy of each model is used as the weight of the model. The weights of the five models are denoted as S1 to S5 in sequence, and then the first to fifth imaging models are fused according to the weights to obtain the imaging prediction model.

[0128] In this case, the classification results output by the first to fifth imaging models can be represented by 0 or 1, where 0 indicates no recurrence and 1 indicates recurrence. When using the imaging prediction model to process the radiomics features of a patient, the radiomics features are processed with the first to fifth imaging models respectively to obtain five classification results corresponding to the five models, denoted as P1 to P5 in sequence. Combining the weights and the classification results, the parameter P0 is calculated. The calculation method is: P0 = S1 * P1 + S2 * P2 + S3 * P3 + S4 * P4 + S5 * P5. After obtaining P0, if P0 is greater than or equal to 0.5, it is determined that the imaging classification result output by the imaging prediction model is recurrence; if P0 is less than 0.5, it is determined that the imaging classification result output by the imaging prediction model is no recurrence.

[0129] In the process of obtaining the above first to fifth imaging models, the five-fold cross-validation method can be used to optimize the hyperparameters involved in the process based on the divided test group to obtain a more accurate imaging model. The specific implementation of the five-fold cross-validation method can refer to the relevant existing technologies and will not be elaborated here.

[0130] Optionally, according to the recurrence situation data and various artificial intelligence classification methods to process the clinical data of the sample object, a clinical prediction model corresponding to the clinical data is obtained, including:

[0131] Screen the clinical data of the sample object based on the log-rank test algorithm and the univariate analysis algorithm to obtain the first screened clinical data;

[0132] Screen the first screened clinical data based on the multivariate analysis algorithm to obtain the second screened clinical data;

[0133] Process the second screened clinical data according to the recurrence situation data and the random forest method to obtain the first clinical model corresponding to the clinical data;

[0134] Process the second screened clinical data according to the recurrence situation data and the support vector machine to obtain the second clinical model corresponding to the clinical data;

[0135] Process the second screened clinical data according to the recurrence situation data and the logistic regression method to obtain the third clinical model corresponding to the clinical data;

[0136] Process the second screened clinical data according to the recurrence situation data and the decision tree to obtain the fourth clinical model corresponding to the clinical data;

[0137] Process the second screened clinical data according to the recurrence situation data and the gradient boosting tree to obtain the fifth clinical model corresponding to the clinical data;

[0138] Fuse the first clinical model, the second clinical model, the third clinical model, the fourth clinical model and the fifth clinical model to obtain the clinical prediction model (C-model) corresponding to the clinical data.

[0139] When obtaining the first screened clinical data, the log-rank test algorithm and the univariate analysis algorithm can be used to screen the clinical data of each sample object to obtain the first screened clinical data corresponding to each sample object. Among them, when using the univariate analysis algorithm for screening, the clinical data with the corresponding probability value (P-value) less than 0.1 can be retained as the first screened clinical data.

[0140] The specific principles of the log-rank test algorithm and the univariate analysis algorithm can refer to the relevant existing technologies.

[0141] When obtaining the second screened clinical data, the first screened clinical data can be further screened through a multivariate analysis algorithm to obtain the second screened clinical data. When using the multivariate analysis algorithm for screening, the clinical data with a corresponding probability value (P-value) less than 0.05 can be retained as the second screened clinical data.

[0142] Before obtaining the model, the data corresponding to multiple sample objects can be divided into a training group and a test group according to a ratio of 2:1. The division method can refer to the aforementioned method for obtaining the imaging prediction model and will not be elaborated here.

[0143] For the specific process of obtaining the first clinical model by processing the second screened clinical data of the sample objects according to the recurrence situation data and the random forest method, reference can be made to the relevant descriptions of the random forest method in the prior art.

[0144] For the specific process of obtaining the second clinical model by processing the second screened clinical data of the sample objects according to the recurrence situation data and the support vector machine, reference can be made to the relevant descriptions of the support vector machine in the prior art.

[0145] For the specific process of obtaining the third clinical model by processing the second screened clinical data of the sample objects according to the recurrence situation data and the logistic regression method, reference can be made to the relevant descriptions of the logistic regression method in the prior art.

[0146] For the specific process of obtaining the fourth clinical model by processing the second screened clinical data of the sample objects according to the recurrence situation data and the decision tree, reference can be made to the relevant descriptions of the decision tree in the prior art.

[0147] For the specific process of obtaining the fifth clinical model by processing the second screened clinical data of the sample objects according to the recurrence situation data and the gradient boosting tree, reference can be made to the relevant descriptions of the gradient boosting tree in the prior art.

[0148] In the process of obtaining the above clinical models, the recurrence situation data of the sample objects can be used as the classification labels of the sample objects, classifying the sample objects into two categories: recurrent samples and non-recurrent samples. The second screened clinical data of the sample objects can be used as the input data for the above various artificial intelligence classification methods. Each clinical model obtained by the above method can receive the second screened clinical data of any object as the input and output the classification result of the object, and this classification result represents whether the osteosarcoma of the object will recur after treatment.

[0149] In the process of obtaining the above first to fifth clinical models, the method of five-fold cross-validation can be used to optimize the hyperparameters involved in the process based on the divided test group to obtain a more accurate clinical model. The specific implementation method of the five-fold cross-validation method can refer to the relevant prior art.

[0150] For the method of integrating the first clinical model, the second clinical model, the third clinical model, the fourth clinical model, and the fifth clinical model to obtain the clinical prediction model, reference can be made to the method of integrating the first to fifth imaging models to obtain the imaging prediction model in the foregoing embodiments.

[0151] Optionally, integrating the imaging prediction model and the clinical prediction model to obtain a multimodal processing model, including at least one of the following:

[0152] Integration method 1: Connect the imaging prediction model and the clinical prediction model in series to obtain a multimodal processing model. Among them, the input of the clinical prediction model includes the output of the imaging prediction model, and the output of the clinical prediction model serves as the output of the multimodal processing model;

[0153] Integration method 2: Connect the imaging prediction model and the clinical prediction model in parallel to obtain a multimodal processing model. Among them, the output of the multimodal processing model is determined according to the output of the clinical prediction model and the output of the imaging prediction model.

[0154] In integration method 1, an image classification result can be added to the input data of the clinical prediction model to connect the imaging prediction model and the clinical prediction model in series.

[0155] The method of making a prediction using the multimodal processing model obtained according to integration method 1 is as follows:

[0156] Use the imaging prediction model in the multimodal processing model to process the radiomics features of the patient to obtain the classification result of the imaging prediction model, and then input the classification result of the imaging prediction model and the second-screened clinical data into the clinical prediction model in the multimodal processing model. The classification result output by the clinical prediction model in the multimodal processing model is used as the classification result of the multimodal processing model.

[0157] Optionally, it can also be connected in series in another way, that is, placing the clinical prediction model before the imaging prediction model. The output of the clinical prediction model serves as the input of the imaging prediction model, and the output of the imaging prediction model serves as the output of the multimodal processing model.

[0158] In this way, a neural network model capable of updating radiomics features, denoted as the image update model, can be trained first using the training set in combination with the clinical prediction model and the imaging prediction model obtained in the foregoing manner. The image update model can obtain the classification result of the clinical prediction model and the radiomics features of any object, and then update the radiomics features based on the classification result of the clinical prediction model and output the updated radiomics features. In this way, the classification result of the clinical prediction model is integrated into the radiomics features.

[0159] Then, the clinical prediction model, the imaging update model, and the imaging prediction model can be connected in series in sequence to obtain a multimodal processing model.

[0160] After being connected in series in the above manner, for any object whose recurrence needs to be predicted, the second-screened clinical data of the object can be processed by the clinical prediction model in the multimodal processing model to obtain the classification result of the clinical prediction model, and the radiomics features of the object and the classification result of the clinical prediction model can be processed by the imaging update model to obtain the updated radiomics features of the object. Finally, the updated radiomics features of the object can be processed by the imaging prediction model in the multimodal processing model, and the classification result output by the imaging prediction model in the multimodal processing model is used as the classification result of the multimodal processing model.

[0161] In integration method two, the accuracy of the imaging prediction model and the clinical prediction model can be evaluated according to the test group, and the accuracy is used as the weight of the model. The imaging prediction model and the clinical prediction model are connected in parallel according to this weight to form a multimodal processing model. The method for evaluating the accuracy can refer to the foregoing embodiments.

[0162] The method of prediction by the multimodal processing model obtained according to integration method two is as follows:

[0163] The accuracy of the imaging prediction model is denoted as L1, and the accuracy of the clinical prediction model is denoted as T2. The radiomics features of the patient are processed by the imaging prediction model in the multimodal processing model to obtain the classification result T1 of the imaging prediction model, and the second-screened clinical data of the patient are processed by the clinical prediction model in the multimodal processing model to obtain the classification result T2 of the clinical prediction model. The T0 value of the patient is calculated according to the formula T0 = L1 * T1 + L2 * T2. If this value is greater than or equal to 0.5, it is determined that the classification result of the multimodal processing model is recurrence. If this value is less than 0.5, it is determined that the classification result of the multimodal processing model is non-recurrence.

[0164] Among them, both T1 and T2 are integers of 0 or 1, where 0 indicates non-recurrence and 1 indicates recurrence.

[0165] In step S105, the target object can be any osteosarcoma patient after treatment. The method for obtaining the radiomics data and clinical data of the target object can refer to the method for obtaining the radiomics data and clinical data of the sample object.

[0166] In S106, the radiomics data and clinical data of the target object can be processed in the same way as the radiomics data and clinical data of the sample object in the foregoing embodiments to obtain the radiomics features and the second screened clinical data of the target object. Then, the radiomics data of the target object is processed by the imaging prediction model to obtain an imaging classification result, the second screened clinical data of the target object is processed by the clinical prediction model to obtain a clinical classification result, and the radiomics data and clinical data of the target object are processed by the multimodal processing model to obtain a multimodal classification result of the target object. Finally, the processing result of the target object is determined by combining the imaging classification result, the clinical classification result, and the multimodal classification result.

[0167] The manner of determining the processing result of the target object by combining the imaging classification result, the clinical classification result, and the multimodal classification result may be:

[0168] Count the number of classification results indicating recurrence and the number of classification results indicating non-recurrence among the three classification results, and determine the classification result with the majority as the processing result.

[0169] For example, if two or three classification results indicate recurrence, the processing result is determined to be recurrence; if two or three classification results indicate non-recurrence, the processing result is determined to be non-recurrence.

[0170] Optionally, after obtaining the above imaging prediction model, clinical prediction model, and multimodal processing model, the model performance of the above models in the training group and the test group can be evaluated through the receiver operating characteristic (ROC) curve and the area under the curve (AUC), and whether to fine-tune the model can be determined based on the evaluation results. At the same time, the sensitivity, specificity, and accuracy of each of the above models can be used to provide a comprehensive evaluation of the model performance.

[0171] Further, the clinical net benefit at different prediction thresholds can be quantified through decision curve analysis (DCA). It mainly focuses on one-year recurrence (ER), and at the same time includes three-year follow-up data to further verify the prediction accuracy of the above models.

[0172] Optionally, one or two best models can also be determined from the imaging prediction model, clinical prediction model, and multimodal processing model based on the DCA curve, and Kaplan-Meier (KM) survival analysis and Log-Rank test are performed on the best models. Statistical analysis can be carried out using Python software and R software.

[0173] Further, in order to better confirm the universality and generalization ability of the model, internal and external cohorts can be used to evaluate various machine learning metrics.

[0174] For example, Kaplan-Meier survival analysis can be used to detect the prognostic differences between high-risk and low-risk patients in an imaging prediction model in internal and external cohorts.

[0175] The receiver operating characteristic curve (AUC) and C-index can be used to verify the prediction specificity and accuracy of imaging prediction models, clinical prediction models, and multimodal processing models in internal and external cohorts.

[0176] In internal and external cohorts, calibration curves can be applied to verify the prediction accuracy and reliability of imaging prediction models, clinical prediction models, and multimodal processing models for 12-month and 24-month progression-free survival (PFS).

[0177] In internal and external cohorts, decision curve analysis (DCA) can be used to evaluate the robustness of imaging prediction models, clinical prediction models, and multimodal processing models in clinical practice.

[0178] Exemplarily, the evaluation results obtained in the above manner can be referred to Figure 3 and Figure 4 , from Figure 3 It can be seen that the DCA curve shows that the imaging prediction model obtained based on the random forest method and the multimodal processing model obtained based on the random forest method have the best effect in evaluating the early recurrence of osteosarcoma. Among them, the multimodal processing model obtained based on the random forest method consists of an imaging prediction model obtained based on the random forest method and a clinical prediction model obtained based on the random forest method.

[0179] The method provided in this embodiment, based on the imaging, clinical, and pathological data of osteosarcoma patients, applies the method of artificial intelligence machine learning to establish a prediction model for the early recurrence of osteosarcoma, so as to estimate the recurrence risk of osteosarcoma patients at an early stage. The invention helps to guide the clinical medication strategy of osteosarcoma patients, can improve the quality of life of patients as early as possible, reduce the burden on families and society, and at the same time can also save the medical costs and treatment time of patients, and has extremely high clinical application value.

[0180] The method of this embodiment uses the non-enhanced tumor MRI examination image data most commonly used by clinical osteosarcoma patients, combined with the clinical and pathological data of the patients for integrated analysis. It is a non-invasive, convenient and easy-to-implement common technical means, with low acquisition and use costs, excellent clinical application prospects, and is generally applicable to medical units at all levels.

[0181] Moreover, this embodiment uses the latest multiple artificial intelligence machine learning algorithms to construct a prediction model, avoiding the errors in manual screening. Compared with traditional inspection and detection analysis methods, its accuracy and credibility are higher.

[0182] In summary, the technology of the present invention can provide an important reference for the selection of precise treatment plans for clinical patients with osteosarcoma lung metastasis, guide the clinical medication of patients, and ultimately improve the prognosis of osteosarcoma patients.

[0183] The embodiment of the present application also provides a multi-modal osteosarcoma data processing system based on artificial intelligence. Please refer to Figure 5 This system may include the following modules.

[0184] An acquisition module 501, configured to acquire radiomics data, clinical data, and recurrence data of multiple sample objects. The radiomics data includes T1-weighted images and T2-weighted images, and the field of view of the T1-weighted image is consistent with the field of view of the T2-weighted image on the corresponding plane. The clinical data includes epidemiological data and tumor-related data, and the recurrence data indicates whether the osteosarcoma of the corresponding sample object recurs;

[0185] An extraction module 502, configured to extract radiomics features of the radiomics data of the sample object;

[0186] A construction module 503, configured to process the radiomics features of the sample object and the clinical data of the sample object according to the recurrence data and multiple artificial intelligence classification methods, to obtain an image prediction model corresponding to the radiomics features and a clinical prediction model corresponding to the clinical data;

[0187] An integration module 504, configured to integrate the image prediction model and the clinical prediction model to obtain a multi-modal processing model;

[0188] The acquisition module 501 is configured to acquire radiomics data and clinical data of a target object;

[0189] A prediction module 505, configured to process the radiomics data and clinical data of the target object according to the image prediction model, the clinical prediction model, and the multi-modal processing model, to obtain a processing result of the target object, and the processing result characterizes the recurrence risk of the osteosarcoma of the target object.

[0190] Optionally, when the extraction module 502 extracts the radiomics features of the radiomics data of the sample object, it is configured to:

[0191] Perform image segmentation processing on the radiomics data of the sample object to obtain radiomics data of the lesion area in the radiomics data of the sample object;

[0192] Extract the first-order features, texture features, and shape features of the radiomics data of the lesion area, and perform feature transformation processing on the first-order features and texture features of the radiomics data of the lesion area to obtain transformed features of the radiomics data of the lesion area;

[0193] Screen the first-order features, texture features, shape features, and transformation features of the radiomics data of the lesion area based on the intraclass correlation coefficient to obtain the radiomics features of the radiomics data of the sample object.

[0194] Optionally, when the construction module 503 processes the radiomics features of the sample object according to the recurrence situation data and multiple artificial intelligence classification methods to obtain the imaging prediction model corresponding to the radiomics features, it is used for:

[0195] Process the radiomics features of the sample object according to the recurrence situation data and the random forest method to obtain the first imaging model corresponding to the radiomics features;

[0196] Process the radiomics features of the sample object according to the recurrence situation data and the support vector machine to obtain the second imaging model corresponding to the radiomics features;

[0197] Process the radiomics features of the sample object according to the recurrence situation data and the logistic regression method to obtain the third imaging model corresponding to the radiomics features;

[0198] Process the radiomics features of the sample object according to the recurrence situation data and the decision tree to obtain the fourth imaging model corresponding to the radiomics features;

[0199] Process the radiomics features of the sample object according to the recurrence situation data and the gradient boosting tree to obtain the fifth imaging model corresponding to the radiomics features;

[0200] Fuse the first imaging model, the second imaging model, the third imaging model, the fourth imaging model, and the fifth imaging model to obtain the imaging prediction model corresponding to the radiomics features.

[0201] Optionally, when the construction module 503 processes the clinical data of the sample object according to the recurrence situation data and multiple artificial intelligence classification methods to obtain the clinical prediction model corresponding to the clinical data, it is used for:

[0202] Screen the clinical data of the sample object based on the log-rank test algorithm and the univariate analysis algorithm to obtain the first screened clinical data;

[0203] Screen the first screened clinical data based on the multivariate analysis algorithm to obtain the second screened clinical data;

[0204] Process the second screened clinical data according to the recurrence situation data and the random forest method to obtain the first clinical model corresponding to the clinical data;

[0205] Process the second screened clinical data according to the recurrence situation data and the support vector machine to obtain the second clinical model corresponding to the clinical data;

[0206] Processing the clinically relevant data after the second screening according to the recurrence data and the logistic regression method to obtain a third clinical model corresponding to the clinically relevant data;

[0207] Processing the clinically relevant data after the second screening according to the recurrence data and the decision tree to obtain a fourth clinical model corresponding to the clinically relevant data;

[0208] Processing the clinically relevant data after the second screening according to the recurrence data and the gradient boosting tree to obtain a fifth clinical model corresponding to the clinically relevant data;

[0209] Fusing the first clinical model, the second clinical model, the third clinical model, the fourth clinical model, and the fifth clinical model to obtain a clinical prediction model corresponding to the clinically relevant data.

[0210] Optionally, when the integration module 504 integrates the image prediction model and the clinical prediction model to obtain a multimodal processing model, it is used for at least one of the following:

[0211] Connecting the image prediction model and the clinical prediction model in series to obtain a multimodal processing model, where the input of the clinical prediction model includes the output of the image prediction model, and the output of the clinical prediction model is used as the output of the multimodal processing model;

[0212] Connecting the image prediction model and the clinical prediction model in parallel to obtain a multimodal processing model, where the output of the multimodal processing model is determined based on the output of the clinical prediction model and the output of the image prediction model.

[0213] For the multi-modal osteosarcoma data processing system based on artificial intelligence in this embodiment, its working principle can refer to the relevant steps in the multi-modal osteosarcoma data processing method based on artificial intelligence provided in the foregoing embodiment.

[0214] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other.

[0215] For the convenience of description, when describing the above system or device, various modules or units are described separately according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0216] As can be seen from the description of the above embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of this application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0217] Finally, it should also be noted that in this text, relational terms such as first, second, third, and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such a process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0218] The above are only the preferred embodiments of this application. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of this application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of this application.

Claims

1. A multimodal osteosarcoma data processing method based on artificial intelligence, characterized in that: include: Model building phase: Obtaining radiomics data, clinical data, and recurrence data of multiple sample subjects, wherein the radiomics data includes T1-weighted images and T2-weighted images, the field of view of the T1-weighted images is consistent with the field of view of the T2-weighted images on the corresponding plane, the clinical data includes epidemiological data and tumor-related data, and the recurrence data indicates whether the osteosarcoma of the corresponding sample subject has recurred; Extracting radiomics features of radiomics data of the sample object; According to the recurrence data and a plurality of artificial intelligence classification methods, the imaging omics features of the sample object and the clinical data of the sample object are processed respectively to obtain an imaging prediction model corresponding to the imaging omics features and a clinical prediction model corresponding to the clinical data; Integrating the image prediction model and the clinical prediction model to obtain a multimodal processing model; Data processing stage: Obtain radiomics data and clinical data of target subjects; The imaging genomics data and clinical data of the target object are processed according to the imaging prediction model, the clinical prediction model and the multimodal processing model to obtain a processing result of the target object, wherein the processing result represents the osteosarcoma recurrence risk of the target object.

2. The method according to claim 1, characterized in that The step of extracting radiomics features of the radiomics data of the sample object comprises: Performing image segmentation processing on the radiomics data of the sample object to obtain radiomics data of the lesion area in the radiomics data of the sample object; Extracting first-order features, texture features, and shape features of the imaging omics data of the lesion area, and performing feature transformation processing on the first-order features and texture features of the imaging omics data of the lesion area to obtain transformation features of the imaging omics data of the lesion area; The first-order features, texture features, shape features and transformation features of the imaging omics data of the lesion area are screened based on the intra-group correlation coefficient to obtain the imaging omics features of the imaging omics data of the sample object.

3. The method according to claim 1, characterized in that Processing the radiomics features of the sample object according to the recurrence data and multiple artificial intelligence classification methods to obtain an image prediction model corresponding to the radiomics features, including: Processing the radiomics features of the sample object according to the recurrence data and the random forest method to obtain a first imaging model corresponding to the radiomics features; Processing the radiomics features of the sample object according to the recurrence data and a support vector machine to obtain a second imaging model corresponding to the radiomics features; Processing the imaging features of the sample object according to the recurrence data and the logistic regression method to obtain a third imaging model corresponding to the imaging features; Processing the radiomics features of the sample object according to the recurrence data and the decision tree to obtain a fourth imaging model corresponding to the radiomics features; Processing the radiomics features of the sample object according to the recurrence data and the gradient boosting tree to obtain a fifth imaging model corresponding to the radiomics features; The first image model, the second image model, the third image model, the fourth image model and the fifth image model are integrated to obtain an image prediction model corresponding to the radiomics feature.

4. The method according to claim 1, characterized in that: Processing the clinical data of the sample subject according to the recurrence data and a plurality of artificial intelligence classification methods to obtain a clinical prediction model corresponding to the clinical data includes: Screening the clinical data of the sample subjects based on the log-rank test algorithm and the univariate analysis algorithm to obtain first post-screening clinical data; Screening the first post-screening clinical data based on a multivariate analysis algorithm to obtain second post-screening clinical data; Processing the second post-screening clinical data according to the recurrence data and the random forest method to obtain a first clinical model corresponding to the clinical data; Processing the second post-screening clinical data according to the recurrence data and the support vector machine to obtain a second clinical model corresponding to the clinical data; Processing the second post-screening clinical data according to the recurrence data and the logistic regression method to obtain a third clinical model corresponding to the clinical data; Processing the second post-screening clinical data according to the recurrence data and the decision tree to obtain a fourth clinical model corresponding to the clinical data; Processing the second post-screening clinical data according to the recurrence data and the gradient boosting tree to obtain a fifth clinical model corresponding to the clinical data; The first clinical model, the second clinical model, the third clinical model, the fourth clinical model and the fifth clinical model are integrated to obtain a clinical prediction model corresponding to the clinical data.

5. The method according to claim 1, characterized in that The integrating the image prediction model and the clinical prediction model to obtain a multimodal processing model comprises at least one of the following: Connecting the image prediction model and the clinical prediction model in series to obtain a multimodal processing model, wherein the input of the clinical prediction model includes the output of the image prediction model, and the output of the clinical prediction model serves as the output of the multimodal processing model; The image prediction model and the clinical prediction model are connected in parallel to obtain a multimodal processing model, wherein the output of the multimodal processing model is determined according to the output of the clinical prediction model and the output of the image prediction model.

6. A multimodal osteosarcoma data processing system based on artificial intelligence, characterized in that: include: An acquisition module, for obtaining radiomics data, clinical data and recurrence data of multiple sample objects, wherein the radiomics data includes T1-weighted images and T2-weighted images, the field of view of the T1-weighted images is consistent with the field of view of the T2-weighted images on the corresponding plane, the clinical data includes epidemiological data and tumor-related data, and the recurrence data indicates whether the osteosarcoma of the corresponding sample object has recurred; An extraction module, used for extracting radiomics features of the radiomics data of the sample object; A construction module is used to process the radiomics features of the sample object and the clinical data of the sample object respectively according to the recurrence data and multiple artificial intelligence classification methods to obtain an image prediction model corresponding to the radiomics features and a clinical prediction model corresponding to the clinical data; An integration module, used to integrate the image prediction model and the clinical prediction model to obtain a multimodal processing model; The acquisition module is used to obtain radiomics data and clinical data of the target object; A prediction module is used to process the radiomics data and clinical data of the target object according to the image prediction model, the clinical prediction model and the multimodal processing model to obtain a processing result of the target object, wherein the processing result represents the osteosarcoma recurrence risk of the target object.

7. The system according to claim 6, characterized in that When the extraction module extracts the radiomics features of the radiomics data of the sample object, it is used to: Performing image segmentation processing on the radiomics data of the sample object to obtain radiomics data of the lesion area in the radiomics data of the sample object; Extracting first-order features, texture features, and shape features of the imaging omics data of the lesion area, and performing feature transformation processing on the first-order features and texture features of the imaging omics data of the lesion area to obtain transformation features of the imaging omics data of the lesion area; The first-order features, texture features, shape features and transformation features of the imaging omics data of the lesion area are screened based on the intra-group correlation coefficient to obtain the imaging omics features of the imaging omics data of the sample object.

8. The system according to claim 6, characterized in that The construction module processes the radiomics features of the sample object according to the recurrence data and multiple artificial intelligence classification methods to obtain an image prediction model corresponding to the radiomics features, and is used to: Processing the radiomics features of the sample object according to the recurrence data and the random forest method to obtain a first imaging model corresponding to the radiomics features; Processing the radiomics features of the sample object according to the recurrence data and a support vector machine to obtain a second imaging model corresponding to the radiomics features; Processing the imaging features of the sample object according to the recurrence data and the logistic regression method to obtain a third imaging model corresponding to the imaging features; Processing the radiomics features of the sample object according to the recurrence data and the decision tree to obtain a fourth imaging model corresponding to the radiomics features; Processing the radiomics features of the sample object according to the recurrence data and the gradient boosting tree to obtain a fifth imaging model corresponding to the radiomics features; The first image model, the second image model, the third image model, the fourth image model and the fifth image model are integrated to obtain an image prediction model corresponding to the radiomics feature.

9. The system according to claim 6, characterized in that The construction module processes the clinical data of the sample object according to the recurrence data and multiple artificial intelligence classification methods to obtain a clinical prediction model corresponding to the clinical data, and is used to: Screening the clinical data of the sample subjects based on the log-rank test algorithm and the univariate analysis algorithm to obtain first post-screening clinical data; Screening the first post-screening clinical data based on a multivariate analysis algorithm to obtain second post-screening clinical data; Processing the second post-screening clinical data according to the recurrence data and the random forest method to obtain a first clinical model corresponding to the clinical data; Processing the second post-screening clinical data according to the recurrence data and the support vector machine to obtain a second clinical model corresponding to the clinical data; Processing the second post-screening clinical data according to the recurrence data and the logistic regression method to obtain a third clinical model corresponding to the clinical data; Processing the second post-screening clinical data according to the recurrence data and the decision tree to obtain a fourth clinical model corresponding to the clinical data; Processing the second post-screening clinical data according to the recurrence data and the gradient boosting tree to obtain a fifth clinical model corresponding to the clinical data; The first clinical model, the second clinical model, the third clinical model, the fourth clinical model and the fifth clinical model are integrated to obtain a clinical prediction model corresponding to the clinical data.

10. The system according to claim 6, characterized in that When the integration module integrates the image prediction model and the clinical prediction model to obtain a multimodal processing model, it is used for at least one of the following: Connecting the image prediction model and the clinical prediction model in series to obtain a multimodal processing model, wherein the input of the clinical prediction model includes the output of the image prediction model, and the output of the clinical prediction model serves as the output of the multimodal processing model; The image prediction model and the clinical prediction model are connected in parallel to obtain a multimodal processing model, wherein the output of the multimodal processing model is determined according to the output of the clinical prediction model and the output of the image prediction model.

Images