Radiomics feature selection method and analysis device for constructing artificial intelligence model

Abstract

This radiomics feature selection method for constructing an artificial intelligence model comprises the steps in which: an analysis device calculates a statistically relevant indicator by performing univariate analysis with an output value of a learning model with respect to each of a plurality of radiomics features extracted from a medical image; the analysis device removes, from among the radiomics features, a feature for which the degree of relevance with the output value is lower than a threshold value, on the basis of the statistically relevant indicator; the analysis device removes, from among the radiomics features, a feature having an inter-feature collinearity equal to or higher than the threshold value; the analysis device additionally removes, from among the radiomics features, a feature for which the degree of relevance with the output value is lower than the threshold value, on the basis of the statistically relevant indicator; and the analysis device selects final radiomics features by applying an embedded technique to the remaining radiomics features.

Description

Radiomics feature selection method and analysis device for building artificial intelligence models

[0001] The technology described below relates to a technique for selecting significant radiomics features from medical images.

[0002] In the medical field, imaging is a key tool for disease diagnosis and prognosis. Representative medical imaging modalities include ultrasound, X-ray, computed tomography (CT), positron emission tomography (PET), and magnetic resonance imaging (MRI).

[0003] Radiomics analysis is the study of extracting quantitative and statistical disease-related features from imaging data such as ultrasound, CT, PET, and MRI, and using these features to diagnose and classify diseases. Radiomics features consist of a variety of factors categorized into several categories, including shape, intensity, texture, high-order, and filter. Hundreds or even thousands of radiomics features can be extracted from a single medical image.

[0004] Recently, applications utilizing artificial intelligence (AI), such as machine learning, have emerged in the medical field. AI models can classify lesions by receiving radiomics features extracted from medical images. As mentioned above, radiomics features consist of a large number of items. Therefore, prior to building an AI model, radiomics features relevant to a specific AI model must be selected to prevent overfitting, improve model performance, and reduce model development time and costs.

[0005] Conventional radiomics feature selection techniques include filtering, wrapper, and embedded techniques. Filtering is a statistical technique that analyzes the significance of each feature with the analysis objective (e.g., Pearson's Correlation). Wrapper techniques input a specific subset into a learning model and find the optimal subset among all combinations. Embedding techniques select the optimal subset during the learning process under certain constraints (e.g., LASSO). However, each feature selection technique has its own advantages and disadvantages. Previous studies have empirically applied feature selection techniques.

[0006] The technique described below is intended to provide an automated methodology for selecting significant radiomics features.

[0007] A method for selecting radiomics features for building an artificial intelligence model includes a step of an analysis device receiving a plurality of radiomics features extracted from a medical image as input, a step of the analysis device performing a univariate analysis with an output value of a learning model for each of the radiomics features to produce a statistical correlation index, a step of the analysis device first removing a feature among the radiomics features whose correlation with the output value is lower than a threshold value based on at least one of the statistical correlation indexes, a step of the analysis device second removing a feature among the radiomics features whose collinearity between features is higher than a threshold value after the first removal, a step of the analysis device third removing a feature among the radiomics features whose correlation with the output value is lower than a threshold value based on at least one of the statistical correlation indexes after the second removal, and a step of the analysis device applying an embedded technique to the radiomics features after the third removal to select final radiomics features.

[0008] An analysis device for selecting radiomics features includes an interface device that receives a medical image as input, a computational device that extracts a plurality of radiomics features from the medical image, and selects features for constructing a learning model among the plurality of radiomics features. The computational device (i) performs univariate analysis on each of the radiomics features with respect to an output value of a learning model to produce a statistical correlation index, (ii) firstly removes features among the radiomics features whose correlation with the output value is lower than a threshold value based on at least one of the statistical correlation indexes, (iii) secondly removes features among the radiomics features whose collinearity between features is higher than a threshold value after the first removal, (iv) thirdly removes features among the radiomics features whose correlation with the output value is lower than a threshold value based on at least one of the statistical correlation indexes, and (v) thirdly selects final radiomics features by applying an embedding technique to the radiomics features after the third removal.

[0009] The technology described below utilizes a feature selection methodology that utilizes a combination of techniques to reduce the shortcomings of individual methodologies and contribute to the construction of a high-performance learning model.

[0010] Figure 1 is an example of the process of building a learning model using radiomics features.

[0011] Figure 2 is an example of a process for selecting radiomics features.

[0012] Figure 3 is an example of a complex process for selecting radiomics features.

[0013] Figure 4 shows the radiomics feature selection process using an actual data set.

[0014] Figure 5 is an example of an analysis device that selects radiomics features.

[0015] The technology described below is susceptible to various modifications and embodiments. Specific embodiments are illustrated and described in detail in the drawings. However, this does not limit the technology described below to specific embodiments, and it should be understood that all modifications, equivalents, and alternatives fall within the spirit and scope of the technology described below.

[0016] Terms such as first, second, A, and B may be used to describe various components, but these components are not limited by these terms and are used solely to distinguish one component from another. For example, without departing from the scope of the technology described below, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0017] As used herein, the singular expressions should be understood to include the plural expressions unless the context clearly dictates otherwise, and the term "comprises" and the like should be understood to mean the presence of a described feature, number, step, operation, component, part, or combination thereof, but not to exclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0018] Before going into a detailed description of the drawings, it should be made clear that the division of components in this specification is merely a division based on the main function of each component. In other words, two or more components described below may be combined into a single component, or a single component may be further subdivided into two or more components with more detailed functions. In addition to its own main function, each component described below may additionally perform some or all of the functions of other components, and of course, some of the main functions of each component may be exclusively performed by other components.

[0019] Additionally, in performing a method or method of operation, each process constituting the method may occur in a different order than the stated order, unless the context clearly indicates a specific order. That is, each process may occur in the same order as the stated order, may be performed substantially simultaneously, or may be performed in the opposite order.

[0020] The technique described below is a technique for selecting radiomics features that are significant for specific applications, such as lesion classification in medical images.

[0021] Medical images can be any of a variety of images, including ultrasound, X-ray, CT, PET, PET-CT, MRI, and MRA.

[0022] Hereinafter, a device that selects significant radiomics features from medical images is referred to as an "analysis device." An analysis device is a computing device capable of running algorithms or models for image processing and feature selection. For example, an analysis device can be implemented as a PC, smart device, or server. Furthermore, the analysis device can also construct learning models and perform performance verification.

[0023] First, we outline the process by which the analysis device builds a learning model utilizing radiomics features. Figure 1 illustrates an example of the process (100) for building a learning model utilizing radiomics features. Radiomics analysis can typically be comprised of seven steps.

[0024] The analysis device receives medical images (110). The medical images may be any of various types of images, such as CT, PET, or MRI.

[0025] The analysis device can segment regions of interest (ROIs) from the medical image (120). This ROI segmentation can be performed manually or automatically using a deep learning-based segmentation model. Furthermore, ROI segmentation can be performed using a hybrid approach utilizing both manual and automatic techniques.

[0026] The analysis device extracts radiomics features from medical images (130). The radiomics features can be categorized into categories such as shape, intensity, texture, high-dimensionality, and filter.

[0027] The analysis device can select radiomics features using any one of a filter technique, a wrapper technique, and an embedded technique (140).

[0028] The analysis device can build a certain learning model using the selected radiomics features (150). At this time, the learning model includes logistic regression (LR), decision tree, random forest (RF), XGBoost (eXtreme Gradient Boosting), LightGBM (Light Gradient Boosting Machine), CatBoost (Categorical Boosting), KNN (K-nearest neighbor), Naive Bayes, SVM (support vector machine), artificial neural network, etc. Artificial neural network includes deep learning models such as CNN (Convolutional Neural Network).

[0029] The analysis device can perform verification on the constructed model (160). Model verification can be performed through internal verification and / or external verification.

[0030] Furthermore, the analysis device can evaluate the model's performance based on the validation results (170). The performance indicator may be at least one of the following: Area Under the ROC Curve (AUC), sensitivity, and specificity.

[0031] Figure 2 illustrates an example of a radiomics feature selection process. The specific goals of selection are to remove data noise, prevent overfitting, reduce training time and cost, and improve the performance of the learning model. Radiomics feature selection techniques can be categorized into filtering, wrapping, or embedding techniques.

[0032] Filtering techniques screen for significance between features and the desired outcome without using a separate learning algorithm. Compared to other methods, filtering techniques are simple, easy to use, and fast to compute. However, they have the disadvantage of not being able to identify collinearity between features. Filtering techniques select features by comparing a threshold value with an indicator indicating the significance of a specific feature through univariate analysis.

[0033] The wrapper technique uses a learning algorithm to evaluate performance across all possible feature subsets and select the optimal feature subset. Therefore, while the wrapper technique takes into account collinearity between features, it can be time-consuming to find the optimal combination and can be prone to overfitting.

[0034] Embedded techniques select features using penalty or tree-based learning algorithms with feature selection built into the model itself. Embedded techniques are more efficient than wrapper techniques because they don't evaluate every subset. However, they are model-dependent, meaning their results vary depending on parameter settings.

[0035] Figure 3 is an example of a complex process (200) for selecting radiomics features.

[0036] The analysis device receives radiomics features extracted from the medical image being analyzed (210). It is assumed that the analysis device has previously acquired information regarding the relationship between each feature and the output value to be predicted or classified by the learning model. The analysis device stores the radiomics features input into the initial candidate feature set. The medical image may be any of various types of images. The radiomics features extracted from the image may also vary depending on the type of medical image.

[0037] The analysis device generates a feature map for the radiomics features included in the candidate feature set (220). The feature map stores all information produced during the feature selection process for each feature. The analysis device can perform a univariate analysis on the relationship between each feature and the output value. The feature map can store the category and univariate analysis results for each feature. For example, the analysis device can store statistical correlations (e.g., p-value, AUC, odds ratio (OR), etc.) for each feature and basic information of the corresponding feature (category of the radiomics feature, name of the feature) in the feature map. The category of the radiomics feature can be classified into morphological features, intensity, texture, etc. The analysis device updates the feature map during the radiomics feature selection process, and other information is accumulated.

[0038] The analysis device performs filtering based on the likelihood of each radiomics feature included in the candidate feature set (230). The analysis device examines the basic likelihood of significance based on the relationship between the feature and the output value for each radiomics feature included in the candidate feature set (230). The analysis device can filter (remove) features with low correlation with the output value. The analysis device can determine the likelihood for each feature based on the p-value or AUC, and remove features with low likelihood from the candidate feature set. For example, the analysis device can remove features with a p-value greater than or equal to a first threshold (e.g., 0.5) or an AUC less than a second threshold (e.g., 0.7) from the candidate feature set. In other words, the analysis device can leave only features with a high correlation with the model's output value based on the p-value or AUC in the candidate feature set.

[0039] The analysis device filters features based on the collinearity between the remaining radiomics features in the candidate feature set (240). Accordingly, the analysis device can produce an index indicating the correlation between features. For example, the analysis device can calculate the absolute correlation (e.g., Pearson correlation) that considers the relationship between variables only and / or the Variance Inflation Factor (VIF) that considers the relationship between independent variables only. The analysis device can remove features from the candidate feature set that have an absolute correlation greater than or equal to a third threshold or a VIF greater than or equal to a fourth threshold. In other words, the analysis device can leave only features that have low correlations (i.e., are independent) between features based on the absolute correlation or VIF in the candidate feature set.

[0040] The analysis device can perform significance-based filtering based on the relationship between the feature and the output value for each radiomics feature remaining in the candidate feature set (250). At this time, the analysis device can select features based on the odds ratio along with the "p-value or AUC." For example, the analysis device can remove from the candidate feature set features that (i) have a p-value greater than or equal to a fifth threshold value or an AUC less than or equal to a sixth threshold value, and (ii) have an odds ratio (OR) of 1 (removing features whose variable value increases and decreases do not affect the output of the learning model). In other words, the analysis device can keep in the candidate feature set only features that have a high correlation with the output value of the learning model based on the "p-value or AUC," and at the same time, features whose variable value increases and decreases affect the learning model.

[0041] Finally, the analysis device can select the final radiomics features using an embedded technique from the remaining radiomics features in the candidate feature set (260). For example, the analysis device can select the final radiomics features using a learning model such as LASSO, EN (Elastic-Net), or RF. The analysis device can select a certain number of features ranked high in the RF classification as the final radiomics features.

[0042] The researchers verified the performance of the selection technique for the aforementioned radiomics features.

[0043] Figure 4 shows the radiomics feature selection process using an actual data set.

[0044] Dataset 1 contains radiomics features extracted from PET-CT images of patients with metabolic syndrome collected at Soonchunhyang University Seoul Hospital. Using Dataset 1, the researchers built a model to predict metabolic syndrome after bariatric surgery. Dataset 1 has a sample size of 79 individuals and consists of 854 features across 55 categories. The researchers used 70% of the entire dataset for model building and the remaining 30% for validation. The researchers used LR, RF, and SVM as learning models. The model performance was evaluated using accuracy, sensitivity, specificity, balanced accuracy (BA), and area under the curve (AUC). BA was used to determine the optimal cutoff point for the prediction model.

[0045] The analysis tool generated a feature map with p-values, AUC, and OR for each of the 854 features. The statistical values ​​for these features could be either information already included in the data set or values ​​calculated by the analysis tool through univariate analysis of the features.

[0046] The researcher selected the feature values ​​using the process described in Figure 3. At this time, the researcher set the first threshold for the p-value to 0.5 in process 230, the third threshold for the absolute correlation (COR) to 0.8 in process 240, the third threshold for the VIF to 4, the fifth threshold for the p-value to 0.2 in process 250, the sixth threshold for the AUC to 0.55, and the upper criterion to 10 in process 260.

[0047] The analyzer retained 530 features with p-values ​​< 0.5 among 854 features in the feature set. The analyzer then separately selected features based on absolute COR and VIF. The analyzer retained 211 features with absolute COR < 0.8 in feature set 1. Additionally, the analyzer retained 109 features with VIF < 4 in feature set 2. The analyzer then selected features with (i) p-values ​​< 0.2 or AUC > 0.55 and (ii) ORs other than 1. This process left 127 features in feature set 1 and 75 features in feature set 2. Finally, the analyzer selected the final radiomics features using embedded techniques such as LASSO, EN, and RF, respectively. In feature set 1, LASSO selected 43 features, EN selected 35 features, and RF selected 10 features. In feature set 2, LASSO selected 26 features, EN selected 30 features, and RF selected 10 features. RF used a top score of 10. Table 1 below shows the performance of the LR, RF, and SVM models built through cross-validation using the feature set.

[0048] Model TypeFeature SetNumber of FeaturesPerformance IndicatorAUCAccuracySensitivitySpecificityBALRCOR(Feature Set 1)LASSO430.4790.5220.6250.4670.546EN350.5580.6520.5000.7330.617RF100.8000.8260.7500.8670.808VIF(Feature Set 2)LASSO260.5080.5650.6250.533 0.579EN300.4420.6520.0001.0000.500RF100.7920.8700.6251.0000.812RFCOR(Feature Set 1)LASSO430.5870.7390.5000.8670.683EN350.5460.7390.2501.0000.625RF100.6750.6520.7500 .6000.675VIF(Feature Set 2)LASSO260.6670.8260.5001.0000.750EN300.6500.7830.5000.9330.717RF100.6620.8260.5001.0000.750SVMCOR(Feature Set 1)LASSO430.6500.8260.6250.9330.779EN 350.6500.7390.6250.8000.713RF100.6920.6520.8750.5330.704VIF(Feature Set 2)LASSO260.6920.7390.6250.8000.713EN300.7170.7830.5000.9330.717RF100.7830.8260.6250.9330.779

[0049] Looking at Table 1, the LR model showed the highest performance with an AUC of 0.8 for radiomics features selected by COR and RF, the RF model showed the highest performance with an AUC of 0.675 for radiomics features selected by COR and RF, and the SVM model showed the highest performance with an AUC of 0.783 for radiomics features selected by VIF and RF.

[0050] Figure 5 illustrates an example of an analysis device (300) that selects radiomics characteristics. The analysis device (300) may be physically implemented in various forms. For example, the analysis device (300) may take the form of a computer device such as a PC, a network server, a data processing chipset, or the like.

[0051] The analysis device (300) may include a storage device (310), a memory (320), a computing device (330), an interface device (340), a communication device (350), and an output device (360).

[0052] The storage device (310) can store medical images that are the subject of analysis of the learning model.

[0053] The storage device (310) can store a program for extracting radiomics features from medical images.

[0054] The storage device (310) can store an algorithm, program, or learning model used in the radiomics feature selection process.

[0055] The storage device (310) can store the analysis results (significant radiomics features).

[0056] The storage device (310) can store a learning model built using selected radiomics features.

[0057] The memory (320) can store data and information generated during the process of selecting radiomics features by the analysis device (300).

[0058] The interface device (340) is a device that receives certain commands and data from the outside.

[0059] The interface device (340) can receive specific medical images from a physically connected input device or an external storage device. In this case, the specific medical images may be composed of images of a patient population with a specific disease and a normal population, based on values ​​predicted by the learning model (e.g., diagnosis of a specific disease, classification of a specific lesion, prediction of a disease prognosis, etc.).

[0060] The interface device (340) may also receive a plurality of radiomics features extracted from a specific medical image.

[0061] The interface device (340) can receive threshold values, parameters, etc. applied to the radiomics feature selection process.

[0062] The interface device (340) can transmit the selected radiomics features to an external object.

[0063] A communication device (350) refers to a configuration that receives and transmits certain information through a wired or wireless network.

[0064] The communication device (350) can receive specific medical images from an external object.

[0065] The communication device (350) can receive a plurality of radiomic features extracted from a specific medical image from an external object.

[0066] The communication device (350) can receive threshold values, parameters, etc. applied to the radiomics feature selection process from external objects.

[0067] The communication device (350) may also transmit the selected radiomix characteristics to an external object such as a user terminal.

[0068] Meanwhile, the interface device (340) is meant to include an interface that transmits data or information received from the communication device (350) into the analysis device (300).

[0069] An output device (360) is a device that outputs certain information. The output device (360) can output interfaces, analysis results, etc. required for the data processing process.

[0070] The computational device (330) can extract radiomics features (candidate radiomics features) from medical images used to build a specific learning model. The computational device (330) can extract candidate radiomics features from medical images using a commercial program. At this time, the computational device (330) can determine a name and category for each extracted radiomics feature.

[0071] The computational device (330) can generate a feature map for the initial candidate radiomics features. In this process, the computational device (330) can perform a univariate analysis (regression analysis) on each candidate radiomics feature to calculate a statistical correlation between each feature and the output value (predicted value) of the learning model. For example, the computational device (330) can calculate a p-value, an AUC, an OR, etc. The computational device (330) can store statistical correlations (p-value, AUC, OR, etc.) and basic information (name and category) for each radiomics feature to generate a feature map.

[0072] The computational unit (330) performs initial filtering based on the relationship between the feature and the output value for each candidate radiomics feature. The computational unit (330) determines the likelihood of each feature based on the p-value or AUC, and can remove (filter out) features with low likelihood (primary filtering). For example, the computational unit (330) can remove features with a p-value greater than or equal to a first threshold or an AUC less than or equal to a second threshold.

[0073] Thereafter, the computational unit (330) can perform filtering based on the collinearity between the candidate radiomics features remaining as a result of the first filtering (secondary filtering). The computational unit (330) can remove features with an absolute correlation (COR) greater than or equal to the third threshold or a value-in-factor (VIF) greater than or equal to the fourth threshold.

[0074] Thereafter, the computational unit (330) can perform significance-based filtering on each of the candidate radiomics features remaining as a result of the secondary filtering based on the relationship between the output values ​​(tertiary filtering). The computational unit (330) can remove features from the candidate feature set that (i) have a p-value greater than or equal to the fifth threshold value, or an AUC less than or equal to the sixth threshold value, and (ii) have an OR of 1.

[0075] Afterwards, the computational unit (330) can select the final radiomix features by applying an embedded technique to the candidate radiomix features remaining after the third filtering. The embedded technique can select the final radiomix features using at least one of various learning models (such as LASSO, EN, RF, etc.). For example, the computational unit (330) can select a certain number of features from the RF as the final radiomix features.

[0076] The computational device (330) can construct a learning model for a specific application based on the final radiomics features.

[0077] The computing device (330) may be a device such as a processor, AP, or chip embedded with a program that processes data and performs certain operations.

[0078] Additionally, the radiomics feature selection technique described above or the method for constructing a learning model using the selected radiomics features may be implemented as a program (or application) including an executable algorithm that can be executed on a computer. The program may be stored and provided on a temporary or non-transitory computer-readable medium.

[0079] A non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the various applications or programs described above may be stored and provided on a non-transitory readable medium, such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM (read-only memory), PROM (programmable read only memory), EPROM (Erasable PROM, EPROM), EEPROM (Electrically EPROM), or flash memory.

[0080] Temporarily readable media refers to various types of RAM, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous DRAM (Synclink DRAM, SLDRAM), and Direct Rambus RAM (DRRAM).

[0081] The present embodiment and the drawings attached to the present specification only clearly illustrate a part of the technical idea included in the above-described technology, and it is obvious that all modified examples and specific embodiments that can be easily inferred by a person skilled in the art within the scope of the technical idea included in the specification and drawings of the above-described technology are included in the scope of the rights of the above-described technology.

Claims

1. A step in which the analysis device receives a plurality of radiomics features extracted from medical images; A step in which the above analysis device performs univariate analysis with the output value of the learning model for each of the above radiomics features to produce a statistically related index; A step in which the above analysis device first removes features among the radiomics features whose correlation with the output value is lower than a threshold value based on at least one of the statistically related indicators; A step in which the above analysis device secondarily removes features among the radiomics features whose collinearity between features is higher than a threshold value after the first removal; A step in which the analysis device thirdly removes features among the radiomics features whose correlation with the output value is lower than a threshold value based on at least one of the statistically related indicators after the second removal; and A method for selecting radiomics features for building an artificial intelligence model, comprising a step of selecting final radiomics features by applying an embedded technique to the radiomics features after the third removal by the above analysis device.

2. In paragraph 1, In the first removal step above The above analysis device is a radiomics feature selection method for building an artificial intelligence model that removes at least one feature among the radiomics features whose p-value is greater than or equal to a first threshold value.

3. In paragraph 1, In the above second removal step The above analysis device is a radiomics feature selection method for building an artificial intelligence model that removes at least one feature among the radiomics features whose absolute correlation is greater than or equal to a second threshold value or whose VIF (Variance Inflation Factor) is greater than or equal to a third threshold value.

4. In paragraph 1, In the above 3rd removal step The above analysis device is a radiomics feature selection method for building an artificial intelligence model that removes (i) at least one feature among the radiomics features whose p-value is greater than or equal to a fourth threshold value or whose AUC (Area Under ROC Curve) is less than or equal to a fifth threshold value, and (ii) whose odds ratio is 1 at the same time.

5. Interface device that receives medical images; A method for extracting a plurality of radiomics features from the above medical image and including a computational device for selecting features for building a learning model among the plurality of radiomics features, The above-mentioned computational device (i) performs univariate analysis on the output value of the learning model for each of the radiomics features to produce a statistical correlation index, (ii) firstly removes a feature among the radiomics features whose correlation with the output value is lower than a threshold value based on at least one of the statistical correlation indexes, (iii) secondly removes a feature among the radiomics features whose collinearity between features is higher than a threshold value after the first removal, (iv) thirdly removes a feature among the radiomics features whose correlation with the output value is lower than a threshold value based on at least one of the statistical correlation indexes, and (v) thirdly selects final radiomics features by applying an embedded technique to the radiomics features after the third removal.

6. In paragraph 5, In the first removal step above The above-mentioned computational device is an analysis device that selects radiomics features by removing at least one feature among the radiomics features whose p-value is greater than or equal to a first threshold value.

7. In paragraph 5, In the above second removal step The above-mentioned calculation device is an analysis device that selects radiomics features by removing at least one feature among the radiomics features whose absolute correlation is greater than or equal to a second threshold value or whose VIF (Variance Inflation Factor) is greater than or equal to a third threshold value.

8. In paragraph 5, In the above 3rd removal step The above-mentioned computational device is an analysis device that selects radiomics features by removing (i) at least one feature among the radiomics features whose p-value is greater than or equal to a fourth threshold value or whose AUC (Area Under ROC Curve) is less than or equal to a fifth threshold value, and (ii) whose odds ratio is 1 at the same time.

Images