Biomedical Data Debiasing Framework for Faster Drug Discovery Predictions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The rapid increase in biomedical data has made data processing technologies evolve to handle big data efficiently, but existing methods consume substantial computing resources and time, especially in preprocessing and feature selection for predictive models, which are often costly and inefficient.
Innovation Solution
A machine learning (ML) framework that automates and optimizes preprocessing algorithms, combines datasets, and utilizes a parallel computing network to enhance computational capacity, reducing resource consumption and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple preprocessing routings are executed to achieve optimal results, then prediction accuracy is improved, but computational time and resource consumption increase substantially
Solution Approach 1:
The system performs preliminary action by automatically selecting and configuring the optimal preprocessing routing before the actual predictive modeling task. The ML algorithm evaluates multiple preprocessing routings in advance, identifies the best combination, and caches the results, so that when a prediction task arrives, the system can directly use the pre-determined optimal routing without executing all possible preprocessing combinations, thus saving substantial computational time while maintaining high prediction accuracy
Solution Approach 2:
The system creates copies of preprocessing configurations and caches them for reuse. Once the optimal preprocessing routing is identified through ML algorithm evaluation, this configuration is stored and replicated for subsequent similar tasks, eliminating the need to re-execute the same preprocessing evaluations and reducing computational time for repeated prediction tasks while preserving the accuracy benefits of the optimized preprocessing pipeline
2Measurement precision
If multiple preprocessing routings are executed to achieve optimal results, then prediction accuracy is improved, but computational resource consumption increases substantially
Solution Approach 1:
The system performs preliminary action by automatically selecting and configuring the optimal preprocessing routing before the actual predictive modeling task. The ML algorithm evaluates multiple preprocessing routings in advance, identifies the best combination, and caches the results, so that when a prediction task arrives, the system can directly use the pre-determined optimal routing without executing all possible preprocessing combinations, thus saving substantial computational time while maintaining high prediction accuracy
Solution Approach 2:
The system creates copies of preprocessing configurations and caches them for reuse. Once the optimal preprocessing routing is identified through ML algorithm evaluation, this configuration is stored and replicated for subsequent similar tasks, eliminating the need to re-execute the same preprocessing evaluations and reducing computational time for repeated prediction tasks while preserving the accuracy benefits of the optimized preprocessing pipeline
3Measurement precision
If manual feature selection is performed in large datasets with numerous variables, then relevant features can be identified, but the procedure becomes difficult and time-consuming to execute
Solution Approach 1:
The system implements self-service by enabling the ML algorithm to automatically perform feature selection without requiring manual intervention from biomedical scientists. The algorithm autonomously evaluates numerous variables, identifies relevant features, and integrates them into the predictive model, thereby maintaining high feature selection accuracy while completely eliminating the operational difficulty and time consumption associated with manual feature selection in large datasets
Solution Approach 2:
The system replaces the mechanical manual feature selection process with an automated ML-based system. Instead of relying on human experts to manually examine and select features from large datasets, the ML algorithm automatically performs the selection based on data-driven patterns, thereby maintaining or improving feature selection accuracy while dramatically reducing the operational burden and time required
4Reliability
If integration of multiple datasets is performed to increase variable reliability, then prediction reliability is improved, but the complexity of preprocessing and feature selection increases
Solution Approach 1:
The system applies universality by designing a unified ML-based preprocessing and feature selection framework that can handle multiple types of biomedical datasets simultaneously. The ML algorithm automatically adapts to different data sources and formats, performing consistent feature selection and integration across diverse datasets without requiring separate manual preprocessing procedures for each data type, thereby improving prediction reliability through multi-dataset integration while keeping the system complexity manageable through automation and standardization
Data Source
AI summary
Some embodiments relate to methods, systems, and frameworks for data analytics using machine learning, such as methods and systems for preprocessing of biomedical data, using machine learning, for input to a predictive model. The method may include receiving data from a data source, using at least one machine learning (ML) algorithm from a plurality of ML algorithms to obtain at least one combination of preprocessing steps, and computing an accuracy score for each of the at least one combination based on accuracy of prediction of the predictive model. The method may further include using at least one ML algorithm to optimize the feature selection of the predictive model, combining a plurality of datasets into a single dataset, and using a parallel computing network to provide a framework for executing such predictive model.