Dependency Graph Workflow Management for Bioinformatics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data analysis processes in computer systems, such as bioinformatics workflows, often require significant processing and memory resources, leading to high electrical power consumption, especially in shared systems where multiple users execute overlapping workflows, resulting in redundant computations due to changes in parameters or inputs.
Innovation Solution
A computer-implemented method that uses a dependency graph to identify and manage dependencies between analysis steps, allowing for parallel execution and tracking of existing results to avoid redundant computations, thereby reducing resource demands and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a full analysis sequence is performed again upon parameter or input update, then the desired target entity can be recreated with updated parameters, but processing resources and memory resources are wasted due to redundant computations
Solution Approach 1:
The system performs preliminary checks by querying the workflow database and traversing the dependency graph before executing the full analysis sequence. It identifies which entities already exist and which actions have been previously scheduled, allowing the system to skip redundant computations and only re-execute necessary portions of the workflow when parameters actually change.
Solution Approach 2:
The system implements feedback mechanisms by continuously monitoring entity existence status in the workflow database and tracking action scheduling status. This feedback allows the system to determine whether a full re-execution is necessary or if previously computed results can be reused, thereby reducing unnecessary power consumption from redundant computations.
2Productivity
If multiple users or systems execute overlapping workflow sequences in shared analysis systems, then system utilization is improved, but processing resource demands and memory resource demands increase substantially
Solution Approach 1:
Before executing workflow actions, the system performs preliminary queries to the workflow database to check if the desired target entity or intermediate entities already exist. It traverses the dependency graph to identify which actions have been previously scheduled or are currently executing, allowing multiple users to share computational results and avoid duplicating resource-intensive processing.
Solution Approach 2:
The system merges workflow executions by allowing multiple users to share common computational paths. When multiple workflows require the same intermediate entities or target entities, the system identifies these overlaps through dependency graph traversal and ensures that computations are performed only once, with results shared across multiple workflows, thereby reducing total processing resource demands.
3Ease of operation
If redundant requests to execute the same action are allowed, then system responsiveness is improved, but processing resources are wasted due to duplicate execution of the same analysis steps
Solution Approach 1:
The system implements feedback by tracking the scheduling status of each action in the workflow database. Before scheduling an action for execution, the system queries the database to determine if the action is already scheduled, preventing duplicate execution. This feedback mechanism maintains system responsiveness by allowing users to submit workflow requests conveniently while ensuring that redundant computations do not waste processing resources.
Solution Approach 2:
The system performs preliminary checks of the workflow database and action scheduling status before allowing workflow execution to proceed. This preliminary action prevents redundant requests from causing duplicate execution of the same analysis steps, thereby maintaining both ease of operation and effective workflow completion productivity.
Data Source
AI summary
Aspects include querying, by a processing system, a workflow database for a desired target entity. The processing system traverses a dependency graph that defines one or more entity and action sequences of a workflow that result in creating the desired target entity based at least in part on failing to locate the desired target entity in the workflow database. The workflow can be a bioinformatics workflow to analyze one or more digital representations of biological data. The processing system queries the workflow database to determine an entity existence status of each entity in the one or more entity and action sequences. The processing system schedules execution of an associated action that outputs each entity identified as non-existent based at least in part on the entity existence status and the one or more entity and action sequences to create the desired target entity.