Parallel Genome Segmentation for Scalable Sequence Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current genome sequencing analysis methods are unable to efficiently process large volumes of next-generation sequencing data, leading to prolonged analysis times and inefficiencies in computational resource use, especially when distributing the workload across multiple compute nodes.
Innovation Solution
A parallel processing system that divides the genome into evenly sized subregions, utilizing hardware processors to perform segment processing, deduplication, realignment, and genotyping in a deterministic manner, ensuring data integrity and load balancing across processors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If sequential data analysis process is used, then data integrity is maintained, but analysis time becomes excessively long and cannot keep up with population-scale sequencing
Solution Approach 1:
The genome is divided into multiple segments that can be processed in parallel across multiple compute nodes. Each segment is independently analyzed while maintaining coordination through a shared file system, enabling population-scale sequencing analysis without sacrificing data integrity.
Solution Approach 2:
A shared file system acts as an intermediary between parallel processing tasks, allowing multiple compute nodes to access and coordinate on segment data without direct communication overhead. This mediator enables efficient parallel processing while maintaining data consistency across the distributed system.
2Productivity
If computational resources are increased for faster analysis, then processing speed improves, but resource utilization efficiency decreases
Solution Approach 1:
By segmenting the genome into manageable chunks, the system can distribute work evenly across computational resources. This segmentation allows for efficient load balancing and maximizes resource utilization while maintaining high processing speeds through parallel execution.
Solution Approach 2:
The system processes segments in parallel with coordinated access patterns, allowing computational resources to be fully utilized without requiring excessive overhead for coordination. The shared file system enables partial actions on different segments to proceed simultaneously with minimal synchronization costs.
Data Source
AI summary
An apparatus includes a memory configured to store a sequence. The sequence includes an estimation of a biological sequence. The sequence includes a set of elements. The apparatus also includes a set of hardware processors. Each hardware processor is configured to implement a segment processing module. The apparatus also includes an assignment module implemented in a hardware processor. The assignment module is configured to receive the sequence from the memory, and assign each element to at least one segment from a set of segments, including, when an element maps to at least a first segment and a second segment, assigning the element to both the first segment and the second segment. The segment processing module is configured to, for each segment from a set of segments specific to that hardware processor, and substantially simultaneous with the remaining hardware processors, remove at least a portion of duplicate elements in that segment to generate a deduplicated segment. The segment processing module is further configured to reorder the elements in the deduplicated segment to generate a realigned segment that has a reduced likelihood for alignment errors.