Sequencing Data Storage Control for Archive and Re-Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The large size of raw sequence data files, typically occupying 2-10 GB each, poses a challenge for long-term storage and management, especially on a population scale, due to capacity and expense constraints.
Innovation Solution
A dynamic storage solution that moves raw sequence data between an external data repository and archive storage based on analysis needs, allowing for long-term storage and re-analysis as required, with data management controlled by a processor that determines storage locations and triggers analysis tools for initial and re-analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If raw sequence data is stored in external data repository for long-term storage, then data availability for re-analysis is maintained, but storage capacity requirements and expenses increase substantially
Solution Approach 1:
The patent segments storage into two distinct locations: external data repository for actively analyzed data and archive storage for long-term retention. This segmentation allows the system to maintain data availability for re-analysis while reducing the storage capacity burden on the external repository by moving completed data to archive storage.
Solution Approach 2:
The patent introduces a data management controller as an intermediary that manages data movement between the external data repository and archive storage. This mediator coordinates the transfer of raw sequence data based on analysis status, ensuring data availability when needed while optimizing storage capacity utilization across both locations.
2Adaptability or versatility
If raw sequence data is kept in external data repository for re-analysis, then flexibility for new analysis types is maintained, but storage expenses increase
Solution Approach 1:
The patent implements dynamic data movement where raw sequence data is transferred from external data repository to archive storage based on the completion of analysis. This dynamic approach allows the system to maintain re-analysis flexibility by enabling data retrieval from archive storage when new analysis types are needed, while reducing storage costs by keeping data in the lower-cost archive location rather than continuously occupying expensive external repository space.
3Quantity of substance
If raw sequence data is moved to archive storage after initial analysis, then storage costs are reduced, but data retrieval time increases for re-analysis
Solution Approach 1:
The patent performs preliminary actions by maintaining metadata information about raw sequence data in the external data repository even after data is moved to archive storage. This preliminary indexing allows the system to quickly locate and retrieve specific datasets from archive storage based on analysis requests, reducing the effective retrieval time despite the physical data being stored in the archive location.
Data Source
AI summary
Apparatus and method of managing genomic sequencing data. In an embodiment, a data management controller is communicatively coupled to an external data repository over a communication network. The data management controller is configured to detect a first event triggering initial analysis on raw sequence data encoded in a standard file format and stored in the external data repository. The data management controller is configured to launch one or more analysis tools to perform the initial analysis on the raw sequence data stored in the external data repository, and to output initial analysis results. The data management controller is configured to determine whether the initial analysis results pass quality control, and control electronic storage of the raw sequence data when the initial analysis results pass quality control by storing the raw sequence data in archive storage, and deleting the raw sequence data from the external data repository.