Sequencing Data Storage Switching for Cost-Effective Re-Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The large size of raw sequence data files, typically occupying 2-10 GB each, poses a challenge for long-term storage and management, especially on a population scale, due to capacity and cost constraints.
Innovation Solution
A dynamic storage solution that moves raw sequence data between an external data repository and archive storage based on analysis needs, allowing for long-term storage and re-analysis as required, with data management controlled by a processor that determines storage location and triggers analysis tools for initial and re-analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If raw sequence data is stored in external data repository for long-term storage, then data availability for re-analysis is maintained, but storage costs and capacity requirements increase substantially
Solution Approach 1:
The patent segments data storage into two distinct locations: external data repository for actively accessed data and archive storage for long-term storage. This segmentation allows the system to maintain data availability for re-analysis while reducing the storage capacity burden on any single storage system, thereby resolving the contradiction between reliability and quantity of substance.
2Speed
If raw sequence data is kept in external data repository, then quick access for analysis is enabled, but storage expenses increase significantly
Solution Approach 1:
The patent implements a dynamic data management system that automatically moves data between external data repository and archive storage based on access patterns and analysis requirements. This dynamic approach enables quick access when data is needed while reducing storage costs when data is not actively being accessed, thereby resolving the contradiction between speed and loss of energy.
3Loss of energy
If raw sequence data is moved to archive storage, then storage costs are reduced, but data access time increases
Solution Approach 1:
The patent employs preliminary actions by pre-moving data to archive storage after initial analysis is complete, and by implementing automated data retrieval processes when re-analysis is needed. This preliminary organization and automated retrieval minimizes the actual data access time while maintaining cost-effective archive storage, thereby resolving the contradiction between loss of energy and loss of time.
Data Source
AI summary
Apparatus and method of managing genomic sequencing data. In an embodiment, a data management controller is communicatively coupled to an external data repository over a communication network. The data management controller is configured to detect a first event triggering initial analysis on raw sequence data encoded in a standard file format and stored in the external data repository. The data management controller is configured to launch one or more analysis tools to perform the initial analysis on the raw sequence data stored in the external data repository, and to output initial analysis results. The data management controller is configured to determine whether the initial analysis results pass quality control, and control electronic storage of the raw sequence data when the initial analysis results pass quality control by storing the raw sequence data in archive storage, and deleting the raw sequence data from the external data repository.