Metadata Driven File System for Data Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data management systems face challenges in efficiently managing and regulating electronic data storage due to reliance on path names, leading to excessive storage costs, complexity, and difficulty in controlling sensitive information distribution across networks.
Innovation Solution
A meta-data driven intelligent file system that uses profiles and policies to manage files, providing an abstraction layer between the file system and user storage, allowing for customizable data storage, duplication prevention, and intelligent searching, while reducing unnecessary storage through de-duplication and retention policies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If path names are used to manage files in traditional file systems, then file access and storage are simple and direct, but data duplication increases and storage efficiency decreases
Solution Approach 1:
The patent introduces a file system translator as an intermediary layer between the file system and the storage device. This translator intercepts file access requests, translates path names into unique identifiers, and manages data storage centrally. The intermediary resolves the contradiction by maintaining the simplicity of path-based access while enabling efficient deduplication and centralized control of stored data.
Solution Approach 2:
The patent creates a virtual copy of the file system namespace through the translator, which maps path names to unique identifiers without duplicating the actual data. Multiple path names can reference the same underlying data through this virtual mapping, eliminating data duplication while preserving the familiar path-based access interface for users.
2Reliability
If multiple copies of the same document are stored by different users, then each user has independent access control, but storage costs increase and data regulation becomes difficult
Solution Approach 1:
The patent merges the storage of identical documents into a single physical copy while maintaining multiple logical references through unique identifiers. The file system translator enables multiple users to access the same data through different path names without creating duplicate storage copies, thereby reducing storage costs while preserving individual access control through the mapping mechanism.
Solution Approach 2:
The unique identifier system serves multiple functions simultaneously: it acts as a deduplication key, an access control identifier, and a reference pointer. This multi-functional approach allows the system to maintain independent access control for different users while storing only one copy of each unique document, addressing both reliability and storage cost concerns.
3Productivity
If sensitive business information is transmitted electronically across networks, then communication efficiency increases, but data security and regulation become more difficult
Solution Approach 1:
The file system translator implements feedback mechanisms by monitoring and logging all file access requests, translations, and operations. This feedback system enables centralized tracking of sensitive data flows across the network, allowing administrators to regulate and audit data access while maintaining efficient electronic communication through the transparent translation layer.
4Reliability
If backup systems are expanded to handle increasing data volumes, then data protection improves, but backup systems become cumbersome and complex
Solution Approach 1:
The patent extracts the complexity of data management and deduplication from the backup system itself and places it in the file system translator layer. By handling data normalization, unique identifier assignment, and deduplication upstream in the file system, the backup system receives already-optimized data, significantly reducing its complexity while maintaining or improving data protection capabilities.
Data Source
AI summary
A system and method for managing data is provided. The system includes a network for interconnecting a plurality of computers. A data storage means is connected to the network to receive, store and transmit a plurality of files to and from the network. A plurality of computers is also connected to the network. Each computer is configured for originating and for receiving files. Each of the files has a unique identifier associated therewith. Each computer may retrieve a file from the data storage means using the unique identifier.