Metadata Tree Backup Access via Microservice Mediator
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data restoration methods are inefficient as they require administrator intervention and lack user access control, leading to poor deduplication and accessibility of backed-up data objects.
Innovation Solution
A system that generates a metadata tree from backup data, allowing users to access and restore their own data by converting security attributes into a universal access control list, enabling efficient deduplication and compression/undeduplication of data objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is backed up using traditional linear writing methods, then backup storage is simple, but data accessibility and deduplication efficiency deteriorate
Solution Approach 1:
The backup data set is segmented into multiple data objects with individual offset tracking, allowing efficient random access without sequential scanning. Each data object maintains its own offset information, enabling parallel processing and improved deduplication efficiency while maintaining manageable structural complexity.
Solution Approach 2:
Offsets of all data objects are recorded and indexed in advance during the backup process, creating a metadata structure that enables fast random access during restoration. This preliminary indexing action eliminates the need for sequential walking during access operations.
2Ease of operation
If backup is performed under administrator authority, then data security is maintained, but user accessibility and autonomy deteriorate
Solution Approach 1:
A microservice layer acts as an intermediary between users and the backup data set. The microservice handles security authentication and authorization, verifying user credentials against security attributes before allowing access to specific data objects. This intermediary layer maintains security while enabling user autonomy.
Solution Approach 2:
Security attributes and access control lists are copied and associated with each data object in the backup, allowing users to access their own data without requiring administrator privileges. The security context is preserved alongside the data, enabling authenticated user access while maintaining security.
3Ease of operation
If absolute offsets of all data objects are recorded in advance, then data accessibility is improved, but deduplication for additional backups deteriorates
Solution Approach 1:
The offset recording mechanism is made dynamic rather than static. Offsets are recorded relative to the backup data set structure, which can change with subsequent backups. This dynamic offset management allows efficient access in the current backup while preserving the ability to perform deduplication across multiple backup iterations.
Data Source
AI summary
Embodiments are described for presenting a backed up data object to a client, in a native file format, in response to a request from the client to retrieve the requested data object. The request includes security attributes of the requested data object. If the security attributes in the request match the security attributes stored with the backed up data object, then the requested data objet is returned to the requesting client. When a backup is performed, metadata associated with each backed up data object is stored in the backup data set. When a request is received to restore a backed up data object, a metadata tree is constructed that represents the directory structure of the backed up data objects, and further includes security attributes for each data object and an offset into the backup data set where the requested data object can be found.