Shared File Cache Using ML Prediction and Symbolic Links
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The inefficiency in file storage systems leads to redundant copies of files, wasting resources, complicating maintenance, and risking inconsistencies due to increased data duplication and inconsistent updates across multiple storage locations.
Innovation Solution
A method and system utilizing a machine learning model to predict which files should be stored in a cloud-based cache, allowing multiple applications to access these files without creating multiple copies, thereby optimizing storage and reducing redundant data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If files are stored at multiple storage locations to enable parallel application access, then application accessibility is improved, but storage space is wasted due to redundant copies
Solution Approach 1:
The patent uses symbolic links (soft links) instead of actual file copies to enable multiple applications to access the same file. The symbolic link contains only a reference path to the original file, not the file data itself, thus providing accessibility without duplicating storage content.
Solution Approach 2:
The patent introduces a file management system that acts as an intermediary between multiple applications and the master storage. This system creates and manages symbolic links, allowing applications to access files through the link structure rather than directly copying files to multiple locations.
2Speed
If redundant copies of files are created for local deployment, then read/write operation speed is improved, but computer resources are wasted
Solution Approach 1:
The patent creates lightweight symbolic links instead of full file copies for local deployment. These symbolic links are minimal in size (containing only path information) compared to actual file copies, thus providing fast local access while consuming minimal computer resources.
Solution Approach 2:
The patent separates the file access function from file storage by using symbolic links. The link structure is segmented from the actual file data, allowing the system to maintain fast local access pointers without duplicating the heavy file content locally.
3Reliability
If multiple copies of files are maintained across different storage locations, then application availability is improved, but data consistency becomes error-prone and time-consuming to maintain
Solution Approach 1:
The patent uses symbolic links that reference the original file location rather than maintaining independent copies. This ensures that all applications access the single source of truth, eliminating data consistency issues while maintaining high availability through the link structure.
Solution Approach 2:
The patent merges multiple file access paths into a single unified storage location through symbolic links. Instead of maintaining separate file copies across different locations, the system combines access routes while preserving a single master copy, thus ensuring consistency and simplifying maintenance.
Data Source
AI summary
The present disclosure discloses the system and method for efficient file storage and management. The method includes collecting information on access of different files by different applications on a master storage. The collected information is then normalized within a predetermined range. Additionally, the method includes training a machine learning model with the normalized information to predict which of the files should be stored in a cache. The cache is ten implemented by storing predicted files in the cache. Consequently, the different applications then access one of the predicted files in the cache without making multiple copies of the predicted files.