File Path Reference Refresh for Deleted Physical Files
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud computing environments, applications continue to write data to deleted physical files due to stale references, leading to significant data loss, as the applications are unaware of the deletion and do not update the file paths, resulting in lost log data and other types of data.
Innovation Solution
Implementing a mechanism where applications periodically check the validity of physical file paths using asynchronous threads, creating new file paths and updating references when deletions are detected, ensuring data is written to valid files.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If applications continuously write data to physical files using stored file paths, then data writing efficiency is improved, but data loss increases when files are deleted
Solution Approach 1:
The system performs preliminary actions by periodically checking file validity before data writing occurs. The validity check mechanism proactively identifies deleted files and updates references in advance, preventing data loss without interrupting normal data writing operations.
Solution Approach 2:
The system implements feedback by continuously monitoring file validity through periodic checks and using this information to update file path references. When a file is detected as deleted, the system receives feedback about the invalid reference and automatically updates to a valid file, ensuring data is written to correct locations.
2Reliability
If applications periodically check file validity using separate threads, then data loss is reduced, but system complexity increases
Solution Approach 1:
The system segments the data writing process from the validity checking process by using separate threads. The main thread handles data writing operations while a dedicated thread performs validity checks, allowing both functions to operate independently and improving reliability without blocking the data writing path.
Solution Approach 2:
The validity check thread acts as an intermediary between the data writing process and the file system. It periodically samples file validity status and communicates this information back to the main writing process, enabling indirect coordination that improves reliability with minimal complexity overhead.
3Loss of substance
If applications immediately respond to file deletions, then data loss is minimized, but detection speed is reduced
Solution Approach 1:
The system uses periodic action by checking file validity at regular intervals rather than continuously or immediately upon deletion. This periodic checking balances detection speed with system resource usage, ensuring files are detected and replaced within acceptable time frames while maintaining operational efficiency.
Data Source
AI summary
In some implementations, a server device may identify a first file location for storing data associated with an application and a reference to a file path for the first file location. The server device may store a data stream associated with the application to the first file location using the reference. The server device may periodically execute first instructions associated with checking whether a valid physical file exists in the first file location by opening a connection via the file path, wherein the first instructions are executed independently of second instructions associated with storing the data stream. The server device may selectively cause the data stream to be stored in the first file location or a second file location based on determining whether the valid physical file exists in the first file location.