Zero-byte filename telemetry for cloud storage cost reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing telemetry systems face challenges in efficiently generating, storing, and retrieving telemetry data, particularly in cloud-based data storage and retrieval systems, due to high costs, resource intensiveness, and time-consuming data retrieval processes.
Innovation Solution
The proposed solution involves using a processing device to generate telemetry data and store it in the file name of a zero-byte file, allowing the data to be retrieved efficiently by enumerating the file system's directories and files without accessing individual file contents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If telemetry data is stored in file contents of traditional storage systems, then data can be stored persistently, but data retrieval is time-consuming and resource-intensive
Solution Approach 1:
The patent extracts telemetry data from traditional file content storage and relocates it to file metadata (specifically the file name field). This extraction allows the data to be stored in a more accessible location that can be enumerated quickly without reading entire file contents, thereby reducing retrieval time and improving efficiency.
Solution Approach 2:
The patent transitions telemetry data from the content dimension (file contents) to the metadata dimension (file name/attributes). This dimensional change enables the data to be accessed through file system enumeration operations rather than requiring file content parsing, significantly improving retrieval performance.
2Ease of manufacture
If telemetry data is stored in cloud-based distributed storage systems, then data can be stored remotely, but storage costs increase proportionally with data volume
Solution Approach 1:
The patent extracts telemetry data from the file content storage model and places it in file metadata. Since metadata typically consumes minimal storage space compared to actual file contents, this extraction dramatically reduces the storage volume required for telemetry data, thereby lowering cloud storage costs while maintaining remote storage accessibility.
Solution Approach 2:
The patent changes the storage parameter from storing data in file contents (large volume) to storing data in file metadata (minimal volume). This parameter change transforms the storage requirement from proportional to data size to a fixed minimal overhead, significantly reducing storage costs.
3Adaptability or versatility
If multiple users access the same storage system simultaneously, then collaboration is enabled, but file contention issues arise
Solution Approach 1:
The patent extracts telemetry data from file contents to file metadata. Since metadata operations are separate from file content operations, multiple users can access and enumerate files simultaneously without contending for file content locks, thereby improving access reliability while maintaining multi-user capability.
4Ease of operation
If telemetry data is stored using complex storage databases with logging services, then data can be queried efficiently, but system complexity increases
Solution Approach 1:
The patent extracts telemetry data from complex database storage to simple file metadata. File system enumeration provides sufficient querying capability for telemetry data without requiring complex database logging services, thereby reducing system complexity while maintaining operational ease.
Solution Approach 2:
The patent enables the file system itself to provide the querying capability through standard enumeration operations. Instead of requiring external complex logging services, the file system's native metadata structure and enumeration functionality serve the telemetry data access needs directly, simplifying the overall system.
Data Source
AI summary
Techniques for generating and storing application telemetry data are described. An example method includes generating, by an application, a unit of telemetry data comprising metrics related to a runtime state of the application. The method also includes generating a character string comprising the metrics. The method also includes writing, by a processing device executing the application, a zero-byte file to a storage system using the character string as a file name of the zero-byte file.