Cloud Metrics Storage Using Binary Encoding and Indexed Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing volume of metric data generated by cloud system monitoring overwhelms hardware resources and leads to poor performance in storage and processing systems.
Innovation Solution
Transform metric data into compact binary files using mapping tables to encode fields as short integer variables, merge byte arrays into binary files, and generate indexes for efficient retrieval without scanning the entire dataset.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If metric data is stored in raw format, then data integrity is maintained, but storage space consumption increases significantly
Solution Approach 1:
The patent changes the parameter representation by transforming metric data fields into encoded formats using mapping tables that assign compact integer identifiers to field names and types. This parameter transformation reduces storage space while maintaining data integrity through the reversible encoding process.
Solution Approach 2:
The patent creates a compressed copy of the metric data by transforming each field into a compact representation using mapping tables. The original data is preserved in the mapping tables while the transformed data is stored in the binary file, enabling space-efficient storage without losing information.
2Productivity
If the entire metric data dataset is scanned for retrieval, then complete data access is achieved, but processing time and system performance deteriorate
Solution Approach 1:
The patent performs preliminary action by creating indexes during the data storage process. The indexes are built in advance and stored alongside the binary data, enabling fast retrieval operations without scanning the entire dataset when queries are executed.
Solution Approach 2:
The patent extracts the retrieval function from the full data scan by introducing separate index structures that store only the necessary information for quick lookup. This extraction allows the system to retrieve data by querying the index rather than scanning the entire binary file.
3Quantity of substance
If mapping tables with field IDs and field type IDs are used, then storage compression is achieved, but data transformation complexity increases
Solution Approach 1:
The patent applies universality by using mapping tables that serve multiple functions: they define field names, data types, and encoding rules in a single structure. This multi-functional approach consolidates complexity into a reusable reference structure rather than requiring separate transformation logic for each field.
Data Source
AI summary
Methods, systems, and computer-readable storage media for receiving metric data of a cloud system periodically; transforming the metric data of each type into a byte array using mapping tables, wherein the byte array is an encoded format of the metric data, where each field of the metric data is encoded as a field ID and a field type ID that are short integer variables; merging and storing the byte arrays of multiple metric data into a binary file, wherein the binary file comprises multiple blocks with each block comprising multiple byte arrays; generating indexes for common fields of different metric data in the binary file; receiving a retrieval request requesting metric records including a common field of a particular value; determining storage locations of one or more metric records satisfying the retrieval request; and obtaining the one or more metric records from the binary file using the corresponding storage locations.