Simulation Data Compression Using Master Values and Integer Offsets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Simulation models generate vast amounts of data, particularly with millions or billions of cells, leading to challenges in efficient processing and storage due to the large size of real number data types like float and double precision data structures, which are difficult to compress without compromising precision.
Innovation Solution
A processor-implemented method that identifies a master data value to cluster similar data values, converting them into integer representations while maintaining the master value in floating point form, thereby reducing data size through compression and efficient transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If real number data types (float and double precision) are used to store simulation data, then measurement precision is maintained, but data size becomes excessively large making compression difficult
Solution Approach 1:
The patent segments simulation data into groups based on spatial proximity and similarity of values. By dividing the large dataset into smaller manageable groups, the system can apply different compression strategies to each group while maintaining overall precision. Each group is represented by a master data value and offset values, creating a hierarchical structure that reduces total data size.
Solution Approach 2:
The patent changes the parameter representation from full floating-point precision to a hybrid approach where master data values retain floating-point precision but offset values use integer representations. This parameter transformation allows significant compression while maintaining acceptable precision for engineering simulations by exploiting the fact that many cells have similar values.
2Measurement precision
If all simulation data is transmitted with full precision, then measurement precision is maintained, but transmission time increases
Solution Approach 1:
The patent segments data into groups that are transmitted together with a single master data value and multiple offset values. This segmentation reduces the total number of transmissions required compared to sending each floating-point value individually, thereby reducing transmission time while maintaining precision through the master-offset structure.
Solution Approach 2:
By changing the transmission format from all floating-point values to a hybrid master-offset format with integer representations, the patent reduces the byte size of transmitted data. This parameter change directly decreases transmission time while preserving measurement precision through the retained floating-point master values.
3Quantity of substance
If data is compressed using integer representations, then data size is reduced, but measurement precision is compromised
Solution Approach 1:
The patent segments data into groups where only the master data value retains full floating-point precision while offset values use integer representations. This segmentation strategy preserves precision for the representative value of each group while using compression for the differential values, achieving a balance between size reduction and precision maintenance.
Solution Approach 2:
The patent applies parameter changes selectively: master data values maintain their floating-point parameter representation for precision, while offset values are transformed to integer parameters for compression. This selective parameter transformation resolves the contradiction by applying compression only where it doesn't significantly impact precision.
4Measurement precision
If simulation data is processed and stored in full precision format, then measurement precision is maintained, but processing speed and storage efficiency decrease
Solution Approach 1:
The patent segments simulation data into groups with master values and offsets, creating a more compact internal representation. This segmentation reduces memory bandwidth requirements and improves cache utilization during processing, thereby increasing processing speed while maintaining precision through the master-offset structure that preserves full precision when needed.
Solution Approach 2:
By changing the storage format from all floating-point to hybrid master-offset with integer representations, the patent reduces memory footprint and improves storage efficiency. The integer offsets require less storage space and can be processed more efficiently by hardware, increasing productivity while the floating-point master values preserve measurement precision.
Data Source
AI summary
Systems and methods are provided for a processor-implemented method for compressing, storing, and transmitting simulation data. The simulation data including a set of floating point data values is received, the simulation data characterizing simulated physical properties of a physical object. A first master data value is identified from the set to cluster one or more data values from the set as a first group of data values based on a comparison between a data value of the set and the first master data value. Compressed simulation data is transmitted, where the compressed simulation data includes a floating point representation of the first master data value and integer representations of other data values of the first group of data values.