Ordered-Tree Bytestream Encoding for Sparse Data Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for encoding and decoding sparse data sets, such as sparse matrices, require significant memory usage and inefficient reconstruction due to the need to store extensive metadata, limiting data compression and processing efficiency.
Innovation Solution
A method and apparatus for decoding and encoding data using an ordered tree structure that intersperses node symbols and data symbols, where node symbols indicate the presence of data symbols in the tree, allowing for implicit location and reconstruction of data without explicit metadata, and utilizing different entropy decoding parameters for different symbol types.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If sparse data sets are encoded using traditional techniques, then data compression is achieved, but significant memory usage and inefficient reconstruction occur due to extensive metadata storage requirements
Solution Approach 1:
The patent extracts and eliminates the separate metadata structure from traditional sparse data encoding. By using an ordered tree where node symbols and data symbols are interspersed in a single bytestream without separate metadata sections, the invention removes the burden of storing extensive metadata while maintaining compression efficiency.
Solution Approach 2:
The invention merges the data symbols and structural information into a single integrated bytestream represented by an ordered tree. Instead of separating data and metadata into different storage structures, the ordered tree combines both functions into one unified representation, reducing memory usage and simplifying reconstruction.
2Measurement precision
If extensive metadata is stored to represent sparse data sets, then data reconstruction accuracy is maintained, but memory usage increases significantly
Solution Approach 1:
The ordered tree structure is self-descriptive, where each node symbol contains embedded information about its children nodes. The tree structure itself serves as the metadata, eliminating the need for separate metadata storage. Each node's indicator elements provide self-contained information about data symbol locations, enabling accurate reconstruction without external metadata references.
3Quantity of substance
If traditional encoding techniques are used for sparse data, then data compression is achieved, but processing efficiency during reconstruction is reduced
Solution Approach 1:
The ordered tree structure performs preliminary organization of data symbols during encoding, arranging them in a predetermined traversal order. This pre-organization enables efficient reconstruction by allowing the decoder to directly traverse the tree and retrieve data symbols in the correct sequence without complex metadata interpretation or additional processing steps.
Data Source
AI summary
There may be provided a method of decoding a received set of encoded data, the method comprising: receiving the encoded data set representing information that has been compressed; identifying from the encoded data set an ordered set of interspersed node symbols and data symbols; processing an ordered tree having a predetermined number of layers from the ordered set of identified node symbols and data symbols, wherein each node symbol comprises a set of indicator elements which indicate if a further symbol is expected later in the dataset for a branch in the tree, such that the ordered set of identified node symbols determine the tree and such that data symbols are arranged in data locations in the tree based on the indicator elements, and wherein each data symbol represents one or more values associated with the information that has been compressed; and, reconstructing the information that has been compressed using the values represented by the data symbols based on location of the data symbols in the ordered tree and using a predetermined value when the node symbols indicate that no data symbol is included in the data set corresponding to a location in the ordered tree.