Binary Object Buffer Encoding Format for Precision Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data formats, such as JSON and Protocol Buffers, are inefficient for encoding and decoding large volumes of data in JavaScript, particularly when dealing with 3D or video data, as they can lose precision for integers larger than 32 bits and are slow to decode repeated data types like strings.
Innovation Solution
A data format using a type-length-value scheme with type codes and a binary format that includes a list of values as a look-up table, allowing for efficient encoding and decoding by reusing identifiers and reducing data size through compression and precise encoding of data types like BigInt integers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If JSON or Protocol Buffers are used for encoding data, then data can be shared between different components, but precision is lost for integers larger than 32 bits and decoding speed is slow
Solution Approach 1:
The patent changes the data type parameters from standard 32-bit integers to arbitrary-precision integers (BigInt), allowing representation of integers larger than 32 bits without precision loss. This parameter change enables exact representation of large integers while maintaining encoding efficiency through the custom binary format with type-length-value scheme.
2Productivity
If repeated data types like strings are encoded using traditional formats, then data can be represented, but decoding speed is slow due to repeated processing
Solution Approach 1:
The patent performs preliminary action by creating a lookup table during encoding that stores the positions of repeated string values. During decoding, these pre-computed positions are used to quickly retrieve repeated strings without reprocessing them, significantly reducing decoding time for data with repeated patterns.
Solution Approach 2:
The patent uses copying by storing references to previously encoded string values in the lookup table. Instead of decoding repeated strings multiple times, the system copies references to the original string positions, enabling fast retrieval of repeated data through simple table lookups rather than full decoding operations.
3Measurement precision
If data is encoded in a detailed format to preserve all information, then precision is maintained, but data size increases
Solution Approach 1:
The patent extracts only the necessary information for precise representation. Instead of encoding complete string values repeatedly, it extracts and stores only the positions of strings in the lookup table. This extraction approach maintains full precision of the original data while significantly reducing the quantity of encoded data through reference-based storage.
Solution Approach 2:
The patent applies asymmetry in its encoding strategy by treating first occurrences of strings differently from repeated occurrences. First occurrences are encoded with full string data, while repeated occurrences are encoded with compact position references. This asymmetric approach optimizes the balance between precision and data size by applying different encoding methods based on occurrence pattern.
Data Source
AI summary
A computer-implemented method is provided. The computer-implemented method comprises: determining data to be encoded, the data comprising data elements that each have a corresponding type, length, and value; determining a list of values of data elements having a first type; encoding the list as a first portion of encoded data including an indication of the first type, the length of each value, and the value; encoding the plurality of data elements as a second portion of encoded data, wherein for each data element whose value is included in the list of values, the second portion includes an indication of a position of the value in the list, and wherein for each data element having a second type, the second portion includes an indication of the type, an indication of the length, and an indication of the value; and transmitting the encoded data for storage and/or decoding.