Compact Binary Encoding with Variable-Length Length Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data encoding schemes, such as ASN-1 and UTF8, require multiple reads to determine the length of data blocks and have limitations in encoding data of arbitrary sizes, with ASN-1 having a minimum size of three bytes and a maximum size of 2^1008 bytes, and UTF8 encoding only up to 31 bits.
Innovation Solution
A method and apparatus for encoding data of arbitrary size by dividing it into data blocks and generating length blocks with predetermined sizes, allowing the number of length blocks to be determined without examining all of them, and transmitting both blocks, with encoding rules that can encode data in a minimal number of octets, such as rules 1A through 6A, which can encode data in 1 to 2048 octets with minimal reads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If ASN-1 encoding rules are used, then data structure independence and formal description capability are improved, but the number of reads required to determine length increases and minimum encoding size becomes three bytes
Solution Approach 1:
The encoding is divided into distinct segments: a length field occupying specific bit positions (e.g., bits 7-5 for short form, or multiple octets for long form) and data fields. This segmentation allows the length to be determined by reading only the necessary number of leading octets rather than requiring multiple reads through the entire structure, thus resolving the contradiction between formal description capability and reading efficiency.
2Adaptability or versatility
If ASN-1 encoding rules are used, then data structure independence is improved, but encoding overhead increases with minimum size of three bytes
Solution Approach 1:
The encoding scheme applies different quality characteristics to different parts of the data structure. The length field uses a compact representation that adapts to the actual data size: short form length encoding for small values (reducing overhead) and long form length encoding for larger values. This local optimization reduces the minimum encoding overhead while maintaining data structure independence.
3Adaptability or versatility
If UTF8 encoding is used, then multi-byte character set encoding capability is improved, but the maximum encoding size is limited to 31 bits
Solution Approach 1:
The encoding structure employs a nested organization where a universal length field can accommodate arbitrarily large values by using multiple octets in a hierarchical manner. The length field itself can encode values beyond 31 bits by utilizing extended octet sequences, allowing the same encoding framework to handle both character data and arbitrarily large binary objects without size limitations.
4Productivity
If compact encoding is implemented with minimal reads, then processing efficiency is improved, but the ability to encode arbitrary size data may be limited
Solution Approach 1:
The encoding scheme dynamically adapts its structure based on the data size. The length field automatically adjusts its representation: using short form encoding (fewer octets) for small data sizes to maximize processing efficiency, and transitioning to long form encoding (more octets) for larger data sizes to maintain arbitrary size capability. This dynamic adaptation resolves the contradiction between processing efficiency and arbitrary size encoding.
Data Source
AI summary
A method and apparatus for encoding data of arbitrary length. Data of arbitrary size is divided into one or more data blocks. One or more length blocks are generated that include length information identifying a quantity of the one or more data blocks into which the data is divided, wherein the length information can be determined by a number of reads of the length blocks and without examining the data blocks. The length blocks and the data blocks are transmitted.