Hierarchical Bitmask Compression for Sparse Serialized Fields
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression methods for bitmasks in communication systems are inefficient, leading to network congestion and resource wastage due to the overhead of bitmasks, which negatively impact communication speed and resource consumption.
Innovation Solution
A hierarchical approach for compressing and decompressing bitmasks is implemented, utilizing a representative tree structure to minimize the number of bits used, especially for sparse bitmasks, by optimizing the representation of strings of zeroes and inverting bitmasks to convert dense bitmasks to sparse ones, and adapting field ordering for improved compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If bitmasks are used to decrease the size of messages, then message size is reduced, but the overhead of bitmasks increases the size of each compressed message
Solution Approach 1:
The patent divides the bitmask into multiple segments or groups, where each segment represents a portion of the data fields. This segmentation allows the system to only transmit segments that contain relevant information (where bits are set to 1), rather than transmitting the entire bitmask structure. The hierarchical grouping enables progressive refinement of which fields are present, reducing the effective overhead by eliminating unnecessary segment delimiters and structure markers for empty segments.
Solution Approach 2:
The patent extracts and transmits only the essential information from the bitmask structure. Instead of transmitting the complete bitmask including all structural overhead and delimiters, the system extracts only the meaningful bit patterns that indicate field presence. This extraction approach removes redundant structural elements while preserving the core functionality of indicating which data fields are present in the message.
2Productivity
If bitmasks are appended to each message to indicate field presence, then data transmission efficiency is improved, but communication speed is negatively affected
Solution Approach 1:
The patent inverts the traditional bitmask approach by using a sparse representation where only the presence of fields is indicated through compact bit patterns rather than explicit delimiters and structure markers. Instead of appending a complete structured bitmask that slows down transmission, the system uses inverted logic where the absence of delimiters and structure elements for empty segments actually speeds up transmission while still providing the necessary field presence information through the bit patterns themselves.
3Reliability
If all information is passed through the network without compression, then data integrity is maintained, but network congestion occurs and resources are wasted
Solution Approach 1:
The patent applies preliminary compression using bitmasks at the source before transmission. By pre-processing the data to identify and mark only the present fields using bitmask technology, the system reduces the amount of data that needs to be transmitted through the network. This preliminary action maintains data integrity because the bitmask structure preserves the essential information about which fields are present, while significantly reducing network resource consumption by eliminating the transmission of unnecessary data elements.
Data Source
AI summary
Disclosed are systems and methods for communicating with a sender machine and a receiver machine, and analyzing a bitmask associated with a message to be sent from the sender machine to the receiver machine, such that the bitmask comprises one or more presence bits and one or more absence bits, such that the presence bits indicate that an associated field of the message are included in the message and the absence bits indicate that an associated field of the message are omitted in the message. The systems and methods also dynamically determine compression instructions to compress the bitmask, and provide the compression instructions to the sender machine, wherein the compression instructions reduce size of the bitmask associated with the message to be sent to the receiver machine.