Elastic Data Packing With Compression Profiles for Sparse Tables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Memory constraints in computing apparatuses, such as network routers, limit their performance due to the need for large tables of packet headers, policies, and instructions, where conventional software-based compression is costly and inefficient for certain usage patterns.
Innovation Solution
Implementing a hardware-level compression and decompression system using Field Programmable Gate Arrays (FPGAs) or Application Specific Integrated Circuits (ASICs) to compress and decompress data units by generating and using compression profiles that indicate which fields in a data unit carry values, allowing for the omission of non-value carrying fields and variable-length field compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional software-based compression algorithms are used to compress data before storing in memory, then memory usage is reduced, but processing overhead and complexity increase significantly
Solution Approach 1:
The patent replaces software-based compression algorithms executed by general-purpose processors with hardware-level compression circuits implemented in FPGAs or ASICs. This substitution of mechanical/software systems with dedicated hardware circuits eliminates the processing overhead and complexity associated with software execution, while achieving the same data compression objective. The hardware circuits perform compression operations natively at the circuit level, bypassing the need for software interpretation and processor overhead.
2Productivity
If additional memory and higher-speed memory are included in computing apparatuses to improve performance, then data storage capacity and processing speed increase, but device cost and physical constraints are exceeded
Solution Approach 1:
The patent changes the fundamental parameter of data representation by implementing compression at the hardware circuit level rather than storing uncompressed data. This parameter change in the data storage approach allows the same physical memory capacity to effectively store larger quantities of data, thereby improving processing speed and capacity without adding physical memory components or exceeding device size constraints.
3Quantity of substance
If general-purpose processors execute software decompression algorithms before data processing, then data can be accessed from compressed storage, but processing time and performance are degraded
Solution Approach 1:
The patent performs compression operations preliminarily at the hardware level during data storage, creating compressed data structures that are optimized for subsequent retrieval. The hardware compression circuits prepare the data in advance by encoding it in a compact format, so that when data needs to be accessed, the compressed structure can be efficiently utilized without requiring time-consuming software decompression operations. This preliminary hardware-based preparation eliminates the time loss associated with software decompression.
Data Source
AI summary
This disclosure relates to compressing and/or decompressing a group of similar data units, such as a table or queue of data units processed by a networking device or other computing apparatus. Each data unit in the group may only have values for fields in a master set. The described systems are particularly suited for hardware-level processing of groups of sparsely-populated data units, in which a large number of the data units have values for only a small number of the fields. In an embodiment, non-value carrying fields in a data unit are compressed based on a compression profile selected for the data unit. The compression profile indicates, for each master field, whether the compressed data unit includes a value for that field. Non-value carrying fields are omitted from the compressed data unit. The compression profile also permits compression of value-carrying fields using variable-width field lengths specified in the profile.