Hardware Range Coding With Speculative Probability Fetch
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Context-based range coders in data processing units face inefficiencies due to repeated memory access operations for retrieving probability values, leading to slowed throughput and increased memory bandwidth usage, especially when switching between applications and storing/retrieving state information.
Innovation Solution
Implementing a method where speculative probability values are retrieved in response to a single read instruction, reducing memory access frequency and using compression techniques to minimize latency associated with state information storage and retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If context-based range coder retrieves probability values through repeated memory access operations on a bit-by-bit basis, then coding accuracy is maintained, but processing throughput decreases and memory bandwidth efficiency worsens
Solution Approach 1:
The patent applies preliminary action by pre-fetching and storing multiple probability values into a local register file before they are actually needed for coding. When the range coder needs probability values for multiple bits, they are already available in the register file from a previous pre-fetch operation, eliminating the need for repeated memory access during the actual coding process.
Solution Approach 2:
The patent transitions from a one-dimensional sequential memory access model (one probability value at a time) to a multi-dimensional batch access model. By organizing probability values in a register file with multiple entries and using batch pre-fetch operations, the system accesses probability values in parallel groups rather than sequentially, effectively adding a temporal batching dimension to the access pattern.
2Measurement precision
If context-based range coder performs frequent memory read operations to retrieve probability values, then coding precision is maintained, but memory bandwidth consumption increases
Solution Approach 1:
The system performs preliminary pre-fetching of probability values into a local register file, so that when coding operations need these values, they are already available locally. This eliminates the need for frequent memory read operations while maintaining access to the precise probability values needed for accurate context-based coding.
Solution Approach 2:
The register file acts as an intermediary between the memory system and the range coder. Instead of the range coder directly accessing memory frequently, it accesses the local register file which holds pre-fetched probability values. This intermediary layer provides the precise values needed while decoupling the coding operation from frequent memory access.
3Reliability
If state information is stored and retrieved when switching between applications, then coding continuity is maintained, but latency increases
Solution Approach 1:
The system maintains state information in a persistent storage structure that can be quickly restored. When switching between applications, the previously saved state is already available in storage, allowing for rapid restoration without recomputing. The state includes context values and probability tables that are preserved and quickly reloaded into the register file.
Solution Approach 2:
The system discards the current state when switching applications and recovers the previously saved state from storage. Instead of maintaining all possible states in memory simultaneously or recomputing states, the system efficiently discards the current context and recovers the appropriate previous state, minimizing the time spent on state management during application switches.
Data Source
AI summary
A highly programmable device, referred to generally as a data processing unit, having multiple processing units for processing streams of information, such as network packets or storage packets, is described. The data processing unit includes one or more specialized hardware accelerators configured to perform acceleration for various data-processing functions. This disclosure describes examples of retrieving speculative probability values for range coding a plurality of bits with a single read instruction to a on-chip memory that stores a table of probability values. This disclosure also describes examples of storing state information used for context-coding packets of a data stream so that the state information is available after switching between data streams.