Asymmetric Multi-Level Cache for Low-Latency Order Book Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Electronic trading systems face challenges in efficiently storing and retrieving large data sets from order books due to the latency issues associated with external memory caches like DRAM, which are slow on initial access and require frequent access operations, while internal caches like BRAM are faster but inadequate for large data sets.
Innovation Solution
Implementing an asymmetric multi-level caching structure with two external DRAMs, where the first cache allocates space for actively traded price ranges using a circular buffer array and the second cache uses a sparse hash array for outlier prices, minimizing external memory accesses through efficient tile relocation based on price behavior changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If external DRAM is used to store large datasets of orders, then storage capacity is improved, but memory access delay increases
Solution Approach 1:
The external memory storage is segmented into multiple levels: a first external memory cache for actively traded price ranges and a second external memory cache for less frequently traded prices. This segmentation allows the system to store large datasets while optimizing access patterns by keeping frequently accessed data in the first cache level, reducing overall memory access delay.
Solution Approach 2:
Different quality levels of storage are assigned to different data characteristics. The first external memory cache provides high-speed access for actively traded prices (local quality optimization), while the second external memory cache handles less frequently traded prices. This local quality differentiation resolves the contradiction by matching storage performance to data access patterns.
2Ease of manufacture
If a simple hash array is used for indexing, then implementation simplicity is improved, but hash collisions increase
Solution Approach 1:
The patent transitions from a simple one-dimensional hash array to a two-dimensional hash index structure by introducing a secondary hash function. This dimensional change allows the system to maintain implementation simplicity while effectively resolving hash collisions through a multi-stage indexing approach.
Solution Approach 2:
A secondary hash function acts as an intermediary mechanism that resolves collisions in the primary hash array. When hash collisions occur, the secondary hash function provides an additional indexing dimension, ensuring reliable data retrieval without compromising implementation simplicity.
3Stability of the object's composition
If fixed cache allocation is used, then system stability is improved, but adaptability to price behavior changes deteriorates
Solution Approach 1:
The patent implements dynamic cache allocation where the first external memory cache automatically adjusts its allocation based on current price behavior and trading activity. This dynamic adaptation allows the system to maintain stability through automated rebalancing while responding to changing market conditions, resolving the contradiction between fixed stability and adaptive versatility.
Data Source
AI summary
Systems and methods are provided herein for efficient data relocation within a multi-level memory cache. In particular, the method can include (i) storing specific data structures, or tiles, in a first memory cache that contain open order data representing open orders for one of buying or selling a financial instrument at consecutive price levels that fall within a defined price range between a most aggressive price and a least aggressive price; (ii) storing multiple tiles in a second memory cache that contain open order data representing open orders for one of buying or selling the financial instrument at prices that fall outside the defined price range; and relocating tiles between the first memory cache and the second memory cache in response to changes in the open order data for one of buying or selling the financial instrument at the most aggressive price.