Market Data Gateway Coalescing Updates for Trading Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Electronic trading platforms face challenges in maintaining near real-time processing of market data due to high message loads, leading to delays and bottlenecks, especially during periods of high market activity.
Innovation Solution
A method for controlled market data delivery where market updates are processed and coalesced in a queue before being sent to client terminals, with the number of updates to be processed dynamically adjusted based on priority levels, allowing for efficient delivery of the most up-to-date information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If market updates are processed and delivered in real-time without filtering, then data timeliness is improved, but network bandwidth and client device processing capacity are overwhelmed
Solution Approach 1:
The system applies partial action by selectively filtering and coalescing market data updates before delivery. Instead of delivering every single update, the system delivers only the most relevant coalesced updates, reducing network bandwidth consumption while maintaining data timeliness for critical market movements.
Solution Approach 2:
The gateway device acts as an intermediary between the exchange and client devices. It receives raw market updates, processes them through coalescing logic, and delivers filtered results to clients. This intermediary function reduces the processing burden on client devices while maintaining near real-time delivery.
2Loss of information
If all market depth levels are provided to traders, then information completeness is improved, but processing load on gateways increases significantly
Solution Approach 1:
The system applies local quality by providing different levels of market depth information to different clients based on their subscription levels and needs. Full market depth is provided to premium clients who need it, while standard clients receive coalesced summaries. This resolves the contradiction by matching information completeness to actual client requirements rather than providing universal full-depth data.
Solution Approach 2:
The gateway dynamically changes the parameter of market depth granularity based on client subscriptions and market conditions. During high volatility, more detailed depth information may be provided; during normal conditions, coalesced summaries suffice. This flexible parameter adjustment maintains information quality while managing processing load.
3Productivity
If market data is coalesced before delivery, then network efficiency is improved, but data delivery latency increases
Solution Approach 1:
The system uses periodic action by implementing time-based coalescing intervals. Market updates are coalesced over short, fixed time windows (e.g., 100ms intervals) rather than being processed individually. This periodic approach balances the need for coalescing efficiency with the requirement for timely delivery, as the intervals are short enough to maintain near real-time performance.
Solution Approach 2:
The coalescing interval and aggressiveness are made dynamic rather than static. During periods of high market volatility, the system reduces coalescing intervals to minimize latency. During stable market conditions, longer intervals can be used to maximize bandwidth efficiency. This dynamic adjustment resolves the contradiction by adapting to actual market conditions.
Data Source
AI summary
A system and methods for controlled market data delivery are described. According to one example embodiment, upon detecting a preset threshold of updates in a queue at a gateway, a predetermined number of market updates may be processed before any data is sent to a client terminal. According to one method, update types may be used to control how many updates are processed before any data is sent. Also, once the number of updates to be processed is set, a market update having a higher priority update type may be used to trigger a change in the number of market updates that was set for processing.