Lead Match Engine Synchronization for Distributed Trading

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Electronic trading systems face challenges in maintaining deterministic processing and synchronization of order books across geographically distributed match engines, leading to inefficiencies and potential manipulation risks.

Innovation Solution

The system designates a lead match engine for each product, synchronizing order books in real-time across multiple venues by recreating the order book state in follower match engines, minimizing latency and avoiding complex synchronization schemes, and allowing instantaneous role switching between venues.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If multiple match engines operate independently across distributed venues, then system capacity and geographic distribution are improved, but synchronization complexity and latency increase

Engineering Contradiction:

Improvesystem capacityVSAvoidsynchronization complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system divides the order book management into segments: each venue maintains its own local order book state independently, while a lead venue coordinates cross-venue synchronization. This segmentation allows parallel processing at each venue while maintaining overall consistency through the lead venue's arbitration of message ordering.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The lead match engine pre-establishes message ordering and sequencing before messages are propagated to follower venues. By determining the authoritative order of messages in advance at the lead venue, the system avoids complex real-time synchronization conflicts at follower venues, reducing overall synchronization complexity.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If complex synchronization schemes are implemented across distributed match engines, then order book consistency is improved, but processing latency increases

Engineering Contradiction:

Improveorder book consistencyVSAvoidprocessing latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The lead match engine performs preliminary ordering of messages and determines the authoritative sequence before propagation. Follower venues simply replay messages in the predetermined order without complex conflict resolution, ensuring consistency while minimizing latency.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Instead of having each venue independently determine message order and then reconcile conflicts (which would be complex and slow), the system inverts the approach: the lead venue determines the order first, and followers passively receive and replay in that predetermined order, dramatically simplifying synchronization.

Inventive Principle:

Principle #13The other way round (Inversion)

3Speed

If message ordering is not strictly enforced across distributed venues, then processing speed is improved, but deterministic processing and market integrity deteriorate

Engineering Contradiction:

Improveprocessing speedVSAvoiddeterministic processing

Core Design Contradiction:

SpeedVSReliability

Solution Approach 1:

The lead match engine establishes the definitive message order in advance before messages are distributed to follower venues. This preliminary ordering ensures that all venues process messages in the same deterministic sequence, maintaining market integrity while allowing parallel processing at each venue.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The lead match engine acts as an intermediary that arbitrates message ordering between different venues. It receives messages, determines their authoritative sequence, and propagates them to followers in that order, ensuring deterministic processing across the distributed system without requiring direct coordination between all venues.

Inventive Principle:

Principle #24Intermediary (Mediator)

4Device complexity

If lead match engine role is fixed to one venue, then synchronization simplicity is improved, but system adaptability and fault tolerance worsen

Engineering Contradiction:

Improvesynchronization simplicityVSAvoidvenue role flexibility

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The lead match engine role is dynamic rather than fixed: any venue can become the lead venue, and roles can switch between venues as needed. This dynamic assignment maintains synchronization simplicity by always having a single lead, while simultaneously providing adaptability and fault tolerance by allowing any venue to assume the lead role if needed.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS20200402170A1Deterministic Message Processing in a Distributed Network

Publication Date: 2020.12.24 CHICAGO MERCANTILE EXCHANGE INC

AI summary

Message processing is described, in which each of first and second computer systems have a message operator configured to process messages to manage a data structure having a state determined via the processing. The message operator of the first computer system is configured to reroute unprocessed messages to the message operator of the second computer system such that the message operator of the second computer system establishes an order of an input message stream and a current state of the data structure arising from the input message stream. The message operator of the second computer system is configured to, once the current state is established, send the messages of the input message stream to the message operator of the first computer system in accordance with the established order such that the current state of data structure is recreated by the message operator of the first computer system.