Delimiter-Based Messaging Structure for Market Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current messaging structures in financial systems are inefficient due to overhead from padding and the use of keys or references, straining computer systems and networks as the number of orders and trades increases.
Innovation Solution
A messaging structure using delimiters to demarcate data and repeating groups, with templates defining message types and data ordering, allowing for efficient extraction and interpretation of data without conveying meaning within the message, and enabling flexible addition, deletion, or reordering of data elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If fixed length messaging structures are used, then message format simplicity is improved, but message size efficiency deteriorates due to padding overhead
Solution Approach 1:
The message is segmented into variable-length fields separated by delimiters, allowing each field to contain only the necessary data without padding. This segmentation enables efficient message sizing while maintaining structured format through delimiter-based organization.
Solution Approach 2:
The messaging structure transitions from static fixed-length fields to dynamic variable-length fields. Each field's length adapts to the actual data content, eliminating the need for padding while allowing receivers to dynamically parse messages based on delimiter positions and field length indicators.
2Loss of information
If key-value paired structures are used, then data meaning conveyance is improved, but processing overhead increases
Solution Approach 1:
Data meanings are preliminarily defined through externally maintained symbol tables and templates rather than being embedded in each message. This preliminary action allows messages to contain only raw data values, significantly reducing processing overhead while maintaining complete data meaning through reference to the pre-established symbol tables.
Solution Approach 2:
The meaning-conveying metadata (keys, symbols, definitions) is extracted from the message body and placed in separate symbol tables. This extraction eliminates unnecessary overhead from each message while preserving data meaning through the separated symbolic references that are resolved using pre-loaded symbol tables.
3Measurement precision
If message structures include all data meaning, then data interpretation accuracy is improved, but message size increases
Solution Approach 1:
Symbol tables serve as intermediaries between the compact message data and the full meaning required for interpretation. The symbol tables store comprehensive data definitions and meanings externally, allowing messages to reference meanings through compact symbols rather than embedding complete meaning descriptions, thus maintaining interpretation accuracy while minimizing message size.
4Productivity
If templates are disseminated prior to message sending, then message processing efficiency is improved, but system complexity increases
Solution Approach 1:
Templates and symbol tables serve multiple functions: they define message structure, provide data meaning, enable validation, and support parsing. This multi-functionality consolidates what would otherwise require separate mechanisms into unified template structures, improving processing efficiency without proportionally increasing complexity.
Data Source
AI summary
Systems and methods are provided for communicating and processing market data. The market data may comprise quotes, orders, trades and/or statistics. A messaging structure allows for adding, re-ordering and/or expanding data, within the printable character set of any language. One or more delimiters are defined and used to delimit data elements within the message structure. The data is interpreted based on templates which may be disseminated prior to the sending of messages and used as an abstraction so that the meaning of data need not be conveyed in the message.