High-throughput parallel processing platform

Abstract

A market data processing device may include a single-instruction, multiple-data (SIMD) circuit adapted to perform SIMD operations and a storage medium having encoded thereon executable instructions that cause the SIMD circuit to perform financial market data processing (MDP) on data of financial market data messages. Performing the financial market data processing may include performing the SIMD operations on the data of the financial market data messages.

Description

[0001] HIGH-THROUGHPUT PARALLEL PROCESSING PLATFORM BACKGROUND

[0002] Low-latency, high-throughput data processing technology is used in a wide variety of fields, including artificial intelligence (Al), machine learning (ML), and market data processing (MDP). Users of MDP technology often implement strategies based on the most-up-to-date, or real time, market data. Such strategies may be used in a few milliseconds after receiving updated market data, in an environment in which one or two milliseconds of delay can make the difference between a winning strategy (e.g., a successful trade) and a losing strategy.

[0003] BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The accompanying drawings illustrate a number of example implementations and are a part of the specification. Together with the following description, these drawings demonstrate and explain various principles of the present disclosure.

[0005] FIG. 1A is a block diagram of an example computer system including an example market data processing (MDP) engine.

[0006] FIG. IB is a block diagram of an example computer system for generating code for a market data processing (MDP) engine.

[0007] FIG. 2A is a flowchart of an example method for performing SIMD-based financial market data processing.

[0008] FIG. 2B is a flowchart of an example method for processing fields of financial market data messages using SIMD-based hardware.

[0009] FIG. 3 is a flowchart of an example method for generating code for SIMD-based financial market data processing based on configuration data.

[0010] FIG. 4A is a block diagram illustrating a first portion of the method of FIG. 2B.

[0011] FIG. 4B is a block diagram illustrating a second portion of the method of FIG. 2B.

[0012] Throughout the drawings, identical reference characters and descriptions indicate similar, but not necessarily identical, elements. While the examples described herein are susceptible to various modifications and alternative forms, specific implementations have been shown by way of example in the drawings and will be described in detail herein. However, the example implementations described herein are not intended to be limited to the particular forms disclosed. Rather, the present disclosure covers all modifications, equivalents, and alternatives falling within the scope of the appended claims.

[0013] DETAILED DESCRIPTION

[0014] In a financial exchange, parties wishing to buy and / or sell financial instruments publicly declare to other members of the exchange their bids to buy instruments or offers to sell instruments, and following a trade may inform the other members of the exchange of the terms of the trade. Such information may include details of orders submitted to the market, quotes of prevailing bids and offers available in the market, and trades that may have been executed, and may include terms, such as the financial instrument (e.g., stock or other security, or other instruments) to which the quote or trade relates, a price or price range, a volume of the financial instrument that the party wants to trade or that was traded, and / or other information regarding a potential or concluded trade.

[0015] When such exchanges were first created, this information was shared in an open outcry system, where parties would shout quotes or conclude trades by voice and then turn in records of agreed-upon trades to a clearing house for the exchange.

[0016] Starting decades ago, an ever-growing fraction of such quote, order, and trade data has been communicated electronically. Orders are sent to an exchange electronically and order, quote, trade, and other information is distributed by the exchange electronically.

[0017] Other parties are interested in receiving this information for a variety of their own purposes. If a party is contemplating a trade, receiving bids / offers from other parties may help them with determining whether to make a bid or offer of their own, or whether to accept one of the bids or offers they receive and execute a trade. As another example, messages regarding recently executed trades may be used to track activity in an exchange, such as to publish an individual stock price or a set of stock prices, or to calculate averages or other statistics for stock prices for an exchange. In many such cases, parties, particularly traders, are looking to receive each of the updates from the exchange, such as each order, each quote, and each trade.

[0018] Exchanges therefore offer feeds to which a party can subscribe to receive the information being published by the exchange regarding quotes, trades, or other activity. Given the volume of activity in some exchanges, particularly electronic trading activity, the number of updates being published by some exchanges can be in the range of tens of millions of messages per second. For example, one feed from one popular exchange publishes peak data rates of more than 50 million trading activity messages per second. One exchange can, and often does, have additional feeds transmitting other messages and having similar volume, and there are many exchanges that each have their own feeds. In many cases, a particular financial instrument might be traded on multiple different exchanges or otherwise have updates regarding the status of the instrument that are available via multiple different feeds. Overall, the scale of messages regarding financial activity that are being published by exchanges is into the hundreds of millions of messages per second. A party that is looking to maintain a broad view of the market and ensure they know the current state of different financial instruments may need to receive and evaluate such hundreds of millions of messages per second to gain proper context and ensure they are not at a competitive disadvantage with respect to another party (e.g., a competitor in the financial markets) or are not missing out on market opportunities.

[0019] For market participants undertaking time-sensitive activities (e.g., real-time or other highspeed or high-frequency trading), the data is to be processed in the shortest time possible.

[0020] Accordingly, traders and other market participants for decades have been developing tools to electronically receive and evaluate financial market data via feeds from exchanges. Receiving the data from the exchanges involves not just receipt over a network but also provision through computer hardware to analyze it. Exchanges often publish messages with multiple different contents and formats, even over the same feed. Given the scale of messages to be communicated in a short time and cost and constraints of network bandwidth, exchanges often distribute the messages with very little contextual or descriptive information, without even labels for individual values in the messages. Instead, messages are provided that are a bare-bones stream of bits containing flags and values (or other information) that would be useless without a key informing a recipient of how to parse a message having that format. These messages include huge amounts of information, but the fields are of highly varying sizes, formats, and contents, and are often encoded in complex ways to further reduce size. In addition, there are large numbers of message types, even from the same exchange or for the same feed from the exchange. Accordingly, to receive a message, a recipient must first identify the message source and type from information that is contained in that message, then obtain the format for messages of that type from that source, and then use that format information to parse the message to obtain the values and flags (or other information) set out therein. Obtaining the values and flags may include non-trivial interpretation, such as converting encoded values back into their unencoded form or expanding flags (which might be true / false, or an indicator for one of a set of options, or the like) to be a corresponding value. Once the values are obtained for a message, the values may then need to be contextualized and / or stored, such as by retrieving a most recent prior value (e.g., from a database or other data set of past data) so as to determine whether, e.g., a price is going up or going down, how a new quote compares to a current best quote, or accumulate the effects of incremental changes, such as adding individual trade sizes to keep track of total daily trading volume. This may involve retrieving more values from a data set of information regarding each financial instrument and each exchange so as to contextualize data with things like current best quote values, last trade values, and so on. The data set may then need to be updated, such as if a new quote is a current best quote or if a trade message indicates that a trade has been completed and a recent trade price or recent trade volume is to be updated. And the received values (potentially together with contextualizing information) may need to be communicated to recipients, which may be software or hardware agents executing on a same computer doing the receiving, other agents in the same data center, other receivers across local- or wide-area private networks, the Internet, or other receivers. Such agents may wish to receive information about a status of an exchange or one or more financial instruments, or other information, once messages from an exchange have been processed. In such cases, a message would need to be generated by retrieving data to be communicated, and the message would need to be prepared (including formatted), and the recipient(s) of the message and the manner to communicate with them (e.g., an identifier for the recipient) would need to be identified, and the message would need to be sent.

[0021] Some or all of these actions are taken in response to each received message for each exchange, at a rate that reaches hundreds of millions of times per second. Time is more and more of the utmost importance in these processes, because as time passes, the messages become older and less relevant as other parties act on the messages and as newer messages replace older messages. For example, if a message indicates a party is willing to buy a specified volume of a specified financial instrument at a specified price, a party who is interested in selling that instrument at that price needs to receive, process, and respond to the message before another party beats them to the trade. As another example, a real-time risk management system needs to track market conditions as precisely as possible in order to perform accurate risk assessment and avoid potentially serious consequences for the market participant, and possibly the broader financial system.

[0022] With hundreds of millions of message processing operations needed per second, dense message formats that need interpretation, variability between message formats and between message sources, and high variation in the sizes and formats of specific contents of messages, even if the financial context of the information contained within messages and the complexity of that information were to be completely set aside (e.g., such that the meaning of the information is disregarded and the focus is instead put on the bits and bytes of the messages), the computing demands of timely processing messages of this type and scale are immense. The financial industry has therefore long investigated options for solving this computing resource problem and processing these messages at scale and speed.

[0023] Complicating this search for suitable processing speeds, though, and of high concern to market participants and regulators, is the high rate of growth in global message rates since the el ectronifi cation of markets decades ago. For example, peak message rates have risen at a rate of over 45% year over year between 2001 and 2024. This results in steeply increasing costs of processing for all market participants. Message rates are expected to continue to grow, at potentially even higher rates, as the number of trading venues and data feeds continues to grow, more granular pricing increments lead to more frequent price changes, and the trend towards higher trading volumes with lower average trade sizes continues. Processing systems need high throughput, such as a high number of messages processed per second, to process the required data at acceptable cost. Managing scale and speed while controlling cost is therefore of key importance.

[0024] The traditional approach to processing of financial market data was and is processing the data using standard software executing on a standard, generally-programmable central processing unit (CPU). Modern CPUs operate at very high speeds and can perform many operations per second. Unfortunately, though, they have many attributes that make them ill-suited to the low latency and high throughput needed in this domain. A detailed understanding of computer architectures would reveal the source and extent of the difficulty that arises in this computing task.

[0025] For example, a device may have (among other components) a CPU to perform operations on data in accordance with instructions, memory to store data and instructions, a network adapter to receive data from a network and transfer it to the CPU for processing, and communication buses on a motherboard to convey the data between these components. As different software operations are to be performed by the CPU over time, the CPU needs to locate in memory the specific executable code for those instructions, send a request over the communication bus to the hard drive or to the memory to retrieve the executable code, load the executable code into on-chip short term memory (memory cache and registers) on the CPU, and then configure the CPU to execute the retrieved executable code.

[0026] This process is time-consuming, and as a result modern CPUs continuously try to predict which section of executable code will be required next. This is termed branch prediction. When the CPU predicts that some executable code is to execute next, the CPU will try to load that code before the CPU finishes executing a previous section of code. The CPU may, however, make an incorrect prediction. In this case, significant performance costs can be incurred, as the CPU needs to stop executing already-retrieved code (which was wrongly predicted to be executed next) and instead start the retrieval process anew for the correct code, and operations on the CPU may largely cease until the correct code is retrieved and made available for execution.

[0027] While branch prediction may have success in many other areas, the structure and variability of market data messages and the variability of the data make branch prediction notoriously hard in this field. Prediction failures are common. This can mean that despite the speed requirements of market data, typical computing hardware is even slower than normal for market data.

[0028] Similar operations as the code-retrieval operations discussed above are needed to load data onto the CPU for processing using retrieved executable code, though potentially more often as each message includes additional and different data. CPUs try to identify data which is often needed and hold it in relatively small, local, short-term memory caches. Again, while this is often successful in other fields, the structure and variability of market data leads to frequent memory cache misses, again with significant performance implications and typical computing hardware operating more slowly for market data than for other fields.

[0029] It should be appreciated that the processing of market data messages includes operations to parse and interpret data as well as operations to compare the data to previous values, update datastores, and prepare responsive actions including preparing output messages, meaning potentially multiple rounds of loading executable code and loading data, for each message received. In addition, even prior to any of these operations for loading instructions and providing data to execution hardware of the CPU for processing, when data is first received over a network by a device, before it could even be available for processing by the CPU the data would need to be processed and error-checked by the network adapter, prepared for transmission within the computer, and sent from the network adapter over the communication bus to memory, from which it is available for retrieval by the CPU.

[0030] Each of these operations takes time. When compared to the speed of actually executing a single instruction with the CPU, these operations may take (relatively speaking) huge lengths of time, i.e., significant fractions of a second. Particularly if multiple software operations are to be performed in response to a received message (e.g., parse a received message to identify values therein, obtain contextualizing data, compare the received values to the contextualized data, update the data set from which the contextualized data was obtained, generate and send a new message to another recipient, or other messages noted above or not), the time needed to process a received message with a CPU can be very significant, resulting in high latencies and poor throughput.

[0031] Without the right computer architecture, the time needed to process a message can be higher than the necessary latency with which the operations must be done to avoid the message becoming outdated and any response being moot. Worse, high latency often gives rise to poor throughput, as fewer operations complete (and therefore fewer messages are processed) each second. Choosing the right computer architecture to ensure sufficient speed has therefore been a significant investment of funds, personnel, time, and other resources by the financial industry — many billions of dollars have been spent on this evaluation and selection in recent decades, with the investment growing over time and importance. Having the right computer architecture has become a principle point of remaining competitive in the industry. Financial institutions now have in-house technology departments of a size and depth of talent that rival many technology companies and are a major employer of the best scientific and engineering graduates.

[0032] The financial industry has, for example, evaluated improving the speed and scale of operations for processing of financial market data using multiple CPUs, including multiple devices in parallel that each handled at least some of the incoming messages and / or having multiple CPUs within one device. In such an operation, the load of incoming messages could be balanced between different devices and / or between different CPUs within one device. Having multiple devices or CPUs operating in parallel can improve efficiency and throughput. However, this option was still limited by the difficulties inherent to use of software executing on generally-programmable CPUs, including the delays and architecture inefficiencies noted in the immediately above.

[0033] When multi-core chips were developed for generally-programmable CPUs, efforts were also made to use the additional cores within CPUs, including by using multiple CPUs with multiple cores in multiple devices. However, though this approach is now widely used, as each core is still similar in architecture and usage to standard generally-programmable CPUs, the use of multi-core CPUs still suffered from the efficiency limitations noted above, and the approach has not kept pace with industry desires for lower latencies and higher throughputs.

[0034] Given the efficiency limitations of generally-programmable CPUs that provided difficulties in processing financial market data in a manner that would meet the latency and throughput demands of this industry, the financial industry has for the last 20 years focused on identifying alternative computing hardware that could meet these computing demands.

[0035] Graphics processing units (GPUs) have been the focus of efforts for building computing platforms to meet the resource and speed demands of processing financial market data. GPUs are special-purpose computing hardware that are designed to perform mathematical operations quickly on large volumes of data. They were initially designed to perform the mathematical transformation operations on graphics data when graphics rendering for video games or other contexts increased in scale and complexity, but their mathematical processing capabilities have been successfully applied in other data areas, such as scientific computing and artificial intelligence. The speed advantage of a GPU lies first in having hardware that is optimized for performance of certain mathematical operations and certain data types, and often having within one GPU different specialized hardware for different mathematical operations. Given the optimized hardware, GPUs are able to perform certain operations much faster than general-purpose CPUs. Another advantage of GPUs is that they often have a wide (typically wider than a standard CPU) processing pipeline that is able to process large volumes of data together in parallel, which also increases their throughput capabilities. Accordingly, some firms have attempted to use GPUs for processing of market data messages.

[0036] While GPUs can offer some advantages, they have also suffered from disadvantages. GPUs can perform some operations more quickly than traditional CPUs, but are not configured to perform other operations quickly and some GPUs may not be able to perform certain operations at all. GPUs are also largely configured for the data types needed for computer graphics (which are similar to what is often used in scientific computation) and are not designed for and not efficient at processing many of the data types used in market data processing. Also, GPUs are typically implemented as a CPU co-processor, meaning that data are communicated back and forth over a bus between the CPU and the GPU. There can be significant delays to such back-and-forth communications, which can add to latency. A significant drawback to using GPUs for market data, though, is that the computing model of a GPU is not well suited to processing the structure of data received on market data feeds. In keeping with their computer graphics processing origins, GPUs are designed to operate on large blocks of similar information that need to be processed as a batch or at a regular interval or when a condition is met (e.g., performing a task at a regular interval to update a screen at typically 50 to 100 Hertz (once every 10 to 20 milliseconds)). This mode of operation also suits many scientific computation tasks, but it does not match the needs of market data processing, where fields of many different types and lengths arrive in a sequence that can’t be accurately predicted and need to be processed at very low latency.

[0037] As a result, while GPUs have been discussed or evaluated for use in market data processing, GPUs are not in use for market data processing, though they are used in many adjacent areas of financial technology (for example, running theoretical market pricing and volatility calculations).

[0038] Given that processing units (generally-programmable CPUs or GPUs) executing software loaded into them have suffered from speed and other disadvantages, efforts in the financial industry to identify dedicated computing hardware that is able to process the scale and types of messages communicated in this industry and address the speed and scale processing needs of the industry.

[0039] Field-programmable gate arrays (FPGAs) are one type of configurable logic device in which different blocks of hardware within the FPGA can be configured to perform certain operations. FPGA engineering is very different from software development and is closely related to the field of electronic circuit design. An FPGA designer has much more flexibility to tailor the operation of the device than a programmer of a CPU or GPU, and as a result designs can be more efficiently matched to the process at hand. Also, as the name suggests, the design can be updated if necessary, and easily re-loaded into the FPGA, changing the functionality. The expertise and engineering effort required to implement an FPGA solution is typically very high versus writing software for a CPU or GPU, but for some kinds of workload a performance advantage can be obtained. Groups of FPGAs can be deployed that are configured to perform a variety of tasks surrounding financial market data messages, including having different FPGAs configured to perform processing tasks for parsing and interpreting messages, comparing to previously-received data, updating data sets, and performing responsive actions.

[0040] To increase the speed with which financial market data can be processed, engineers also sought to leverage FPGAs in novel ways to eliminate some of the delays inherent in CPU and GPU-based systems. For example, as mentioned above, to move data into a processor or GPU, the data would need to be moved from a network adapter that received it, across a communications bus to memory, and then into the CPU and / or GPU. Early FPGA accelerators suffered from the same problem, until the development of network adapters with FPGAs on the adapter. With this architecture, the FPGA could perform operations on data received by the network adapter without the data needing to cross the communications bus or reach the CPU. In some cases, the FPGAs on the network adapter could also take responsive actions, such that the network adapter was able to receive and process data, and generate and send a responsive message out over the network, without the data being sent over the communications bus of the device to the CPU. This had the ability to further reduce the number of fraction-of-a-second delays in the processing of the data and further meet the size and scale demands of the industry.

[0041] Efforts have therefore been made by the financial industry to move beyond configurable hardware and into special-purpose hardware. While the configurable hardware of FPGAs was faster than the more general-purpose hardware of CPUs and GPUs, the execution speed of FPGAs is much still slower than what could be achieved with specially-designed hardware. Application-specific integrated circuits (ASICs) are not programmable and do not execute software or instructions. Instead, the ASIC has fixed electrical components that are arranged in a manner to perform the required operations. The speed of the fixed electrical components performing these operations exceeds that achievable with CPUs, GPUs, FPGAs, or other approaches. A difficulty, however, is that they take large amounts of time and are very costly to design and manufacture, requiring highly specialized skillsets and facilities, and that (given their fixed nature) they cannot be upgraded if bugs / errors are found or improvements are to be made. As such they are only a viable option for the largest, highest performance financial firms (typically, firms engaged in High-Frequency Trading) for whom the payoff of tiny improvements in latency justifies the costs and complexities involved. Such firms have made continual investments in ASIC design and development over the last decade.

[0042] As developers have labored to find new ways to increase speed, market participants have also looked outside the device. Institutions may adjust network hardware, such as replacing traditional network devices with low-latency variants, replacing traditional cables with fiber optics or replacing fiber optics with wireless microwave links, implementing novel network architectures, and more. Or market participants may try to reduce the time of network communication by reducing the distance data must flow along a network line or eliminating routing hardware. For example, institutions looking for a competitive advantage may place their devices in “co-location” centers where they can receive a direct wireline connection to the exchange’s devices that are present in the same data center, or otherwise ensure the data must travel mere feet within a building rather than miles across a city — even with the speed of electrical signals, given the time constraints of this market a shorter distance can provide important fractions of a second in time savings.

[0043] As a result of these efforts, players in the financial industry today leverage one or some combination of these technologies: CPUs, GPUs, FPGAs, ASICs, and networking techniques. These computer architecture options have remained largely the same for over a decade. Despite this, the speed and scale pressures continue to mount as the scale of electronic trading increases. The investments being made also grow as developers find ways to slightly increase speed.

[0044] Trillions of dollars are being traded via these technologies and billions of dollars are being spent to find competitive advantage. There would be huge competitive advantage and a great deal of money to be made if the industry could find new architecture options that could be faster. But despite the available opportunity, the industry is and has been for over a decade at something of a standstill with respect to architecture choices.

[0045] The inventors have recognized and appreciated, however, that a new architectural option is available for processing of financial market data, and which might be able to exceed the speeds available with some other available architectures. This architecture option can also mitigate some of the disadvantages of available architectures and make speed increases available to more members of the financial industry. But despite that this architecture was first made commercially available by major processor manufacturers in the early 1990’s and remains an important aspect of modern, high-performance CPUs today, this architecture option has not been leveraged for processing of financial market data because the architecture itself is designed and operates in a manner that, to experts in computer architectures and in financial market data, appears impractical for the particular challenges of the messages types and formats of financial market data. As explained below, however, the inventors have recognized and appreciated that this architecture could be adapted for use with financial market data and provide a new architecture option for processing of financial market data.

[0046] This other architecture is special-purpose hardware for executing single instruction, multiple data (SIMD) operations. Such SIMD circuitry is available in add-on cards and, since 2011, has also been built into many commercially available CPUs from Intel Corporation and Advanced Micro Devices, Inc., where it goes by a number of trade names, including various iterations of Advanced Vector Extensions including AVX2 and AVX-512.

[0047] These circuits have a general operation akin to a GPU and for at least this reason (as should be appreciated from the discussion of GPUs above), this hardware has been regarded as unsuitable for processing of financial market data. This hardware differs from traditional CPU architectures in that, as its name (single instruction, multiple data) suggests, the hardware is specifically arranged to be configured with software to perform the same operation in bulk on a batch of data. Such SIMD hardware may therefore be arranged to store data for processing arranged as vectors, which may be vector registers, and can in one step perform a single instruction in parallel on multiple items of received data. By performing such operations in parallel, the throughput of these circuits can be higher than the throughput of traditional CPU processors.

[0048] One of the key originally-proposed uses for this hardware was performing graphics operations on computers that did not have the power, space, or other resources or demands to justify including a separate GPU. By performing some operations using the parallel hardware of the SIMD circuitry, this circuitry was designed to perform the graphics operations faster than performing the same operations on only standard CPU hardware though not as fast as performing those operations on the wider buses and more highly parallelized hardware of a GPU.

[0049] The use of SIMD hardware (e.g., AVX2) therefore would have been viewed as being unsuitable for use with financial market data, and slower than the available GPU options due to more constrained bus widths and other configuration constraints, if one of ordinary skill had even considered SIMD for use with financial market data. As noted above, despite the strong financial incentive to consider every available possibility for processing of financial market data, SIMD has not been used with financial market data to date.

[0050] In addition to the perceived speed disadvantages, a major reason those of ordinary skill would not have considered SIMD for use with financial market data is that the unpredictable sequence of message formats and types of data for financial market data (not the values of the data, but the formats and types of messages and the specific data fields contained therein) do not obviously match with the capabilities of SIMD and do not match with how financial market data has tended to be processed. SIMD hardware, as noted immediately above, includes data paths that are narrow when compared to data paths available with other parallelized circuits such as GPUs. As a result of these narrow paths, the processing pipelines of SIMD hardware cannot process a full message at once in parallel. Instead, to process the full content of a message, the message data would need to be processed in two or more sets of operations. This naturally means delays as compared to other available options, as needing to wait to process additional elements of a message could and would introduce delays into a processing pipeline.

[0051] More than that, though, SIMD hardware is best suited for performing operations on batches of items of similar size and type. This is very much not the case for financial market data. As discussed above, financial market data messages have varying formats and include different data and different data types in different message formats, and within a message format the data fields and the data types of those fields (signed or unsigned integers, floating points, Booleans, or other data types) vary. The data types and data sizes thus vary greatly from one field to the next. Market data messages arrive in an unpredictable sequence. As market data messages arrive for processing, then, the received data fields would vary in size and type, which is not well-suited for processing with SIMD.

[0052] A traditional way in which messages (whether financial market data or otherwise) are processed at high speed and / or scale is known as “transaction” processing. This might be referred to as “row” processing if a list of messages is envisioned where each message includes content extending horizontally across the page / list and the list has each message in one row. In such a row-based processing, once hardware begins processing a row, that hardware processes a message in full and performs all operations related to processing of that message are performed. Typically, in such a message- or row-based processing, the message is processed as received to increase speed, and the fields of the message are processed in the order they are received. This has a benefit of enabling the start of a message to be processed before a message is fully received, which may reduce processing latency in some circumstances.

[0053] The alternative manner of processing a message, using the same list mindset as in the previous paragraph, is known as analytic, field-based or “column” processing. In such a manner of processing, many messages (financial market data or otherwise) are received together and aggregated, such that many copies of a single “field” of those messages can be viewed and processed together. This can have some advantages in some types of data processing, such as when seeing changes in a specific field over time is a chief goal of processing. This can work well for “offline” or non-time-critical analysis of data. But it has the obvious disadvantage that aggregating messages in this way requires waiting for multiple messages to be received so that they can be aggregated and viewed together. For financial market data, this is usually unacceptable for latency reasons, as it increases a risk that messages may become outdated and stale before they are processed. This delay becomes even more pronounced when the types of messages that are received over time are considered. As mentioned above, financial market data contains multiple message formats with diverse data contents and formats. If a delay is inserted to receive a certain number of messages such that a common set of data fields (e.g., quote prices) can be processed together, this is not simply a matter of waiting for that number of messages to be received. Different messages with different formats and contents, and from different sources, may be received over time. It may take a large number of messages to be received before sufficient messages with similar content are received so as to process them together. This may introduce even more delays, which increases the chances of messages becoming stale and irrelevant (and trading opportunities being lost) before processing even begins, which further discourages use of field-based processing in time-critical applications.

[0054] Returning to the discussion of SIMD above, the SIMD hardware was discussed as being arranged to efficiently process data fields of identical width. For message-based / row-based processing, the SIMD hardware is not arranged for processing of data where message fields of varying size / length and type are adjacent to one another, given the variation. This may encourage a developer toward using field-based / column-based processing, such that the same field (having the same length / size and similar type) from multiple messages could be processed by the SIMD hardware. However, doing so would insert unacceptable delays (noted above) needed to receive and aggregate multiple messages having the same data field(s) to be processed, after which each of the fields would be processed one after another so as to process the entirety of each message. Such an approach raises a risk that messages would be stale by the time they were processed.

[0055] Given that SIMD hardware is (as discussed above) optimally configured for processing vectors containing data having the same data width and similar type, and given the variability of data field lengths in financial market data, if one of ordinary skill in the art had looked into using SIMD hardware for processing of financial market data, to gain the advantages of that hardware they may have looked to use field-based / column-based processing so as to process a batch of data having the same data widths. However, one of ordinary skill also would have appreciated that, as discussed above, using field-based / column-based processing would have inserted undesirable delays into the processing so as to enable aggregation of messages and batch processing of a set of the same field.

[0056] Accordingly, despite being commercially available for well over 30 years, and despite the constant and increasing pressures in the market to explore new and different architectures for use in processing financial market data, developers have not used SIMD for financial market data processing. If anyone of ordinary skill in the art had considered SIMD hardware for use with these message formats and types, the reasons discussed above would have led that one of ordinary in the art to conclude SIMD was not suitable for use in processing financial market data.

[0057] The inventors have recognized and appreciated, however, that SIMD could be put to use in processing of financial market data and could be arranged to perform processing of financial market data that meets or exceeds the speeds available through use of FPGAs.

[0058] Accordingly, described herein are various examples of techniques for use of SIMD hardware to process financial market data and techniques for generating software for execution on SIMD hardware to process financial market data. In various examples described below, one or more, single- or multi-core central processing units (CPU) each include one or more sets of SIMD hardware and are arranged to perform financial market operations on financial market data received in financial market data messages. The one or more sets of SIMD hardware may be within the integrated circuit packaging of the CPU, and for a multicore CPU may be within each or one or more of the cores of the multicore processor. In the specific case of SIMD integrated with CPU hardware (CPU SIMD), the SIMD hardware may be configured to execute instructions for processing of vector data on the SIMD hardware, which may be specified in software that is provided to the CPU for execution.

[0059] While some examples below focus on systems that include such CPUs with SIMD hardware, it should be appreciated that embodiments are not limited to include solely CPUs with integrated SIMD hardware. Some embodiments may additionally include use of standalone SIMD hardware, in addition to or as an alternative to CPUs with integrated SIMD hardware. Some embodiments may include, in addition to CPUs with integrated SIMD hardware and / or standalone SIMD hardware, CPUs without SIMD hardware, GPUs, FPGAs, ASICs, and / or other computer hardware to perform financial market data operations, with tasks divided between the computing hardware in various ways.

[0060] In accordance with some techniques described herein, in some embodiments CPU SIMD hardware may be configured to manage a storage of a financial market data, to be processed and / or updated based on received financial market data messages, in multiple vectors. The vectors may relate to a financial instrument that is traded on an exchange and may contain data specific to that instrument and that exchange. The vectors may each include one or more data fields related to financial market data for that instrument and that exchange, such that at least some (or, in some cases, all) of the fields from financial market data messages related to that instrument and exchange are arranged in the vectors, with each of the fields in at least one of the vectors. In accordance with some techniques described herein, the vectors can be processed by CPU SIMD hardware to perform operations related to processing of financial market data, such as in response to receipt of financial market data messages that relate to activity for that instrument and exchange. Examples of operations that can be performed on the vectors are discussed above in the context of processing of financial market data, and are discussed below in the context of some illustrative CPU SIMD implementations.

[0061] In some embodiments, each vector (of a set of vectors) to be processed by the CPU SIMD hardware includes data fields having identical lengths / sizes. To achieve this, the fields that relate to a financial instrument and to an exchange are arranged into the vectors such that different fields from messages for that instrument / exchange are sorted to different vectors based on the size of the fields. As an example, 64-bit wide fields may be assigned to one or more vectors having fields of that width, 32-bit wide fields may be assigned to one or more vectors having fields of that width, and 16-bit and 8-bit wide fields may be respectively assigned to vectors having fields of that width. Dividing and rearranging fields in this manner is not customary in the case of financial market data, where for latency reasons processing traditionally follows the sequence in which fields appear in messages, which is set by an exchange and usually influenced both by logically grouping related fields together (such as all fields related to a new bid, all fields related to a new ask, and so on) and by the frequency with which the fields / messages are expected to arrive. Rearranging the fields into a set of vectors based on size of the fields, without respect to an order in which the fields arrived in an incoming message, is not a traditional manner of processing financial market data and is counter-intuitive when prioritizing latency, as market data processing often does. However, the inventors have recognized and appreciated that splitting fields related to an instrument / exchange into multiple vectors for analysis with CPU SIMD vector-processing hardware can generate efficiency and speed advantages later in the processing that can offset (or more than offset) any potential delay from field splitting. As mentioned above, in some implementations that use such a division of message fields into multiple vectors, the vectors may each have fields of identical length.

[0062] Accordingly, when a financial market data message is received and the message type and its format identified, vectors having fields of different lengths / sizes may be retrieved and loaded into CPU SIMD hardware for processing in response to the financial market data message. In some embodiments, the multiple fields of a vector may be of different data types having a same size, such as including in the same vector signed and unsigned data types. Accordingly, while a vector may include multiple data fields of a same length, it may include data fields of different type.

[0063] In some embodiments, each data field that may be received in a financial market data message or otherwise relate to a financial instrument traded on an exchange may be defined in the system as an element to be processed by CPU SIMD hardware. The vectors, which may in some cases be termed record vectors, can then be defined as a collection of such elements, in some cases without specific regard to the SIMD hardware on which the data will be analyzed. In some such embodiments, when financial market data is divided into multiple record vectors, those record vectors for the data may be associated with one another in the system, and a set of associated record vectors may in some cases be termed a vector group. The vector group may be a collection of vectors for a record, which in some embodiments may relate to a financial instrument and / or an exchange. In some such embodiments, the exchange may be termed a “participant” in a wider market for the financial instrument, such as in a case that a particular financial instrument is traded on multiple exchanges.

[0064] A particular record for a financial instrument may, in some cases, relate to a participant, and there may be other records that relate to the same financial instrument but other participants. In some embodiments that operate with record vectors, market data for a financial instrument may be processed in the system using a record that is an association of multiple vectors, each of which may include data elements (and, in some cases, each vector may include data elements of a same size / length and / or multiple different data types).

[0065] In some embodiments that divide financial market data between vectors using record vectors, one or more of the record vectors may be defined to include one or more data fields that may be relevant to the processing of multiple records created from financial market data messages and / or to processing of vectors related to the financial instrument. For example, in a case of a financial instrument that is traded on multiple exchanges or otherwise has multiple participants, there may be certain data fields that may be relevant to processing of financial market data that relates to the financial instrument on multiple different exchanges or otherwise with respect to multiple participants. In such a case, these fields could be allocated to different record vectors for different participants and redundantly processed. However, doing so could increase memory storage requirements and could cause memory cache inefficiencies. By placing the fields for a record that may be used together with fields for other records in one or more specific record vectors (which may be termed “shared” record vectors), the shared record vectors may be more readily available for use together with other record vectors. In some embodiments that operate with shared record vectors, the shared record vector may include vectors having only data fields of the same sizes, as discussed above with respect to vectors for other record vectors. Shared record vectors may be associated using vector groups in the same way as record vectors, and may contain data types of equal length.

[0066] The grouping of data by vectors, record vectors (including shared record vectors), and vector groups and records may have advantages in various embodiments. For example, this structure may provide advantages in some embodiments with respect to caching and ensuring memory cache hits. As another example, this structure may provide advantages for flexibility between different CPU SIMD hardware with which different implementations may operate, such as having different bus widths or handling operations in slightly different ways.

[0067] Turning first to caching, the proposed structure of vectors, record vectors, vector groups, and records can have advantages with respect to memory cache management in some embodiments. The memory hierarchy for a CPU may include a hard drive or other large-volume storage remote from the CPU and memory that is accessible by the CPU via a bus and that is in a separate integrated circuit from the package, but may also include one or more memory caches that are located within the same integrated circuit as the CPU. Such on-chip memory caches may be for the entire CPU chip, for a specific core of the CPU (for a multi-core CPU), and / or for SIMD hardware of the CPU. Memory (including caches, main memory, and hard drives) are arranged in a hierarchy, in which the speed with which data can be retrieved for processing decreases as the data gets farther away from the execution hardware, but the size of the storage increases. Accordingly, during processing of a financial market data message, there could be delays in processing data as the execution hardware waits for data to be retrieved, if the necessary data is not close to the execution hardware in the memory hierarchy. An advantage of an arrangement of data in a vector group is that, in some cases and for some caches, the entire vector group might be retrievable and placed in memory cache at one time. This may make it more likely that all data fields in the vector group are available in memory cache and available for quick access. This may be helpful for some financial market data operations that may include, for a financial instrument, analyzing activity for the instrument across multiple participants. For example, an overnight analytics operation that looks at all activity for an instrument, across participants from the prior trading day may need quick access to activity across participants and may benefit from being retrievable together. As another example, operations connected to an auction process during a trading day (e.g., at market open or close, or as a scheduled or unscheduled intraday event) may benefit from quick access to data for multiple contributors to an order book. Or, for routine financial activity, an increase in trading of a financial instrument in one exchange is often accompanied by (or triggers) an uptick of trading of that financial instrument in other exchanges, as parties look to trade the financial instrument everywhere it is available, based on the new activity. Accordingly, in some embodiments a data structure may be maintained that may be movable into a memory cache, in accordance with a vector group that is an association of records and their associated vectors. In some embodiments, while a vector group may be a concept related to cache maintenance, a last value cache or logical cache of market data (which is itself stored in a memory cache and / or stored elsewhere in a memory hierarchy, such as by being partially stored in a memory cache) may not be maintained precisely as a vector group or including a vector group. Instead, in some embodiments, a cache may be maintained as a set of record hives, where each record hive includes the data for a particular financial instrument. For example, the record hive may include data about a financial instrument in addition to data about each exchange where that instrument is traded (e.g., for a financial instrument and a set of one or more participants), but without specific regard to vectors or the SIMD hardware on which the data will be analyzed. Accordingly, a record hive may contain or be associated with a set of record arrays, where each entry in a record array may be a list of fields that relate to a record (e.g., a financial instrument and a participant) and that are to be stored together. In other cases, a record hive may store other kinds of data, such as the orders associated with a given order book. In some embodiments, as discussed above with respect to a shared record vector, the record hive may contain one or more shared record arrays that include data fields that are relevant to multiple different (non-shared) records, or that are relevant to the record hive independent of any nonshared record. In such an organization, a record hive may have a group key or other group identifier, which for example may be an identifier for a financial instrument to which the contained or associated data relate. Each record array may also be associated with a sub-key, and each entry within a record array may be associated with its own key, which may correspond to a participant (e.g., exchange) identifier. Accordingly, the financial instrument and participant identifiers can be used to obtain certain record array entries, and thereby certain vectors, from the cache (e.g., last value cache) maintained in memory cache or otherwise in the memory hierarchy.

[0068] A record array for a financial instrument may, in some embodiments, include all of the data fields for a particular instrument and identifier. In other embodiments, a record for a financial instrument and for a participant may be divided into two or more record arrays, each of which may hold some of the data fields for the record. In some such embodiments, a record array may store data fields for a record (e.g., for a financial instrument and for a participant) that are to be analyzed together, to maximize chances they can be obtained together. In some such embodiments, the division of data fields for a record into different record arrays may be done based on a division of fields into record vectors and vector groups. For example, data fields that are included in the same record vector, or that may be included in record vectors that are associated with the same vector group, may be included in the same record array to maximize chances of being stored together in memory cache.

[0069] Techniques have been discussed so far of building record vectors, vector groups, and record arrays so as to encourage efficiency in processing of financial market data using CPU SIMD hardware and increase likelihood of data being in a memory cache (or otherwise in the memory hierarchy) close to execution hardware when needed. In some embodiments, the sorting of data fields into vector groups, record vectors, and so on may remain fixed over time, or at least until an administrator or other user reconfigures data fields. In other embodiments, a system may be configured to use different arrangements of data fields into vector groups and record arrays over time. This may be reflective of, for example, different financial market data messages that may be retrieved at different times. For some exchanges, it may be more likely that messages of certain formats or having certain data fields may be received at one time (e.g., morning) and that messages of other formats or having certain other data fields may be received at another time (e.g., afternoon). Accordingly, in some embodiments, in response to certain conditions being met (e.g., time, volume of activity, types of messages being received, types of instruments being traded) a system may reconfigure an arrangement of data fields into vector groups and / or into record arrays.

[0070] Returning to the discussions above of advantages in certain embodiments of arranging data fields as record vectors, vector groups, and record arrays, in some embodiments this arrangement may provide flexibility with respect to computing hardware on which techniques will be used. CPU SIMD hardware can have certain variations between hardware, such as by having 256-bit wide hardware for parallel operations or 512-bit wide hardware for parallel operations, and the like. CPU SIMD programming and hardware is unfamiliar for many programmers, particularly when operating with multiple discrete and varying data fields rather than with the types of large groups of similar data for which SIMD hardware was designed. If programmers were made to learn the intricacies of not just one type of SIMD hardware but of each variation on which their code might execute, this would be a significant barrier to adoption of SIMD (on top of the barriers discussed above). The inventors have thus recognized and appreciated the benefits of a system that allows for a level of abstraction from SIMD hardware and masks some of the lowest-level implementation details from a user, including by identifying data types as elements in a system and grouping the elements into record vectors, vector groups, and record arrays. When software instructions to be run on CPU SIMD hardware are generated, in some embodiments a code generator may identify (from input from a user or otherwise) the hardware on which the executable code is to be run and may adjust the generation of code accordingly.

[0071] When adjusting generation of code, in some embodiments the code generator may adjust the precise way in which memory is allocated (e.g., the size of memory allocated to certain fields), the precise way in which data is assigned to SIMD registers of the SIMD hardware, the precise SIMD instructions that are generated, and other parameters of the generated code.

[0072] Accordingly, when data elements and vectors are to be generated for messages, the precise allocation of data fields to vectors, vector groups, and record arrays may be set by a code generator, in connection with input from a user as well as details (e.g., configuration data) regarding the hardware. In some such embodiments, data fields that are to be processed or groups of data fields that are to be processed together may be identified in input from a user (e.g., a programmer or administrator, or other user). A code generator, upon receiving the identification of data fields and / or the fields to be processed together and an assignment of fields to record vectors and vector groups, may map the record vector and vector group inputs to specific SIMD register and type definitions, to specific record arrays (for the last value cache or other logical cache), and to processing in the SIMD hardware, including by generating code to effect these assignments and executions. Doing so may aid in increasing execution efficiency through having efficiency rules configured in the code generator and used to process instructions and automatically make memory and register assignments, among others. Doing so may aid in improving execution efficiency of financial market data with CPU SIMD hardware, as the programmers responsible for writing the code may be insulated for idiosyncrasies of the SIMD hardware or SIMD operations and can focus on specifying the data they want processed rather than the precise manner in which the SIMD hardware is to be used to process the data.

[0073] In fact, in some embodiments, a code generator may be configured to generate SIMD code (e.g., code for execution on SIMD hardware) without receiving as input a specification of corresponding instructions in another programming language. In some examples, the code generator includes a compiler suitable for compiling SIMD source code, and also performs other code generating functions. In other examples a code generator may generate source code that can be run directly in an interpreted environment (e.g. Java) or compiled into executable code with a standard compiler (e.g. C++), and also performs other code generating functions. Typically, a code generator may generate code (e.g., executable code) by processing of non-executable code, often called human-readable code or source code. The source code specifies in detail operations to be performed and the code generator translates that detailed specification into other code (e.g., modified source code or executable code that can be executed by hardware). In some embodiments, however, a code generator may receive as input configuration data including an identification of data fields to be processed, certain characteristics of those data fields, and indications of how data fields are to be grouped. In response, a code generator may generate code that allocates memory to storing the data, generate code for storing data parsed from an incoming message into that memory, and generate code for updating previously-stored data per data received in an incoming message.

[0074] Accordingly, in some embodiments operations like feed handler operations that parse incoming financial market data messages and perform specified operations on them (e.g., to compare received data to pre-existing data and potentially take responsive actions) may be defined by the code generator through generation of code, based on an input specification. Any or all of the examples of operations performed above with respect to processing of financial market data may be operations performed by a “feed handler.” In some embodiments, a feed handler to be executed on CPU SIMD hardware may be generated and configured automatically by a code generator based on configuration data including a specification of message formats of interest to a user and / or of data fields of those message formats that are of interest to a user. With a specification of the message formats and / or data fields to be processed, the code generator can be informed of the types of data to be processed, which given similarities in how feed handlers operate (e.g., to perform baseline operations for financial market data), the code generator can generate feed handler executable code that is designed for fast execution on CPU SIMD hardware. In some embodiments, in addition to or as an alternative to generating code for execution on CPU SIMD hardware, a code generator may generate code for execution on standalone SIMD hardware. Or, in some embodiments a code generator that is able to generate code for execution on SIMD hardware may additionally or alternatively generate “traditional” code that runs on other computing hardware (e.g., not SIMD hardware).

[0075] In some embodiments that operate with such a code generator, a specification of message formats and data fields may include an identification of one or more message fields of a message format for an exchange feed and indicate how those fields are to be mapped to particular record vectors or vector groups for processing. The indication of how message fields relate to record vectors or vector groups may indicate how a field maps to a particular pre-defined field in a predefined record vector or vector group. In some embodiments and in some cases, a user may desire that a particular field be processed using conventional CPU processing rather than SIMD processing, the specification may identify that a particular field is to be processed with non-SIMD or scalar processing. A specification may also specify certain transformations or interpretations to be performed on specified fields. For example, a specification may indicate that one or more mathematical or logical operations are to be performed on a field received as input in a message, before the resulting value is to be used in processing (by SIMD hardware or otherwise), and may identify the operation(s) by an identifier to where such operation(s) are predefined or may set out the operations explicitly. As another example, a specification may indicate that an order of bytes in a field is to be modified (e.g., from big endian to little endian). Other examples include that a data type is to be transformed (e.g., from one type of integer to another), that a decoding is to be performed to interpret one value or flag into another, or other operations that can be performed on data.

[0076] Some such embodiments that have such a code generator generating code based on configuration data indicating message formats and / or data fields to be processed may generate all of the code to be executed for handling financial market data for a particular entity or to be run on particular CPU SIMD hardware. In the financial industry, the level of competition at a technical level and the need to eke out nanoseconds to have a competitive edge over others means that (as discussed above) many financial institutions have large in-house technical teams that are generating code and hardware at the highest speeds possible. That emphasis on internal expertise and speed means that the technical teams at many institutions will not purchase already-generated code from others, to minimize the risk of purchasing inefficient code. This means, however, that these teams need in-house expertise for all hardware, which is not currently available for CPU SIMD hardware given that it has not been available for use in processing financial market data. This lack of expertise, combined with what would have been natural skepticism of CPU SIMD hardware for the reasons noted above, would have impeded adoption if anyone had tried to use CPU SIMD hardware for financial market data. The inventors have recognized and appreciated that as they develop tools for use of CPU SIMD hardware to process financial market data, customers may be reluctant to use pre-generated code. Accordingly, in some embodiments, some or potentially all code that is run on CPU SIMD hardware may be code that is generated in connection with the user’s instructions on how the code should be generated.

[0077] In some such embodiments, as noted above, the input received from a user regarding how code should be generated may include an indication of what message types and / or data fields are to be processed, and an indication of an identity of the messages (e.g., by source), and an indication of which record vector elements, if any, should be affected by the fields and how. In other embodiments, however, a finer-grained level of control may be offered to a user, for at least some fields. For example, when specifying data fields that are to be stored within vectors for processing on CPU SIMD hardware, some embodiments may provide a user an option to specify whether a data field, a record vector, and / or a vector group should be stored using aligned memory storage or nonaligned memory storage. With aligned memory storage, data fields are stored in chunks that match a natural configuration size of memory, such as that matches a hardware capability such as a bus width in memory or a register size in memory, and such a size may be used even if a data field is smaller than that size. In such a case that a data field is smaller than the natural configuration size of the memory, there may be at least some and potentially many wasted bits in memory, as the memory space is padded to make up for the smaller size of the data unit to be stored. Aligned memory storage may be used because, in some cases, storage and retrieval may be faster than for unaligned memory storage — consistent with discussions above, “faster” here may mean small fractions of a second, but these differences may be significant in this industry. A disadvantage to use of aligned memory storage for small data fields (which is particularly important for financial market data, given the number of small data fields and variability of data between messages) is that the extra wasted bits must still be retrieved and put into a memory cache, which means at least some and potentially much of the memory cache could be wasted storing irrelevant extra bits. As noted above, speed of processing of financial market data may hinge in part on whether data is in a memory cache, so having more of the memory cache wasted is typically a bad choice. A user may thus be faced with a choice of whether to prioritize the slightly faster operations that can be achieved with aligned memory operations, or the slightly faster operations that can be achieved with more efficient memory cache usage. A user may analyze operations to be performed and potentially perform speed experiments and then, using the interface that allows for specification of whether and where to use aligned or nonaligned memory options, specify the type of memory operation to be performed. In some such embodiments, a code generator, when interpreting input provided by a user, may leverage that input when creating code and decide between use of the code instruction for aligned memory and the code instruction for unaligned memory. In some embodiments, the code generation process may automatically select the most appropriate performance options, including alignment or other options. In embodiments that perform such an automation of selection, the selection may be made based on any suitable factors. In some embodiments, the selection may be made based on heuristic techniques, or one or more other conditions or criteria. In some embodiments, a learning technique may be employed and a system may make an assessment of instrumentation from one or more prior runnings of the system. In other embodiments, other techniques may be used, as embodiments are not limited in this respect.

[0078] In some aspects, the techniques described herein relate to an apparatus including: at least one single-instruction, multiple-data (SIMD) circuit adapted to perform one or more SIMD operations; and at least one storage medium having encoded thereon executable instructions that cause the at least one SIMD circuit to perform financial market data processing on data of one or more financial market data messages, wherein performing the financial market data processing includes performing at least one of the SIMD operations on the data of the one or more financial market data messages.

[0079] In some aspects, the techniques described herein relate to an apparatus, further including at least one central processing unit (CPU), wherein: the at least one CPU includes the at least one SIMD circuit; and the executable instructions further include executable instructions that can be executed by the at least one CPU to configure the at least one SIMD circuit to perform the financial market data processing on the data of the one or more financial market data messages and to make the data of the one or more financial market data messages available for the financial market data processing by the at least one SIMD circuit. In some aspects, the techniques described herein relate to an apparatus, wherein: the at least one CPU is a plurality of CPUs; and each CPU of the plurality of CPUs includes one or more SIMD circuits of the at least one SIMD circuit.

[0080] In some aspects, the techniques described herein relate to an apparatus, wherein: the at least one SIMD circuit includes a first SIMD circuit; the first SIMD circuit includes a vector register; and the first SIMD circuit is adapted to execute at least one of the SIMD operations in parallel on data stored in the vector register.

[0081] In some aspects, the techniques described herein relate to an apparatus, wherein performing the financial market data processing on the data of the one or more financial market data messages includes performing at least one of the SIMD operations to read, write, delete or update one or more values of a datastore based on one or more values of and / or determined from the data.

[0082] In some aspects, the techniques described herein relate to an apparatus, wherein the at least one SIMD operation to read, write, delete, or update the one or more values of the datastore includes at least one SIMD operation to determine, from the data and using the at least one SIMD circuit, a particular value to be stored in a last value cache.

[0083] In some aspects, the techniques described herein relate to an apparatus, wherein the at least one SIMD operation to determine the particular value to be stored in the last value cache includes: a first SIMD operation to load one or more first values from the last value cache; and a second SIMD operation to determine, based on the one or more first values from the last value cache and the one or more values of and / or determined from the data, the particular value to be stored in the last value cache.

[0084] In some aspects, the techniques described herein relate to an apparatus, wherein performing the financial market data processing on the data of the one or more financial market data messages includes performing at least one of the SIMD operations to convert at least some of the data from a first format to a second format.

[0085] In some aspects, the techniques described herein relate to an apparatus, wherein performing the financial market data processing on the data of the one or more financial market data messages further includes performing at least one of the SIMD operations to output at least one message including second data in the second format generated using the at least some of the data of the one or more financial market data messages. In some aspects, the techniques described herein relate to an apparatus, wherein the at least one of the SIMD operations to output the at least one message includes: at least one first SIMD operation to load one or more first values from a last value cache; and at least one second SIMD operation to determine, based on the one or more first values from the last value cache and the at least some of the data of the one or more financial market data messages, the at least one message including second data in the second format.

[0086] In some aspects, the techniques described herein relate to an apparatus, wherein performing the financial market data processing on the data of the one or more financial market data messages includes performing at least one of the SIMD operations on a one or more vectors, each vector of the one or more vectors including a plurality of fields, and the plurality of fields of a vector each being of a same length.

[0087] In some aspects, the techniques described herein relate to an apparatus, wherein: the one or more vectors include a first vector and a second vector; all fields of the first vector are represented using a first number of bits; and all fields of the second vector are represented using a second number bits.

[0088] In some aspects, the techniques described herein relate to an apparatus, wherein performing the at least one SIMD operation on the data of the one or more financial market data messages includes: performing one or more SIMD operations to map values from a plurality of fields of an incoming financial market data message to a plurality of fields of the one or more vectors, wherein mapping the values includes copying a subset of the plurality of fields of the incoming financial market data message from one or more source vector registers of the SIMD hardware to a target vector register of the SIMD hardware configured to store a first vector of the one or more vectors, the subset of the plurality of fields of the incoming financial market data message being of equal length.

[0089] In some aspects, the techniques described herein relate to an apparatus, wherein performing the one or more SIMD operations to map values from the plurality of fields of the incoming financial market data message to the plurality of fields of the one or more vectors further includes: copying one or more parts of the incoming financial market data message from a memory to the one or more source vector registers of the SIMD hardware, wherein the memory is a memory of a network interface card (NIC), a memory of the CPU, or a cache of the CPU. In some aspects, the techniques described herein relate to an apparatus, wherein mapping the values further includes: copying one or more fields of the subset of the plurality of fields of the incoming financial market data message from the one or more parts of the incoming financial market data message in the one or more source registers of the SIMD hardware to the target vector register; and the executable instructions that cause the at least one SIMD circuit to perform the financial market data operations on the plurality of vectors include operations to cause the at least one SIMD circuit to perform the financial market data operations on the target vector register after all fields of the first vector across the one or more parts of the incoming financial market data message have been copied from the one or more source vector registers to the target vector register.

[0090] In some aspects, the techniques described herein relate to an apparatus, wherein the operations to cause the at least one SIMD circuit to perform the financial market data operations on the target vector register after all fields of the first vector across the one or more parts of the incoming financial market data message have been copied to the target vector register include operations to cause the at least one SIMD circuit to perform the financial market data operations on the target vector register after all parts of a plurality of parts of the financial market data message have been copied to the one or more source vector registers.

[0091] In some aspects, the techniques described herein relate to an apparatus, wherein for a financial market data message of the one or more financial market data messages, the plurality of vectors include: a first plurality of vectors that store data specific to a financial instrument and to a financial market exchange on which the financial instrument is traded, the financial market data message relating to the financial instrument and to the financial market exchange; and a second plurality of vectors that store data generic to one or more financial instruments including the financial instrument and / or generic to one or more financial market exchanges including the financial market exchange.

[0092] In some aspects, the techniques described herein relate to an apparatus, wherein: for a second financial market data message of the one or more financial market data messages, the second financial market data message relating to the financial instrument and to a second financial market exchange, the financial market data processing is performed on a third plurality of vectors; and the third plurality of vectors include the second plurality of vectors and a fourth plurality of vectors specific to the financial instrument and to the second financial market exchange.

[0093] In some aspects, the techniques described herein relate to an apparatus, wherein: the first and fourth plurality of vectors are maintained in a data structure representing a vector group; and the instructions include instructions to manage caching of the first and fourth plurality of vectors by the CPU, such that the first and fourth plurality of vectors are both resident in a cache of the CPU.

[0094] In some aspects, the techniques described herein relate to an apparatus, wherein: the data structure is associated with a data structure cache key; and different sets of vectors within the data structure may be associated with sub-keys of the data structure cache key.

[0095] In some aspects, the techniques described herein relate to an apparatus, wherein: the data structure cache key is associated with the financial instrument; and different data structures are maintained for and correspond to different financial instruments, each data structure being associated with a respective data structure cache key that is associated with the corresponding financial instrument.

[0096] In some aspects, the techniques described herein relate to a method including: performing financial market data processing on data of one or more financial market data messages, wherein performing the financial market data processing includes performing at least one singleinstruction, multiple-data (SIMD) operation of SIMD hardware on the data of the one or more financial market messages.

[0097] In some aspects, the techniques described herein relate to a method, wherein performing the financial market data processing on the data of the one or more financial market data messages includes performing at least one SIMD operation of the SIMD hardware to read, write, delete or update one or more values of a datastore based on one or more values of and / or determined from the data.

[0098] In some aspects, the techniques described herein relate to a method, wherein the at least one SIMD operation to read, write, delete, or update the one or more values of the datastore includes at least one SIMD operation to determine, from the data and using the SIMD hardware, a particular value to be stored in a last value cache.

[0099] In some aspects, the techniques described herein relate to a method, wherein the at least one SIMD operation to determine the particular value to be stored in the last value cache includes: a first SIMD operation to load one or more first values from the last value cache; and a second SIMD operation to determine, based on the one or more first values from the last value cache and the one or more values of and / or determined from the data, the particular value to be stored in the last value cache.

[0100] In some aspects, the techniques described herein relate to a method, wherein performing the financial market data processing on the data of the one or more financial market data messages includes performing at least one SIMD operation of the SIMD hardware to convert at least some of the data from a first format to a second format.

[0101] In some aspects, the techniques described herein relate to a method, wherein performing the financial market data processing on the data of the one or more financial market data messages further includes performing at least one SIMD operation of the SIMD hardware to output at least one message including second data in the second format generated using the at least some of the data of the one or more financial market data messages.

[0102] In some aspects, the techniques described herein relate to a method, wherein performing the financial market data processing on the data of the one or more financial market data messages includes performing at least one SIMD operation of the SIMD hardware on one or more vectors, each vector of the one or more vectors including a plurality of fields, and the plurality of fields of a vector each being of a same length.

[0103] In some aspects, the techniques described herein relate to a method, wherein performing the at least one SIMD operation on the data of the one or more financial market data messages further includes: performing one or more SIMD operations to map values from a plurality of fields of an incoming financial market data message to a plurality of fields of the one or more vectors, wherein mapping the values includes copying a subset of the plurality of fields of the incoming financial market data message from one or more source vector registers of the SIMD hardware to a target vector register of the SIMD hardware configured to store a first vector of the one or more vectors, the subset of the plurality of fields of the incoming financial market data message being of equal length.

[0104] In some aspects, the techniques described herein relate to a method, wherein performing the one or more SIMD operations to map values from the plurality of fields of the incoming financial market data message to the plurality of fields of the one or more vectors further includes: copying one or more parts of the incoming financial market data message from a memory to the one or more source vector registers of the SIMD hardware, wherein the memory is a memory of a network interface card (NIC) or a cache of the CPU.

[0105] In some aspects, the techniques described herein relate to a method, wherein mapping the values further includes: copying one or more fields of the subset of the plurality of fields of the incoming financial market data message from the one or more parts of the incoming financial market data message in the one or more source registers of the SIMD hardware to the target vector register; and the executable instructions that cause the at least one SIMD circuit to perform the financial market data operations on the plurality of vectors include operations to cause the at least one SIMD circuit to perform the financial market data operations on the target vector register after all fields of the first vector across the one or more parts of the incoming financial market data message have been copied from the one or more source vector registers to the target vector register.

[0106] In some aspects, the techniques described herein relate to a method, wherein the operations to cause the at least one SIMD circuit to perform the financial market data operations on the target vector register after all fields of the first vector across the one or more parts of the incoming financial market data message have been copied to the target vector register include operations to cause the at least one SIMD circuit to perform the financial market data operations on the target vector register after all parts of a plurality of parts of the financial market data message have been copied to the one or more source vector registers.

[0107] In some aspects, the techniques described herein relate to a method, wherein for an incoming financial market data message related to a financial instrument, the plurality of vectors include: a first plurality of vectors that store data specific to the financial instrument and to a financial market exchange on which the financial instrument is traded, the incoming financial market data message relating to the financial instrument and to the financial market exchange; and a second plurality of vectors that store data generic to one or more financial instruments including the financial instrument and / or generic to one or more financial market exchanges including the financial market exchange.

[0108] In some aspects, the techniques described herein relate to a method, wherein: for a second financial market data message of the one or more financial market data messages, the second financial market data message relating to the financial instrument and to a second financial market exchange, the financial market data processing is performed on a third plurality of vectors; and the third plurality of vectors include the second plurality of vectors and a fourth plurality of vectors specific to the financial instrument and to the second financial market exchange.

[0109] In some aspects, the techniques described herein relate to a method, wherein performing at least one SIMD operation to output at least one message includes: performing at least one first SIMD operation to load one or more first values from a last value cache; and performing at least one second SIMD operation to determine, based on the one or more first values from the last value cache and the at least some of the data of the one or more financial market data messages, the at least one message including second data in the second format.

[0110] In some aspects, the techniques described herein relate to a method including: generating a feed handler for processing of incoming financial market data messages, wherein generating the feed handler includes generating instructions for the feed handler based at least in part on a first input identifying SIMD hardware on which the instructions are to execute and a second input identifying how a plurality of fields of the incoming financial market data messages are to be mapped to a plurality of vectors, wherein the first and second input are received separate from the instructions.

[0111] In some aspects, the techniques described herein relate to a method, further including: determining, based on the first input identifying the SIMD hardware, one or more properties of the SIMD hardware including one or more widths of one or more vector registers of the SIMD hardware, and one or more widths of one or more elements of the one or more vector registers on which one or more SIMD instructions of the SIMD hardware can operate, wherein the generating the instructions for the feed handler based at least in part on the first input identifying the SIMD hardware includes generating the instructions for the feed handler based at least in part on the determined one or more properties of the SIMD hardware.

[0112] In some aspects, the techniques described herein relate to a method, wherein: the second input identifying how the plurality of fields are to be mapped to the plurality of vectors includes, for one or more fields of the plurality of fields, an indication of whether to use aligned memory storage or unaligned memory storage for storing data of the one or more fields; and the generating the instructions for the feed handler based at least in part on the second input includes generating the instructions in a manner that outputs aligned or unaligned memory storage instructions for storage operations related to the one or more fields consistent with the indication. In some aspects, the techniques described herein relate to a method, wherein: the generating the instructions for the feed handler based at least in part on the second input includes generating, using the second input, instructions that define data structures that store the plurality of vectors and instructions that store, in the plurality of vectors, data from one or more fields of the plurality of fields of the incoming financial market data messages; and wherein generating the instructions that store the data from the one or more fields of the incoming financial market data messages include generating instructions to divide the one or more fields of an incoming financial market data message between multiple vectors of the plurality of vectors, each vector of the multiple vectors including a respective plurality of fields of a same width.

[0113] In some aspects, the techniques described herein relate to a method, wherein the generating the instructions for the feed handler based at least in part on the second input includes generating, for each vector of the multiple vectors and using the second input, instructions to assign the plurality of fields of the respective vector to SIMD registers of the SIMD hardware.

[0114] In some aspects, the techniques described herein relate to a method, wherein the generating the instructions for the feed handler includes generating instructions to read, write, delete or update one or more values of a datastore based on one or more values included in the incoming financial market data messages.

[0115] In some aspects, the techniques described herein relate to a method, wherein the generating the instructions to read, write, delete, or update the one or more values of the datastore includes generating instructions to determine, using the SIMD hardware and based on the one or more values included in the incoming financial market data messages and / or the one or more values of the datastore, a particular value to be stored in a last value cache.

[0116] In some aspects, the techniques described herein relate to a method, wherein generating instructions for the feed handler based at least in part on the second input includes: receiving, in the second input, data identifying one or more transformations or interpretations to be performed on received data that is to be stored in a field of a vector; and generating instructions to perform the one or more transformations or interpretations.

[0117] In some aspects, the techniques described herein relate to a method, wherein generating the feed handler further includes: generating second instructions to be executed by non-SIMD hardware. In some aspects, the techniques described herein relate to a method, wherein the generating the instructions to read, write, delete, or update the one or more values of the datastore includes generating instructions to determine, using the SIMD hardware and based on the one or more values included in the incoming financial market data messages and / or the one or more values of the datastore, a particular value to be included in an output message.

[0118] Illustrative Embodiments

[0119] Described below are various illustrative embodiments of techniques described herein. It should be appreciated, however, that embodiments are not limited to operating in accordance with any of the examples below. Other embodiments are possible.

[0120] Use cases for market data are diverse, from high-performance electronic trading to updating a stock price on a company’s investor-relations website, and may have very different performance, functional and commercial requirements.

[0121] The complexities and rapid pace of change in the financial markets have attracted third party companies offering products and services such as: (1) connectivity - the ability to connect to a source, whether local or overseas, to receive real-time data and / or interact with their trading systems; (2) hosting - the ability to place servers and other equipment at a chosen location, typically proximate to a particular financial center; (3) data wholesaling - receiving, transporting and distributing data on behalf of the content owner, possibly also managing end-user accounts and collecting data licensing fees on their behalf; (4) technology - e.g. for company- wide distribution of data, order and execution handling, risk management, trading analytics, or MDP; and (5) managed services - hands-on field service or service management, hardware provisioning, equipment leasing, consultancy and system design, contract development, etc.

[0122] Such companies may be generally referred to below as vendors. Those vendors that offer market data processing technology or services may be referred to below as market data vendors.

[0123] Advantages that may be offered by some of the embodiments described below include some or all of the following. Some embodiments achieve FPGA-equivalent performance, or better, using software running standard server CPUs and systems. Some embodiments enable the client to specify the functionality that they require, and selectively generate code according to their wishes. This approach allows for fine-grained customization by the client and ensures that there is no run-time cost for unwanted or unused features. As new features are developed, for example in response to technological advances, they can be added to a code generator, for the client to use or not as they see fit for their use case. The code generator can be updated to implement exchange- driven and other industry changes (EDCs). High-performance networking technologies in use in the industry can be supported. Clients can be permitted to use their own configuration control, deployment and management tools and maintain full control over the operational environment. In some embodiments, binaries are created by the client for their own use. A vendor of the code generator may not provide binaries in some such embodiments. A vendor of technologies described herein may in some embodiments not need any kind of access to a client’s network.

[0124] Referring to FIG. 1 A, a system 100 for financial market data processing (MDP) may include one or more source devices 110, one or more networks 120, a MDP engine 130 (e.g., MDP facility), one or more datastores 140, and one or more client devices 150. In some examples, the source device(s) 110 produce and / or transmit market data messages 112 to the network(s) 120. The MDP engine 130 can receive market data messages 112 via the network(s) 120 and processes the market data messages 112 to extract raw market data and / or produce refined market data (collectively, market data 132), which the MDP engine 130 can store in a datastore 140 and / or send to one or more client devices 150. In some examples, the MDP engine 130 processes the market data messages 112 in combination with existing market data 132 to produce updated market data 132. Also, based on the market data 132, the MDP engine 130 and / or the client devices 150 may generate responsive market data, such as bids and / or offers for financial instruments, determine whether to accept or deny other parties’ bids and / or offers, and transmit market activity messages 134 regarding such bids, offers, acceptances, and denials to one or more market exchanges via the network(s) 120. The client(s) 150 may receive market data 132 from the MDP engine 130 and / or from the datastore 140. In some examples, the client(s) 150 may also transmit configuration data 152 to the MDP engine 130, which may control how the MDP engine 130 processes the market data messages 112. Some non-limiting examples of the components of the system 100 and their functions are described in further detail below.

[0125] In some examples, the source device(s) 110 transmit market data messages 112 to the network(s) 120. These market data messages 112 can be messages of real-time data feeds provided by sources, which may include participants (e.g., exchanges), brokers, other financial execution venues or reporting facilities, etc. Such messages can pertain to quotes, trades, or other market activity. Given the volume of activity in some exchanges, particularly electronic trading activity, the rate of updates provided by some sources (e.g., exchanges) can be in the range of tens of millions of messages per second. In many cases, a particular financial instrument may be traded on multiple different exchanges or otherwise have updates regarding the status of the instrument that are available via multiple different feeds. Overall, the scale of messages regarding financial activity provided by sources is into the hundreds of millions of messages per second. Some non-limiting examples of such data feeds and feed messages are described herein.

[0126] Each source device 110 can be, or can be included in, any suitable computing device. A source device may be or include a server-class machine, such as a rack-mounted server, a blade server, a cloud-based virtual server instance, etc. A source device 110 may include one or more multi-core processors, graphics processing units (GPUs), or specialized accelerators (e.g., FPGAs or ASICs) to support parallel processing and low-latency operations. In some implementations, a source device 110 may operate within a distributed cluster, a high-availability configuration, or a load-balanced environment to ensure scalability and fault tolerance. A source device 110 may support secure networking protocols for communication with other devices and datastores. These examples are illustrative and not limiting.

[0127] As illustrated in FIG. 1 A, the source device(s) 110 and the MDP engine 130 may communicate via one or more networks 120. A network 120 may include a wide variety of communication infrastructures and technologies. For instance, a network 120 may be or include a local area network (LAN), a wide area network (WAN), or a metropolitan area network (MAN). In some examples, a network 120 may include public networks, such as the Internet, or private networks, such as enterprise intranets. A network 120 can also encompass wireless networks, including Wi-Fi, cellular networks (e.g., 3G, 4G, 5G), and satellite communication systems, as well as wired networks, such as Ethernet or fiber-optic connections. In certain implementations, the network may utilize virtual private networks (VPNs), peer-to-peer (P2P) architectures, or cloud-based networking environments. In some examples, the network 120 is contained within a co-location center. These examples are provided for illustration and are not intended to limit the scope of techniques described herein.

[0128] The system 100 can include a MDP engine 130. In some embodiments, in accordance with techniques described herein, the MDP engine includes or provides a data pipeline specialized for financial market data processing. The MDP engine can be implemented as software executing on one or more processing devices. At least one such processing device includes SIMD hardware (e.g., SIMD execution units). The MDP engine 130 can utilize the SIMD hardware (and, optionally, other specialized computing techniques) to increase throughput and reduce processing latency of market data processing.

[0129] The MDP engine 130 can receive market data messages 112 via the network(s) 120 and process the market data messages 112 to extract raw market data and / or produce refined market data (collectively, market data 132). The MDP engine 130 can store the market data 132 in a datastore 140 and / or send the market data 132 to one or more client devices 150. In some examples, the processing of the market data messages 112 by the MDP engine is based, at least in part, on existing market data 132. Thus, in some examples, the refined market data produced by the MDP engine 130 is based on raw market data extracted from the market data messages 112 and existing market data retrieved from the datastore 140.

[0130] In some examples, the MDP engine 130 receives market data messages 112 of real-time data feeds and converts them to an appropriate form for end-use by a user. The actual processing steps undertaken depend on the use case, but the various functions performed are commonly described, and are described below, under the term “Feed Handlers.” A feed handler might be analogized to a television at the end of a cable broadcast; the broadcast isn’t useful until it has been received, decoded and processed.

[0131] The market data processing (MDP) engine 130 may be hosted on any suitable computing device or system that includes SIMD hardware and is configured to provide high-performance data processing capabilities. In some embodiments, the computing device may comprise a server-class machine, such as a rack-mounted server, a blade server, or a cloud-based virtual server instance. The computing device may include one or more multi-core processors, which may include or be coupled to SIMD hardware (e.g., SIMD circuitry). In some examples, the SIMD hardware can be integrated with non-SIMD hardware (e.g., scalar data processing circuitry) in a central processing unit (CPU). The CPU can have any suitable architecture or features, including but not limited to a superscalar pipeline, a very long instruction word (VUIW) architecture, in-order scheduling of instructions for execution, out-of-order scheduling of instructions for execution, etc. The SIMD hardware can be configured to process market data messages 112 and / or market data 132 using any of the techniques described herein. In some examples, portions of the market data processing can be performed on non-SIMD hardware (e.g., non-SIMD hardware of the same CPU that includes the SIMD hardware, non-SIMD hardware of another CPU, other any other suitable non-SIMD hardware. Some non-limiting examples of non-SIMD hardware can include some types of graphics processing units (GPUs), multiple-input multiple-data (MIMD) hardware, single-input single-data (SISD) hardware, multiple-input single-data (MISD) hardware, etc.

[0132] In some examples, the MDP 130 can use a feed handler to process captured network data (e.g., market data messages 112) from files, e.g., in PCAP format, and can also output market data 132 to files rather than a live network, a capability widely used for trading analytics and quantitative research.

[0133] SIMD hardware can execute a single instruction across multiple data elements in parallel, thereby improving throughput and reducing latency for computationally intensive workloads. In some embodiments, the SIMD hardware may include one or more vector registers, each capable of storing multiple data elements, such as price values, trade volumes, or other market indicators. These vector registers enable the simultaneous execution of arithmetic, logical, or comparison operations on multiple data points using a single instruction.

[0134] In some examples, SIMD architectures provide vector instructions that operate on the contents of vector registers in parallel. For example, a single addition instruction may add corresponding elements from two vector registers and store the results in a third register, rather than performing multiple scalar addition operations sequentially. This parallelism may be particularly advantageous for tasks such as computing moving averages, normalizing data, or applying mathematical transformations across large datasets. The instruction set may include operations for addition, subtraction, multiplication, division, and other functions suitable for vectorized execution.

[0135] In some implementations, the SIMD hardware may support both aligned and unaligned memory access for loading and storing data into vector registers. Aligned memory access ensures that data elements are stored at memory addresses that match the natural alignment of the vector size, which can improve performance by reducing memory access cycles. However, the hardware may also allow unaligned memory access to accommodate data structures that do not conform to alignment requirements, providing flexibility in handling diverse data layouts. These features can enable efficient processing of financial market data streams, regardless of how the data messages are organized in memory. In some embodiments, the SIMD hardware may implement a vector instruction set (e.g., an Advanced Vector Extensions instruction set), which provides wide vector registers and instructions suitable for high-throughput parallel processing. The AVX2 and AVX-512 variants of the Advanced Vector Extensions instruction set support 256-bit and 512-bit vector registers, respectively, and support simultaneous operations on multiple double-precision floating-point values, single-precision floating-point values, integer values, etc. In some examples, the vector instruction set includes operations such as vector addition, subtraction, multiplication, division, fused multiply-add (FMA), and various logical and comparison instructions, all of which can be applied across multiple data elements during the execution of a single vector instruction. The vector instruction set can also provide mechanisms for loading and storing aligned or unaligned data from memory into vector registers, allowing flexibility in handling diverse data layouts while maintaining performance. The foregoing examples are illustrative and not limiting.

[0136] In some examples, the computing device further includes graphics processing units (GPUs) or specialized accelerators (e.g., FPGAs or ASICs) to support parallel processing and low-latency operations. In some examples, some of the computer-executable instructions that perform the market data processing can be executed on such hardware of the computing device. The device may further include volatile and non-volatile memory components, such as DRAM, SSD storage, and persistent storage arrays, to enable rapid access to large volumes of financial market data. In some implementations, the computing device may operate within a distributed cluster, a high-availability configuration, or a load-balanced environment to ensure scalability and fault tolerance. The computing device may support networking protocols (e.g., secure or non-secure networking protocols) for communication with datastores 140 and / or client devices 150. These examples are illustrative and not limiting.

[0137] In some examples, a source device 110 and the MDP engine 130 may be physically located in close proximity to each other (e.g., within a co-location center, on the same floor of a co-location center, or within the same room of a co-location center). In other examples, the MDP engine 130 may be remote from the source device 110. In some cases, when a conventional MDP device is located a first distance from a source device 110 and an embodiment of the MDP engine 130 is located a second distance from the source device 110, where the second distance is somewhat greater than the first distance, the MDP engine 130 can (1) receive market data messages 112 from the source device 110, (2) process those messages, and (3) provide a market activity message 134 to the source device 110 based on the market data message 112 before the conventional MDP device completes similar data processing activities (1 )-(3), even though the conventional MDP device benefits from reduced data transmission times during steps (1) and (3).

[0138] The system 100 can include a datastore 140 configured to store market data 132. In some examples, the datastore 140 includes a last value cache (LVC) 142. Some non-limiting examples of LVC’s and techniques for updating LVC’s are described herein. In some examples, the datastore 140 is hosted by the same computing device (e.g., server) or set of computing devices that hosts the MDP engine 130. In some examples, the datastore 140 is hosted in a computing device separate from the computing device(s) that hose the MDP engine 130.

[0139] The datastore 140 may include any suitable data storage system configured to store, organize, and provide access to market data 132. In some embodiments, the datastore can include a relational database, a non-relational (NoSQL) database, an object-oriented database, or a distributed data storage system. The datastore may be implemented using technologies such as SQL-based systems, key-value stores, document databases, or columnar databases, and can reside on volatile or non-volatile memory. The datastore may be hosted on any suitable computing device or environment, including but not limited to physical servers, virtual machines, cloud-based platforms, or containerized environments. In certain implementations, the datastore may be deployed in a centralized architecture, a distributed architecture, or a hybrid configuration, and may utilize redundancy, replication, and / or sharding techniques to ensure scalability and fault tolerance. These examples are provided for illustration and are not limiting.

[0140] The system 100 can include one or more client devices 150. In some examples, a client device 150 may execute any suitable financial market data analysis and / or trading software, which may be configured to process the market data 132 and provide trading functionality or other time-sensitive functionality based on the market data 132. In some examples, the client device 150 executes trading software that, based on the market data 132, can determine whether to accept or deny other parties’ bids and / or offers, generate market activity messages 134, such as bids and / or offers for financial instruments, and transmit the market activity messages 134 regarding such bids, offers, acceptances, and denials to one or more market exchanges via the network(s) 120.

[0141] In some examples, software executed by the client devices 150 can provide visualization and / or decision-support functionality. Such software may include charting tools, technical analysis facilities, risk assessment engines, and / or algorithmic trading interfaces. In some embodiments, the software may support real-time data streaming, historical data retrieval, and customizable dashboards for monitoring market conditions. The analysis software may implement various computational techniques, such as statistical modeling, predictive analytics, and machine learning algorithms, to identify trends and generate insights. The software may be deployed as a standalone application, a web-based platform, or a cloud-hosted service, and may integrate with external data sources, APIs, and trading systems. These examples are illustrative and not limiting.

[0142] A client device may include any suitable computing device configured to retrieve, present, and analyze market data 132. Non-limiting examples of client devices can include dedicated servers (e.g., in data centers), desktop computers, laptop computers, tablet computers, smartphones, and other mobile computing devices. In some examples, the client device is implemented as a dedicated workstation, a thin client, or a virtualized client environment. The client device may include one or more processors, memory components, display interfaces, and network interfaces for communicating with the MDP engine 130, a datastore 140, or other remote devices over a network. The client device may support communication protocols (e.g., secure or non-secure communication protocols) for transmitting and receiving market data 132.

[0143] It is to be appreciated that FIG. 1 A is intended neither to be a depiction of necessary components for a computer system or device to operate in accordance with the principles described herein, nor a comprehensive depiction. The computer system 100 can include additional components, omit one or more of the described components, include multiple instances of a component (even if only one instance is shown in FIG. 1 A), and / or can be organized in any other suitable manner.

[0144] Referring to FIG. IB, a system 160 for creating an MDP engine may include a MDP code generator 180 and one or more client devices 170. In some examples, a user can operate the client device 170 to prepare configuration data 172 for the MDP code generator 180. The output of the MDP code generator 180 may include source code or executable code for a feed handler of a MDP engine 130. The configuration data 172 may control how the output of the MDP code generator 180 processes market data messages 112. Such configuration data 172 can include, without limitation, an identification of data fields to be processed, certain characteristics of those data fields, indications of how data fields are to be grouped, a specification of message formats of interest to a user and / or of data fields of those message formats that are of interest to a user, etc. In some examples, based on such configuration data, the MDP code generator 180 can generate code (e.g., code for a customized feed handler) suitable for efficient, SIMD-based processing of market data messages 112 in accordance with the user preferences or interests indicated by the configuration data, and can use that customized feed handler to process market data messages on behalf of the user. Some non-limiting examples of code generation techniques suitable for generating code for a customized feed handler based on configuration data are described herein.

[0145] An example has been described in which a client device 170 prepares configuration data 172 for an MDP code generator 180, which generates code for the MDP engine 130 based on the configuration data 172.

[0146] A client device 170 may include any suitable computing device configured to prepare configuration data 172. Non-limiting examples of client devices can include desktop computers, laptop computers, tablet computers, smartphones, and other mobile computing devices. In some examples, the client device is implemented as a dedicated workstation, a thin client, or a virtualized client environment. The client device may include one or more processors, memory components, display interfaces, and a means of submitting the configuration data into the MDP code generator, e.g. via a network connection.

[0147] It is to be appreciated that FIG. IB is intended neither to be a depiction of necessary components for a computer system or device to operate in accordance with the principles described herein, nor a comprehensive depiction. The computer system 160 can include additional components, omit one or more of the described components, include multiple instances of a component (even if only one instance is shown in FIG. IB), and / or can be organized in any other suitable manner.

[0148] Below, some examples are discussed based on a feed handler for the Consolidated Quote System (CQS) data feed, one of the key data feeds forming the Consolidated Tape Association (CTA) Securities Information Processor (SIP). It should be appreciated that CQS from CTA is only illustrative of a number of different feeds with which embodiments may operate. CQS was chosen here as an example feed as the specifications are freely available and it’s a widely-used feed that, while generated in the U. S., is delivered worldwide. The CTA delivery protocols and message types used exemplify features which appear in many other market data feeds, although the technical details (e.g., the fields included in each type of message and the ordering of those fields, etc.) can differ significantly from one market data feed to another.

[0149] The information of a market data feed includes a stream of pricing and other market information, formatted according to a data feed specification and sent via a network transmission protocol. Specifications vary by exchange, but CQS (like some others) uses UDP Multicast as a transmission mechanism, with a bespoke format comprising a series of blocks (one per UDP datagram), each containing one or more messages of various types depending on the market activity being published. Each block begins with a block header giving essential information such as transmission timestamp, block sequence number (for detecting missed blocks) and the number of messages contained in the block. Messages also have their own header which, among other things, indicates the message type and length of the message data. Armed with the information from the headers, the remainder of the data can be decoded and acted upon before moving on to the next block.

[0150] Peak message rates for the busiest data feeds can exceed 50 million / second, so performance is critically important in terms of managing footprint and meeting system latency targets.

[0151] The CQS Specification defines thirteen different message types; some are used much more than others. One of the more frequently used message types is called Long Quote, which is sent to provide pre-trade pricing (a.k.a., quote) information, and we have selected this as the focus of our examples. The Short Quote message type is also used frequently.

[0152] The Long Quote message contains the recent pre-trade prices (bid and offer) for a given instrument (for example, a common stock such as Microsoft). Data fields in the Long Quote message include the symbol identifying the instrument (e.g., MSFT), a single-letter participant code indicating which regional stock exchange is the source of the message, and the bid and offer prices, sizes and publication times.

[0153] The purpose of a feed handler is to process the ‘raw’ incoming feed (e.g., market data messages 112) into another form (e.g., market data 132) for programmers or other end-users. Functions of a feed handler may include one or more of the following (as well as potentially other functions): (1) Removing network transmission protocol information from each incoming data packet, to leave the data feed information (e.g., information of the market data messages 112). (2) Parsing the remaining data and copying it into an in-memory data structure for further processing. Various kinds of validity check may be performed at this stage. (3) Performing format conversions and ‘normalization’, e.g., converting values from their on-feed format (e.g., the format used in the market data messages 112) into a different format preferred by an end user or suitable for processing by the end user’s software. (4) Maintaining a Last Value Cache (LVC) (which may be a logical cache, which may be partially or wholly stored in a memory cache) of the most recent value of one or more fields (e.g., all fields) for each instrument. (5) Performing value-added calculations in real-time, e.g., comparing the incoming message to the values in the LVC to determine whether a price just moved up or down. (6) Writing any modified values back to the LVC. (7) Sending the results of processing to a user, e.g., over a network connection. Note that feed handlers can also be used to process captured network data from files, e.g., in PCAP format, and can also output to files rather than a live network, a capability widely used for trading analytics and quantitative research.

[0154] Described below are examples of feed handler functions that can offer high throughput and low latency. As noted previously, some examples below relate to one message type from the CQS data feed (Long Quote) This message accounts for a significant percentage of the messages on that feed and has quite a rich structure with varied processing needs, such that a small number of test messages exercise many features described below.

[0155] Referring to FIG. 2A, an example method 200 for performing SIMD-based financial market data processing (MDP) may include steps 210-230. The method 200 may be performed, for example, by the MDP engine 130.

[0156] In step 210, the MDP engine receives financial market data messages 112 provided by one or more source devices 110. As described above, these messages are often received at a very high rate (e.g., tens of millions of messages per second).

[0157] In step 220, the MDP engine performs financial market data processing (MDP) on fields of the financial market data messages 112 using SIMD hardware, thereby obtaining financial market data 132. In some examples, performing the MDP includes converting the data contained in the financial market data messages 112 from a first format to a second format. Such conversion may involve extracting the fields of the financial market data messages from CPU memory (e.g. a CPU cache) and transferring to registers of SIMD hardware in an order that differs from the order of the fields within the messages. In some examples, such conversion may also involve applying data transformations to the extracted fields. Some non-limiting examples of techniques for converting the data of the financial market data messages from a first format to a second format are described herein.

[0158] In some examples, the process of converting the data contained in the financial market data messages from a first format to a second format may be in full or in part influenced by (e.g., based on) data fetched from a last value cache (LVC) in response to processing of the message (e.g., the header or the payload of the message). Such LVC data may in some examples be fetched using SIMD instructions.

[0159] Performing the MDP may include determining, based at least in part on the contents or types of the messages and / or on data fetched from a last value cache (LVC) based on processing of the message, one or more fields of the last value cache (LVC) to be modified (e.g., initialized or updated) and / or one or more values (e.g., initial values or updated values) to be stored in those fields. At least some of the steps involved in making these determinations (e.g., steps involving processing of the fields of the messages or accessing the LVC) may be performed using SIMD hardware (e.g., performed by executing SIMD instructions with operands that can include SIMD registers, mask registers, general-purpose registers and / or memory operands). In some examples, determining the field(s) of the LVC to be modified can be performed before or after the data contained in the market data messages 112 is transformed from a first format to a second format. Determining the values to be stored in those fields can be based on the values of the fields of the market data messages 112 and the current values of the fields of the LVC. Some non-limiting examples of techniques for determining values to be stored in the LVC are described herein.

[0160] In some examples, performing the MDP includes performing SIMD operations (e.g., executing SIMD instructions) on vectors stored in vector registers of the SIMD hardware. The elements of the vectors may be fields of the financial market data messages 112 or values derived from those fields. In some examples, the fields (or derived values) stored in the elements of a vector are all of the same width (e.g., 8 bits, 16 bits, 32 bits, or 64 bits). In some examples, before performing the SIMD operations on the vectors, the fields of the financial market data messages 112 are copied to vector registers of the SIMD hardware. For example, in a system in which the vector registers have a width of N bytes, data of the market data messages 112 can be extracted (e.g., from the memory of a network interface card (NIC), system memory, or a CPU cache) in chunks of M*N bytes (where M is determined by the number of available vector registers and their widths), the fields of the extracted data can be rearranged (e.g., “permuted,” “shuffled,” or “swizzled”) in accordance with the configuration data 172, and the rearranged fields can be copied into vector registers of the SIMD hardware. In some examples, this process does not involve an intermediate step of storing the rearranged fields back to system memory or to the CPU cache before copying the rearranged fields to the vector registers. The next N-byte chunk of data extracted from the market data message 112 can then be processed in the same way. In some examples, preparing the data of the financial market data messages for SIMD processing and performing SIMD operations on the prepared data involves performing the steps of the method illustrated in FIG. 2B.

[0161] In step 230, the MDP engine stores at least a portion of the financial market data in a datastore 140 and / or provides at least a portion of the financial market data to a client device. Storing a portion of the financial market data in a datastore may include storing the portion of financial market data in a last value cache (LVC) maintained in the datastore. Providing a portion of the financial market data to the client device may include transmitting the portion of data to the client device via a network, a shared memory area, or any other suitable mechanism. As described herein, the client device may process the data (e.g., analyze the data, make trading decisions based on the data, etc.) and / or present the data to a user.

[0162] Referring to FIG. 2B, an example method 250 for processing fields of financial market data messages using SIMD-based hardware may include steps 252-256. The method 250 may be performed, for example, by the MDP engine 130. In some examples, step 220 of the method 200 of FIG. 2A is carried out by performing the method 250.

[0163] In step 252, the MDP engine makes the fields of the financial market data messages 112 available for the financial market data processing by the SIMD hardware. Making the fields of the financial market data messages available for processing by the SIMD hardware may include placing the message data in a memory accessible to the SIMD hardware (e.g., a memory of a network interface card (NIC) of the system 100, or a memory of a CPU that includes the SIMD hardware, such as a cache memory of the CPU).

[0164] In step 254, the MDP engine 130 transfers the fields of the financial market data messages to elements of vector registers of the SIMD hardware, such that each vector registers stores a respective subset of the fields and all fields in the respective subset have the same width. In some examples, transferring the fields of the market data messages to the vector registers may include copying values from the fields of the incoming financial market data messages to elements of vector registers in the SIMD hardware. In some examples, this mapping (e.g., simultaneous transferring and rearranging of the fields) is performed based on configuration data that specify which fields of the financial market data messages map to which fields of the vectors. In some examples, performing the mapping includes storing different subsets of the fields of the incoming messages to different vectors based on the widths of the fields of the incoming messages, such that fields of the incoming messages having the same width are mapped to vectors in which all fields have that same width, and fields of the incoming messages of different widths are mapped to different vectors. As described herein, mapping the fields of the messages to the vectors in this manner can involve delaying the processing of the vectors until all portions (e.g., ‘chunks’) of an incoming message have been processed and the fields of interest from those portions of the message have been copied to a set of vector registers. In some examples, fields may be processed in order of size with all fields of a given size being copied to vector registers and SIMD processing triggered before moving on the next size.

[0165] In step 256, the MDP engine performs SIMD operations on the vector registers. In some examples, at least one vector stored in the vector registers contains data specific to a financial instrument and to a financial market exchange. Such vectors are sometimes referred to herein as “record vectors.” In some examples, at least one vector stored in the vector registers contains data generic to one or more financial instruments (e.g., the financial instrument to which the record vector pertains) and / or generic to a financial market exchange (e.g., the financial market exchange to which the record vector pertains). Such vectors are sometimes referred to herein as “shared record vectors.” In some examples, at least one vector stored in the vector registers contains data specific to one or more financial instruments (e.g., the financial instrument to which the record vector pertains) and specific to a second financial market exchange (e.g., not the financial market exchange to which the record vector pertains).

[0166] There is another item not yet addressed: a set of engine settings that control functionality supported by the MDP engine.

[0167] Together, the data structures and configuration data described herein constitute metadata that can be (and in this example are) used to direct code generation during the process of building the MDP engine (e.g., a code generation process). The engine emits specific code in accordance with the metadata, and hard-coded choices regarding which code to emit are made at code generation time in accordance with the functionality specified in the metadata, for example: (1) Generic programming methods can be used extensively to determine code emission at build time, though other methods could be used. (2) Code that does not require customization according to the metadata, can be imported directly from library code - examples include code for basic network operations or file input and output operations.

[0168] The following steps may occur during the build process, in this example: (1) Relevant library code is imported. This includes basic I / O code for network or file input and output, depending which I / O options are selected, and a feed-specific(e.g., specific to a data feed provided by one or more source devices 110 of a specific source) loop for iterating the incoming data messages. (2) Abstract representations of internal data structures (e.g. RecordVector) are converted into concrete implementations (e.g., for workspace and LVC structures) and code is emitted for them. (3) Code is emitted to transfer fields from the incoming feed message into the SIMD processing pipeline, according to the set of mappings supplied. (4) Code is emitted to perform optional functions of the processing pipeline, e.g., synthetic national best bid and offer (NBBO) generation, output type, different levels of normalization, support for an explicit defined / undefined indicator for fields, if these have been selected. (5) If last value caching is enabled, code is generated to fetch, compare, modify and store last value cache data, and to use the LVC data in any calculations (e.g. to detect price changes) which may affect the output. (6) Code is generated to build user output. (7) The emitted and library code are compiled and linked into an executable.

[0169] In some examples, this process is implemented using any suitable software engineering build tools.

[0170] In some examples, code that is not needed is not generated. The result of this process can be an extremely lightweight, highly customized feed handler that is largely branch-free and has very little code overhead. Revisions may be made to the metadata and a new custom build generated at any time. Because the MDP engine and libraries are relatively static over time and are reused in many different scenarios, a very high level of integrity can be assured.

[0171] Referring to FIG. 3, an example method 300 for generating code for SIMD-based financial market data processing may include steps 310-320. The method 300 may be performed, for example, by the MDP engine 130.

[0172] In step 310, the MDP engine receives first configuration data identifying SIMD hardware and second configuration data identifying a mapping from fields of incoming financial market data messages to vectors. The first configuration data and second configuration data (collectively, configuration data 172) can be provided by a client device 170.

[0173] In some examples, based on the first configuration data, the MDP engine can determine one or more properties of the SIMD hardware (e.g., a width of a cache line in a data cache accessible to the SIMD hardware, widths of vector registers of the SIMD hardware, and widths of elements of the vector registers on which SIMD instructions of the SIMD hardware can operate, etc.).

[0174] In some examples, the second configuration data indicate whether to use aligned memory storage or unaligned memory storage to store data extracted from fields of incoming financial market data messages. Some non-limiting examples of second configuration data are described herein (e.g., with reference to Tables 8 and 10).

[0175] In step 320, the MDP engine generates a feed handler for processing the incoming financial market data messages, including generating code of a feed handler based on the first configuration data and the second configuration data. In some examples, the first and second configuration data are provided by a user, but the source code instructions of the feed handler are not provided by the user.

[0176] Generating code based on the first configuration data may include generating the source code instructions based on the properties of the SIMD hardware ascertained from the first configuration data.

[0177] Generating code based on the second configuration data may include emitting aligned or unaligned memory storage instructions for storage operations consistent with the alignment indications contained in the second configuration data.

[0178] In some examples, generating code for the feed handler based on the second configuration data includes generating, based on the second configuration data, instructions that allocate memory for the vectors and instructions that store, in the vectors, data from fields of the incoming financial market data messages. Generating the instructions that store the data from the fields of the incoming financial market data messages may include generating instructions to divide the fields of an incoming financial market data message between multiple vectors. Each vector of the multiple vectors may include fields of a same width. In some examples, generating the instructions for the feed handler based on the second configuration data includes generating, for each vector of the multiple vectors, instructions to assign the fields of the respective vector to SIMD registers of the SIMD hardware.

[0179] In some examples, generating the instructions for the feed handler includes generating instructions to update a datastore based, at least in part, on values included in the incoming financial market data messages. Updating the datastore can include storing new or updated values to one or more fields of the last value cache (LVC). In some examples, updating the datastore includes reading current values of one or more fields Fi of the LVC, calculating new or updated values for one or more fields Fj of the LVC (e.g., based on the values in the market data messages and the current values of the fields Fi of the LVC), and storing the new or updated values to the one or more fields Fj of the LVC. In some examples, the fields Fi and Fj may overlap at least in part, such that the current values of one or more fields read from the LVC are replaced by the new or updated values.

[0180] In some examples, generating instructions for the feed handler based on the second configuration data includes identifying, based on the second configuration data, one or more transformations or interpretations to be performed on received data that is to be stored in a field of a vector, and generating instructions to perform the one or more transformations or interpretations.

[0181] In some examples, generating instructions for the feed handler further includes generating other instructions to be executed by non- SIMD hardware (e.g., instructions to be executed using a CPU’s scalar hardware).

[0182] Some Examples

[0183] Below, some examples of aspects and implementations of MDP engines, methods for performing SIMD-based financial market data processing, and methods for generating code for SIMD-based financial market data processing based on configuration data are described. These examples are provided for purposes of illustration and are not limiting.

[0184] Some Examples of Methods for Performing SIMD-based Financial MDP

[0185] In connection with the discussion of aspects and implementations of some examples, it may be helpful to understand the format of the incoming message for that feed. Six tables (Tables 1 -6) are shown below, with each table representing a different UDP packet received on the feed. Each table shows the CQS payload of the packet, e.g., a block header, message header and Long Quote message. (In general, it is possible to receive multiple messages within a single UDP packet, but for simplicity the examples are restricted to a single message.) In each table, the first three columns give the CQS name of each data field, a reference to the appropriate section of the CQS Specification, and a naming for that field for the engine of these examples. In each table, the block header appears at the beginning of the UDP pay load and contains the size of the block along with information useful for versioning, lost packet detection and recovery, timestamping and data integrity. A message header contains the length, category and type of the message and a byte identifying the participant exchange (one of the 29 SEC-registered stock exchanges in the US) from which the data originates, among other info.

[0186] The Long Quote message identifies the instrument (e.g., a stock) to which the data applies by security symbol, along with the actual quote prices and related information. See the Consolidated Quotation System Multicast Binary Output Specification, published by the Securities Industry Automation Corporation, (“CQS Specification”) for the allowable values and meanings of these fields.

[0187] The Long Quote message contains a field called the National BBO Indicator. This is regulatory information which determines the current best trading price across all US exchanges (i.e. the best price available nationally). Generally, financial agents dealing on behalf of clients are not permitted to trade at a worse price than the NBBO. The NBBO is not sent explicitly on the feed, but is communicated through a system of indicators and appendages to save bandwidth.

[0188] The rules for determining the current NBBO are covered in the CQS Specification, but note that, depending on the National BBO Indicator, there may be appendages attached to the Long Quote message containing additional NBBO information (in the example messages, the fourth message, illustrated by Table 4, has 2 appendages added). The NBBO can be derived and tracked in a synthetic NBBO record, which can be given a pseudo-participant code to distinguish it from a real participant exchange. Because the National Best Bid and National Best Offer might be from two different exchanges, separate Bid Participant ID and Offer Participant ID fields can be maintained in the NBBO record. Table 1.

[0189] -

[0190] .. 1?. _

[0191] .. _.

[0192] sr -

[0193]

[0194] Table 2.

[0195] -

[0196] .. 1?. _

[0197] .. _.

[0198] -

[0199]

[0200] Table 3.

[0201] . _

[0202] "0" [2® _

[0203]

[0204] Table 4

[0205] _

[0206] _

[0207] .

[0208]

[0209] Table 5.

[0210] CQS Header and Field Naming V2.9g CQS Ref BSFT Struct and Member Field Naming Type Size Default Override Hex Offset from Block Header or MESSAGE HEADER 2 02 00

[0211] b 108 1006c |01 02

[0212] " Q" 51 03

[0213] c 1 "0" 4f 04

[0214] b 4 b0blb2b3 b0blb2b3 05 06 07 08

[0215] 1 01 09

[0216] b 4 b4b5b6b7 b4b5b6b7 0a 0b 0c 0d

[0217] b 4 b8b9babb b8b9babb 0e 0f 10 11

[0218] b baad 0x3896 |3896 |12 13

[0219] 87 10057 |00 01

[0220] " Q" 51 02

[0221] " L" 4c 03

[0222] c 1 " Z" |5a 104

[0223] bcbdbebf bcbdbebf 05 06 07 08

[0224] c0clc2c3 c0clc2c3 09 0a 0b 0c

[0225] 1 01 0d

[0226] c4c5c6c7 c4c5c6c7 0e 0f 10 11

[0227] 8081828384858687 8081828384858687 12 13 14 15 16 17 18 19

[0228] " BSFT" 4253465420202020202020 la lb 1c Id le If 20 21 22 23 24 60 60 25

[0229] c 1 61 61 26

[0230] c 1 SP |20 |27

[0231] 88898a8b8c8d8e8f 88898a8b8c8d8e8f 28 29 2a 2b 2c 2d 2e 2f b 4 c8c9cacb c8c9cacb 30 31 32 33

[0232] *9091929394959697 "9091929394959697 34 35 36 37 38 39 3a 3b b 4 cccdcecf cccdcecf 3c 3d 3e 3f

[0233] *62 62 '40

[0234] *63 *63 *41

[0235] *64 64 42

[0236] d0dld2d3 d0dld2d3 43 44 45 46

[0237] c 1*65 %5 *47

[0238] b 4:d4d5d6d7 d4d5d6d7 48 49 4a 4b

[0239] b 4:d8d9dadb d8d9dadb 4c 4d 4e 4f

[0240] c 1;66 *66 50

[0241] *67 *67 *51

[0242] *68 68 52

[0243] *69 *69 *53

[0244] 6a 6a *54

[0245] c 1 6b 6b *55

[0246] c 1 " A",41 fs 6

[0247] 0 00

[0248]

[0249] Table 6.

[0250] CQS Header and Field Naming V2.9g CQS Ref BSFT Struct and Member Field Naming Type Size Default Override Hex Offset from Block Header or MESSAGE HEADER b 1 *2 02 "00

[0251] b 2 "108 [006c 01 02

[0252] c 1 " Q" Vl 03

[0253] c 1 "0" 4f *04

[0254] b 4 b0blb2b3 b0blb2b3 05 06 07 08

[0255] b 1 '1 '01 *09

[0256] b 4 b4b5b6b7 b4b5b6b7 0a 0b 0c 0d

[0257] b 4 b8b9babb b8b9babb 0e 0f 10 11

[0258] b 2 baad 0x389c |389c 12 13

[0259] b 2 87 10057 00 01

[0260] c 1 " Q" 51 *02

[0261] c 1 " L" 4c "03

[0262] c 1 " Z" |5a f04

[0263] b 4 bcbdbebf bcbdbebf 05 06 07 08

[0264] b 4 c0clc2c3 c0clc2c3 09 0a 0b 0c

[0265] b 1 *1 *01 0d

[0266] b 4 c4c5c6c7 c4c5c6c7 0e 0f 10 11

[0267] b 8 *8081828384858687 *8081828384858687 12 13 14 15 16 17 18 19 c[ ] 11 " BSFT" *4253465420202020202020 la lb 1c Id le If 20 21 22 23 24 i

[0268] 00 c 1 *60 60 "25

[0269] c 1 "61 *61 "26

[0270] c 1 SP [20 [27

[0271] b 8 88898a8b8c8d8e8f 88898a8b8c8d8e8f 28 29 2a 2b 2c 2d 2e 2f b 4 c8c9cacb c8c9cacb 30 31 32 33

[0272] b 8 "9091929394959697 *9091929394959697 34 35 36 37 38 39 3a 3b b 4 cccdcecf cccdcecf 3c 3d 3e 3f

[0273] c 1 "62 *62 *40

[0274] c 1 *63 63 "41

[0275] c 1:64 *64 *42

[0276] c[ ] 4;d0dld2d3 d0dld2d3 43 44 45 46

[0277] c 1*65 *65 *47

[0278] b 4 d4d5d6d7 d4d5d6d7 48 49 4a 4b

[0279] b 4:d8d9dadb d8d9dadb 4c 4d 4e 4f

[0280] c 1 "66 "66 *50

[0281] c 1 *67 *67 "51

[0282] c 1 "68 *68 *52

[0283] c 1 *69 69 "53

[0284] c 1 6a 6a *54

[0285] c 1 6b 6b "55

[0286] c 1 " G" (47 "56

[0287] b 1 "0 "00

[0288] 108

[0289]

[0290] 87 When a Long Quote message is processed, it may (e.g., not if the exchange is being filtered out) update the contents of the appropriate participant record and, depending on the value of the National Best Bid and Offer (NBBO) Indicator field, may additionally update the NBBO record.

[0291] The Long Quote message may also contain a Security Status Indicator, the contents of which can also have a significant effect on processing. The third sample message (Table 3) demonstrates this.

[0292] The Long Quote message (ignoring the block header but including the message header) is just 87 bytes long while containing 30 fields of different sizes, types and semantics (we decompose the Timestamp 2 field into two fields resulting in 31 fields). Optional appendages to the message come in several variations and can add between 5 and 36 bytes to the message while adding between 3 and 10 more fields.

[0293] During processing, data is held in temporary workspace, in a mixture of CPU registers and memory. In accordance with some techniques described herein, data structures can be defined for the workspace that are well-suited to SIMD processing.

[0294] Data can optionally also be stored in a last value cache (LVC) in connection with processing. As such, LVC data structures can also be defined. It can be advantageous in some embodiments for these to be similar to the workspace formats to avoid unnecessary conversions when moving data to and from LVC.

[0295] Identified below are some design choices that can improve performance in some embodiments, after which specific illustrative data structures are discussed:

[0296] 1. Represent the basic unit of storage (both for workspace and LVC) as a structure (“record”) containing data expressed in basic SIMD-compatible elements.

[0297] 2. In some examples, a record is or includes a collection of abstract record vectors. Each record vector holds fields of the same size (they may be different types) and each data field is present in exactly one record vector. The use of record vectors supports efficient SIMD processing and facilitates: (a) Dissociating the logical record definition from the details of the target SIMD implementation (e.g. AVX-512 or AVX2) in a way that can be automatically mapped onto different SIMD targets, (b) Supporting multiple field types in the same record vector e.g. in a vector of 64-bit element sizes, support storing of both u64 element types and i64 element types. This leverages the ability of many integer SIMD instructions e.g. load / add / compare / store etc. to process a mixture of signed and unsigned elements. Going further, other 64-bit types can be stored in the record vector, e.g., fixed-point integer quantities that might want to support non-standard widening / upscaling conversions from shorter e.g. 16-bit versions of those fixed-point numbers, (c) Allowing automatic improvement (e.g., optimization) of placement of data regardless of which data fields are selected into the build, allowing efficient customization.

[0298] 3. Support grouping the record vectors into vector groups as a mechanism for supporting memory cache optimization.

[0299] 4. Support the notion that different records can be related to each other and may need to be retrieved from LVC and processed as part of a single processing function. The related records can support these additional properties: (i) Support an 0(1) operation to find a related record, (ii) Support a common or shared set of fields between the related records to minimize total storage size and reduce processing requirements. An update to one record may then update these shared fields which are also visible to all other related records.

[0300] 5. Support partial record access.

[0301] 6. Support transient or temporal fields that can be processed like regular fields but are not stored in the LVC.

[0302] 7. Support a range of record and vector memory alignment options for the storage of the records, including highly dense packing options when a table places a high demand on the memory cache but also more sparse options that can facilitate faster data access when processing a data feed that incurs lower demands on the memory cache. In some embodiments, elements can be a building block of data structures and have a type and size. Element type may, in some embodiments, be one of unsigned integer, signed integer or floating point. Element size in such an embodiment can be one of 8-bit, 16-bit, 32-bit or 64-bit. The range of available types and sizes may be different in other embodiments.

[0303] A list of elements may therefore include: u8 (unsigned 8-bit integer), ul6 (unsigned 16-bit integer), u32 (unsigned 32-bit integer), u64 (unsigned 64-bit integer), i8 (signed 8-bit integer), i 16 (signedl6-bit integer), i32 (signed 32-bit integer), f32 (single-precision 32-bit floating point), i64 (signed 64-bit integer), f64 (double-precision 64-bit floating point). Some embodiments may also support defining array-elements that are fixed-length arrays of the above element types, e.g.: u8[n] (array of n elements of type u8), etc. These elements (and arrays thereof) are compatible with currently commercially available SIMD processors (e.g., AVX-512 and AVX2 SIMD processors).

[0304] Fields represent the individual data items that are parsed from an incoming feed. They can be assigned a name and a numeric identifier or field id. When included in a record vector (see below) each field is associated with an element type and length.

[0305] Market data feeds typically define (in the feed specification) their own lists of field names and field ids to use within their messages, and these apply to incoming feed data (e.g., incoming market data messages 112). In some embodiments, in the context of a market data processing engine operating in accordance with some techniques described herein data structures may use definitions provided herein of field; feed data structures use the feed spec definition.

[0306] In some embodiments of the engine, a record vector is an array of (1 to 64) elements. 64 is chosen as it’s the largest number of bits permitted in an AVX-512 bitmask. This is sufficient for most market data purposes, though can be extended to larger sizes or for other iterations of AVX if desired.

[0307] Each record vector represents fields of a particular element size. Zero or more record vectors may be used for each element size. Any shared fields are allocated to a distinct shared (set of) record vector(s). Mixing of types of equal size within each record vector can be permitted.

[0308] During compilation, each record vector can be transformed into a list of concrete SIMD vector definitions (e.g., vector definitions based on the features of the SIMD hardware on which the code will execute). For example, a relevant feature of SIMD hardware that can affect the SIMD vector definitions is the width of the SIMD hardware’s vector registers, which may be 512-bit, 256-bit or 128-bit or any other supported width. In aggregate, such vector registers may hold all the fields defined by the (abstract) record vector definitions. This transformation depends on the SIMD target implementation and is automated.

[0309] The result (a record) is a set of data structures tailored to a particular concrete SIMD implementation, which represents all the data fields of interest compactly, and separates shared fields from ‘regular’ (e.g., non-shared) fields but otherwise gathers elements of equal size together. The record structure is well-suited to use in the workspace and the LVC, bearing in mind that the LVC excludes any transient fields.

[0310] Using our CQS example and analyzing the various message types in the feed specification, the following element sizes are available to process and store:

[0311] Element size (bits) Count Comment

[0312] 64 13 1 transient field

[0313] 64 2 2 shared fields

[0314] 32 19 1 transient field

[0315] 16 0

[0316] 8 20

[0317] 8 [4] 4 4 array fields of type u8[4]

[0318] From these the following record vector definitions are derived, separating the shared from non-shared fields:

[0319] u64_l

[0013] (1 transient)

[0320] u32_l

[0019] (1 transient)

[0321] u8_l

[0036] (the 4 u8[4] of the array fields can be stored within here) shared_u64_l [2] (2 shared fields)

[0322] For an AVX-512 build of the above vector definitions, the engine may transform the above into the following list of SIMD vectors (“#0” could be read as “part 0”, “#1” as “part 1” etc.):

[0323] Record Vector SIMD Vector List SIMD Elements SIMD Register Size Used (bits)

[0324] u64_l u64 l#0 8 / 8 512

[0325] u64 1#1 5 / 8 512

[0326] u32_l u32 l#0 16 / 16 512

[0327] u32 1#1 3 / 4 128

[0328] u8 1 u8 1 36 / 64 512

[0329]

[0330] shared u64 1 shared u64 1 2 / 2 128

[0331] Similarly, an AVX2 build may generate the following lists of SIMD vectors:

[0332] Record Vector SIMD Vector List SIMD Elements SIMD Register Size Used (bits)

[0333] u64 1 u64 l#0 4 / 4 256

[0334]

[0335] u64 1#1 4 / 4 256 u64 1#2 4 / 4 256

[0336] u64 1#3 1 / 2 128

[0337] u32_l u32 l#0 8 / 8 256

[0338] u32 1#1 8 / 8 256

[0339] u32 1#2 3 / 4 128

[0340] u8 1 u8 1#1 32 / 32 256

[0341] u8 1#2 4 / 32 128

[0342]

[0343] shared u64 1 shared u64 1 2 / 2 128

[0344] In some examples, transient field definitions may be included along with the regular fields in a record vector definition, and may in some cases be placed at the end of the definition.

[0345] In some examples, the SIMD Elements Used column and SIMD Vector Size column refer to the space used in the workspace vectors that are used in the processing pipeline. The amount of space reserved in the LVC excludes transient elements. For example, the u64_l record vector definition always allocates LVC space for 12 u64 (64-bit elements), for the 12 non-transient fields.

[0346] In some embodiments, array fields do not span across SIMD vectors e.g., in the AVX-512 definitions, any arrangement of the u8[4] elements in the u8_l vector definition is acceptable as all the elements fit inside the single SIMD vector (u8_l). In the case of an AVX2 build with such an implementation, it may have performance advantages to ensure that no u8[4] array field definitions span the u8_l#l and u8_2#2 SIMD vectors.

[0347] Shared fields are put into their own vectors that are accessible by any of the records that are related to each other.

[0348] Should the required set of fields change (due, say, to an exchange driven change) the process can be re-run and will again result in a compact vector representation. An exchange driven change can include any change to the content, format, timing, semantics, or other properties of the market data messages 112.

[0349] Record vectors have additional properties defining their group membership (see Vector Groups), memory alignment and element type.

[0350] With vectors aligned per examples above, a record vector structure is partitioned in a way that can increases efficiency of placement of the data in the memory cache hierarchy. To do this, a vector group can be configured and maintained. In creating a vector group, some embodiments may optimize for the most frequent message types, of which our CTA Long Quote message is a good example. From the example above, vector definitions can be partitioned into three vector groups: a ‘hot’ group containing all the fields from the Long Quote message; a ‘cold’ group for remaining fields; and a group containing only shared fields:

[0351] hot group cold group shared group

[0352] u64_l[3] (1 transient) u64_2

[0010] shared_u64_l [2]

[0353] u32_l[7] (1 transient) u32_2

[0012]

[0354] u8_l

[0012] u8_2

[0024]

[0355] For an AVX-512 build of the above vector definitions, the engine may create the following set of SIMD vectors (AVX2 definitions are omitted from this example):

[0356] Record Vector SIMD Vector List SIMD Elements SIMD Register Size (bits)

[0357] u64 1 u64 1 3 / 4 256

[0358] u32 1 u32 1 7 / 8 256

[0359] u8 1 u8 1 12 / 16 128

[0360] u64_2 u64 2#0 8 / 8 512

[0361] u64 2#1 2 / 2 128

[0362] u32 2 u32 2 12 / 16 512

[0363] u8 2 u8 2 24 / 32 256

[0364]

[0365] shared u64 1 shared u64 1 2 / 2 128

[0366] In the SIMD register size column, the size of the SIMD register may be automatically selected from available registers to increase overall efficiency in use of the registers for the available data. Although two three-vector group definitions (plus a shared group) are used in this example, in some embodiments any number of vector groups can be defined. During the processing of any specific message from the feed (e.g., the feed of market data messages 112), the processing engine may update any number of private or shared SIMD vectors that exist.

[0367] Vector groups may have further properties, including (1) a group type indicating whether the LVC holds a single instance of the vectors in the group, or an array thereof; (2) a property controlling the placement of the group with respect to other vector groups; and (3) a property controlling the memory alignment of the group.

[0368] In some embodiments, a last value cache (LVC) may operate in accordance with at least some of the following design specifications. The LVC may store record arrays in a structure known as a record hive. Each record hive can have a user-defined key. A record array may contain one or more records, and each record in the record array may have a sub-key (e.g., a user- defined sub-key). For each record in the record array, the LVC may store a set (call it {v}) of record vectors, where each record vector in {v} includes (as per the discussion of record vectors above) a collection of elements of a particular size. The fields to be contained in the record array, among other characteristics, may be determined by a vector group definition or otherwise.

[0369] Each record hive can contain one or more shared record arrays, where each shared record array entry contains one or more shared record vectors. The purpose of the shared record array(s) is to hold fields that may relate to more than one (non-shared) record array entry, or that are unrelated to any record array entry, or for other purposes. The shared record vectors may be determined by a vector group definition. Each record hive has an associated index of every record array (including any shared record arrays) in the hive. The time to access the index for a given record array may be 0(1). Each record array can have an associated index of every record vector in the array. In some embodiments, any given record vector may be converted to one or more concrete SIMD vectors, in which case an index of a record vector includes an index to each concrete SIMD vector. Access to any item (hive, record array, record vector or concrete SIMD vector) can be achieved with 0(1) complexity (e.g., in constant time).

[0370] Embodiments that use this LVC design may have parameters controlling how the LVC data is partitioned and stored in memory. Two possibilities are shown in Table 7 below. On the left, each record hive contains two record arrays (hot array and cold array) plus one shared record array. Note that in this example, all hot array data for a given record hive is stored together, (separately) all cold array data for a given record hive is stored together, and (again separately) all shared array data is stored together. On the right, each record hive contains only a single record array, plus one shared record array. Depending on the use case, either can give efficiencies. The example explains a case relevant to market data.

[0371] The design is novel, in that data for a given financial instrument and participant can be partitioned into any number of parts via the record array mechanism. This allows the MDP engine to group fields in a way that increases storage density and avoids wasting memory cache space. Also, the design naturally lends itself to situations, common in market data, where a group of records is known to update at a similar time. Table 7. Array Storage.

[0372] Participant i hot^array » i. ". [. <i8 1. [ ■■■^ '2 ■ ■•. u32"2 ■]. u’g’z’ P i; r ■.~tfM. Aw.'A i. A732 m''r.~'! I. <78 m'1. F <."1 g4 w'2..'"7 J."" S32."w2."" T i.7'8 m'2...'

[0373] sharsd TNeUTj hot_arrsy and scared

[0374]

[0375] For the example CQS data feed, the National Best Bid and Offer (NBBO) and regional records may be well served by this architecture in some embodiments, for one or more of the following reasons. First, the record hive key naturally maps onto the instrument identifier (e.g., BSFT) while the record sub-keys naturally map onto the participant codes. Second, processing a regional Long Quote message frequently updates the NBBO record e.g., the second and fourth sample messages’ (illustrated in Tables 2 and 4) records update both the P participant and the NBBO record. Third, the third message (from Table 3), with a Security Status Indicator of 0 or Limit Up-Limit Down (LULD) Price Band is information that is at security level rather than participant level, i.e. applies to each and every participant record for the instrument, and therefore is naturally represented by a shared record array. Fourth, a change in quote price at one exchange often correlates with a cascade of quotes from other exchanges, for a given instrument. There are therefore efficiencies in storing the quote-related fields from all the participant records together, especially if stored densely such that activity from one participant keeps other participants’ data in memory cache.

[0376] Fifth, there can be advantages to storing fields that are often accessed separately from fields that are seldom accessed, to avoid filling the memory cache with unused data. This can be achieved, for example, using the hot and cold vector groups in the left-hand side of Table 7. Sixth, there may be times when the ‘cold’ fields are more likely to change than the ‘hot’ fields, e.g. during overnight processing outside trading hours. These fields also benefit from greater storage density and therefore better memory cache performance. Seventh, in practice, there may be many different sets of fields that can benefit from being grouped this way (e.g., batch processing of PCAP files is one example of a scenario in which it may be useful to define many groups), and the proposed LVC design permits an arbitrary number of groups.

[0377] In addition to the above- described techniques for allocating fields to record vectors, vector groups, and record arrays, memory performance may be increased in some embodiments with granular control of memory alignment. For example, a parameter value(s) may determine a memory alignment policy when allocating a vector group; a memory alignment policy to apply to all the vectors in the group; and / or a policy override at the individual vector level for any exception cases.

[0378] The memory alignment parameter value may determine whether memory allocation and retrieval techniques use aligned memory storage or unaligned memory storage. The combination of these options may allow for very fine-grained performance tuning and optimization. In some such embodiments, MDP engine code chooses at code generation time whether to use aligned (faster) or unaligned (slower) SIMD loads / stores dependent on the vector’s alignment.

[0379] Some embodiments may offer some or all of the following vector group alignment options: standard, cache line, and exact< N>. The standard alignment option aligns vector groups according to the default alignment of elements in a C struct. The default alignment of elements in a C struct is alignment to boundaries corresponding to the largest element size from these arrays. Thus the hot array in our example above would be 8-byte-aligned as the u64_l vector stores 8 byte values. The cache Jine alignment option aligns vector groups to boundaries between blocks of memory equal in size to cache lines. The size of a cache line depends on CPU architecture. On x86 architecture the CPUID instruction can be used to determine the cache line size, which is 64 bytes for our examples. The exact< N> alignment option aligns vector groups at boundaries between N-byte blocks of memory, where N is a power of two.

[0380] Some embodiments may offer some or all of the following vector alignment options: standard, cache_line, exact< N>, register_size, and register_size_or_min< N>. The standard option aligns vectors according to the default alignment of elements in a C struct. The cache line alignment option aligns vectors to boundaries between blocks of memory equal in size to cache lines. The exact< N> options aligns vectors at boundaries between N-byte blocks of memory, where N is a power of 2. The register size option aligns vectors to boundaries between blocks of memory equal in size to the registers in which the vectors are loaded for processing. The register size or min< N> option aligns vectors to boundaries between N-byte blocks of memory (where N is a power of 2) or, if register_size is larger than N bytes, to blocks of memory equal in size to the registers in which the vectors are loaded for processing.

[0381] With the foregoing data structures described, an application of those data structures to the CQS Long Quote message example can be shown. Accordingly, below are described, for this example, participants ids, fields, record vectors and vector groups for the CQS Long Quote message and the in-memory structures to hold this data. Additional details of operation of some embodiments of the engine are also described.

[0382] See Table 8 below. Note that some tables below have been shortened for brevity, e.g. the Field Detail section doesn’t list the fields from the u64_2, u32_2 or u8_2 vectors.

[0383] One feature of some embodiments of the MDP engine is its option to filter certain participants. Referring to Table 8, a user may select participants that they wish to include, in this example just P and W, and decide in which order these participant records are to appear in memory. A user may also be able to indicate that they would like the MDP engine to generate a synthetic NBBO, by enabling the pseudo-participant. The relevant settings include (i) part and (ii) in use. The part setting refers to the Participant ID, with the NBBO Participant ID at the top of the list. The rest are ordered according to the total number of message updates received for each Participant ID from the CQS feed in a single day, to keep the most active participants close together in memory. The in use setting is a boolean value and is set to 1 if records from that Participant ID / exchange are to be held in the last value cache.

[0384] Table 8 below assigns a unique field-name and numeric ID to the CQS fields of interest in this example. Table 8. Participant, Field, Vector Group, Record Vector, and Field Detail Definitions. Partfcipimi: Field Veclcx Group pan tlald_nisn« fieldjd veclorjjroup §rt?up_typa ureter group_alignment < 5 feied!_sertuenes_nutnbsr p 1 1 tojtrray; array 0 cscns Jins P | par tisigantJO 1 1 i totorray i array 1 standard £ toasts nto_spoch 2 1 ii shsto i single 3 | standard T | gmastemp1„natwsec®nd 3 | K ^participant.rsfefense„rtumber Y * RsoorcJ Vector syrtod J sfto„BBte„resteetto Jtocsfcs' § | wcanw vactor.group type vecioGaagnraent N | prima ty Jlst in§_mar Mj>«r8srpaftL’d 7 1 1 ufSj; h«t„array tofOdj 1 regiSar_®K8 8 Stesnc! BLsisius.„lndlcaitsf g 1 1 vSZ.1 i rtcs jrrsy VniOSJ ragsstacsize X atoto.jtoatjater<anc<a jsnee 0 I I i hw..array yintgj register..sfee | aysiionjtoSar^-ppsrjtb'esixsid^teica 10 i I a84_2: cofd_artay uinl84_t standard u Jbrstosid_pr:ce 11 i pGGJ i tod„array uinlSS J standard A n«rttor_cf_apc1i<»t_eKtensiCins 12 | h‘6..2 i cto..array tentgj standard H | & Ktoy„PtosJntecatsr 13 I I shared uS~ 1 i shsto ttoS4_t | standard V feid..ptos. wsnrfeion 13 i M bid _p«S8 15 1 0 l»!d_price_d®ncminat«-„c<xia 15 I L bid_size 17 | 0 5 W

[0385] Field Detail vector^rranw hetti jwis length endian isjrans pad unmap Initlatoalue inhiatostoty «64„1 1 bdjtos 1 1 0 1 5 1 SitdG i 0 OxOOOOOOOOOCOSOOOO I invalidates u>34,.1 nfferjstice 1 0 0 0x05 i 0 0x0550000530000005 rwaiidates u64„1 pato.ipss!itoateraasa„nismber 1 -3 1 0x00 i 0 «32„t bd„stes t o o oxoo i o 0x00000300 invalidates v32.. J byjsmestarrsp..eposh 1 0 0 oxoo i o «32„ 1 bidJintestBmp_aariC5BC<>nci t o o 0x00 i 0 u32„1 affer...®ze 1 0 0 0x05 i 0 0x05000003 invalidates u32„1 orter Jjmestamp_epoah 1 5 0 0x00 i 0 u33..i offer,.tsmestamp.,naaes»«»td 1 0 0 0x00 i 0 o32„.1 bl3CK. BB<^eac®. number 1 0 1 0x05 i 0 u8„i partKiparrtJd 1 3 0 0x00 i 1 «8„1 bfd_j?a'he.ipant„fd 1 0 0 0x00 i 0 0x20 invalidates uS„.1 offer.participant Jd 1 0 0 0x05 i 0 0x23 invalidates uS_i shcat_saia_rstade:^8»jB< ScMor 1 5 0 0x00 i 0 0x30 invalidates «8_1 primaryjiatieg^msrkal^ssrticipantjts i o o oxoo i o 0x20 invalidates <sS. J 8 nancfeLsistusJridicaus 1 0 0 0x00 i 0 0x33 invalidates u8_1 brd_suBte_cor!sjtic-fi 1 5 0 0x00 i <3 3x20 invalidates afferjpHXejstradilusn 1 0 0 0x03 i 0 0x20 invalidates u8_J muiity_statbsja$catcr 1 0 0 oxoo i o 0x20 invalidates «8_1 rararijrrtBmsiJ-asate 1 5 0 0x00 i 0 0x20 invalidates v3_1 wtfemsni^sarsdi&pn 1 0 0 0x05 i 0 0x23 invalidates yS„l j iyidjedicaitsr. 1 ^ sharad_cS^_1 towraGimhjmcaJrand

[0386]

[0387] shsrad <: S4 1 j upper smit price band 1 0 I 0 j 0x05 i 0

[0388] The Vector Group table of Table 8 define vector groups for the example. The fields of the Vector Group table include (a) vector group, which is the name of the group; (b) group type, which can include (i) array, which holds an array of vector groups (one for each participant exchange), or (ii) single, which holds one vector group (for the shared record vectors, in the example of Table 8); (c) order, which is an unsigned integer with vector groups having lower order values being stored earlier in memory; and (d) (vector J group alignment, which indicates the memory alignment policy for each vector group. The Record Vector table of Table 8 assigns record vectors to vector groups. The fields of the Record Vector table include (a) vector jiame,' (b) vector group, which identifies a valid group from the Vector Group table; (c) type, which defines the width of the vector elements (as discussed earlier, signed integer types, unsigned integer types, and other types of the same width are allowed in a vector); and (d) vector alignment, which indicates the memory alignment policy for each vector.

[0389] The Field Detail table of Table 8 also links specific fields to specific record vectors and includes several other field properties. Note that the order in which fields appear in this table may determine where that field is placed in the record vector. For example, bid_price may be the first element in the u64_l record vector and offer_price may be the second, etc. The fields of the Field Detail table include the following:

[0390] a) vector name - must match a definition in the Record Vectors table.

[0391] b) field name - must match a definition in the Field table.

[0392] c) length - the field's length in terms of units of the vector’s element type, which is 1 unless the field is an array field. Array fields are only used for arrays of u8 elements in the CQS example.

[0393] d) endian - the source endian-ness of the field, so appropriate byte re-ordering instructions can be generated.

[0394] e) is transient) - designates fields to be processed and passed to the user in real-time but not cached in the LVC. An example might be m block sequence number, which is a protocol field used for detecting dropped packets and usually isn’t of interest afterwards.

[0395] f) (short Jpad(value) - padding for short fields if the pad byte is not 0. This is not used in the CTA feed.

[0396] g) unmap(pable) - if set this means that the field is unmappable i.e. it can’t appear in any mappings performed when processing messages (mappings are defined below), and therefore can’t change after it is created. This may be used for the m_participant_id field, for example.

[0397] h) initial value - Initial value for a field that is set on record creation or reset. (Reset is a process of resetting last value cache and other values to default values after the end of a trading session, to prepare for the next session.) If an initial value is not provided, the field is “zeroed” (as appropriate for its type) and set invalid. Many market data feeds use sentinel values (e.g. ASCII SPACE to mean an undefined indicator field) so this feature allows us to provide the feed-appropriate sentinel value. In some examples, an explicit ‘undefined’ indicator can be provided.

[0398] a) Initial( value) validity - 4 policies are supported for how the initial value interacts with field validity (if enabled). These four policies may include: (a) always valid - the field can never become invalid (if validity bitmasks are enabled, the validity bit for this field is always set); (b) valid - the field is initially marked as valid, but can be set invalid; (c) invalid - the field is initially marked as invalid, but can be set valid; and (d) invalidates - the field is initially marked as invalid and set to the supplied initial value. If the field is subsequently marked as valid, then marked invalid again, the field value can be reset to the initial value. This also defines a sentinel value that is used during the mapping process, and sets the field to that value.

[0399] In CTA there are some explicit ’invalidate’ message types that don't contain sentinel values (e.g. NBBO condition ‘O’ invalidates both sides of the NBBO but does not include zero prices for bid or offer). In these cases, the field validity bits are cleared, but for data consistency, in some embodiments, the fields can be reset to their explicitly defined initial values, if these exist.

[0400] Having established a manner of storing data for processing of messages, the manner of storing data received from the network in order to parse it can be covered. Fields, field lengths and offsets within the data are to be defined for parsing. As shown in Table 9, doing this for the CQS Long Quote message provides the following: Table 9. Long Quote Message Definition.

[0401] #pragma pack(push, 1)

[0402] struct_block_ header / / See CQS 3.5 Block Header

[0403] {

[0404] uint8_ t m_version;

[0405] uint16_ t m_block_size;

[0406] char m data feed indicator;

[0407] char m retransmission indicator;

[0408] uint32_ t m_block_sequence_number;

[0409] uint8_t m_number_of_messages;

[0410] uint32_t m block timestamp epoch;

[0411] uint32_ t m_block_timestamp_nanosecond;

[0412] uint16_ t m_block_checksum;

[0413] };

[0414] struct message header# See CQS 4.0 MESSAGE HEADER

[0415] {

[0416] uint16_t m_length;

[0417] char m_category;

[0418] char m_type;

[0419] char m -participant id;

[0420] uint32_t m timestampl epoch;

[0421] uint32_t m timestampl— nanosecond;

[0422] uint8 t m id;

[0423] uint32_t m_transaction_id;

[0424] int64_t m -participant _reference number,

[0425] };

[0426] struct long_quote: public message_header / / See CQS: 6.4.2 Long Quote - Category Q Type L {

[0427] std:: array<char, 11> m security symbol;

[0428] char m instrument type;

[0429] char m_quote -Condition;

[0430] char m security status indicator;

[0431] uint64 t m bid price or lower limit _price_band;

[0432] uint32 t m bid size;

[0433] uint64 t m offer _price_or_upper_limit _price_band;

[0434] uint32 t m offer size;

[0435] char m retail— interest— indicator;

[0436] char m settlement condition;

[0437] char m market condition;

[0438] std::array<uint8_t, 4> m finra market maker id;

[0439] char m finra bbo indicator;

[0440] uint32 t m timestamp2 epoch;

[0441] uint32 t m timestamp2 nanonsecond;

[0442] char m short sale restiction. indicator;

[0443] char m _primary_listing_market _participant_id;

[0444] char m -financial— status indicator,

[0445] char m sip generated message identifier;

[0446] char m_luld_ indicator,

[0447] char m_nbbo_luld_indicator;

[0448] char m_nbbo_indicator;

[0449] };

[0450] #pragma pack(pop) Table 9 shows of a set of structs and member fields, including: (1) a C / C++ struct representing the Block Header from the data feed specification; (2) another struct representing the Message Header; and (3) another struct representing the Long Quote message.

[0451] Assuming an incoming network packet has been copied to a CPU memory buffer from the NIC (typically using some kernel-bypass mechanism) and that a pointer to this buffer is available, the data can be accessed by casting the pointer to the appropriate struct type and accessing through the member fields. Starting with the block header, at least m number of mes-sages is read. The pointer can then be advanced to the beginning of the first message header, the type of the message can be determined (from m category and m type, per the CQS Specification) and the message can be parsed, noting the m length so that the pointer can be advanced to the following message, if there is one.

[0452] For brevity, the appendage structs that can follow the long quote structs are not defined in the example above, although in some embodiments they may use the same methodology. In the fourth sample message in Table 4, the third column (labeled “BSFT Struct and Member Field Naming”) defines a struct long quote double short appendage to represent the National Best Bid Short Appendage and National Best Offer Short Appendage appendages that are received for a Long Quote message that has a National BBO Indicator field of “T”.

[0453] In the CQS Block Header, there is an 8 byte SIP Block Timestamp field. In this example, it can be decomposed into its 2 constituent integer values as described in the specification to m_block_timestamp_epoch and m_block_timestamp_nanosecond as a user convenience. Other timestamps in the specification can be similarly decomposed.

[0454] The fields in Table 9 are coded as follows: (i) fields neither underlined nor italicized (e.g., uint8_t in version) are protocol fields that used in the example to parse the CQS datagram / message block or to decide how to validate and process the message; (ii) fields underlined (e.g., char m data feed indicator) are not used in this example; and (iii) fields in italics (e.g., uint32_t m block sequence number) are mapped to the fields defined in the previous section.

[0455] An administrator may in some embodiments be at liberty to create fields in our Field definitions that are not received on the feed at all, e.g. to hold a derived or calculated value.

[0456] The type of each struct member above maps onto a list of supported SIMD element types. Reflection capabilities of a chosen computer language may be used to calculate the size of each field and the offset of each field from the start of the message in memory. In some embodiments, mappings are a key mechanism by which the actions of the engine in response to the incoming feed are defined.

[0457] For each message type a set of mappings may be created, where each mapping details how a particular feed message impacts the fields in corresponding record vectors. If there are no mappings for a particular message type in a given implementation, an engine may be configured to ignore messages of that type. If one or more mappings are present for a given message type, each mapping is applied every time a message of that type is received. The mappings are implemented using SIMD functions as far as possible, as described later (Message Processing).

[0458] Each mapping may have the following attributes:

[0459] (i) mapping_name - a name given to the mapping.

[0460] (ii) field name - a field from the Field Detail definitions that is the target of the mapping.

[0461] (iii) transform - an optional user-definable calculation which may be applied to the value during the mapping process. A mapping may specify multiple transforms (daisy-chained) to be executed in the declared sequence.

[0462] (iv) mapping type - one of 4 mapping types currently, see below.

[0463] (v) member_field - for a mapping type of member field, this is the source of the mapping.

[0464] (vi) non_member_field - the source for a mapping type of non member field.

[0465] (vii) Implied value - supplies the constant value for an implied value mapping.

[0466] (viii) is valid - used with implied field or invalidates field mappings, sets or clears the validity bit for the field.

[0467] (ix) m transient - if set, this means that mapping has a transient effect on the field, such that the field is processed as usual but not stored back to the LVC.

[0468] In some examples, the mapping type is one of member_field, non_member_field, implied_value or invalidate_field. The member field value indicates that contents of the field in member field column are copied into the SIMD vector field given in the field name column. The non member field value indicates a field that is outside the range of using SIMD parallel instructions, as described in more detail above. The effect is the same as member field, but the implementation is scalar. This restriction may not be present in all embodiments, as mem-ber field may be used for most non member field mappings, though the mapping type may remain in some such embodiments to handle special cases. The implied value is used for fields that are set to a predetermined value on receipt of an indicator, for example the bid and offer quote condition fields which are set to the ASCII value R (0x52) if an NBBO indicator T is received (see the CQS Specification section 6.4.5). The invalidate field marks the field invalid, and if a sentinel value is defined in the field definition, (e.g., initial value = ‘invalidates’) also set the value of the field to the sentinel value.

[0469] The mapping type for a given field can differ between mappings.

[0470] In some examples, source and target fields in general can differ in the following ways: (a) the byte order may differ; (b) the position within the target data structure may differ; and / or (c) the width of the field may differ. In these cases, the source data is permuted to place it in the correct position and byte order. As there are multiple SIMD instructions that can be used to perform these functions (e.g. permute, shuffle, etc.) for brevity the term permute can be used to mean any of them. A scalar implementation of permute may also be used in some fallback cases. The code generated will use the most applicable SIMD instruction to the circumstances.

[0471] If the field is defined with an initial validity of invalidates (see Table 8) then a mem-ber field, non member field or implied value mapping can cause the field to be marked invalid if the supplied value equals the field’s sentinel value.

[0472] The SIMD vector which is target of the mapping can be deduced automatically at build time from our record vector and vector group metadata. Table 10. Message-field Mapping Definitions,

[0473] mappings_name field_name mapping_type non_member_field imp_value is_valid m_transient

[0474] m_timestamp2_nanosecond m_nbbo_bid_price

[0475] 0x52 0x52

[0476] non_member_field implied_value

[0477]

[0478] In the case of Long Quote, in this example, the m nbbo indicator field prescribes a multitude of ways in which the fields in our NBBO record are to be updated by the message. So a multitude of mappings are created for the Long Quote message, some of which are used to update the appropriate participant and some to update the MDP en-gine’s NBBO record. Note that a Long Quote message may use one mapping to update to a regional record and may use another mapping to update the NBBO record.

[0479] See Table 10 for two example mappings for the Long Quote message. The first example mapping is used when the when the m nbbo indicator field is T and updates the NBBO record. The second mapping is used when the m security status indicator is a 0 or a 9 and the m nbbo indicator is an O. This is a price band message that updates a regional record including the price band fields in the shared vectors.

[0480] In the first mapping, the m_nbbo_bid_price in the received long_quote message will be copied to the bid_price field in the SIMD vector (after appropriate permutation). Note that member_field can be used for any field contained within 128 bytes from MESSAGE HEADER. In this mapping, the mul<10000> transform is used to convert a 2-byte bid and offer price to 8-bytes, for consistency with other bid and offer price fields on the feed (CQS uses these two different sizes for on-wire bandwidth savings). The transform is applied after the source bytes are permuted into the correct SIMD vector positions.

[0481] Ultimately all the member fields will be permuted into place by vector operations.

[0482] Summarizing the previous sections, above is defined, for this example: (1) layouts for processing and optional last-value-caching of each field in an element (or an array of elements) of a SIMD vector of a certain element size (8, 16, 32 or 64 bits); (2) sets of fields through an abstract record vector definition that supports up to 64 elements, for flexibility targeting different CPU implementations; (3) how record vectors can be arranged in groups with the option to store related records together in arrays or multiple arrays; and (4) the concept of shared vectors to allow efficient sharing of fields between records.

[0483] Also described above are examples of: (1) the structure of the incoming feed in terms of header and data field offsets, lengths and types (presented above in c-struct form) with sufficient info to let us parse the incoming data; (2) a mechanism of mappings for each message type defining how fields in records change when that specific type of message is received, and how to ignore a message by not defining any mappings, or how to process a message in multiple ways by defining many mappings; and (3) four message mapping types, namely member field, non member field, implied field and invali-date_field to represent common actions.

[0484] Metadata Generation

[0485] At code generation time, the feed-related definitions and MDP engine settings including user overrides are processed to determine an assortment of new fields and values that can be used to generate pipeline code. For example, list of fields with alignment policies are converted into indexes, offsets and lengths so the location and size of each object in memory can be determined precisely. Configuration data describing vectors are converted into vector definitions for the workspace / image view and the last value cache view, and these vector definitions are specialized for the specific SIMD target (e.g., AVX2 or AVX-512). In addition, the configuration data describing vectors are combined with the mapping definitions to produce a vector-level view of the mappings and transforms (e.g., various bitmasks, indexes for rearranging fields, and vector properties) rather than the field-level views of the individual fields. This information can be used during code generation. Table 11 shows a vector view of vector mappings generated in accordance with the example of Tables 8 and 10 for the CTA Long Quote message. Table 11. Message-Vector Mapping.

[0486]

[0487]

[0488] The columns in Table 11 are described below:

[0489] (i) mapping_name - as per Message-field Mapping definitions in Table 10.

[0490] (A) event_id - a 1 byte enumeration.

[0491] (B) can create records - indicates whether a message mapping allows records to be created.

[0492] (C) vector_name - indicates the name of the vector.

[0493] (D) field_bitmask - a bitmask of indexes used by this mapping / vector.

[0494] (E) uncached_field_bitmask - bitmask of indexes that do not appear in the cache (e.g., record-level transient fields used by this mapping / vector).

[0495] (F) cache update bitmask - bitmask of indexes for fields which should be stored back to the LVC upon receiving this message (e.g., field bitmask but excluding record-level and mapping-level transient fields).

[0496] (G) transient_field_bitmask - bitmask of indexes of record-level transient fields.

[0497] (H) non_member_field bitmask - bitmask to handle some scalar corner cases

[0498] (I) invalidate_field itmask - bitmask of indexes which are invalidated by the message (irrespective of value).

[0499] (J) maybe_valid_field_bitmask - bitmask of indexes which can be valid, but some fields may have their values checked against sentinel ‘invalidate’ values.

[0500] (K) can value invalidate bitmask - bitmask of indexes which should have their value checked. If 0, maybe valid field bitmask can be considered as definitely valid field bitmask.

[0501] (L) endian_shuffle_index_sequence -indexes for the endian shuffle.

[0502] (N-P) permute method, permute index sequence and permute mask - the short or long permute method used along with indexes and masks.

[0503] Table 11 also includes a vectorized view of a transform that can be used to upscale 16-bit price fields received in the specific Long Quote message that generates the long_quote_nbbo_T_indi-cator mapping. Mappings are applied after permutes. FIGS. 4A-4B illustrate aspects of the example method 250 for processing fields of financial market data messages using SIMD-based hardware (e.g., AVX-512 hardware). In the example of FIGS. 4A-4B, the example method 250 is applied to an example market data message 400.

[0504] In the example of FIG. 4A, the market data message 400 is stored in a memory 405 (e.g., a memory of a network interface card (NIC), a cache (e.g. L2 cache) of a CPU of the MDP engine 130, etc.) of the system 100. In the example of FIG. 4A, the SIMD hardware includes two source vector registers of width N bytes, and the width of the market data message 400 is 4N bytes. In FIG. 4A, at step 252 of the method 250, the first N bytes (first “chunk”) of the message 400 are copied to source vector register 410a of the SIMD hardware, and the second N bytes (second “chunk”) of the message 400 are copied to source vector register 410b of the SIMD hardware.

[0505] In FIG. 4A, at step 254 of the method 250, at least some of the fields of the first and second chunks of the message 400 are copied from the source vector registers 410 to vector registers 425. During the step of copying the fields into the vector registers 425, the fields are rearranged to form vectors 420 of a vector group 430 in accordance with the vector mappings specified in the configuration data for the MDP engine. In the example of FIG. 4A, four fields of a first length (e.g., B bits) are copied to the first four elements of vector register 425c as the first four fields of a vector 420c, one field of a second length (e.g., 2B bits) is copied to the first element of vector register 425b as the first field of a vector 420b, and two fields of a third length (e.g., 4B bits) are copied to the two elements of vector register 425a as the two fields of a vector 420a, As can be seen, each vector register 425 stores fields of the same width, and different vector registers can store fields of different widths. As the fields of the message 400 are copied from the source registers 410 to the registers 425, the relative ordering among fields of the same width can be preserved, but is not necessarily preserved. Also, in the example of FIG. 4A, field 411 is not copied into any of the illustrated vector registers 425 because field 411 is assigned to a vector that is not within the vector group 430. However, field 411 could be copied to a non-illustrated vector register assigned to a vector within a different vector group, or could be discarded.

[0506] In the example of FIG. 4B, the chunks of market data message 400 (or at least the chunks of the message 400 that have not yet been copied to the source registers 410 of the SIMD hardware) are still stored in a memory 405 of the system 100. In FIG. 4B, at step 252 of the method 250, the third N bytes (third “chunk”) of the message 400 are copied to source vector register 410a of the SIMD hardware, and the fourth N bytes (fourth “chunk”) of the message 400 are copied to source vector register 410b of the SIMD hardware.

[0507] In FIG. 4B, at step 254 of the method 250, at least some of the fields of the third and fourth chunks of the message 400 are copied from the source vector registers 410 to vector registers 425. During the step of copying the fields into the vector registers 425, the fields are rearranged to form vectors 420 of the vector group 430 in accordance with the vector mappings specified in the configuration data for the MDP engine. In the example of FIG. 4B, two fields of the first length (e.g., B bits) are copied to the fifth and sixth elements of vector register 425c as the fifth and sixth fields of a vector 420c, and three fields of the second length (e.g., 2B bits) are copied to the last three elements of vector register 425b as the last three fields of a vector 420b, In the example of FIG. 4B, fields 412, 413, and 414 are not copied into any of the illustrated vector registers 425 because these fields are not assigned to any vectors within the vector group 430. However, each of fields 412-414 could be copied to a non-illustrated vector register assigned to a vector within a different vector group, or could be discarded.

[0508] One of ordinary skill will appreciate that example of FIGS. 4A-4B is provided by way of example and is not limiting. For example, the arrangement of fields of the example market data message 400 in the memory 405 of the system 100, the number of source vector registers 410, the number of vectors 420 in the vector group 430, the widths and number of elements in the vector registers 425 storing the vectors 420, the arrangement of fields of the market data message 400 in the elements of the vector registers 425, etc. are non-limiting.

[0509] Some examples have been described in which fields of a market data message 112 are assigned to vectors based, in part, on the widths of the fields, such that fields of the same width can be assigned to the same vector but fields of different widths are not assigned to the same vector. In some examples, the assignment of message fields to vectors can be based not only on field width but also on other factors. Some non-limiting examples of such factors may include the data types of the fields (e.g., in some examples, integer types and floating-point types are not assigned to the same vector), the SIMD operations to be performed on the fields, the semantics of the fields, etc. Some examples of vector groups have been described. In some examples, the rearrangement of message fields into vector groups is facilitated by waiting for subsequent chunks (e.g., all chunks) of a market data message to be made available for processing by the SIMD hardware before initiating the SIMD processing of the message’s fields, rather than initiating the SIMD processing of the message’s fields as soon as those fields are transferred into vector registers of the SIMD hardware.

[0510] In some examples, the use of vector groups enables fine-grained control of the CPU caches and optimization of CPU cache performance (e.g., increased hit rates, reduced average latency per cache read, etc.) based on the workload being processed. Control and optimization of CPU caches arises from the use of vector groups because vector groups allow the developer of the feed handler (e.g., the provider of the configuration data 172) to designate sub-groups of fields based on the nature of the workload, without regard to the order in which those fields are received in market data messages 112, thereby allowing the most-often accessed field data to remain in CPU cache without being evicted by less important field data.

[0511] Some Examples of Methods for Generating Code for SIMD-based MDP

[0512] Described briefly below are example methods for generating code for SIMD-based financial MDP. In some examples, the process for generating the code involves a user providing code generation configuration data (e.g., metadata), the MDP code generator using the configuration data to control a code generation process, and a process of providing executable code to the user based on the output of the code generator, which may in some cases involve a code compilation step. In some examples, the configuration data provided by the user include the instances of Tables 8 and 10 as described above.

[0513] In the remainder of this section, some examples of types of financial market data processing are described. These types of financial MDP can include message processing, block header processing, last value cache processing, output processing, and validity bitmask processing. The code generated by the MDP code generator can include these and other examples of types of financial MDP. Message Processing

[0514] Message type can be determined by ‘peeking’ into relevant parts of the message, usually the header. Once message type is determined, the message header and message body may be processed together by applying the mappings defined in the code generation configuration data (e.g., metadata).

[0515] One example method that may be implemented in the generic source code to process a message includes the steps (1) - (4) as described below.

[0516] In step (1), load the first line of the incoming / 7 message into SIMD source registers. In some examples, one or two 512-bit registers are used for this operation, giving a max of 1024 bits, which is sufficient to hold a complete message header and body for most data feeds. Longer messages can be processed by stepping through the source 1024 bits at a time. The complete message can be loaded, with no distinction between the header and body. Note that the data in the source registers is still in the raw market data feed format (e.g., the format of the market data messages 112).

[0517] In step (2), map the required fields from the source registers into a set of SIMD target registers. The target registers are the concrete SIMD implementations defined by the record vector / vector group metadata, e.g., a group of SIMD vectors distinguished by element size and vector group (see discussion above of workspaces and LVC structures). Byte positions and lengths of fields within these vectors are also known from this metadata. The source field, target field and target register are known from the defined mappings, each mapping defines the behavior for a single field in a particular target register. The byte positions and lengths of the fields in the source registers are known from the feed message definition (see discussion of feed processing).

[0518] Mapping types member field, implied field and invalidate field are implemented with SIMD permute instructions. These instructions can copy any set of bytes from the source registers, they need not be contiguous, and can place these bytes into the target register in any order, and thus can perform endian conversion and can place the bytes for each field in the correct position in the target register; can also copy an immediate (absolute) value into the target register, again in any byte position or order; can extend source values to a wider element in the target; and require a control mask, control register and source register as arguments. The control mask indicates which positions are to be permuted. The control register determines the source register and byte positions of the permute, and for byte positions where no permute is indicated by the control register, indicates an absolute value to be copied to the target register. Appropriate control mask and register contents are automatically generated from metadata. In some examples, the SIMD permute instructions are applied to each register in the target register set in turn. In some examples, these operations are coalesced to reduce instruction count and increase performance.

[0519] Mapping type non member field is a catch-all for corner-cases where a message exceeds the line-size (e.g. 1024 bits) of the processing pipeline. In these cases, the code emitted is scalar code that performs the same function as the SIMD member field code above, non member field mappings are also used to include block header fields in the output of message processing, if this is required.

[0520] In step (3), perform any transforms specified in the mapping on the target vectors. These transforms may include operations to convert or scale data types from raw format to the format required by the record vector definition. Like transforms affecting a particular SIMD vector may be performed in parallel using SIMD. If any daisy-chained transforms are specified, these may be performed in multiple passes, one pass for each element of the chain, and if there are multiple daisy-chained transforms for a given SIMD vector, like transforms are performed in parallel.

[0521] In step (4), write the target vectors back to workspace memory, ready to be passed to the user.

[0522] Block Header Processing

[0523] The block header is processed before any message, since the block header contains information (such as message count) useful for parsing the rest of the data. In some cases, users may wish to include fields from the block header in the output from processing a particular message type. This functionality can be achieved by adding a non member mapping to the metadata for that message type, in which case the following processing code is emitted: (1) load the block header into SIMD source registers; (2) perform endian conversion if required, using permute, shuffle or another more appropriate SIMD instruction; (3) copy source field into appropriate places in a target register set (e.g., using scalar code, or using the permute instruction); and (4) merge the target register(s) with the result of message processing using the blend SIMD instruction.

[0524] The block header can be processed alongside each message as part of the same message processing pipeline, and this method may be adopted in some embodiments.

[0525] Last Value Cache Processing

[0526] Functions performed with respect to the last value cache (LVC) can include load, compare, update, and store. Code for these LVC functions is generated if LVC is enabled and at least one non-transient field is in use.

[0527] LVC Load

[0528] The LVC load function is used to retrieve a set of data from the LVC. In some examples, loading from the LVC includes permuting the symbol from the message into a workspace register and hash it, using the hash to look up the LVC addresses of the SIMD vectors for this record, including any shared vectors. The SIMD vectors required are known in advance from our metadata, given the message type. Under some conditions more vectors may be needed, for example if configured to send a certain fixed field set with every update. The relevant vectors are loaded into SIMD registers using the SIMD load instruction.

[0529] LVC Compare

[0530] The info read from the LVC is in the same format as the corresponding vector in the processing pipeline, except for transient fields which appear at the end of the vector in the pipeline. A comparison is made between the LVC and pipeline data using mask compare. This is masked to restrict the comparison to just those fields that may have been affected as a result of a mapping (also known from the metadata). The compare yields a bitmask of which elements are different between the processing pipeline and the LVC entry. This comparison can be extended to compute differences between values in the cache and the processing pipeline value, e.g. to detect price movement and direction. Other comparisons can be used to give further functionality during the LVC compare function. LVC Update

[0531] The bitmask generated by the compare operation is used to update the working copy of the cache entry with any changed values, leaving other values unaltered. Again, the values transferred could be price differences, etc. if needed. At this point, the workspace copy of the LVC entry contains the latest data, but the LVC entry hasn’t been updated.

[0532] LVC Store

[0533] The store or masked store commands are used to write the revised working copy of the LVC data back to the LVC. Masking may be used to eliminate transient fields or field mappings that are marked ‘do not update cache’.

[0534] Output

[0535] Multiple outputs are supported, including NLO output and, in some examples, 3 types of vector output: (1) regional participants only are output; (2) NBBO participant only is output; or (3) combined regional and NBBO participants are output.

[0536] For each vector the processing pipeline has generated a final result register; a bitmask of fields that have updated; and a bitmask of the transient fields. The mask compress store command is used remove any unwanted fields (fields that haven’t changed and aren’t required in the output by a fixed field rule) and write the remainder to a memory buffer as part of the output message. Data from each vector (u64_l, etc.) are combined into the output buffer. This data can be written to a file output or transmitted onto the network. There’s also a callback which can fire at this point.

[0537] Guaranteed Field Sets / Full Image Output

[0538] It is permitted to specify fields that are sent to the user irrespective of whether they have changed. In some examples, this functionality is implemented using the following steps: (1) deduce which vectors in which groups are needed from the LVC (all of them in the case of full image, otherwise known from metadata); (2) load these vectors from the LVC into the workspace; (3) perform any other processing; and (4) merge the fixed field values into the output using a logical or operation. Validity Bitmasks

[0539] Validity bitmasks are calculated (if enabled) to indicate which fields are valid, instead of relying on sentinel values. For fields that use a sentinel, compare equal mask can be used to determine whether the validity bit is to be set or cleared.

[0540] While the foregoing disclosure sets forth some examples using specific block diagrams, flowcharts, and examples, each block diagram component, flowchart step, operation, and / or component described and / or illustrated herein can be implemented, individually and / or collectively, using a wide range of hardware, software, or firmware (or any combination thereof) configurations. In addition, any disclosure of components contained within other components is to be considered as non-limiting examples since many other architectures can be implemented to achieve the same functionality.

[0541] Included in the discussion above are flowcharts showing steps and acts in accordance with the present disclosure. The processing and decision blocks of the flowcharts above represent steps and acts that can be included in algorithms that carry out these processes. Algorithms derived from these processes (or steps thereof) can be implemented as software integrated with and directing the operation of one or more single- or multi-purpose processors (e.g., central processing units (CPUs), graphics processing units (GPUs), tensor processing units (TPUs), hardware accelerators, etc.) which can include SIMD hardware or be used in combination with SIMD hardware. It is to be appreciated that the flowchart(s) included herein do not depict the syntax or operation of any particular circuit or of any particular programming language or type of programming language. Rather, the flowchart(s) illustrate the functional information one of ordinary skill in the art can use to fabricate circuits or to implement computer software algorithms to perform the processing of a particular apparatus carrying out the types of techniques described herein. It is to also be appreciated that, unless otherwise indicated herein, the particular sequence of steps and / or acts described in each flowchart is merely illustrative of the algorithms that can be implemented and can be varied in implementations and embodiments of the principles described herein. Accordingly, in some embodiments, the techniques described herein can be embodied in computer-executable instructions implemented as software, including as application software, system software, firmware, middleware, embedded code, or any other suitable type of software. Such computer-executable instructions can be written using any of a number of suitable programming languages and / or programming or scripting tools, and also can be compiled as executable code.

[0542] When techniques described herein are embodied as computer-executable instructions, these computer-executable instructions can be implemented in any suitable manner, including as a number of functional facilities, each providing one or more operations to complete execution of algorithms operating according to these techniques. A “functional facility,” however instantiated, is a structural component of a computer system that, when integrated with and executed by one or more computers, causes the one or more computers to perform a specific operational role. A functional facility can be a portion of or an entire software element. For example, a functional facility can be implemented as a function of a process, or as a discrete process, or as any other suitable unit of processing. If techniques described herein are implemented as multiple functional facilities, each functional facility can be implemented in its own way; all need not be implemented the same way. Additionally, these functional facilities can be executed in parallel and / or serially, as appropriate, and can pass information between one another using a shared memory on the computer(s) on which they are executing, using a message passing protocol, or in any other suitable way.

[0543] Generally, functional facilities include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically, the functionality of the functional facilities can be combined or distributed as desired in the systems in which they operate. In some implementations, one or more functional facilities carrying out techniques herein can together form a complete software package. These functional facilities can, in alternative embodiments, be adapted to interact with other, unrelated functional facilities and / or processes, to implement a software program application.

[0544] Some exemplary functional facilities have been described herein for carrying out one or more tasks. It is to be appreciated, though, that the functional facilities and division of tasks described is merely illustrative of the type of functional facilities that can implement the exemplary techniques described herein, and that embodiments are not limited to being implemented in any specific number, division, or type of functional facilities. In some implementations, all functionality can be implemented in a single functional facility. It is to also be appreciated that, in some implementations, some of the functional facilities described herein can be implemented together with or separately from others (i.e., as a single unit or separate units), or some of these functional facilities can be omitted.

[0545] Computer-executable instructions implementing the techniques described herein (when implemented as one or more functional facilities or in any other manner) can, in some embodiments, be encoded on one or more computer-readable media to provide functionality to the media. Computer-readable media include magnetic media such as a hard disk drive, optical media such as a Compact Disk (CD) or a Digital Versatile Disk (DVD), a persistent or non-persistent solid-state memory (e.g., Flash memory, Magnetic RAM, etc.), or any other suitable storage media. Such a computer-readable medium can be implemented in any suitable manner, including as system memory, accelerator memory, and / or system memory of a computer system (e.g., the computer system 100 of FIG. 1 A or the computer system 160 of FIG. IB) or as a stand-alone, separate storage medium. As used herein, “computer-readable media” (also called “computer-readable storage media”) refers to tangible storage media. Tangible storage media are non-transi-tory and have at least one physical, structural component. In a “computer-readable medium,” as used herein, at least one physical, structural component has at least one physical property that can be altered in some way during a process of creating the medium with embedded information, a process of recording information thereon, or any other process of encoding the medium with information. For example, a magnetization state of a portion of a physical structure of a computer-readable medium can be altered during a recording process.

[0546] In some, but not all, implementations in which the techniques can be embodied as computer-executable instructions, these instructions can be executed on one or more suitable computing device(s) operating in any suitable computer system, or one or more computing devices (or one or more processors of one or more computing devices) can be programmed to execute the computer-executable instructions. A computing device or processor can be programmed to execute instructions when the instructions are stored in a manner accessible to the computing de- vice / processor, such as in a local memory (e.g., an on-chip cache or instruction register, a computer-readable storage medium accessible via a bus, a computer-readable storage medium accessible via one or more networks and accessible by the device / processor, etc.). Functional facilities that comprise these computer-executable instructions can be integrated with and direct the operation of a single multi-purpose programmable digital computer apparatus, a coordinated system of two or more multi-purpose computer apparatuses sharing processing power and jointly carrying out the techniques described herein, a single computer apparatus or coordinated system of computer apparatuses (co-located or geographically distributed) dedicated to executing the techniques described herein, SIMD hardware, or any other suitable system.

[0547] Embodiments have been described where the techniques are implemented in circuitry and / or computer-executable instructions. It should be appreciated that some embodiments may be in the form of a method, of which at least one example has been provided. The acts performed as part of the method may be ordered in any suitable way. Accordingly, embodiments may be constructed in which acts are performed in an order different than illustrated, which may include performing some acts simultaneously, even though shown as sequential acts in illustrative embodiments.

[0548] Various aspects of the embodiments described above may be used alone, in combination, or in a variety of arrangements not specifically discussed in the embodiments described in the foregoing and is therefore not limited in its application to the details and arrangement of components set forth in the foregoing description or illustrated in the drawings. For example, aspects described in one embodiment may be combined in any manner with aspects described in other embodiments.

[0549] Use of ordinal terms such as “first,” “second,” “third,” etc., in the claims to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another or the temporal order in which acts of a method are performed, but are used merely as labels to distinguish one claim element having a certain name from another element having a same name (but for use of the ordinal term) to distinguish the claim elements. Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” “having,” “containing,” “involving,” and variations thereof herein, is meant to encompass the items listed thereafter and equivalents thereof as well as additional items.

[0550] The word “exemplary” is used herein to mean serving as an example, instance, or illustration. Any embodiment, implementation, process, feature, etc. described herein as exemplary should therefore be understood to be an illustrative example and should not be understood to be a preferred or advantageous example unless otherwise indicated.

[0551] Having thus described several aspects of at least one embodiment, it is to be appreciated that various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to be part of this disclosure, and are intended to be within the spirit and scope of the principles described herein. Accordingly, the foregoing description and drawings are by way of example only.

Claims

CLAIMSWhat is claimed is:

1. An apparatus comprising:at least one single-instruction, multiple-data (SIMD) circuit adapted to perform one or more SIMD operations; andat least one storage medium having encoded thereon executable instructions that cause the at least one SIMD circuit to perform financial market data processing on data of one or more financial market data messages, wherein performing the financial market data processing includes performing at least one of the SIMD operations on the data of the one or more financial market data messages.

2. The apparatus of claim 1, further comprising at least one central processing unit (CPU), wherein:the at least one CPU comprises the at least one SIMD circuit; andthe executable instructions further comprise executable instructions that can be executed by the at least one CPU to configure the at least one SIMD circuit to perform the financial market data processing on the data of the one or more financial market data messages and to make the data of the one or more financial market data messages available for the financial market data processing by the at least one SIMD circuit.

3. The apparatus of claim 2, wherein:the at least one CPU is a plurality of CPUs; andeach CPU of the plurality of CPUs comprises one or more SIMD circuits of the at least one SIMD circuit.

4. The apparatus of claim 2, wherein:the at least one SIMD circuit comprises a first SIMD circuit;the first SIMD circuit comprises a vector register; andthe first SIMD circuit is adapted to execute at least one of the SIMD operations in parallel on data stored in the vector register.

5. The apparatus of claim 2, wherein performing the financial market data processing on the data of the one or more financial market data messages includes performing at least one of the SIMD operations to read, write, delete or update one or more values of a datastore based on one or more values of and / or determined from the data.

6. The apparatus of claim 5, wherein the at least one SIMD operation to read, write, delete, or update the one or more values of the datastore comprises at least one SIMD operation to determine, from the data and using the at least one SIMD circuit, a particular value to be stored in a last value cache.

7. The apparatus of claim 6, wherein the at least one SIMD operation to determine the particular value to be stored in the last value cache includes:a first SIMD operation to load one or more first values from the last value cache; and a second SIMD operation to determine, based on the one or more first values from the last value cache and the one or more values of and / or determined from the data, the particular value to be stored in the last value cache.

8. The apparatus of claim 2, wherein performing the financial market data processing on the data of the one or more financial market data messages comprises performing at least one of the SIMD operations to convert at least some of the data from a first format to a second format.

9. The apparatus of claim 8, wherein performing the financial market data processing on the data of the one or more financial market data messages further comprises performing at least one of the SIMD operations to output at least one message comprising second data in the second format generated using the at least some of the data of the one or more financial market data messages.

10. The apparatus of claim 9, wherein the at least one of the SIMD operations to output the at least one message comprises:at least one first SIMD operation to load one or more first values from a last value cache; andat least one second SIMD operation to determine, based on the one or more first values from the last value cache and the at least some of the data of the one or more financial market data messages, the at least one message comprising second data in the second format.

11. The apparatus of claim 1, wherein performing the financial market data processing on the data of the one or more financial market data messages comprises performing at least one of the SIMD operations on a one or more vectors, each vector of the one or more vectors comprising a plurality of fields, and the plurality of fields of a vector each being of a same length.

12. The apparatus of claim 11, wherein:the one or more vectors comprise a first vector and a second vector;all fields of the first vector are represented using a first number of bits; andall fields of the second vector are represented using a second number bits.

13. The apparatus of claim 11, wherein performing the at least one SIMD operation on the data of the one or more financial market data messages includes:performing one or more SIMD operations to map values from a plurality of fields of an incoming financial market data message to a plurality of fields of the one or more vectors, wherein mapping the values comprises copying a subset of the plurality of fields of the incoming financial market data message from one or more source vector registers of the at least one SIMD circuit to a target vector register of the at least one SIMD circuit configured to store a first vector of the one or more vectors, the subset of the plurality of fields of the incoming financial market data message being of equal length.

14. The apparatus of claim 13, wherein performing the one or more SIMD operations to map values from the plurality of fields of the incoming financial market data message to the plurality of fields of the one or more vectors further includes:copying one or more parts of the incoming financial market data message from a memory to the one or more source vector registers of the at least one SIMD circuit, wherein the memory is a memory of a network interface card (NIC), a memory of a CPU, or a cache of a CPU.

15. The apparatus of claim 14, wherein mapping the values further includes:copying one or more fields of the subset of the plurality of fields of the incoming financial market data message from the one or more parts of the incoming financial market data message in the one or more source registers of the at least one SIMD circuit to the target vector register; andthe executable instructions that cause the at least one SIMD circuit to perform the financial market data operations on the plurality of vectors comprise operations to cause the at least one SIMD circuit to perform the financial market data operations on the target vector register after all fields of the first vector across the one or more parts of the incoming financial market data message have been copied from the one or more source vector registers to the target vector register.

16. The apparatus of claim 15, wherein the operations to cause the at least one SIMD circuit to perform the financial market data operations on the target vector register after all fields of the first vector across the one or more parts of the incoming financial market data message have been copied to the target vector register comprise operations to cause the at least one SIMD circuit to perform the financial market data operations on the target vector register after all parts of a plurality of parts of the financial market data message have been copied to the one or more source vector registers.

17. The apparatus of claim 11, wherein for a financial market data message of the one or more financial market data messages, the plurality of vectors comprise:a first plurality of vectors that store data specific to a financial instrument and to a financial market exchange on which the financial instrument is traded, the financial market data message relating to the financial instrument and to the financial market exchange; anda second plurality of vectors that store data generic to one or more financial instruments including the financial instrument and / or generic to one or more financial market exchanges including the financial market exchange.

18. The apparatus of claim 17, wherein:for a second financial market data message of the one or more financial market data messages, the second financial market data message relating to the financial instrument and to a second financial market exchange, the financial market data processing is performed on a third plurality of vectors; andthe third plurality of vectors comprise the second plurality of vectors and a fourth plurality of vectors specific to the financial instrument and to the second financial market exchange.

19. The apparatus of claim 18, wherein:the first and fourth plurality of vectors are maintained in a data structure representing a vector group; andthe instructions comprise instructions to manage caching of the first and fourth plurality of vectors by a CPU, such that the first and fourth plurality of vectors are both resident in a cache of the CPU.

20. The apparatus of claim 19, wherein:the data structure is associated with a data structure cache key; anddifferent sets of vectors within the data structure may be associated with sub-keys of the data structure cache key.

21. The apparatus of claim 20, wherein:the data structure cache key is associated with the financial instrument; and different data structures are maintained for and correspond to different financial instruments, each data structure being associated with a respective data structure cache key that is associated with the corresponding financial instrument.

22. A method comprising:performing financial market data processing on data of one or more financial market data messages,wherein performing the financial market data processing comprises performing at least one single-instruction, multiple-data (SIMD) operation of SIMD hardware on the data of the one or more financial market messages.

23. The method of claim 22, wherein performing the financial market data processing on the data of the one or more financial market data messages includes performing at least one SIMD operation of the SIMD hardware to read, write, delete or update one or more values of a datastore based on one or more values of and / or determined from the data.

24. The method of claim 23, wherein the at least one SIMD operation to read, write, delete, or update the one or more values of the datastore comprises at least one SIMD operation to determine, from the data and using the SIMD hardware, a particular value to be stored in a last value cache.

25. The method of claim 24, wherein the at least one SIMD operation to determine the particular value to be stored in the last value cache includes:a first SIMD operation to load one or more first values from the last value cache; and a second SIMD operation to determine, based on the one or more first values from the last value cache and the one or more values of and / or determined from the data, the particular value to be stored in the last value cache.

26. The method of claim 22, wherein performing the financial market data processing on the data of the one or more financial market data messages includes performing at least one SIMD operation of the SIMD hardware to convert at least some of the data from a first format to a second format.

27. The method of claim 26, wherein performing the financial market data processing on the data of the one or more financial market data messages further includes performing at least one SIMD operation of the SIMD hardware to output at least one message comprising second data in the second format generated using the at least some of the data of the one or more financial market data messages.

28. The method of claim 22, wherein performing the financial market data processing on the data of the one or more financial market data messages includes performing at least one SIMD operation of the SIMD hardware on one or more vectors, each vector of the one or more vectors comprising a plurality of fields, and the plurality of fields of a vector each being of a same length.

29. The method of claim 28, wherein performing the at least one SIMD operation on the data of the one or more financial market data messages further includes:performing one or more SIMD operations to map values from a plurality of fields of an incoming financial market data message to a plurality of fields of the one or more vectors, wherein mapping the values comprises copying a subset of the plurality of fields of the incoming financial market data message from one or more source vector registers of the SIMD hardware to a target vector register of the SIMD hardware configured to store a first vector of the one or more vectors, the subset of the plurality of fields of the incoming financial market data message being of equal length.

30. The method of claim 29, wherein performing the one or more SIMD operations to map values from the plurality of fields of the incoming financial market data message to the plurality of fields of the one or more vectors further includes:copying one or more parts of the incoming financial market data message from a memory to the one or more source vector registers of the SIMD hardware, wherein the memory is a memory of a network interface card (NIC) or a cache of a CPU.

31. The method of claim 30, wherein mapping the values further includes:copying one or more fields of the subset of the plurality of fields of the incoming financial market data message from the one or more parts of the incoming financial market data message in the one or more source registers of the SIMD hardware to the target vector register; and performing the financial market data processing on the one or more vectors comprise operations to cause the SIMD hardware to perform the financial market data operations on the target vector register after all fields of the first vector across the one or more parts of the incoming financial market data message have been copied from the one or more source vector registers to the target vector register.

32. The method of claim 31, wherein the operations to cause the SIMD hardware to perform the financial market data operations on the target vector register after all fields of the first vector across the one or more parts of the incoming financial market data message have been copied to the target vector register comprise operations to cause the SIMD hardware to perform the financial market data operations on the target vector register after all parts of a plurality of parts of the financial market data message have been copied to the one or more source vector registers.

33. The method of claim 28, wherein for an incoming financial market data message related to a financial instrument, the plurality of vectors comprise:a first plurality of vectors that store data specific to the financial instrument and to a financial market exchange on which the financial instrument is traded, the incoming financial market data message relating to the financial instrument and to the financial market exchange; and a second plurality of vectors that store data generic to one or more financial instruments including the financial instrument and / or generic to one or more financial market exchanges including the financial market exchange.

34. The method of claim 33, wherein:for a second financial market data message of the one or more financial market data messages, the second financial market data message relating to the financial instrument and to a second financial market exchange, the financial market data processing is performed on a third plurality of vectors; andthe third plurality of vectors comprise the second plurality of vectors and a fourth plurality of vectors specific to the financial instrument and to the second financial market exchange.

35. The method of claim 27, wherein performing at least one SIMD operation to output at least one message comprises:performing at least one first SIMD operation to load one or more first values from a last value cache; andperforming at least one second SIMD operation to determine, based on the one or more first values from the last value cache and the at least some of the data of the one or more financial market data messages, the at least one message comprising second data in the second format.

36. A method comprising:generating a feed handler for processing of incoming financial market data messages, wherein generating the feed handler comprises generating instructions for the feed handler based at least in part on a first input identifying SIMD hardware on which the instructions are to execute and a second input identifying how a plurality of fields of the incoming financial market data messages are to be mapped to a plurality of vectors, wherein the first and second input are received separate from the instructions.

37. The method of claim 36, further comprising:determining, based on the first input identifying the SIMD hardware, one or more properties of the SIMD hardware including one or more widths of one or more vector registers of the SIMD hardware, and one or more widths of one or more elements of the one or more vector registers on which one or more SIMD instructions of the SIMD hardware can operate,wherein the generating the instructions for the feed handler based at least in part on the first input identifying the SIMD hardware includes generating the instructions for the feed handler based at least in part on the determined one or more properties of the SIMD hardware.

38. The method of claim 36, wherein:the second input identifying how the plurality of fields are to be mapped to the plurality of vectors comprises, for one or more fields of the plurality of fields, an indication of whether to use aligned memory storage or unaligned memory storage for storing data of the one or more fields; andthe generating the instructions for the feed handler based at least in part on the second input comprises generating the instructions in a manner that outputs aligned or unaligned memory storage instructions for storage operations related to the one or more fields consistent with the indication.

39. The method of claim 36, wherein:the generating the instructions for the feed handler based at least in part on the second input comprises generating, using the second input, instructions that define data structures that store the plurality of vectors and instructions that store, in the plurality of vectors, data from one or more fields of the plurality of fields of the incoming financial market data messages; and generating the instructions that store the data from the one or more fields of the incoming financial market data messages comprise generating instructions to divide the one or more fields of an incoming financial market data message between multiple vectors of the plurality of vectors, each vector of the multiple vectors comprising a respective plurality of fields of a same width.

40. The method of claim 39, wherein the generating the instructions for the feed handler based at least in part on the second input comprises generating, for each vector of the multiple vectors and using the second input, instructions to assign the plurality of fields of the respective vector to SIMD registers of the SIMD hardware.

41. The method of claim 36, wherein the generating the instructions for the feed handler comprises generating instructions to read, write, delete or update one or more values of a datastore based on one or more values included in the incoming financial market data messages.

42. The method of claim 41, wherein the generating the instructions to read, write, delete, or update the one or more values of the datastore comprises generating instructions to determine, using the SIMD hardware and based on the one or more values included in the incoming financial market data messages and / or the one or more values of the datastore, a particular value to be stored in a last value cache.

43. The method of claim 36, wherein generating instructions for the feed handler based at least in part on the second input comprises:receiving, in the second input, data identifying one or more transformations or interpretations to be performed on received data that is to be stored in a field of a vector; and generating instructions to perform the one or more transformations or interpretations.

44. The method of claim 36, wherein generating the feed handler further comprises:generating second instructions to be executed by non- SIMD hardware.

45. The method of claim 41, wherein the generating the instructions to read, write, delete, or update the one or more values of the datastore comprises generating instructions to determine, using the SIMD hardware and based on the one or more values included in the incoming financial market data messages and / or the one or more values of the datastore, a particular value to be included in an output message.

Images