A financial market data subscription processing system and method

Abstract

The application provides a financial market data subscription processing system and method, and relates to a financial market data subscription processing system, which comprises a subscription rule preprocessing module arranged in an upper computer and a market preprocessing module, a subscription management unit, a market calculation unit, a market snapshot cache module and a market output unit realized by FPGA logic. The modules cooperate with each other to realize low-delay query response of market data in a large-scale subscription scenario and further improve the processing efficiency of financial market data.

Description

Technical Field

[0001] This invention belongs to the field of financial market data processing technology, and in particular relates to a financial market data subscription and processing system and method. Background Technology

[0002] In the field of financial trading, low-latency processing of market data is one of the core technological barriers for institutions to gain a competitive advantage. With the continuous increase in domestic futures / options products and the popularization of high-frequency trading, the high-frequency trading market has put forward sub-microsecond requirements for latency and throughput when decoding, filtering and distributing market data.

[0003] Currently, mainstream financial market data subscription processing methods mainly fall into two technical paths: computer software-based processing solutions and FPGA-based processing solutions. However, existing technical solutions all suffer from significant performance bottlenecks and architectural defects when processing large-scale, multi-contract market data subscriptions. (1) The computer software-based processing scheme receives market data through the network card and transmits all market data to the computer software. The computer software processing path involves operating system kernel network stack processing, kernel mode and user mode switching, data copying and other links, resulting in large delays and jitters. Moreover, the parallelism of computer software decoding is not as good as that of FPGA, resulting in large delays when processing high-concurrency market data. (2) The FPGA-based processing solution requires the subscription contract code to be configured in the FPGA. After the FPGA receives the market data, it traverses the subscription table to see if there is a corresponding contract code. This is only suitable for a small number of contract rules. When the number of subscriptions is large, the time taken to traverse the entire subscription table increases significantly. Moreover, it can only query the new contract after querying the previous contract, which causes the query latency to increase linearly with the number of subscription entries. It is difficult to meet the low latency requirement in a large-capacity subscription scenario. Therefore, it is necessary to provide a financial market data subscription and processing system that can meet the low-latency query response requirements in large-scale subscription scenarios and further improve the processing efficiency of financial market data. Summary of the Invention

[0004] This application provides a financial market data subscription and processing system and method, which achieves low-latency query response in large-scale subscription scenarios and further improves the processing efficiency of financial market data.

[0005] Other objects and advantages of the present invention can be further understood from the technical features disclosed herein.

[0006] To achieve one or more of the above objectives or other objectives, the present invention provides a financial market data subscription and processing system.

[0007] A financial market data subscription and processing system, comprising: The market data preprocessing module is used to receive market data, divide the market data into single contract data, and extract the contract code / encoding from the single contract data. The subscription rule preprocessing module is used to merge and map contract subscription rules into a rule table and output it to the subscription management unit; The subscription management unit is used to store the rule table, perform contract subscription queries based on the contract code / encoding extracted from the single contract data, and output the contract subscription query results to the market data output unit. The market data calculation unit is used to extract the fields required for the full market data snapshot from the single contract data, output the full market data snapshot, and if there is incremental market data, update the full market data snapshot with the incremental market data through the incremental calculation and synthesis algorithm, and output the updated full market data snapshot to the market data snapshot cache module and the market data output unit. The market data snapshot caching module is used to store full market data snapshots and transmit previous market data snapshots to the market data calculation unit; The market data output unit is used to filter the full market data snapshot output by the market data calculation unit based on the contract subscription query results, and output the subscription market data snapshot.

[0008] The specific process of updating the incremental market data to the full market data snapshot is as follows: The market data calculation unit obtains the previous market data snapshot from the market data snapshot cache module, and performs incremental calculations on the incremental market data and the previous market data snapshot using an incremental calculation synthesis algorithm to synthesize the updated full market data snapshot.

[0009] The market data preprocessing module receives market data distributed by the market data gateway, which includes full market data and incremental market data.

[0010] The market data is transmitted via TCP or UDP network transmission protocols.

[0011] The specific process by which the subscription rule preprocessing module merges and maps contract subscription rules into a rule table is as follows: The subscription rule preprocessing module obtains the contract subscription rule and extracts the contract type and contract delivery month based on the contract code / encoding. Map the contract type to row index r, and the contract delivery month to column index c; A two-dimensional array rule table is constructed based on the row index r and the column index c; The identifier value for the index position corresponding to a subscribed contract is 1, and the identifier value for the index position corresponding to a non-subscribed contract is 0.

[0012] The specific process of mapping the contract type to row index r is as follows: The lower N bits of the ASCII codes corresponding to the characters of the contract type are concatenated. If all the characters of the contract type are uppercase letters or all are lowercase letters, then N is 5; otherwise, N is 6.

[0013] The specific process of mapping the contract delivery month to column index c is as follows: Set the year of the transaction date as Y and the month as M; Set the year of the contract delivery date as y and the month as m; If the month m corresponding to the contract delivery date does not exceed 12 months from the transaction date, then the formula for calculating the column index c is: c = (yY) × 12 + (mM); If the month m corresponding to the contract delivery date exceeds 12 months from the transaction date, the formula for calculating the column index c is: c = 7 + (yY) × 4 + (mM) / 3, where (mM) / 3 is an integer quotient.

[0014] The specific process by which the subscription management unit performs contract subscription queries is as follows: The subscription management unit determines the row index r and the column index c based on the contract type extracted by the contract code / encoding and the contract delivery month, and queries the two-dimensional array rule table based on the row index r and the column index c to obtain the identifier value of the corresponding index position in the two-dimensional array rule table, and determines whether the contract has been subscribed.

[0015] The subscription rule preprocessing module is located in the host computer, and the market data preprocessing module, subscription management unit, market data calculation unit, market data snapshot cache module, and market data output unit are all implemented by FPGA logic.

[0016] A method for subscribing to and processing financial market data, implemented based on any one of the aforementioned financial market data subscription and processing systems, includes: The subscription rule preprocessing module merges and maps contract subscription rules into a rule table, and outputs it to the subscription management unit; The market data preprocessing module receives market data, divides the market data into single contract data, and extracts the contract code / encoding from the single contract data. The subscription management unit performs contract subscription queries based on the contract code / encoding extracted from the single contract data and outputs the contract subscription query results to the market data output unit; The market data calculation unit extracts the fields required for the full market data snapshot and the incremental market data from the single contract data, updates the incremental market data to the full market data snapshot through the incremental calculation and synthesis algorithm, and outputs it to the market data snapshot cache module and the market data output unit. The market data output unit filters the full market data snapshot output by the market data calculation unit based on the contract subscription query results, and outputs the subscription market data snapshot.

[0017] Compared with the prior art, the beneficial effects of the present invention mainly include: This application provides a financial market data subscription and processing system and method, which achieves low-latency query response in large-scale subscription scenarios and further improves the processing efficiency of financial market data.

[0018] To make the above and other objects, features and advantages of the present invention more readily understood, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the specific embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the structure of a financial market data subscription processing system provided in Embodiment 1 of this application.

[0021] Figure 2 This is a schematic diagram of the contract subscription mapping process provided in Embodiment 1 of this application. Detailed Implementation

[0022] The foregoing and other technical contents, features, and effects of the present invention will be clearly presented in the following detailed description of a preferred embodiment with reference to the accompanying drawings. The directional terms mentioned in the following embodiments, such as up, down, left, right, front, or back, are merely for reference to the accompanying drawings. Therefore, the directional terms used are for illustrative purposes and not for limiting the present invention.

[0023] The embodiments of this application will now be described in detail with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the embodiments of this application to facilitate a better understanding of the application. However, the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments.

[0024] Example 1

[0025] like Figure 1As shown, a financial market data subscription and processing system includes: a subscription rule preprocessing module, a market data preprocessing module, a subscription management unit, a market data calculation unit, a market data snapshot caching module, and a market data output unit; the subscription rule preprocessing module is located in the host computer, and the market data preprocessing module, the subscription management unit, the market data calculation unit, the market data snapshot caching module, and the market data output unit are all implemented by FPGA logic; Specifically, the market data preprocessing module is used to receive market data distributed by the market data gateway (market data is transmitted via TCP or UDP network transmission protocols, including full market data and incremental market data), and to segment the market data into single contract data, extracting the contract code or contract encoding from the single contract data; wherein, the contract code is a unique identifier string of the contract, and the contract encoding is an integer number mapped from the string. The subscription rule preprocessing module is used to merge and map contract subscription rules into a rule table and output it to the subscription management unit; The subscription management unit stores the rule table and performs contract subscription queries based on the contract code / encoding extracted from the single contract data, and outputs the contract subscription query results to the market data output unit. The market data calculation unit is used to extract the fields required for a full market data snapshot and incremental market data from single contract data. If there is no incremental market data, it outputs a full market data snapshot to the market data snapshot cache module and the market data output unit based on the fields required for the full market data snapshot extracted from the single contract data. If there is incremental market data, it obtains the previous market data snapshot from the market data snapshot cache module, performs incremental calculations on the incremental market data and the previous market data snapshot using an incremental calculation synthesis algorithm, synthesizes an updated full market data snapshot, and outputs the updated full market data snapshot to the market data snapshot cache module and the market data output unit. The market data snapshot caching module is used to store full market data snapshots and to transfer previous market data snapshots to the market data calculation unit when there are incremental market data updates. The market data output unit is used to filter the full market data snapshot output by the market data calculation unit based on the contract subscription query results, and output the subscription market data snapshot.

[0026] The subscription rule preprocessing module in this application is located in the host computer. It obtains contract subscription rules (e.g., Silver AG: contract type and all AG contracts; au00: contract type is Gold AU, contract delivery month is the current trading month) from the host computer, merges and maps these contract subscription rules into a rule table, and configures the rule table into the subscription management unit. The specific process is as follows: Figure 2 As shown, it includes: Obtain the contract subscription rules, extract the contract type and contract delivery month based on the contract code / encoding; map the contract type to the row index r, and the contract delivery month to the column index c, with the identifier value of the index position corresponding to the subscribed contract being 1, and the identifier value of the index position corresponding to the unsubscribed contract being 0; construct a two-dimensional array rule table based on the row index r and the column index c; Specifically, the process of mapping a contract type to a row index r includes: the contract type is usually two uppercase letters (ASCII code: 65-90) or two lowercase letters (ASCII code: 97-122). The lower N bits of the corresponding ASCII code of the contract type are concatenated. If all the characters of the contract type are uppercase letters or all are lowercase letters, then N is 5; otherwise, N is 6. The concatenation results in a value with a width of 2N bits. For example, the contract type ag is mapped to row index r=0xE1 (i.e., (0x67&0x1f)<<5|0x61&0x1f). The specific process of mapping the contract delivery month to column index c includes: Set the year of the trading day as Y and the month as M, and the year of the contract settlement date as y and the month as m; if the month m corresponding to the contract settlement date does not exceed 12 months from the trading day, map the contract month to 12 bits, and calculate the column index c according to the contract type index: c=(yY)×12+(mM). For example: if the transaction date is June 2025, and the subscription contracts are ag2506 and ag2512, then the rule table will have rules[0xE1][0]=1, rules[0xE1][6]=1, and all other index values ​​will be 0; If the month m corresponding to the contract delivery date exceeds 12 months from the transaction date, the interval between the contract delivery date and the current transaction date is: (yY)×12+(mM). The nth quarter month after the contract delivery date exceeds 12 months from the current transaction date is denoted as K: K=[(yY)×12+(mM)-12] / 3=(yY)×4+(mM) / 3-4. rules[0]-rules

[11] stores the rules for the most recent 12 months. The rules for the Kth quarter month are mapped to rules[11+K]. The formula for calculating the column index c is: c=11+[(yY)×4+(mM) / 3-4]=7+(yY)×4+(mM) / 3, where (mM) / 3 is the integer quotient. For example: if the year of the transaction date is Y = 2025, the month is M = 6, and the subscription contracts are ag2506 and ag2609, then the rule table is rules[0xE1][0] = 1, rules[0xE1]

[12] = 1, and all other index values ​​are 0; The subscription management unit of this application obtains the rule table from the subscription rule preprocessing module and performs contract subscription queries based on the contract code / encoding extracted from the single contract data. The specific process includes: Based on the contract type and delivery month extracted from the contract code / encoding, determine the row index r and column index c, and query the two-dimensional array rule table rules based on the row index r and column index c to obtain the identifier value of the corresponding index position in the two-dimensional array rule table rules, and determine whether the contract has been subscribed.

[0027] Example 2

[0028] Embodiment 2 of this application provides a method for subscribing to and processing financial market data, implemented based on the aforementioned financial market data subscription and processing system, including: Step S1: The subscription rule preprocessing module merges and maps the contract subscription rules into a rule table and outputs it to the subscription management unit; Step S2: The market data preprocessing module receives market data, divides the market data into single contract data, and extracts the contract code / contract encoding from the single contract data; Step S3: The subscription management unit performs a contract subscription query based on the contract code / contract code extracted from the single contract data, and outputs the contract subscription query results to the market data output unit; Step S4: The market data calculation unit extracts the fields required for the full market data snapshot and the incremental market data from the single contract data. If there is no incremental market data, the full market data snapshot is output to the market data snapshot cache module and the market data output unit according to the fields required for the full market data snapshot extracted from the single contract data. If there is an incremental market data, the previous market data snapshot is obtained from the market data snapshot cache module. The incremental market data and the previous market data snapshot are incrementally calculated and synthesized using the incremental calculation and synthesis algorithm to synthesize an updated full market data snapshot. The updated full market data snapshot is then output to the market data snapshot cache module and the market data output unit. Step S5: The market data output unit filters the full market data snapshot output by the market data calculation unit based on the contract subscription query results and outputs the subscription market data snapshot.

[0029] This application provides a financial market data subscription and processing system and method, which achieves low-latency query response in large-scale subscription scenarios and further improves the processing efficiency of financial market data.

[0030] The common English terms or letters used in this invention for clarity of description are for illustrative purposes only and are not limiting interpretations or specific uses. They should not be used to limit the scope of protection of this invention based on their possible Chinese translations or specific letters.

[0031] It should also be noted that in this article, relational terms such as “first” and “second” are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

Claims

1. A financial market data subscription and processing system, characterized in that, include: The market data preprocessing module is used to receive market data, divide the market data into single contract data, and extract the contract code / encoding from the single contract data. The subscription rule preprocessing module is used to merge and map contract subscription rules into a rule table and output it to the subscription management unit; The subscription management unit is used to store the rule table, perform contract subscription queries based on the contract code / encoding extracted from the single contract data, and output the contract subscription query results to the market data output unit. The market data calculation unit is used to extract the fields required for the full market data snapshot from the single contract data, output the full market data snapshot, and if there is incremental market data, update the full market data snapshot with the incremental market data through the incremental calculation and synthesis algorithm, and output the updated full market data snapshot to the market data snapshot cache module and the market data output unit. The market data snapshot caching module is used to store full market data snapshots and transmit previous market data snapshots to the market data calculation unit; The market data output unit is used to filter the full market data snapshot output by the market data calculation unit based on the contract subscription query results, and output the subscription market data snapshot.

2. The financial market data subscription and processing system according to claim 1, characterized in that, The specific process of updating the incremental market data to the full market data snapshot is as follows: The market data calculation unit obtains the previous market data snapshot from the market data snapshot cache module, and performs incremental calculations on the incremental market data and the previous market data snapshot using an incremental calculation synthesis algorithm to synthesize the updated full market data snapshot.

3. The financial market data subscription and processing system according to claim 1, characterized in that, The market data preprocessing module receives market data distributed by the market data gateway, which includes full market data and incremental market data.

4. The financial market data subscription and processing system according to claim 3, characterized in that, The market data is transmitted via TCP or UDP network transmission protocols.

5. A financial market data subscription and processing system according to claim 1, characterized in that, The specific process by which the subscription rule preprocessing module merges and maps contract subscription rules into a rule table is as follows: The subscription rule preprocessing module obtains the contract subscription rule and extracts the contract type and contract delivery month based on the contract code / encoding. Map the contract type to row index r, and the contract delivery month to column index c; A two-dimensional array rule table is constructed based on the row index r and the column index c; The identifier value for the index position corresponding to a subscribed contract is 1, and the identifier value for the index position corresponding to a non-subscribed contract is 0.

6. A financial market data subscription and processing system according to claim 5, characterized in that, The specific process of mapping the contract type to row index r is as follows: The lower N bits of the ASCII codes corresponding to the characters of the contract type are concatenated. If all the characters of the contract type are uppercase letters or all are lowercase letters, then N is 5; otherwise, N is 6.

7. A financial market data subscription and processing system according to claim 5, characterized in that, The specific process of mapping the contract delivery month to column index c is as follows: Set the year of the transaction date as Y and the month as M; Set the year of the contract delivery date as y and the month as m; If the month m corresponding to the contract delivery date does not exceed 12 months from the transaction date, then the formula for calculating the column index c is: c = (yY) × 12 + (mM); If the month m corresponding to the contract delivery date exceeds 12 months from the transaction date, the formula for calculating the column index c is: c = 7 + (yY) × 4 + (mM) / 3, where (mM) / 3 is an integer quotient.

8. A financial market data subscription and processing system according to claim 5, characterized in that, The specific process by which the subscription management unit performs contract subscription queries is as follows: The subscription management unit determines the row index r and the column index c based on the contract type extracted by the contract code / encoding and the contract delivery month, and queries the two-dimensional array rule table based on the row index r and the column index c to obtain the identifier value of the corresponding index position in the two-dimensional array rule table, and determines whether the contract has been subscribed.

9. A financial market data subscription and processing system according to claim 1, characterized in that, The subscription rule preprocessing module is located in the host computer, and the market data preprocessing module, subscription management unit, market data calculation unit, market data snapshot cache module, and market data output unit are all implemented by FPGA logic.

10. A method for subscribing to and processing financial market data, implemented based on the financial market data subscription and processing system according to any one of claims 1-9, characterized in that, include: The subscription rule preprocessing module merges and maps contract subscription rules into a rule table, and outputs it to the subscription management unit; The market data preprocessing module receives market data, divides the market data into single contract data, and extracts the contract code / encoding from the single contract data. The subscription management unit performs contract subscription queries based on the contract code / encoding extracted from the single contract data and outputs the contract subscription query results to the market data output unit; The market data calculation unit extracts the fields required for the full market data snapshot and the incremental market data from the single contract data, updates the incremental market data to the full market data snapshot through the incremental calculation and synthesis algorithm, and outputs it to the market data snapshot cache module and the market data output unit. The market data output unit filters the full market data snapshot output by the market data calculation unit based on the contract subscription query results, and outputs the subscription market data snapshot.

Images