Market data processing method and system, electronic equipment and storage medium

Abstract

The invention relates to a market data processing method and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring a market service message in a market transaction process; analyzing the market information service message to obtain initial total information of the market information transaction and an incremental market information in the market information transaction process; using the market information of the incremental market to update initial total information of the market to obtain target total information of the market; and packaging the target full-amount information according to a preset format to obtain a market data packet, and based on pointer jump and FPGA pipeline design thoughts, performing data processing in sequence according to exchange data characteristics in a data flow-in and flow-out process to obtain a market data packet. In the data processing process, the newly-added data is utilized to carry out comprehensive accuracy verification on the existing market information, so that the accuracy of data input and output is ensured, and the stability and reliability of the whole transaction system are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a market data processing method, system, electronic device and storage medium. Background Art

[0002] High-frequency futures trading relies on stable and real-time acquisition of accurate market information, which places strict requirements on the decoding speed of market data and system stability. In the current market, market data decoding is mainly divided into two methods: software decoding and hardware decoding.

[0003] However, although software decoding is more stable and does not need to worry about the consumption of hardware resources, its parsing speed is slow; and although hardware decoding can achieve lower parsing delays, different hardware decoding schemes have significant differences in hardware resource occupation and system stability; the large amount of hardware resources occupied may lead to resource waste, and after integration, the layout and routing of the FPGA becomes complicated, and the decoding module with tight timing is difficult to meet the requirements of hold time and setup time, resulting in frequent timing violations. These factors may ultimately affect the stability of the entire high-frequency trading system.

[0004] Therefore, there is an urgent need to develop a market data processing method, system, electronic device and storage medium to solve one or more of the above-mentioned problems. Summary of the invention

[0005] In view of this, in order to solve the above technical problems or part of the technical problems, an embodiment of the present invention provides a market data processing method, system, electronic device and storage medium.

[0006] In a first aspect, the present application provides a market data processing method, the method comprising:

[0007] Obtain market information during market trading;

[0008] Analyze the market information service message to obtain the initial full information of the market transaction and the incremental market information during the market transaction;

[0009] Using the market information of the incremental market information to update the initial full information of the market, to obtain the target full information of the market;

[0010] The target full amount information is encapsulated according to a preset format to obtain a market data package.

[0011] In a possible implementation, before analyzing the market service message, the method further includes:

[0012] Extracting protocol information of the market service message using the Ethernet message protocol;

[0013] Based on the protocol information, the market service message is cut so that the cut market service message is an application layer message.

[0014] In a possible implementation, the market service message includes a market snapshot message;

[0015] Analyze the market information to obtain the initial full information of the market, including:

[0016] The market snapshot message is analyzed to obtain snapshot information of the market transaction as the initial full information of the market.

[0017] In a possible implementation, the market service message includes a market increment message;

[0018] Analyze the market information service message to obtain the incremental market information during the market transaction process, including:

[0019] Analyze the market price increment message to obtain a valid price range of the market price;

[0020] According to the effective price range of the market information, the incremental market information in the market information trading process is obtained.

[0021] In a possible implementation, the updating process of the initial full information of the market by using the market information of the incremental market to obtain the target full information of the market includes:

[0022] Decode the market information of the incremental market to obtain the price change value and transaction quantity of the newly added market;

[0023] Calculate the transaction price corresponding to the price change value and transaction quantity according to the preset market calculation rules;

[0024] The initial full amount information of the market is updated according to the transaction price and the transaction quantity to obtain the target full amount information of the market.

[0025] In a possible implementation manner, analyzing the market price increment message to obtain a valid price range of the market price includes:

[0026] The market price incremental message is divided into an incremental message header and an incremental message body, wherein the incremental message body includes valid information of multiple newly added transactions;

[0027] The head pointer and the tail pointer point to the incremental message header and the incremental message body respectively;

[0028] Use pointer jumping technology to analyze new market information and obtain the effective price range of the market.

[0029] In a second aspect, the present application provides a market data processing system, the system comprising:

[0030] Acquisition module, to obtain market information during market trading;

[0031] A preprocessing module analyzes the market information to obtain the initial full information of the market transaction and the incremental market information during the market transaction;

[0032] An updating module, using the market information of the incremental market to update the initial full information of the market, to obtain the target full information of the market;

[0033] The encapsulation module encapsulates the target full amount information according to a preset format to obtain a market data package.

[0034] In a possible implementation, the update module includes:

[0035] A decoding unit, used to decode the market information of the incremental market to obtain the price change value and transaction quantity of the newly added market;

[0036] A calculation unit, used to calculate the transaction price corresponding to the price change value and the transaction quantity according to a preset market calculation rule;

[0037] A full quantity updating unit, used for updating the initial full quantity information of the market according to the transaction price and the transaction quantity, and obtaining the target full quantity information of the market;

[0038] A market information storage unit, used to store preset market calculation rules and market price change units;

[0039] The full information storage unit is used to store the initial full information of the market.

[0040] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the market data processing method described in any one of the embodiments of the first aspect are implemented.

[0041] In a fourth aspect, the present application further provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the market data processing method described in any embodiment of the first aspect are implemented.

[0042] The above-mentioned technical scheme provided by the embodiment of the present application has the following advantages compared with the prior art: the market data processing method provided by the embodiment of the present application is based on the design concept of pointer jumping and FPGA pipeline technology, and performs data processing in sequence according to the characteristics of exchange data during data flow in and out. During the data processing process, these new data are used to perform comprehensive accuracy verification of existing market information, thereby ensuring the accuracy of data input and output; during the data processing process, due to the low hardware resource occupancy rate, resource waste is avoided, the overall performance of the system is improved, and the timing optimization of the market decoding module is realized, which is conducive to the rapid convergence of the timing, so as to achieve the effect of improving the stability and reliability of the entire trading system. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0045] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0046] Figure 1 A flowchart of a market data processing method provided in an embodiment of the present application;

[0047] Figure 2 A flowchart of another market data processing method provided in an embodiment of the present application;

[0048] Figure 3 A flowchart of another method for processing market data provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of the format of a market information service layer message provided in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of a pointer jumping technology provided in an embodiment of the present application;

[0051] Figure 6 A structural diagram of a market data processing system provided in an embodiment of the present application;

[0052] Figure 7 A structural diagram of another market data processing system provided in an embodiment of the present application;

[0053] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0055] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0056] In order to solve the technical problems in the prior art that software decoding has high stability without considering the problem of hardware resource occupation, but the decoding penetration delay is high, and the hardware decoding penetration delay is low but the occupation of hardware resources by different hardware decoding schemes may lead to resource waste and poor stability of the system after integration, the present application proposes a market data processing method, system, electronic device and storage medium, which are based on the design concept of pointer jumping and FPGA pipeline technology, and perform data processing in sequence according to the characteristics of exchange data during the data flow in and out. During the data processing process, these new data are used to perform comprehensive accuracy verification of existing market information, thereby ensuring the accuracy of data input and output; the data processing process avoids resource waste due to the low hardware resource occupation rate, the overall system performance is improved, and the timing optimization of the market decoding module is realized, which is conducive to the rapid convergence of the timing, so as to achieve the effect of improving the stability and reliability of the entire trading system.

[0057] Figure 1 A flowchart of a market data processing method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the method specifically includes:

[0058] S101, obtaining market information service messages during market trading;

[0059] In the process of financial market transactions, market information contains real-time market data, such as stock prices, foreign exchange rates, futures contracts, and the latest quotes of other financial instruments. Obtaining market information is crucial for investors and traders, as it can help with investment decisions during the trading process.

[0060] In this embodiment, the market data is processed by acquiring the market service message in the market transaction process.

[0061] Specifically, in order to obtain these market information, it is usually necessary to rely on the real-time data interface of professional financial data providers or exchanges. Data providers transmit real-time market data through high-speed networks to ensure that investors can obtain the latest market information in a timely manner. Market information is usually sent in a standardized format, such as the FIX protocol (Financial Information eXchange), so that various trading systems and software can easily parse and process it; in addition, the acquisition process of market information involves multiple technical links, including data collection, transmission, parsing and display.

[0062] Furthermore, the method for obtaining market business messages includes: first, the data collection system needs to be connected to the server of the exchange or data provider, and obtain the real-time data stream through API (application programming interface) or other communication protocols. Then, the data stream will be transmitted to the back-end server of the trading system, parsed and processed, and finally displayed to investors in a preset form.

[0063] In addition, during the specific implementation, the acquisition of market business messages is not limited to real-time data, but also includes historical data and in-depth data. Historical data can help investors analyze market trends and historical performance, while in-depth data provides more detailed buy and sell order information to help investors better understand market supply and demand conditions.

[0064] S102, analyzing the market information service message to obtain the initial full information of the market transaction and the incremental market information during the market transaction;

[0065] In this embodiment, by analyzing the market business message, the initial full information of the market transaction is extracted, and the incremental transaction market information generated during the market transaction process is captured. The incremental market information is used to reflect the transaction changes in a specific time period of the market; by analyzing the initial full information and incremental market information, the overall market trend and specific transaction behavior are obtained.

[0066] Specifically, the message needs to be parsed, the data fields in the message need to be extracted, and converted into a processable format. Then, the data is analyzed through algorithms to identify key information such as market trends, price fluctuations, and changes in trading volume.

[0067] Among them, the initial full information includes all relevant transaction data and market dynamic data, such as the stock's opening price, highest price, lowest price, closing price, trading volume, etc.; the incremental market information focuses on transaction changes that occur within a certain time interval, such as price fluctuations, increases or decreases in trading volume, etc.

[0068] S103, using the market information of the incremental market to update the initial full information of the market, to obtain the target full information of the market;

[0069] In this embodiment, by analyzing and processing the market information in the incremental market, these incremental data are effectively applied to the initial full information of the market to obtain the latest market target full information, so that the updated full information not only includes all the initial data, but also includes all the latest changes and dynamics; the updated full information provides a comprehensive and real-time market view to help traders better understand and respond to various changes in the market.

[0070] S104: Encapsulate the target full amount information according to a preset format to obtain a market data package.

[0071] In this embodiment, the rules and standards for data encapsulation are predefined as a preset format before data processing, and then the target full information is comprehensively combed and organized to ensure that the data in the full information is accurate, complete and meets business requirements. Finally, according to the requirements of the preset format, the full information is filled into the corresponding data field to obtain a market data package that meets the requirements of the preset format. The market data package is sent to the trader to facilitate the trader's acquisition and analysis of the market.

[0072] Specifically, the preset format may include multiple aspects such as naming rules for data fields, definitions of data types, limits on data length, and the order in which data is arranged, to provide guidance on how to organize various types of information in an orderly and standardized manner.

[0073] During the packaging process, the target full amount of information is comprehensively sorted and organized to ensure that each piece of information is accurate, complete and meets business needs; then, according to the requirements of the preset format, this information is filled into the corresponding data fields one by one;

[0074] During the filling process, it is necessary to remove invalid or redundant information in the target full information, and to perform data conversion to convert the information into the format or type required by the preset format. In addition, data verification is required to ensure that the filled data complies with the format specifications to avoid errors or inconsistencies.

[0075] The market data processing method provided in this embodiment obtains the full information and incremental market information of the transaction by analyzing the market business messages in the market transaction process, then uses the market information of the incremental market to update the initial full information to obtain the target full information, and finally encapsulates the target full information in a preset format to obtain a market data packet and feeds it back to the trading user; the market data processing process is based on the FPGA pipeline design concept, and data processing is performed in sequence according to the characteristics of the exchange data during the data flow in and out, and the newly added data is used to perform comprehensive accuracy verification on the existing market information during the data processing process, thereby ensuring the accuracy of data input and output; and, in the entire data processing process, due to the low hardware resource occupancy rate, resource waste is avoided, and the timing optimization of the market decoding module is realized, which is conducive to the rapid convergence of the timing, so as to achieve the effect of improving the stability and reliability of the entire trading system.

[0076] In an optional solution of the embodiment of the present invention, before analyzing the market service message, the method further includes:

[0077] Extracting protocol information of the market service message using the Ethernet message protocol;

[0078] In this embodiment, before analyzing the market service message, the market service message is deeply extracted and parsed through the Ethernet message protocol to obtain the protocol information in the message. The protocol information includes but is not limited to key information such as the type, length, source address and destination address of the message. Through the above protocol information, the structure and transmission mode of the message can be understood.

[0079] Based on the protocol information, the market service message is cut so that the cut market service message is an application layer message.

[0080] In this embodiment, after obtaining the protocol information, the original market business message is further processed and cut based on the extracted protocol information, and the message is decomposed into smaller and easier to process application layer messages. The application layer message usually contains specific business data and instructions. After cutting, the business data and instructions will become clearer and easier to analyze, and useful information in the message can be more effectively extracted and utilized to provide support for subsequent business processing and decision-making.

[0081] Specifically, the market transaction process includes the diversion of market snapshot messages, market increment messages and other heartbeat messages. According to the Ethernet message protocol, the five-tuple information (source IP, destination IP, source port, destination port, protocol number) is extracted and filtered, and the L2, L3 and L4 layers of the message are trimmed. The remaining application layer data is recombined into a single-cycle 8Byes 10 Gigabit Ethernet axi stream pure business data flow to reduce the processing pressure of the subsequent modules.

[0082] In an optional solution of the embodiment of the present invention, the market service message includes a market snapshot message;

[0083] In this embodiment, the market service message includes other relevant information, such as market update messages, market request messages, etc. These messages together constitute a complete market information transmission system, ensuring the timely and accurate transmission of market information. The market snapshot message not only provides the initial full information and basic market information before the market opens, but also pushes the current full information at an agreed frequency after the market opens. The agreed frequency is determined when the futures company logs in to the exchange.

[0084] Analyze the market information to obtain the initial full information of the market, including:

[0085] The market snapshot message is analyzed to obtain snapshot information of the market transaction as the initial full information of the market.

[0086] Market service messages include market snapshot messages, which are mainly used to provide market data at a specific moment, including but not limited to key information such as prices and trading volumes of various financial products such as stocks, bonds, and foreign exchange. Through market snapshot messages, investors and market participants can quickly understand the latest market trends and make more informed investment decisions.

[0087] In this embodiment, by analyzing the market snapshot message, detailed snapshot information about the market transaction is obtained, and then comprehensive initial full information of the market is constructed based on the snapshot information.

[0088] Specifically, snapshot information not only includes real-time data of the current market, but also covers various key indicators and parameters in the transaction process, such as the price, volume and volatility of various financial products such as stocks, bonds, foreign exchange, etc. It may also include market sentiment and investor behavior patterns. By analyzing snapshot information, we can obtain dynamic changes in the market.

[0089] In an optional solution of the embodiment of the present invention, the market service message includes a market increment message;

[0090] In this embodiment, the market service message includes the market incremental message. The market incremental message refers to the message format used to transmit the latest price changes and related information in the real-time financial market. It usually contains the latest transaction data of financial products such as stocks, bonds, foreign exchange, such as buying and selling prices, trading volumes and other key information; through the market incremental message, investors and transaction participants can quickly obtain the latest market dynamics and make timely investment decisions.

[0091] Analyze the market information service message to obtain the incremental market information during the market transaction process, including:

[0092] Analyze the market price increment message to obtain a valid price range of the market price;

[0093] According to the effective price range of the market information, the incremental market information in the market information trading process is obtained.

[0094] In this embodiment, the above-mentioned market service messages are analyzed in detail to extract the incremental market information generated in the market transaction process. By analyzing and interpreting the incremental market information, the effective price range is identified. The effective price range refers to a price range with practical significance and reference value in the market transaction process. The price range can reflect the actual transaction situation and price fluctuation of the market.

[0095] After identifying the valid price range, the incremental market information in the market trading process is obtained based on the valid price range. The incremental market information refers to the changes and updates of market data in a specific time period compared with the previous time period. By analyzing the incremental market information, we can understand the dynamic changes of the market, grasp the market trends and fluctuations, and provide more accurate and timely decision-making basis for trading participants.

[0096] Figure 2 A flow chart of another method for processing market data provided in an embodiment of the present application is shown as follows: Figure 2 As shown, the method specifically includes:

[0097] S201, decoding the market information of the incremental market to obtain the price change value and transaction quantity of the newly added market;

[0098] In this embodiment, the price change value is usually calculated by comparing the difference between the current price and the previous price, and the transaction quantity is directly taken from the parsed data field; first, the key market information fields, including price, quantity, timestamp, etc., are extracted from the received incremental market data; then, these raw data are formatted and cleaned to ensure the accuracy and consistency of the data.

[0099] For example, the compressed data is sequentially subjected to Varint decoding and ZigZag decoding (the compressed data is mainly compressed using vint encoding (Varint encoding + ZigZag encoding), ZigZag encoding maps signed integers to unsigned integers, and Varint encoding encodes 64-bit binary-encoded unsigned integers with bytes of different lengths according to their size), and the price change value and transaction quantity are parsed.

[0100] In the specific steps of decoding, the extracted market information fields are parsed according to the data protocol or specification, including identifying key elements such as price change identifiers and transaction quantity identifiers in the data, and associating the identifiers with corresponding numerical values, so as to convert the original market data into a structured form that is easy to understand and analyze; once the data is successfully parsed, the price change value and transaction quantity of the newly added market are obtained by calculation, and the price change value and transaction quantity provide information about market trends and transaction activity, which helps to accurately grasp market dynamics.

[0101] S202, calculating the transaction price corresponding to the price change value and the transaction quantity according to a preset market calculation rule;

[0102] In this embodiment, the specific calculation steps include: first, clarifying the preset market calculation rules, which may involve price change factors, weight distribution, time windows and other factors, which together constitute the framework of price calculation; then, according to the market calculation rules, the collected price change values ​​and transaction quantities are input as input data into the preset calculation model, and the transaction price calculated by the model output is finally obtained. The transaction price takes into account both the impact of the price change value and the factor of the transaction quantity, and can accurately reflect the market trading status.

[0103] For example, to calculate the final price, it is necessary to call the market calculation rule formula "Transaction price = calculation basis + minimum price change * price change value" to perform multiplication and addition operations. After calculation, the prices and transaction quantities of all valid price ranges are stored with the contract number index.

[0104] When other incremental market conditions come later, the final results will still be stored in the same location according to this process. The same contract will be continuously updated at different trading times, and the price and transaction quantity updated each time will be written to the same location in the storage. In this way, the latest full contract information can always be stored, thereby realizing the function of maintaining the full information of all contracts.

[0105] S203: Update the initial full amount information of the market according to the transaction price and transaction quantity to obtain the target full amount information of the market.

[0106] In this embodiment, the process of updating and replacing data is executed according to the transaction price and transaction quantity to update the initial full information of the market to ensure that the latest market target full information is obtained. The update process involves reviewing each relevant data in the initial data, and making corresponding modifications and replacements according to the latest transaction price and transaction quantity. Through updating and replacement, the accuracy and timeliness of the market information are ensured to provide the most valuable reference for relevant users.

[0107] Specifically, when the market snapshot message arrives (the market snapshot message is not compressed and does not need to be decoded and calculated), the contract snapshot information is parsed, the contract number index and other full contract information are extracted from the snapshot information, and the contract number index is used to read and write the address to replace the full contract information and the missing valid price domain in the incremental market (the valid price domain that has not changed will not be pushed in the incremental market, so the contract snapshot information is needed to complete it). At the same time, the valid price domain that already exists in the incremental market is verified. If it is different from the relevant valid price domain in the snapshot, it means that the incremental market calculation is wrong, and the incremental market is replaced with the valid price domain in the snapshot.

[0108] In addition, the process of updating the initial full information to the target full information includes before the opening and during the opening.

[0109] Specifically, when a market snapshot message is received before the market opens, the FPGA parses out the initial full information and basic market information, which are used to provide to the multicast incremental message for cumulative calculation; when a market snapshot message is received during the market, the FPGA parses out the full information, which is used to compare and verify with the calculation result of the latest multicast incremental message received.

[0110] The market data processing method provided in this embodiment, in the process of processing incremental market information, extracts the price change value and transaction quantity of the newly added market information by decoding the market information, and then calculates according to the pre-set market calculation rules to determine the corresponding transaction price. After obtaining the transaction price and transaction quantity, the target full information of the market can be obtained according to the initial full information of the data update market; the target full information reflects the latest market situation after the incremental market processing, and provides the latest reference data for investors and transaction participants. Ensure the accuracy and real-time nature of market data, and implement full analysis of udp multicast market and tcp market snapshots based on FPGA, and verify multicast market with market snapshots at the agreed frequency.

[0111] Figure 3 A flow chart of another method for processing market data provided in an embodiment of the present application is shown as follows: Figure 3 As shown, the method specifically includes:

[0112] S301, dividing the market price incremental message into an incremental message header and an incremental message body, wherein the incremental message body includes valid information of multiple newly added transactions;

[0113] In this embodiment, in the process of processing the market incremental message, it is divided into two main parts: the incremental message header and the incremental message body. The incremental message header usually contains some basic information about the message, such as the version number of the message, the length of the message, and the data type contained in the message body, while the incremental message body carries the core data content and contains valid information of multiple newly added transactions.

[0114] Specifically, the valid information of the newly added transactions includes detailed data of each transaction, such as the identification of the two parties to the transaction, transaction time, transaction price, transaction volume and other key information.

[0115] By segmenting, incremental messages can efficiently transmit the latest market dynamics, allowing the recipient to quickly obtain the latest transaction data and make corresponding decisions.

[0116] S302, point the head pointer and the tail pointer to the incremental message header and the incremental message body respectively;

[0117] S303: Use pointer jumping technology to analyze the new information of the market and obtain the effective price range of the market.

[0118] This embodiment uses FPGA pipeline and register with pointer jumping technology to extract the effective value domain.

[0119] Specifically, to ensure accurate data reading, the head pointer and the tail pointer are pointed to the incremental message header and the incremental message body respectively. After pointing to the incremental message header and the body, the pointer jumping technology is used to conduct in-depth analysis of the new market information message.

[0120] The head pointer usually contains basic information of the message, such as the type and length of the message, while the tail pointer is used to mark the end of the message. Pointer jumping technology is an efficient data processing method that allows the program to quickly locate key data areas by jumping the pointer without traversing the entire message. Specifically, in the scenario of adding new messages to the market, pointer jumping technology will be used to identify and extract the valid price domain in the market; the valid price domain usually refers to those price information that are useful for subsequent processing or analysis, such as opening price, closing price, highest price, lowest price, etc.; through pointer jumping, the program can quickly skip those unimportant data parts and directly locate these key price information, thereby greatly improving the efficiency of data processing.

[0121] like Figure 4As shown in the figure, the futures market business layer message is divided into a message header msg_head and a message body msg_body. The msg_body contains valid information contracts of several contracts, and the number of contracts is uncertain. Each contract contains several valid price domain fields, and the number of fields is uncertain. Each field is divided into a domain header field_head and a domain body field_body. The length of field_head is generally 4 bytes, including the domain type field_id and the domain length field_size. The field_id distinguishes different price domains, and the field_size represents the length of the entire price domain; the length of field_body is field_size-4 bytes, which stores the key information contained in this type of domain.

[0122] This embodiment places two groups of pointers when processing the nested content here, and each group of pointers sets a head pointer and a tail pointer. The first group of pointers points to field_head, the head pointer head_hp points to the start byte of field_head, and the tail pointer head_tp points to field_head. The second group of pointers points to field_body, and the head and tail pointers body_hp and body_tp point to the start and end bytes of field_body respectively; the linked list head and tail pointer groups and pointer jumping technology are used to extract the effective price domain in real time, saving a lot of single-byte offset time, thereby ensuring a lower penetration delay; and the time-sharing multiplexing method is used to cyclically parse the market snapshots and multicast increments, and the jumps of a large number of pointers are processed in a pipeline storage manner, thereby saving a large amount of LUT resources in the FPGA.

[0123] like Figure 5 As shown, the pointer jumping technology used in this embodiment is demonstrated. Each pointer group is calculated and stored once, and is retained until the next valid price field is updated and stored. When the pointer group points to the first valid price field of the first contract of a group of messages, since the initial position is fixed, the head pointer position of the field header can be determined to be head_hp, and the pointer group pointing position of the first valid price field field0 can be deduced in turn:

[0124] head_hp0=head_hp

[0125] head_tp0=head_hp+3

[0126] body_hp0=head_tp0+1

[0127] body_tp0=head_tp0+field_size0-4

[0128] The pointer group pointing to the position of the second valid price field field1 can jump according to the offset of field0. When the message is collected into the market_process module, the pointer group of field1 can jump to the determined position in real time:

[0129] head_hp1=body_tp0+1

[0130] head_tp1=body_tp0+4

[0131] body_hp1=head_tp1+1

[0132] body_tp1=head_tp1+field_size1-4

[0133] Any subsequent valid fieldM pointer group of this message can be expressed as:

[0134] head_hpM=body_tp(M-1)+1

[0135] head_tpM=body_tp(M-1)+4

[0136] body_hpM=head_tpM+1

[0137] body_tpM=head_tpM+field_sizeM-4

[0138] What needs special consideration is that since the processing of the axi stream in the FPGA squadron is performed in an 8-byte manner per cycle, the pointer group of the valid value field will jump across cycles. When processing the pointers head_hp, body_hp and body_tp, they need to be discussed separately in 8 different cases, each of which indicates the byte of the current pointer in the 8 bytes of this cycle.

[0139] Figure 6 A structural diagram of a market data processing system provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the system specifically includes:

[0140] The acquisition module 601 acquires the market information during the market transaction process;

[0141] The pre-processing module 602 analyzes the market information service message to obtain the initial full information of the market transaction and the incremental market information during the market transaction;

[0142] An updating module 603 updates the initial full information of the market by using the market information of the incremental market to obtain the target full information of the market;

[0143] The encapsulation module 604 encapsulates the target full amount information according to a preset format to obtain a market data package.

[0144] In a possible implementation, the updating module 603 includes:

[0145] The decoding unit 6031 is used to decode the market information of the incremental market to obtain the price change value and transaction quantity of the newly added market;

[0146] The calculation unit 6032 is used to calculate the transaction price corresponding to the price change value and the transaction quantity according to a preset market calculation rule;

[0147] A full quantity updating unit 6033 is used to update the initial full quantity information of the market according to the transaction price and the transaction quantity to obtain the target full quantity information of the market;

[0148] The market information storage unit 6034 is used to store preset market calculation rules and market price change units;

[0149] The full information storage unit 6035 is used to store the initial full information of the market.

[0150] like Figure 7 As shown in the figure, the process of the system when processing market data is as follows:

[0151] The acquisition module 601 is responsible for the diversion of market snapshot messages, multicast incremental messages and other heartbeat messages. At the same time, it extracts the five-tuple information (source IP, destination IP, source port, destination port, protocol number) according to the Ethernet message protocol for filtering, and cuts off the L2, L3 and L4 layers of the message. The remaining application layer data is recombined with the splicing layer single-cycle 8Byes 10 Gigabit Ethernet axi stream pure business data flow to reduce the processing pressure of the subsequent modules.

[0152] The preprocessing module 602 is the core processing module for decoding futures market information. This module is responsible for parsing market snapshot messages and multicast incremental messages and extracting the valid price domain and key information therein and transmitting them to the subsequent update module 603. The preprocessing module 602 includes a first preprocessing submodule 6021 and a second preprocessing submodule 6022.

[0153] The calculation unit 6032 is responsible for calculating and outputting the full information of the contract. When receiving the basic market information transmitted by the first preprocessing submodule 6021, the update module 603 extracts the contract number index, calculation basis and minimum price change from the basic market information (each contract corresponds to a unique contract number index), and writes the calculation basis and minimum price change into the market information storage unit 6034 with the contract number index as the write address. When receiving the incremental market information transmitted by the second preprocessing unit 6022, the contract number index in the incremental market information is used as the read address to enter the market information storage unit 6034 to read the corresponding calculation basis and minimum price change. At the same time, the decoding unit 6031 is called to perform Varint decoding and ZigZag decoding on the compressed data in sequence (the compressed data is mainly compressed using vint encoding (Varint encoding + ZigZag encoding), ZigZag encoding maps signed integers to unsigned integers, and Varint encoding encodes 64-bit binary-encoded unsigned integers with bytes of different lengths according to their size), and parses out the price change value and transaction quantity.

[0154] To calculate the final price, the calculation unit 6032 needs to be called to perform multiplication and addition operations according to the formula "price = calculation basis + minimum price change * price change value". After calculation, the prices and transaction quantities of all valid price ranges are written into the full information storage unit 6033 with the contract number index as the write address. When other incremental market conditions arrive later, the final result will still be written into the full information storage unit 6033 according to this process. The same contract will be continuously updated at different trading times, and the price and transaction quantity updated each time will be written to the same position in the full information storage unit 6033. In this way, the full information storage unit 6033 can always store the latest full information of the contract, so as to realize the function of the full information storage unit 6033 to maintain the full information of all contracts.

[0155] When the market snapshot message arrives (the market snapshot message is not compressed and does not need to be decoded and calculated), the first pre-processing submodule 6021 will pass the parsed contract snapshot information to the full update unit 6033. The full update unit 6033 extracts the contract number index and other full contract information from the contract snapshot information, and uses the contract number index as the read and write address of the full information storage unit 6033 to replace the full contract information in the full information storage unit 6033, and the valid price domain missing in the incremental market (the valid price domain that has not changed will not be pushed in the incremental market, so the contract snapshot information is needed to complete it). At the same time, the valid price domain that already exists in the incremental market is checked. If it is different from the relevant valid price domain in the snapshot, it means that the incremental market calculation is wrong, then the incremental market is replaced with the valid price domain in the snapshot and written into the full information storage unit 6033.

[0156] The encapsulation module 604 is responsible for managing the contract queue output to the PCIe DMA, processing the full contract packet into the 64-byte data format required by the DMA, and processing the back pressure from the DMA.

[0157] In a possible implementation, the system further includes a cutting module 605 for extracting protocol information of the market service message using an Ethernet message protocol; and cutting the market service message based on the protocol information so that the cut market service message is an application layer message.

[0158] In a possible implementation, the preprocessing module 602 is further configured to analyze the market snapshot message to obtain snapshot information of the market transaction as initial full information of the market.

[0159] In a possible implementation, the preprocessing module 602 is further configured to analyze the market price increment message to obtain a valid price range of the market price; and obtain the incremental market price in the market price transaction process according to the valid price range of the market price.

[0160] In one possible implementation, the update module 603 is also used to decode the market information of the incremental market to obtain the price change value and transaction quantity of the newly added market; calculate the transaction price corresponding to the price change value and transaction quantity according to the preset market calculation rules; update the initial full information of the market according to the transaction price and transaction quantity to obtain the target full information of the market.

[0161] In one possible implementation, the preprocessing module 602 is also used to divide the market incremental message into an incremental message header and an incremental message body, wherein the incremental message body includes valid information of multiple newly added transactions; point the head pointer and the tail pointer to the incremental message header and the incremental message body respectively; use pointer jumping technology to analyze the newly added market message to obtain the valid price range of the market.

[0162] The market data processing system provided in this embodiment can be as follows Figure 6 The market data processing system shown in can be executed as follows Figure 1-5 All steps of market data processing in Figure 1-5 For details, please refer to Figure 1-5 For the sake of brevity, the relevant description is not repeated here.

[0163] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 8As shown, an embodiment of the present application provides an electronic device, including a processor 801, a communication interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804; the memory 803 is used to store computer programs; the processor 801 is used to execute the program stored in the memory 803, and implement the steps of the market data processing method provided by any of the above method embodiments:

[0164] Acquire market information service messages during market trading; analyze the market information service messages to obtain initial full information of market trading and incremental market information during market trading; use the market information of the incremental market information to update the initial full information of the market to obtain target full information of the market; encapsulate the target full information according to a preset format to obtain a market data packet.

[0165] In a possible implementation, the Ethernet message protocol is used to extract the protocol information of the market service message; based on the protocol information, the market service message is segmented so that the segmented market service message is an application layer message.

[0166] In a possible implementation, analyzing the market service message to obtain the initial full information of the market includes: analyzing the market snapshot message to obtain snapshot information of the market transaction as the initial full information of the market.

[0167] In a possible implementation, analyzing the market service message to obtain incremental market information during a market transaction includes: analyzing the incremental market information message to obtain a valid price range for the market information; and obtaining incremental market information during a market transaction based on the valid price range for the market information.

[0168] In one possible implementation, the market information of the incremental market is decoded to obtain the price change value and transaction quantity of the newly added market; the transaction price corresponding to the price change value and transaction quantity is calculated according to a preset market calculation rule; the initial full information of the market is updated according to the transaction price and transaction quantity to obtain the target full information of the market.

[0169] In one possible implementation, the market price incremental message is divided into an incremental message header and an incremental message body, wherein the incremental message body includes valid information of multiple newly added transactions; the head pointer and the tail pointer point to the incremental message header and the incremental message body respectively; and the pointer jumping technology is used to analyze the newly added market price message to obtain the valid price range of the market price.

[0170] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0171] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0172] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0173] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A market data processing method, characterized in that: include: Obtain market information during market trading; Analyze the market information service message to obtain the initial full information of the market transaction and the incremental market information during the market transaction; Using the market information of the incremental market information to update the initial full information of the market, to obtain the target full information of the market; The target full amount information is encapsulated according to a preset format to obtain a market data package.

2. The method according to claim 1, characterized in that Before analyzing the market service message, the method further includes: Extracting protocol information of the market service message using the Ethernet message protocol; Based on the protocol information, the market service message is cut so that the cut market service message is an application layer message.

3. The method according to claim 2, characterized in that The market service message includes a market snapshot message; Analyze the market information to obtain the initial full information of the market, including: The market snapshot message is analyzed to obtain snapshot information of the market transaction as the initial full information of the market.

4. The method according to claim 3, characterized in that The market service message includes a market increment message; Analyze the market information service message to obtain the incremental market information during the market transaction process, including: Analyze the market price increment message to obtain a valid price range of the market price; According to the effective price range of the market information, the incremental market information in the market information trading process is obtained.

5. The method according to claim 1, characterized in that The method of updating the initial full information of the market by using the market information of the incremental market to obtain the target full information of the market includes: Decode the market information of the incremental market to obtain the price change value and transaction quantity of the newly added market; Calculate the transaction price corresponding to the price change value and transaction quantity according to the preset market calculation rules; The initial full amount information of the market is updated according to the transaction price and the transaction quantity to obtain the target full amount information of the market.

6. The method according to claim 4, characterized in that The step of analyzing the market price increment message to obtain a valid price range of the market price includes: The market price incremental message is divided into an incremental message header and an incremental message body, wherein the incremental message body includes valid information of multiple newly added transactions; The head pointer and the tail pointer point to the incremental message header and the incremental message body respectively; Use pointer jumping technology to analyze new market information and obtain the effective price range of the market.

7. A market data processing system, characterized in that: include: Acquisition module, to obtain market information during market trading; A preprocessing module analyzes the market information to obtain the initial full information of the market transaction and the incremental market information during the market transaction; An updating module, using the market information of the incremental market to update the initial full information of the market, to obtain the target full information of the market; The encapsulation module encapsulates the target full amount information according to a preset format to obtain a market data package.

8. The system according to claim 7, characterized in that The update module includes: A decoding unit, used to decode the market information of the incremental market to obtain the price change value and transaction quantity of the newly added market; A calculation unit, used to calculate the transaction price corresponding to the price change value and the transaction quantity according to a preset market calculation rule; A full quantity updating unit, used for updating the initial full quantity information of the market according to the transaction price and the transaction quantity, and obtaining the target full quantity information of the market; A market information storage unit, used to store preset market calculation rules and market price change units; The full information storage unit is used to store the initial full information of the market.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the market data processing method according to any one of claims 1 to 6 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the market data processing method according to any one of claims 1 to 6 are implemented.

Images