Transaction link time delay analysis monitoring method and system

Abstract

The invention provides a transaction link time delay analysis monitoring method and system. The method comprises the following steps: converging transaction link data into the same network port through a switch mirror image; extracting a keyword from the transaction link data as a key value; the network access timestamps of the transaction counter, the market counter, the transaction gateway and the market gateway are used as values; the key value and the value value are stored in a memory in the form of std:: unordedmap and the form of std::: map respectively, and the key value and the value value are stored in the memory in the form of std:: unordedmap; and directly calculating the data with the same key value in the memory to obtain the transaction link delay. According to the transaction link time delay analysis monitoring method and system provided by the invention, the key value and the value value are stored in the memory in the forms of the std:: unordedmap and the std:: map, so that efficient operation in the memory is realized to obtain the transaction link time delay.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and particularly to a method and system for analyzing and monitoring the latency of a trading link. Background Art

[0002] As the proportion of institutional investors and quantitative trading in market transactions gradually increases, the market has put forward higher requirements for the trading latency and stability of securities firms. Existing trading systems usually only monitor the operating status of system components and the effectiveness of network links during intraday trading hours. When users raise trading problems, it is necessary to check each trading link one by one to locate the problem source, resulting in low efficiency. In addition, for occasional problems in the trading link, such as network jitter and abnormal responses, there are no effective monitoring and alarm means, and it is often necessary to wait for customer feedback before conducting investigations, making it difficult to quickly locate problems from a large amount of trading data.

[0003] Therefore, there is a need to provide a method and system for analyzing and monitoring the latency of a trading link that can solve the above problems. Summary of the Invention

[0004] In view of the problems and deficiencies existing in the prior art, the present invention provides a method and system for analyzing and monitoring the latency of a trading link.

[0005] The present invention solves the above technical problems through the following technical solutions:

[0006] A method for analyzing and monitoring the latency of a trading link, the method comprising:

[0007] Aggregating trading link data into the same network port through switch mirroring;

[0008] Extracting keywords from the trading link data as key values;

[0009] Using the network in and out timestamps of the trading counter, market data counter, trading gateway, and market data gateway as value values;

[0010] Storing the key values and the value values in memory in the form of std::unordered_map and std::map respectively;

[0011] Directly performing operations on the data with the same key value in the memory to obtain the latency of the trading link.

[0012] Preferably, the aggregating trading link data into the same network port through switch mirroring includes using the span, rspan, and erspan functions of the switch to copy and send the network data traffic of the monitored port to the specified monitoring port.

[0013] Preferably, with the CAS operation as the core, the operations of reading, comparing, and updating the position of the memory are completed within one instruction;

[0014] In the enqueue operation of the lock-free queue, obtain the tail pointer of the current queue, correctly link the new element to the tail of the current queue through the CAS operation, determine whether the queue status meets the operation conditions, and if so, update the queue; otherwise, re-obtain the tail pointer of the current queue and try again;

[0015] In the dequeue operation of the lock-free queue, obtain the head node of the current queue and the next node of the head node, update the head pointer to the next node through the CAS operation, determine whether the queue status meets the operation conditions, and if so, update the queue; otherwise, re-obtain the head node of the current queue and try again.

[0016] Preferably, the lock-free queue exists in the form of nodes, each node contains data and a pointer to the next node, a new node is created and linked to the tail of the queue during enqueue, and the node is removed from the head of the queue during dequeue.

[0017] Preferably, when it is determined that the memory cannot be immediately released after the node in the lock-free queue is deleted, the memory recycling mechanism tracks the reference situation of the node, and the memory recycling mechanism includes any one of hazard pointers and reference counting.

[0018] Preferably, by loading the cached information of the landing transaction link delay of the corresponding transaction node and gateway, obtain the cached number of items of the previous data statistics, asynchronously traverse the gateway to query the information of the subsequent newly added transaction link delay, and count the average value, maximum value, minimum value, and median of the cached information of the transaction link delay and the information of the newly added transaction link delay.

[0019] Preferably, the transaction node and the gateway are queried through a database connection pool, and the transaction node and the gateway are associated through the foreign key constraint oms_node_id.

[0020] Preferably, count the gateway with the minimum transaction link delay and record the transaction link delay in the cached information.

[0021] Preferably, the gateway information table includes ID number, creation time, modification time, project number, host connection number, IP for interaction between the gateway and the trading system, port number for interaction between the gateway and the trading system, IP for interaction between the gateway and the exchange, exchange IP, exchange port number, market information, platform type, gateway number, and gateway name.

[0022] The present invention also provides a transaction link delay analysis and monitoring system, and the system includes:

[0023] A trading link data aggregation module, which is used to aggregate trading link data into the same network port through switch mirroring;

[0024] A key value acquisition module, which is used to extract keywords from the trading link data as key values;

[0025] A value value acquisition module, which is used to use the network in-and-out timestamps of trading counters, market counters, trading gateways, and market gateways as value values;

[0026] A data storage module, which is used to store the key values and the value values in memory in the forms of std::unordered_map and std::map respectively;

[0027] A trading link latency calculation module, which is used to directly perform operations on the data with the same key value in the memory to obtain the trading link latency.

[0028] Compared with the prior art, the technical solution of the embodiment of the present invention has the following beneficial effects:

[0029] The trading link latency analysis and monitoring method and system provided by the embodiment of the present invention, the method includes: aggregating trading link data into the same network port through switch mirroring; extracting keywords from the trading link data as key values; using the network in-and-out timestamps of trading counters, market counters, trading gateways, and market gateways as value values; storing the key values and the value values in memory in the forms of std::unordered_map and std::map respectively; directly performing operations on the data with the same key value in the memory to obtain the trading link latency, by storing the key value and the value value in the forms of std::unordered_map and std::map in memory, realizing efficient operation in memory to obtain the trading link latency;

[0030] Furthermore, with the CAS operation as the core, the operations of reading, comparing, and updating the position of the memory are completed within one instruction; in the enqueue operation of the lock-free queue, obtain the tail pointer of the current queue, and correctly link the new element to the tail of the current queue through the CAS operation, determine whether the queue state meets the operation conditions, if it meets, update the queue, otherwise re-obtain the tail pointer of the current queue and try again; in the dequeue operation of the lock-free queue, obtain the head node of the current queue and the next node of the head node, update the head pointer to the next node through the CAS operation, determine whether the queue state meets the operation conditions, if it meets, update the queue, otherwise re-obtain the head node of the current queue and try again, by adopting a lock-free queue and a full-memory storage mechanism, ensuring efficient data storage and calculation. Brief Description of the Drawings

[0031] Figure 1 A schematic flowchart of a transaction link delay analysis and monitoring method provided for an embodiment of the present invention;

[0032] Figure 2 A schematic structural diagram of a transaction link delay analysis and monitoring system provided for an embodiment of the present invention. Detailed Embodiments

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0034] The technical solutions of the present invention will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0035] Based on the problems existing in the prior art, as Figure 1 shown, the present invention provides a transaction link delay analysis and monitoring method, and the method includes:

[0036] Step S101: Aggregate transaction link data into the same network port through switch mirroring;

[0037] Step S102: Extract keywords from the transaction link data as key values;

[0038] Step S103: Use the network in and out timestamps of the trading counter, market data counter, trading gateway, and market data gateway as value values;

[0039] Step S104: Store the key values and the value values in memory in the forms of std::unordered_map and std::map respectively;

[0040] Step S015: Directly perform operations on the data with the same key value in the memory to obtain the transaction link delay.

[0041] Specifically, the keywords include, but are not limited to, feature information such as securities accounts, transaction serial numbers, timestamps, and transaction node identifiers.

[0042] In step S105, directly performing operations on data with the same key value in the memory includes calculating the time difference between requests and responses and the processing delays of each node.

[0043] In a specific implementation, aggregating transaction link data into the same network port through switch mirroring includes using the span, rspan, and erspan functions of the switch to copy and send the network data traffic of the monitored port to the specified monitoring port.

[0044] In a specific implementation, with the Compare and Swap (CAS) operation as the core, reading, comparing, and updating the position in the memory are completed within one instruction.

[0045] In the enqueue operation of the lock-free queue, obtain the tail pointer of the current queue, correctly link the new element to the tail of the current queue through the CAS operation, determine whether the queue status meets the operation conditions, and if so, update the queue; otherwise, re-obtain the tail pointer of the current queue and try again.

[0046] In the dequeue operation of the lock-free queue, obtain the head node of the current queue and the next node of the head node, update the head pointer to the next node through the CAS operation, determine whether the queue status meets the operation conditions, and if so, update the queue; otherwise, re-obtain the head node of the current queue and try again.

[0047] In a specific implementation, the lock-free queue exists in the form of nodes. Each node contains data and a pointer to the next node. When enqueuing, a new node is created and linked to the end of the queue. When dequeuing, the node is removed from the head of the queue.

[0048] In a specific implementation, when it is determined that the memory cannot be immediately released after a node in the lock-free queue is deleted, the memory recycling mechanism tracks the reference situation of the node. The memory recycling mechanism includes any one of hazard pointers and reference counting.

[0049] In a specific implementation, by loading the cached information of the transaction link delays of the corresponding transaction nodes and gateways, the number of cached items of the previous data statistics is obtained, and the gateway is asynchronously traversed to query the information of the subsequent newly added transaction link delays, and the average value, maximum value, minimum value, and median of the cached information of the transaction link delays and the information of the newly added transaction link delays are statistically calculated.

[0050] The specific calculation formulas are as follows:

[0051] Average value: Mean = (X1 + X2 + X3 +... + Xn) / n

[0052] Maximum value: Max = max{X1, X2,..., Xn}

[0053] Minimum value: Min = min{X1, X2, …, Xn}

[0054] Median: When n is odd, Median = x(n + 1) / 2; when n is even, Median = (X(n / 2) + X(n / 2 + 1)) / 2

[0055] In a specific implementation, the trading node and the gateway perform a query through a database connection pool, and the trading node and the gateway are associated through a foreign key constraint oms_node_id.

[0056] In a specific implementation, the gateway with the minimum trading link delay is statistically determined and the trading link delay is recorded in the cache information.

[0057] In a specific implementation, the gateway information table includes an ID (Identity Document) number, creation time, modification time, project number, host connection number, IP (Internet Protocol) for the interaction between the gateway and the trading system, port number for the interaction between the gateway and the trading system, IP for the interaction between the gateway and the exchange, exchange IP, exchange port number, market information, platform type, gateway number, and gateway name.

[0058] Specifically, in the case of multiple nodes, multiple counters, and multiple gateways, by matching relevant IPs and ports, the corresponding custom node numbers, gateway names, and gateway numbers are bound to label each link data.

[0059] Based on the problems existing in the prior art, such as Figure 2 As shown, the present invention further provides a trading link delay analysis and monitoring system, and the system includes:

[0060] A trading link data aggregation module 21, which is used to aggregate trading link data into the same network port through a switch mirroring;

[0061] A key value acquisition module 22, which is used to extract keywords from the trading link data as key values;

[0062] A value value acquisition module 23, which is used to use the network in and out timestamps of the trading counter, market counter, trading gateway, and market gateway as value values;

[0063] A data storage module 24, which is used to store the key value and the value value in memory in the forms of std::unordered_map and std::map respectively;

[0064] The transaction link latency calculation module 25 is used to directly perform operations on data with the same key value in the memory to obtain the transaction link latency.

[0065] In summary, for the transaction link latency analysis and monitoring method and system provided by the embodiments of the present invention, the method includes: aggregating transaction link data into the same network port through switch mirroring; extracting keywords from the transaction link data as key values; using the network in and out timestamps of the trading counter, market data counter, trading gateway, and market data gateway as value values; storing the key values and the value values in the memory in the forms of std::unordered_map and std::map respectively; directly performing operations on data with the same key value in the memory to obtain the transaction link latency. By storing the key values and the value values in the memory in the forms of std::unordered_map and std::map, efficient operations in the memory are realized to obtain the transaction link latency.

[0066] Further, with the CAS operation as the core, reading, comparing, and updating operations on the position of the memory are completed within one instruction; in the enqueue operation of the lock-free queue, the tail pointer of the current queue is obtained, and the new element is correctly linked to the tail of the current queue through the CAS operation, and it is judged whether the queue state meets the operation conditions. If it meets, the queue is updated; otherwise, the tail pointer of the current queue is obtained again and the attempt is made again; in the dequeue operation of the lock-free queue, the head node of the current queue and the next node of the head node are obtained, and the head pointer is updated to the next node through the CAS operation, and it is judged whether the queue state meets the operation conditions. If it meets, the queue is updated; otherwise, the head node of the current queue is obtained again and the attempt is made again. By adopting the lock-free queue and the full memory storage mechanism, efficient data storage and calculation are ensured.

[0067] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for analyzing and monitoring the latency of a trading link, characterized in that, The method includes: Aggregating the trading link data into the same network port through switch mirroring; Extracting keywords from the trading link data as key values; Using the network in and out timestamps of the trading counter, market counter, trading gateway, and market gateway as value values; Storing the key values and the value values in memory in the forms of std::unordered_map and std::map respectively; Directly performing operations on the data with the same key value in the memory to obtain the trading link latency.

2. The transaction link delay analysis and monitoring method according to claim 1, wherein The aggregating the trading link data into the same network port through switch mirroring includes using the span, rspan, and erspan functions of the switch to copy and send the network data traffic of the monitored port to the specified monitoring port.

3. The trading link latency analysis and monitoring method according to claim 1, wherein: With the CAS operation as the core, completing the operations of reading, comparing, and updating the position in the memory within one instruction; In the enqueue operation of the lock-free queue, obtaining the tail pointer of the current queue, correctly linking the new element to the tail of the current queue through the CAS operation, determining whether the queue status meets the operation conditions, and if so, updating the queue, otherwise re-obtaining the tail pointer of the current queue and trying again; In the dequeue operation of the lock-free queue, obtaining the head node of the current queue and the next node of the head node, updating the head pointer to the next node through the CAS operation, determining whether the queue status meets the operation conditions, and if so, updating the queue, otherwise re-obtaining the head node of the current queue and trying again.

4. The transaction link delay analysis and monitoring method according to claim 3, characterized in that The lock-free queue exists in the form of nodes, each node contains data and a pointer to the next node, a new node is created and linked to the end of the queue during enqueue, and a node is removed from the head of the queue during dequeue.

5. The transaction link delay analysis and monitoring method according to claim 1, characterized in that When it is determined that the memory cannot be immediately released after the node in the lock-free queue is deleted, tracking the reference situation of the node through a memory recycling mechanism, and the memory recycling mechanism includes any one of a hazard pointer and a reference count.

6. The transaction link delay analysis and monitoring method according to claim 1, characterized in that By loading the cached information of the landed trading link latency of the corresponding trading nodes and gateways, obtaining the number of cached items in the previous data statistics, asynchronously traversing the gateways to query the information of the subsequently added trading link latency, and statistically calculating the average value, maximum value, minimum value, and median of the cached information of the trading link latency and the information of the added trading link latency.

7. The transaction link delay analysis and monitoring method according to claim 6, wherein The trading nodes and gateways are queried through a database connection pool, and the trading nodes and gateways are associated through the foreign key constraint oms_node_id.

8. The transaction link delay analysis and monitoring method according to claim 6, wherein Statistically calculating the gateway with the minimum trading link latency and recording the trading link latency in the cached information.

9. The transaction link delay analysis and monitoring method according to claim 6, characterized in that The gateway information table includes an ID number, creation time, modification time, project number, host connection number, IP for the gateway to interact with the trading system, port number for the gateway to interact with the trading system, IP for the gateway to interact with the exchange, exchange IP, exchange port number, market information, platform type, gateway number, and gateway name.

10. A trading link delay analysis and monitoring system, characterized in that The system includes: Transaction link data aggregation module, which is used to aggregate transaction link data into the same network port through switch mirroring; Key value acquisition module, which is used to extract keywords from the transaction link data as key values; Value value acquisition module, which is used to use the network in and out timestamps of the trading counter, market counter, trading gateway, and market gateway as value values; Data storage module, which is used to store the key value and the value value in memory in the forms of std::unordered_map and std::map respectively; Transaction link latency calculation module, which is used to directly perform operations on the data with the same key value in the memory to obtain the transaction link latency.

Images