Data recovery method and system

Abstract

The embodiment of the present specification provides a data recovery method and system, wherein the data recovery method is applied to a programmable logic component, wherein the programmable logic component comprises a target cache and a target memory, and the method comprises: reading a target service data packet associated with transaction market data in the target cache; in the case that the transaction market data is determined to be abnormal according to the read target service data packet, stopping the data reading operation and executing a data packet recovery task; in the case that the data packet recovery task is in an execution state, writing a plurality of service data packets associated with the transaction market data to the target cache and the target memory in the order of packet number, wherein the writing operation priority of the target cache is higher than that of the target memory; and in the case that the data packet recovery task is in an execution completion state, resuming the data reading operation.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of data processing technology, and in particular to data recovery methods and systems. Background Technology

[0002] In the financial sector, the rapid analysis of trading data and the reliable and fast real-time feedback of accurate trading data to users can help them seize trading opportunities and provide a better trading environment.

[0003] Currently, to expedite the transmission of market data, exchanges use UDP packets, which are susceptible to packet loss due to complex hardware and network environments. Existing technologies mostly address packet loss through purely software-based remediation. This process typically requires business interruption and incurs recovery time. While this time difference may not be readily apparent to users, even a very short delay can impact trading demands in financial trading scenarios. Therefore, an effective solution is urgently needed to address these issues. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a data recovery method. One or more embodiments of this specification also relate to a data recovery apparatus, a data recovery system, and an electronic device, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a data recovery method is provided, applied to a programmable logic component, wherein the programmable logic component includes a target cache and a target memory, comprising:

[0006] Read the target business data packet containing associated transaction market data from the target cache;

[0007] If it is determined that the transaction market data is abnormal based on the target business data packet read, the data reading operation is stopped and a data packet recovery task is executed;

[0008] When the data packet recovery task is in execution state, multiple business data packets associated with the transaction market data are written to the target cache and the target memory in order of packet number, wherein the write operation priority of the target cache is higher than the write operation priority of the target memory;

[0009] If the data packet recovery task is in the completed state, resume the data reading operation.

[0010] According to a second aspect of the embodiments of this specification, a data recovery apparatus is provided, applied to a programmable logic component, wherein the programmable logic component includes a target cache and a target memory, comprising:

[0011] The reading module is configured to read target business data packets containing associated transaction market data from the target cache;

[0012] The stop module is configured to stop the data reading operation and execute a data packet recovery task if it is determined that there is an anomaly in the transaction market data based on the target business data packet read.

[0013] The write module is configured to write multiple business data packets associated with the transaction market data to the target cache and the target memory in order of packet number when the data packet recovery task is in the execution state, wherein the write operation priority of the target cache is higher than the write operation priority of the target memory.

[0014] The recovery module is configured to resume data reading operations when the data packet recovery task is in the completed state.

[0015] According to a third aspect of the embodiments of this specification, a data recovery system is provided, including a server and a terminal, wherein the server is configured with a programmable logic component, the programmable logic component including a target cache and a target memory, comprising:

[0016] The server is configured to read target business data packets associated with market data from the target cache using a programmable logic component; if an anomaly is determined in the market data based on the read target business data packets, the server stops the data reading operation and executes a data packet recovery task; while the data packet recovery task is in the execution state, the server writes multiple business data packets associated with the market data to the target cache and the target memory in order of packet number, wherein the write operation priority of the target cache is higher than the write operation priority of the target memory; when the data packet recovery task is in the execution completion state, the server resumes the data reading operation and sends the read target business data packets to the terminal.

[0017] The terminal is used to perform market data processing tasks based on the target business data packets sent by the server.

[0018] According to a fourth aspect of the embodiments of this specification, an electronic device is provided, the electronic device being a programmable logic component, the programmable logic component being used to perform the steps of the above-described data recovery method.

[0019] The data recovery method provided in this embodiment is applied to a programmable logic component (PLC), which includes a target cache and a target memory. To maximize the performance advantages of the PLC and enable rapid data recovery in the event of data anomalies without interrupting upstream services due to packet loss recovery, specifically, when processing market data using the PLC, after writing the real-time market data to the target cache, the associated target business data packets can be sequentially read from the target cache for subsequent market data distribution. During this process, if the read target business data packets indicate an anomaly in the market data, it suggests potential packet loss. To ensure business continuity and prevent upstream service interruptions, only the data reading operation from the target cache can be stopped, while the market data decoding and data packet recovery tasks will continue simultaneously. Furthermore, while the data packet recovery task is in progress, upstream data writing to the target cache is not restricted. This means that upstream data writing to the programmable logic component will continue, allowing data packets to be written to the cache for waiting. However, the cache's storage space is limited, and the packet loss recovery time is not fixed. Before packet loss recovery, the cache may become unable to continue writing data. Therefore, to ensure uninterrupted upstream data processing, multiple business data packets associated with market data can be written to the target cache and target memory in packet number order, with the write priority to the target cache being higher than that to the target memory. During the packet loss recovery phase, business data packets are prioritized for writing to the target cache, and only when the target cache reaches its storage limit are they written to the target memory. This ensures that upstream data processing and the data writing process remain continuous. Lost data is replenished by executing a data packet recovery task. When the data packet recovery task is complete, it indicates that packet replenishment is finished, allowing data reading from the target cache or target memory to resume. This enables the subsequent distribution of market data packets based on the retrieved target business data packets. In financial trading scenarios, even in the event of packet loss, data integrity and market accuracy are maintained, and this integrity and accuracy are maintained in real-time, achieving highly reliable and efficient data processing to provide users with stable and fast-responding trading services. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a data recovery method provided in one embodiment of this specification;

[0021] Figure 2a This is a schematic diagram of data packet transmission and reception in a data recovery method provided in one embodiment of this specification;

[0022] Figure 2b This is a schematic diagram of a data recovery architecture in a data recovery method provided in one embodiment of this specification;

[0023] Figure 2c This is a schematic diagram illustrating data inflation in a data recovery method provided in one embodiment of this specification;

[0024] Figure 3 This is a flowchart illustrating the processing steps of a data recovery method provided in one embodiment of this specification.

[0025] Figure 4 This is a schematic diagram of the structure of a data recovery system provided in one embodiment of this specification;

[0026] Figure 5 This is a schematic diagram of a data recovery device provided in one embodiment of this specification. Detailed Implementation

[0027] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0028] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0029] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0030] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0031] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0032] FPGA (Field-Programmable Gate Array) is an integrated circuit that can be configured using hardware description languages ​​such as VHDL or Verilog to perform specific functions.

[0033] DDR (Double Data Rate) is a type of DRAM technology that can transmit data once on the rising edge and once on the falling edge of each clock cycle, thus doubling the data transfer rate.

[0034] Cache is a small but very fast memory located between the CPU and main memory. In this embodiment, the cache mentioned is the internal cache of the FPGA.

[0035] HBM (High Bandwidth Memory) is a high-bandwidth memory technology specifically designed for high-performance graphics accelerators, network devices, high-performance computing (HPC), and other fields. Unlike traditional DDR SDRAM, HBM uses 3D stacking technology to vertically stack multiple DRAM chips together and uses a wider data bus to achieve higher data transfer rates and lower power consumption.

[0036] In practical applications, the network environment from the exchange to the customer involves multiple layers of hardware devices such as switches and servers. During multicast market data distribution, packet loss may occur due to the complex hardware environment. After packet loss, market data cannot be recovered quickly, which greatly affects the user's service experience. Therefore, an effective solution is urgently needed to solve the above problems.

[0037] In view of this, this specification provides a data recovery method, and also relates to a data recovery device, a data recovery system, and an electronic device, which will be described in detail in the following embodiments.

[0038] The data recovery method provided in this embodiment is applied to a programmable logic component (PLC), which includes a target cache and a target memory. To maximize the performance advantages of the PLC and enable rapid data recovery in the event of data anomalies without interrupting upstream services due to packet loss, the method is designed to address this issue. Specifically, when processing market data using the PLC, after writing the real-time market data to the target cache, target business data packets associated with the market data can be sequentially read from the target cache for subsequent market data distribution. If, during this process, the read target business data packets indicate an anomaly in the market data, suggesting potential packet loss, to ensure business continuity and avoid upstream service interruption, only the data reading operation from the target cache can be stopped. However, the market data decoding and data packet recovery tasks will still be executed simultaneously. Furthermore, while the data packet recovery task is in progress, the upstream data writing operation to the target cache is not restricted.

[0039] This can be understood as follows: the upstream operation of writing data to the programmable logic component will continue to execute, allowing data packets to be written to the cache for waiting. However, the cache storage space is limited, and the packet loss recovery time is not fixed. Before packet loss recovery, the cache may be unable to continue writing data. Therefore, to ensure that upstream data processing is not interrupted, multiple business data packets associated with the trading market data can be written to the target cache and target memory in order of packet number, and the write operation priority of the target cache is higher than the write operation priority of the target memory. Implementing the packet loss recovery stage allows business data packets to be written to the target cache first, and then written to the target memory when the target cache reaches its storage limit, ensuring that upstream data processing and data writing processes continue to run and are continuous. By executing the data packet recovery task, the lost packet data is replenished. When the data packet recovery task is in the execution completed state, it means that the packet loss replenishment is complete. Therefore, the operation of reading data from the target cache or target memory can be resumed, thereby enabling the subsequent market data distribution processing based on the read target business data packets. In financial trading scenarios, even in the event of packet loss, data integrity and market accuracy can still be maintained. This integrity and accuracy are maintained in real time, achieving highly reliable and efficient data processing to provide users with stable and fast-responding trading services.

[0040] See Figure 1 , Figure 1 A flowchart of a data recovery method according to an embodiment of this specification is shown, applied to a programmable logic component, wherein the programmable logic component includes a target cache and a target memory, and specifically includes the following steps.

[0041] Step S102: Read the target business data packet containing the associated transaction market data from the target cache.

[0042] The data recovery method provided in this embodiment can be applied to scenarios such as order book construction, trading strategies, and trading counters in securities, futures and other trading businesses. It is used to complete data storage by utilizing the memory and cache in the programmable logic component, and at the same time, it is used to complete data recovery operations while ensuring the normal operation of the business in the event of data anomalies, thereby providing users with complete and fast trading services and meeting users' real-time trading needs.

[0043] Furthermore, the data recovery method provided in this embodiment can also be applied to scenarios of sudden surges in market data traffic. For example, when market data enters, the data is short, but after decoding, the fields become longer. At the same time, because the traffic increases, a UDP header needs to be added when sending packets. The data recovery method provided in this embodiment can be used to avoid business interruption caused by a sudden surge in market data traffic.

[0044] Specifically, programmable logic components refer to FPGAs; market data refers to market data in securities or futures scenarios, including snapshot data and tick-by-tick data such as opening price, closing price, historical price trends, turnover rate, lowest price, highest price, trading volume, turnover, and total market capitalization. Correspondingly, business data packets are data structures constructed by reorganizing market data, used for continuous writing to the target cache. This allows downstream stages to continuously consume business data packets from the target cache for market data distribution processing, such as providing users with real-time feedback on the real-time trading price and order count of a stock, or providing users with an order book containing thousands of market data points.

[0045] In practical implementation, because the target cache in the programmable logic component (PLC) has a faster processing efficiency, the market data entering the PLC, after being reorganized, is directly written to the target cache for queuing. Downstream components then read target business data packets from the target cache in real time for market data distribution and processing, thus meeting the high reliability and high efficiency requirements of financial trading scenarios. In other words, in a normal trading processing scenario, if no business problems occur, all market data will enter the target cache, which will process the data at full speed and hand it over to downstream business processes, thus forming a virtuous cycle of real-time market data processing. Specifically, the target business data packet refers to the business data packet that currently reads related market data from the target cache and needs to be used for downstream market data processing. The target cache specifically refers to the cache in the PLC, and the target memory specifically refers to the DDR or HBM of the PLC.

[0046] Therefore, to maximize the performance advantages of programmable logic components (PLCs) and enable rapid data recovery in the event of data anomalies without interrupting upstream services due to packet loss recovery, the following measures are implemented: Specifically, when processing market data using PLCs, after writing real-time market data to the target cache, the associated target business data packets can be sequentially read from the target cache for subsequent market data distribution. During this process, if the read target business data packets indicate an anomaly in the market data, suggesting potential packet loss, to ensure business continuity and avoid upstream service interruption, only the data reading operation from the target cache can be stopped. However, the market data decoding and data packet recovery tasks will continue simultaneously. Furthermore, while the data packet recovery task is in progress, the upstream data writing operation to the target cache is not restricted.

[0047] This can be understood as follows: the upstream operation of writing data to the programmable logic component will continue to execute, allowing data packets to be written to the cache for waiting. However, the cache storage space is limited, and the packet loss recovery time is not fixed. Before packet loss recovery, the cache may be unable to continue writing data. Therefore, to ensure that upstream data processing is not interrupted, multiple business data packets associated with the trading market data can be written to the target cache and target memory in order of packet number, and the write operation priority of the target cache is higher than the write operation priority of the target memory. Implementing the packet loss recovery stage allows business data packets to be written to the target cache first, and then written to the target memory when the target cache reaches its storage limit, ensuring that upstream data processing and data writing processes continue to run and are continuous. By executing the data packet recovery task, the lost packet data is replenished. When the data packet recovery task is in the execution completed state, it means that the packet loss replenishment is complete. Therefore, the operation of reading data from the target cache or target memory can be resumed, thereby enabling the subsequent market data distribution processing based on the read target business data packets. In financial trading scenarios, even in the event of packet loss, data integrity and market accuracy can still be maintained. This integrity and accuracy are maintained in real time, achieving highly reliable and efficient data processing to provide users with stable and fast-responding trading services.

[0048] Step S104: If it is determined that the transaction market data is abnormal based on the target business data packet read, stop the data reading operation and execute the data packet recovery task.

[0049] Specifically, anomalies in market data refer to packet loss. Correspondingly, stopping data reading operations means halting the reading of target business data packets from the target cache to prevent disruptions to downstream market data processing due to data continuity issues. Conversely, the data packet recovery task refers to the task of recovering lost business data packets to ensure data integrity and support normal business operations.

[0050] Based on this, after the upstream writes the business data packets associated with the market data to the target cache, the downstream reads the target business data packets from the target cache for subsequent processing. When writing business data packets to the target cache, the data packets are stored sequentially; therefore, during reading, the target business data packets are read in order. In this process, considering that the programmable logic component may encounter anomalies in the market data due to network issues or complex hardware environments, and that the packet number (the basis for sequentially reading the target business data packets) is assigned by the programmable logic component to the business data packets and cannot detect data anomalies, the read target business data packets can be parsed, and anomaly detection of the market data can be completed based on the parsing results.

[0051] If an anomaly is detected, it means that the current market data cannot be used to complete subsequent business processing. Therefore, it is necessary to perform packet loss data recovery processing. In order to ensure data integrity and allow the business to continue to run stably, the data reading operation can be stopped, and the data packet recovery task can be executed after the data reading operation is stopped. This is to achieve rapid data recovery processing and avoid business interruption that could affect users' transactions.

[0052] Furthermore, when performing anomaly detection on the read target service data packets, this can be accomplished using the sequence identifier corresponding to the service data packets. In this embodiment, the specific implementation is as follows:

[0053] The process involves parsing the read target business data packet to obtain its sequence identifier; detecting any anomalies in the market data based on the sequence identifier; if no anomalies are found, executing a market data distribution task based on the read target business data packet; and if so, stopping the data reading operation and performing a data packet recovery task.

[0054] Specifically, the sequence identifier refers to the sequence number corresponding to each business data packet. This sequence identifier is assigned by the exchange when distributing market data. That is, each business data packet already possesses a corresponding sequence identifier before entering the programmable logic component (PLC). This identifier is used to detect whether the market data is abnormal during processing by the PLC. Correspondingly, the market data distribution task refers to the task of distributing market data to the terminal based on the read target business data packet. In practice, in addition to the market data distribution task, a data processing module can be set in the PLC to perform simple data processing and then distribute it to the terminal. Alternatively, the read target business data packet can be directly distributed to the terminal, where the market data processing task is executed. This embodiment does not impose any limitations on this.

[0055] Based on this, when performing anomaly detection on market data, the target business data packet can be parsed to obtain its sequence identifier. The sequence identifier can then be used to detect any anomalies in the market data. If no anomalies are found, it indicates no packet loss, and a market data distribution task can be performed based on the read target business data packet. If anomalies are found, it indicates packet loss in the target business data packet, requiring the exchange to reissue the lost data packet. In this case, the data reading operation can be stopped, and a data packet recovery task can be performed to ensure high reliability and efficiency in business processing.

[0056] For example, when sending real-time trading data for stock number 1 to users, the stock trading platform configures an FPGA component to improve the efficiency of trading data processing so that users can quickly view the real-time trading data for that stock. In this process, after the stock's trading data is submitted to the FPGA, the FPGA decodes and reassembles the data to construct a business data packet, such as... Figure 2a The diagram shown indicates that 20 service data packets will be obtained at this point. Further, each service data packet is assigned a number according to its packet order, resulting in packet numbers 1-20, where each packet number represents the sending and receiving order of the service data packet.

[0057] Building upon this, if the existing technology of using FPGA's DDR / HBM cache for data packet processing is adopted, the excessively long DDR / HBM cache read time (requiring hundreds of nanoseconds) would hinder the demand for fast transactions. Therefore, this embodiment considers that the FPGA's cache read speed only requires a few nanoseconds, which can support faster data packet processing operations and reduce latency. Thus, business data packets are written into the FPGA's cache in packet number order, enabling rapid processing of business data packets and their transmission downstream. Figure 2bAs shown, this is used to send real-time trading data for stock 1 to the user terminal. Furthermore, since the business data packets are written to the cache in packet number order, downstream sending nodes will read the target business data packets from the cache in packet number order. This can be understood as the sending node first reading the business data packet with packet number 1 from the cache for consumption, and then sending the business data packet with packet number 1 to the detection node. While the business data packets in the cache are being consumed, the upstream node will continuously write new business data packets into the cache, and so on, ensuring the smooth operation of the business.

[0058] Furthermore, the three business data packets corresponding to packet numbers 1, 2, and 3 are currently read from the cache and sent by the sending node to the detection node for packet loss detection. The detection node determines through parsing that the sequence number (seqnum) of the business data packet corresponding to packet number 1 is 8000, the seqnum of the business data packet corresponding to packet number 2 is 8001, and the seqnum of the business data packet corresponding to packet number 3 is 8004. Although the packet numbers are consecutive, the seqnum of the three business data packets does not increase consecutively, further indicating that there is packet loss in the market data before entering the FPGA. Therefore, it is necessary to re-packet the business data packets with seqnum of 8002 and 8003 to support subsequent processing operations. Thus, the detection results are sent to the downstream recovery node, which performs packet loss data recovery operations. At the same time, the detection node will put back pressure on the sending node, causing the sending node to stop reading the target business data packets from the cache.

[0059] If the parsing determines that the value of seqnum is continuous and increasing, it indicates that there is no packet loss. The subsequent packet reading and parsing process continues. That is, the detection node will continue to send the target business data packet to the recovery node. Since there is no packet loss problem at the current moment, the target business data packet will pass through the recovery node at full speed and enter the market data distribution node, which is used to distribute market data to the user terminal, so that the client can browse the latest trading price, number of transactions, and other information of the stock.

[0060] In summary, by detecting the sequence identifier of the target business data packet, abnormalities in the trading market data can be accurately identified. Based on the detection results, decisions can be made on whether to perform data recovery processing, thereby ensuring data integrity and business continuity.

[0061] Furthermore, if the market data is normal, it indicates that downstream devices can consume the target business data packets normally, allowing for subsequent processing. However, since data expansion can lead to the cascading transmission of backpressure signals, to prevent business interruptions caused by data expansion, data expansion detection can be performed before market data distribution. Backpressure processing can then be applied to any signals that fail the detection. In this embodiment, the specific implementation is as follows:

[0062] The process involves determining the packet volume information of the target service data packet and performing data expansion detection based on this information. If the detection passes, the step of executing a market data distribution task based on the read target service data packet is performed. If the detection fails, a backpressure signal is transmitted to the target cache. If the target cache reaches its storage limit threshold, the process switches to the target memory, using it as a secondary cache for the target cache. Specifically, the packet volume information refers to the space occupied by the target service data packet. Correspondingly, data expansion detection refers to the detection process that checks whether the volume of the reassembled target service data packet exceeds a set threshold. This allows for step-by-step backpressure on upstream nodes in the event of data expansion, enabling them to slow down packet output or wait for a specified time (t) before outputting packets. The backpressure signal is a signal transmitted step-by-step from the market data distribution node to its upstream nodes to gradually slow down packet output. The secondary cache refers to the space used to write service data packets instead of the target cache; it can choose target memory as a secondary cache to store more service data packets.

[0063] Therefore, in the absence of packet loss, the target business data packet will be distributed to the market data distribution node level by level to complete the final market data distribution processing. Considering that the expansion of the business data packet can also affect business operations, the packet size information of the target business data packet can be determined, and data expansion detection can be performed based on the packet size information. If the detection passes, it means that the current processing operation does not have any expansion problem, or the current data expansion does not affect business operations, so the step of performing the market data distribution task based on the read target business data packet can continue.

[0064] If the detection fails, it indicates that data bloat is affecting business operations during the current data packet streaming process. This triggers a tiered backpressure strategy, which propagates backpressure signals upstream from the node that identified the data bloat event to control the packet output speed of upstream nodes. Referring to the diagram in 2c, the target business data packet read from the cache passes through the sending node, detection node, and recovery node to reach the market data distribution node. If the market data distribution node detects data packet bloat, it sends a backpressure signal to the recovery node to control its packet output speed. If the bloat is too large, the recovery node continues to send backpressure signals to the detection node, and so on, until the backpressure signal reaches the cache. This indicates that the packet output speed of the target cache needs to be controlled. Since the upstream nodes of the target cache are continuously writing business data packets to it, the target cache may become full at some point. To avoid business interruption, when the target cache reaches its storage limit threshold, the target memory can be used as a secondary deep cache to replace the target cache in processing data, thus ensuring normal business operation.

[0065] In other words, since the data recovery method provided in this embodiment utilizes the cache in the FPGA to process all written business data packets, the more normal processing logic should be that the upstream writes the business data packets to the cache, and the downstream reads the target business data packets from the cache as needed. Furthermore, the downstream reading time and speed should be greater than or equal to the upstream writing time and speed to ensure that the cache is always used for processing business data packets. However, in practical applications, the upstream may reorganize the trading data, leading to an increase in the size of the business data packets—a data bloat problem. This data bloat problem is determined at the market data distribution node. Therefore, a step-by-step transmission of backpressure signals can be used to control the packet output speed of each node corresponding to the upstream node, thereby avoiding the problems caused by data bloat.

[0066] For example, see: Figure 2cAs shown in the diagram, after data packets A, B, C, and D enter the FPGA, they undergo filtering and decoding. The reassembled data packets A, B, C, and D will have different sizes; for example, data packets A and C may be twice as large as when the FPGA received them. Based on this, data packets A, B, C, and D are sequentially written into the cache. When downstream market data distribution is required, data packets A, B, C, and D are read sequentially from the cache, sent through the sending node to the detection node, and then forwarded to the recovery node if no packets are lost. The recovery node will process the packets at full speed until they reach the market data distribution node for further processing. If the market data distribution node determines that data packet C has doubled in size, it will immediately send a backpressure signal to the recovery node, controlling it to continue sending data packets to the market data distribution node. This backpressure signal is passed step-by-step, passing through the detection node and the sending node before reaching the cache, at which point the cache stops sending data packets to the sending node. When the cache reaches a set threshold at a certain point, DDR / HBM can be used as a secondary cache, so that data packets sent from upstream to the cache will enter the secondary cache, thereby solving the impact of data bloat.

[0067] In summary, by detecting any situation that could lead to congestion of the target cache, business interruptions that could restrict front-end transaction processing can be avoided, thereby effectively improving the user's real-time transaction experience.

[0068] Furthermore, after stopping the operation of reading target business data packets from the target cache, in order to ensure the normal and orderly operation of subsequent business, a data packet recovery task can be quickly executed. Executing the data packet recovery task actually requests the transaction data source node to resend the lost business data packets. In this embodiment, the specific implementation method is as follows:

[0069] By executing the data packet recovery task, a request is made to the transaction data source node to send supplementary business data packets; upon receiving the supplementary business data packets, it is determined that the data packet recovery task is in a completed state, and the steps of restoring data reading operation are performed.

[0070] Specifically, the source node for market data refers to the exchange, and the supplementary business data packet refers to the business data packet obtained by reconstructing the lost packet data issued by the source node for market data.

[0071] Based on this, after stopping the reading operation of business data packets, a data packet recovery task can be executed to request the transaction data source node to send supplementary business data packets. If the supplementary business data packets are received, it indicates that the loss of data packets has been replenished. Therefore, the steps of confirming that the data packet recovery task is in the completed state and executing the recovery data reading operation can continue.

[0072] Based on this, when requesting supplementary business data packets from the transaction data source node, this can be done by combining anomaly identification information, anomaly channel information, and anomaly length information. In this embodiment, the specific implementation is as follows:

[0073] By executing the data packet recovery task, the abnormal identification information, abnormal channel information, and abnormal length information associated with the trading market data are determined; a data supplementation request is constructed based on the abnormal identification information, abnormal channel information, and abnormal length information, and the data supplementation request is sent to the trading market source node; supplementary trading market data is received from the trading market source node in response to the data supplementation request, and a supplementary business data packet is constructed based on the supplementary trading market data.

[0074] Specifically, the anomaly identification information refers to the sequence identifier corresponding to the lost business data packet; the anomaly channel information refers to the data processing channel to which the current market data belongs; and the anomaly length information refers to the number of lost business data packets, which can be calculated based on the sequence identifiers of the business data packets before and after the loss. The data supplementation request specifically refers to a request submitted to the market data source node to request supplementary market data. Correspondingly, supplementary market data refers to the market data issued by the exchange in response to the request, and the supplementary business data packet refers to the business data packet constructed based on the supplementary market data. The construction process can be found in the section on constructing business data packets described above, and will not be elaborated upon further in this embodiment.

[0075] Therefore, to avoid consuming network resources, the source node can be requested to send only the lost market data. Thus, by executing a data packet recovery task, the abnormal identification information, abnormal channel information, and abnormal length information associated with the market data can be determined. At this point, a data supplementation request can be constructed based on the abnormal identification information, abnormal channel information, and abnormal length information, and sent to the source node. The source node determines the supplementary market data based on the request and then sends it to the programmable logic component (PLC). After receiving the supplementary market data from the source node in response to the data supplementation request, the PLC constructs a supplementary business data packet based on the supplementary market data and writes it into the recovery area for subsequent recovery processing.

[0076] In practice, when the FPGA sends a request to the exchange, it can send the request to the software side via DMA. The software side then forwards the request to the exchange, which then resends the lost business data packets based on the request. After receiving the supplementary market data, the software side sends it to the FPGA via DMA, and the FPGA then performs subsequent processing and use.

[0077] Following the example above, see [link to example]. Figure 2b The diagram illustrates that, in the event that business data packets with seqnum 8002 and 8003 are lost, the corresponding data processing channel number for the stock can be determined. With seqnum 8002 and 8003 and a packet length of 2, a data replenishment request is constructed and sent to the exchange. The exchange replenishes the lost market data. Once the exchange replenishes the lost data to the recovery module in the FPGA via software, the packet replenishment is considered complete. Subsequent market data processing can then continue using business data packets with seqnum 8002 and 8003. If the sequence numbers of the parsed business data packets are determined to be 8000, 8001, 8002, 8003, and 8004, the integrity of the business data packets is further confirmed. Therefore, subsequent processing can continue by reading new business data packets from the cache according to packet number 4 to complete parsing and processing operations.

[0078] In summary, when requesting the source node of the transaction data to send supplementary business data packets, in order to ensure the accuracy of the supplementary data, the abnormal identification information, abnormal channel information, and abnormal length information can be combined. This allows the recovery area in the programmable logic component located in the cache or memory to quickly complete the reconstruction operation of the lost data, thereby meeting the subsequent rapid processing requirements.

[0079] Step S106: When the data packet recovery task is in execution state, multiple business data packets associated with the transaction market data are written to the target cache and the target memory in order of packet number, wherein the write operation priority of the target cache is greater than the write operation priority of the target memory.

[0080] Specifically, in the case where reading target business data packets from the target cache is stopped and the data packet recovery task is executed, it indicates that a data loss recovery operation is required. To ensure the integrity of the business data packets, downstream processing operations must wait for the data loss to be recovered before they can continue. During the waiting period (during the execution of the data packet recovery task), upstream nodes continuously write market data to the programmable logic component, causing the target cache to continuously receive new business data packets. However, the target cache has limited storage space. As the time for stopping data reading operations increases, the target cache will eventually be unable to write new business data packets, leading to congestion at upstream nodes and resulting in business interruption. In a trading scenario, business interruption will severely impact the real-time requirements of trading.

[0081] Therefore, to avoid business interruptions affecting real-time trading needs, while the data packet recovery task is in progress, the target memory can be used as a secondary deep cache of the target cache. Multiple business data packets associated with the trading market data can be written to both the target cache and target memory in packet number order, with the write priority of the target cache being higher than that of the target memory. In other words, after obtaining multiple business data packets, and given a current data read stoppage, these packets can be written sequentially to the target cache and target memory in packet number order. Because the write priority of the target cache is higher than that of the target memory, the data packets will be written to the target cache first, and then automatically switched to the target memory when the cache space is insufficient. This avoids upstream business interruption. While the target memory is being used as a secondary cache for writing business data packets, the data packet recovery task will be executed or completed, thus supporting rapid downstream data reading and ensuring the corresponding market data distribution processing.

[0082] Specifically, the packet number refers to the number assigned to each business data packet when constructing multiple business data packets. The allocation order follows the sorting order of the business data packets, and the packet numbers are consecutive and unique. This ensures that data is stored and retrieved in a first-in, first-out (FIFO) manner when subsequently written to the target cache and target memory, thus guaranteeing the sequential processing of data. Correspondingly, the write priority refers to the priority at which data is written to the target cache or target memory, ensuring that data is preferentially entered into the target cache. Only when a dynamic switch to memory is needed is the data written to the target memory.

[0083] Furthermore, when writing business data packets to the target cache and target memory, in order to improve data processing efficiency, it is possible to prioritize writing business data packets to the target cache, and when the target cache space is insufficient, it can automatically switch to the target memory to meet the needs of fast data processing. In this embodiment, the specific implementation is as follows:

[0084] The system acquires multiple business data packets associated with the transaction market data, assigns packet numbers to the multiple business data packets, and writes the business data packets to the target cache in the order of the packet numbers. If the target cache reaches its storage limit threshold, the system switches to the target memory and continues to write business data packets to the target memory in the order of the packet numbers. During the process of writing business data packets to the target memory, the system detects the available cache space of the target cache. If the available cache space is greater than a preset space threshold, the system switches to the target cache and continues to write business data packets to the target cache in the order of the packet numbers.

[0085] Specifically, multiple business data packets refer to business data packets related to transaction market data that need to be continuously written to the target cache at the current moment. The storage upper limit threshold refers to the threshold number of business data packets that the target cache can store. Correspondingly, the available cache space refers to the cache space that is detected in real time for the target cache to determine whether it can cache business data packets. The preset space threshold refers to the set threshold for detecting whether the available cache space can store new business data packets. It can be set according to actual needs, and this embodiment does not impose any limitations here.

[0086] Based on this, considering that multiple business data packets need to be prioritized for processing efficiency, and that a dynamic switch to memory will occur when the target cache is full, a packet number can be assigned to each business data packet to ensure orderly processing and normal business operation. This packet numbering follows the packet order. Business data packets are then written to the target cache in packet number order. If the target cache reaches its storage limit during this process, it cannot store new data packets. To ensure normal business operation, the system switches to target memory and continues writing data packets in packet number order. Furthermore, the available cache space of the target cache can be monitored in real time during this process. If the available cache space exceeds a preset threshold, the system switches to the target cache promptly, fully utilizing its performance to continue writing data packets in packet number order. This allows the target cache and target memory to alternately store data packets.

[0087] In practical implementation, the alternating storage of business data packets between the target cache and target memory, with the write priority of the target cache being higher than that of the target memory, can be achieved through the FPGA's switching module. The business data packets cached in the FPGA can achieve fast in-fast out-of-flight functionality. When the cache is not full, data can be written to the cache at full speed. When the FPGA cache is full, it can dynamically switch to DDR or HBM. In practical applications, the switching between cache and DDR / HBM can be performed quickly based on sudden surges in traffic or subsequent backpressure. When data in the cache is consumed, some cache space is released. By checking the cache's empty / full signal to determine if there is available cache space, the system can automatically switch from the target memory to the target cache. When there is a surge in traffic and the cache's full indicator signal is triggered, DDR or HBM secondary cache can be used to ensure that business data packets are not lost during the storage phase. In addition, under normal business processes, if the cache's full indicator signal is not triggered, the cache can continue to achieve fast packet sending and receiving. This can be understood as the cache being able to quickly store data, and downstream data can be quickly sent after reading data, ensuring that the sending speed is greater than the receiving speed, thus enabling the business to run fast and stably.

[0088] Continuing with the previous example, during the process of requesting the exchange to restore the business data packets with seqnum of 8002 and 8003, the FPGA's cache will stop data reading operations, meaning it cannot continue reading the business data packet with packet number 4 for subsequent business processing. During this process, the upstream will continue writing business data packets to the cache. Since the cache's data packet buffering limit is 10, after the cache continuously stores the business data packet with packet number 13, the business data packet with packet number 14 will no longer be written to the cache. At this point, it can automatically switch to the FPGA's DDR or HBM to continue storing the remaining data packets. That is, starting from packet number 14, business data packets are written to the DDR or HBM. For example, after the DDR or HBM stores the business data packet corresponding to packet number 18, the data recovery operation is completed. At this time, the business data packet with packet number 4 in the cache will be consumed downstream, and new business data packets can be written again. Therefore, the business data packet corresponding to packet number 19 can be switched to the cache for writing, and so on. The business data packets received and constructed in real time are dynamically switched between the cache and DDR / HBM for subsequent use.

[0089] In summary, by dynamically switching between the target cache and target memory to store business data packets, and prioritizing the entry of data packets into the target cache, combined with the fast response speed of the target cache, data processing efficiency can be effectively improved. At the same time, secondary caching through the target memory ensures the integrity and continuity of business data.

[0090] Furthermore, considering that the data obtained in a transaction scenario may come from different nodes, the obtained data can be preprocessed and decoded according to a set protocol. After decoding, the data can be reassembled to obtain a business data package that can be stored in the target memory or target cache. In this embodiment, the specific implementation method is as follows:

[0091] Acquire multi-source heterogeneous data and preprocess the multi-source heterogeneous data to obtain transaction market data; wherein, acquiring multiple business data packets associated with the transaction market data includes: determining the current transaction market data in the transaction market data, and decoding the current transaction market data according to a binary protocol or information exchange protocol to obtain transaction-by-transaction data and / or snapshot data; reconstructing the transaction-by-transaction data and / or the snapshot data based on a preset format, and generating multiple business data packets based on the data reconstruction results.

[0092] Specifically, multi-source heterogeneous data refers to data structures issued by multiple upstream nodes when inputting into programmable logic components. Because these data originate from different upstream nodes, the formats of the data produced by each node may differ, thus requiring preprocessing. Preprocessing primarily involves filtering the multi-source heterogeneous data to obtain the required market data. Correspondingly, current market data refers to the market data that needs to be processed at the current moment, i.e., the market data that still needs to be written after the data reading operation has stopped. Correspondingly, binary protocol and information exchange protocol refer to the decoding formats preset by the exchange. The binary protocol is called Binary Protocol, and the information exchange protocol is called FAST protocol (Financial Information eXchange). The transaction-by-transaction data includes order data and transaction data. Order data refers to the fields in the market data used for order placement, transaction data refers to the fields in the market data used for recording transaction information, and snapshot data refers to the fields used to record the trading status or market conditions at a specific point in time, used for subsequent construction of business data packages. Correspondingly, the preset format specifically refers to the format used for data reorganization of market data, so that the constructed business data package can be easily used by downstream users. It should be noted that, for order data, transaction data and snapshot data in market data, the protocol format is different when constructing business data packages because the data types are different, and the structure of the reorganized business data package is also different.

[0093] Based on this, after obtaining multi-source heterogeneous data, considering that the data comes from different upstream nodes, preprocessing can be performed to obtain market data. On this basis, the market data will be continuously consumed. If packet loss causes the data reading operation of the target cache to stop, the current market data within the market data can be decoded according to the exchange's preset binary protocol or information exchange protocol to obtain tick-by-tick data and / or snapshot data when processing continues. Furthermore, to facilitate downstream use, the tick-by-tick data and / or snapshot data can be restructured according to a preset format. This allows the data contained in the market data to be restructured to meet the storage and usage requirements of downstream applications, enabling direct reading and use. The restructured data can then generate multiple business data packets for subsequent continuous writing to the cache or memory, supporting downstream use.

[0094] It should be noted that, without stopping the data reading operation, the transaction data is processed in the same way as the current transaction data after being written to the programmable logic component. The relevant or corresponding descriptions can be found in the description of this embodiment, which will not be elaborated on further here.

[0095] In other words, after receiving the transaction data submitted by the upstream node, the FPGA can decode the corresponding fields of each order and each transaction according to the business encoding format of the exchange's securities market information. This can be done through the decoding module in the FPGA itself, which does not cache data and can achieve full-speed data throughput for subsequent use.

[0096] In practical applications, exchange-defined market data messages may include: a Timestamp field describing the message timestamp (accurate to microseconds or nanoseconds), in the format YYYYMMDDHHMMSSsss; a SecurityID field describing the security code, following exchange code rules; BidPrice1-10 fields describing the bid-ask prices (1st to 10th highest); BidVolume1-10 fields describing the volume of the bid-ask prices (1st to 10th highest); AskPrice1-10 fields describing the ask-ask prices (1st to 10th highest); AskVolume1-10 fields describing the volume of the ask-ask prices (1st to 10th highest); LastPrice field describing the latest transaction price; OpenPrice field describing the opening price of the day; HighPrice field describing the highest price of the day; LowPrice field describing the lowest price of the day; ClosePrice field describing the closing price of the previous day; TotalVolume field describing the cumulative trading volume; TotalValue field describing the cumulative trading amount; and TradeType field describing the transaction type (e.g., regular transaction, block trade).

[0097] After obtaining the aforementioned market data, when decoding using a binary protocol, the following encoding rules can be followed: Fixed-length fields: such as Timestamp (8 bytes), SecurityID (4 bytes). Floating-point encoding: Price fields (such as BidPrice1) use IEEE 754 format (4 or 8 bytes). Integer encoding: Volume fields (such as BidVolume1) use little-endian or big-endian format. Compressed fields: Duplicate fields (such as the order of ten buy / sell orders) are reduced for redundancy through bit compression or differential encoding before decoding. When decoding using the FAST protocol, the following encoding rules can be followed: Template definition: Define static templates for message fields (such as the field order of the ten buy / sell orders). Differential encoding: Only transmit field values ​​that differ from the previous message (such as only updating the changed AskPrice1). Compression algorithms: Use Delta encoding (differential compression) or zip compression and variable-length integers (VarInt) to reduce data volume before decoding. This allows the decoded data to be stored later.

[0098] In summary, by preprocessing multi-source heterogeneous data and then decoding it using a set protocol, we can obtain entrustment data, transaction data, and snapshot data, which facilitates subsequent data reorganization and improves data processing efficiency.

[0099] Step S108: If the data packet recovery task is in the completed state, resume the data reading operation.

[0100] Specifically, during the aforementioned data packet recovery task, to ensure uninterrupted service, the target cache and target memory will be automatically switched to process the business data packets to be written. During this process, the execution of the data packet recovery task will complete the replenishment of lost data. When the data packet recovery task is in the completed state, it means that the replenishment of lost data has been completed, and therefore data reading operations can be resumed. This allows subsequent reading of the target business data packets based on the recovered data reading operations for market data distribution processing. After the target business data packets are distributed to the terminal, the terminal can support the execution of market data processing tasks corresponding to its own attributes. For example, after being handed over to the trading desk, it can execute and manage various financial asset buying and selling operations.

[0101] Specifically, the market data processing task refers to the task to be executed in response to a request from a terminal. The execution of this task relies on target business data packets read from the target cache or target memory. For example, if the terminal is a user terminal, real-time market data can be displayed on the user terminal; if the terminal is a device held by a trading party, it can perform buying and selling operations. In specific implementation, the market data processing task can be determined according to actual needs, and this embodiment does not impose any limitations.

[0102] Furthermore, due to anomalies in the trading data, the upstream process writes business data packets to the target cache or target memory. After resuming the data read operation, it is necessary to switch between the target cache and target memory to complete the read operation of the target business data packets. In this embodiment, the specific implementation method is as follows:

[0103] The process involves: determining the packet number to be read in the current polling period by polling the packet number; detecting service data packets at the target location in the target cache and the target memory according to the packet number to be read; if the detection result indicates that the packet number to be read is associated with the target cache, reading the target service data packet at the target location in the target cache; if the detection result indicates that the packet number to be read is associated with the target memory, reading the target service data packet at the target location in the target memory; determining the packet number to be read in the next polling period based on the packet number to be read, and performing the step of detecting service data packets at the target location in the target cache and the target memory according to the packet number to be read; and executing a market data distribution task based on the read target service data packet.

[0104] Specifically, the packet number to be read refers to the packet number corresponding to the business data packet to be read determined in the current polling cycle; correspondingly, the target position refers to the first position of the business data packet to be read in the target cache and target memory. Since the target cache and target memory write business data packets in the order of packet number and follow the first-in-first-out rule for data packet transmission and reception, the packet number of the business data packet that determines the target position can be directly compared with the packet number to be read without traversing the business data packets in the cache or memory, which can improve the efficiency of business data packet reading.

[0105] Therefore, after the data packet writing operation is completed by dynamically switching between the target cache and target memory, it is necessary to switch to reading business data packets from the target cache and target memory after the data recovery read operation. Specifically, when reading target business data packets sequentially by polling packet numbers, the packet number to be read in the current polling period can be determined in a preset stage by polling packet numbers. That is, the packet number corresponding to the business data packet that needs to be read from the target cache and target memory at the current moment can be determined. Then, the business data packets at the target location in the target cache and target memory can be detected according to the packet number to be read.

[0106] If the detection results determine that the packet number to be read is associated with the target cache, then the service data packet recorded at the first position in the target cache is the service data packet with the corresponding packet number to be read. Therefore, the target service data packet can be read directly at the target position in the target cache. Conversely, if the detection results determine that the packet number to be read is associated with the target memory, then the service data packet recorded at the first position in the target memory is the service data packet with the corresponding packet number to be read. Therefore, the target service data packet can be read directly at the target position in the target memory.

[0107] After reading the target service data packet, the number of the packet to be read in the next polling cycle can be determined based on the packet number to be read, and the step of detecting the service data packets at the target location of the target cache and the target memory according to the packet number to be read can be executed. By repeating the above process, the market data distribution task can be executed based on the read target service data packet.

[0108] In practice, the packet number to be read for each polling cycle can be determined by incrementing the packet number from the previous polling cycle by 1, and the first read starts from 1, thus ensuring the sequential reading of business data packets. Since the target cache and target memory store business data packets through automatic switching, data is read sequentially from the target cache and target memory according to the packet number, ensuring data continuity.

[0109] Continuing with the previous example, when requesting the exchange to replenish data packets, business data packets with package numbers 4 to 13 will be written to the cache, while business data packets with package numbers 14 and later will be written to DDR or HBM. After the exchange recovers the business data packets with seqnum of 8002 and 8003, DDR or HBM will then write the business data packet with package number 18. Based on this, downstream processes can read business data packets according to their package numbers for subsequent processing. During this process, business data packets can be read from the cache and DDR / HBM by polling the package numbers. Specifically, the first position in the cache and DDR / HBM can be searched according to package number 4 to determine that the business data packet corresponding to package number 4 is stored in the cache. Therefore, the business data packet corresponding to package number 4 can be read from the first position in the cache. This process continues, automatically completing the continuous reading of business data packets from the cache and DDR / HBM according to the package number corresponding to each polling cycle. For example, when reading the business data packet with package number 15, it is determined that it is stored in DDR or HBM, so the process switches to reading the business data packet corresponding to package number 15 from DDR or HBM. Finally, the read business data packets can be sent to the user terminal to display the real-time market data of the stock, such as the buy and sell prices at different levels, as well as the quantity corresponding to different price levels.

[0110] In summary, by reading the target business data packets from the cache and memory according to the packet number corresponding to each polling cycle, it is possible to adapt to the dynamic switching operation of the upstream, thereby enabling the target business data packets to be read continuously for subsequent market data distribution processing.

[0111] In summary, the data recovery method provided in this embodiment can be found in [reference needed]. Figure 2bThe diagram illustrates an application of a programmable logic component (PLC). This PLC includes a target cache and target memory. After market data is written to the FPGA, it can decode and reassemble the data to obtain multiple business data packets. If no anomalies exist at the current moment, these packets are written sequentially to the cache. Downstream systems then read the target business data packets from the cache for market data distribution. If an anomaly occurs (such as packet loss or data bloat), DDR or HBM can be used as a secondary cache based on requests from downstream sending nodes. This allows upstream data packets written to the cache to be switched to DDR or HBM for writing new data packets if the cache becomes unusable. In case of packet loss, a recovery node can request supplementary business data packets from the exchange. Upon receiving these supplementary packets, the sending node dynamically switches between reading business data packets sequentially from the cache and DDR / HBM to complete subsequent market data distribution, ensuring uninterrupted trading operations.

[0112] In other words, to maximize the performance advantages of programmable logic components (PLCs) and enable rapid data recovery in the event of data anomalies without interrupting upstream services due to packet loss recovery, the following steps are taken: Specifically, when processing market data using PLCs, after writing real-time market data to the target cache, the associated target business data packets can be sequentially read from the target cache for subsequent market data distribution. During this process, if the read target business data packets indicate an anomaly in the market data, suggesting potential packet loss, to ensure business continuity and avoid upstream service interruption, only the data reading operation from the target cache can be stopped. However, the market data decoding and data packet recovery tasks will continue simultaneously. Furthermore, while the data packet recovery task is in progress, the upstream data writing operation to the target cache is not restricted. This means that the upstream data writing operation to the PLC will continue to execute, allowing data packets to be written to the cache for waiting. However, the cache's storage space is limited, and the packet loss recovery time is not fixed. Before packet loss recovery, the cache may become unable to continue writing data. Therefore, to ensure uninterrupted upstream data processing, multiple business data packets associated with market data can be written to the target cache and target memory in packet number order, with the write priority to the target cache being higher than that to the target memory. During the packet loss recovery phase, business data packets are prioritized for writing to the target cache, and only when the target cache reaches its storage limit are they written to the target memory. This ensures that upstream data processing and the data writing process remain continuous. Lost data is replenished by executing a data packet recovery task. When the data packet recovery task is complete, it indicates that packet replenishment is finished, allowing data reading from the target cache or target memory to resume. This enables the subsequent distribution of market data packets based on the retrieved target business data packets. In financial trading scenarios, even in the event of packet loss, data integrity and market accuracy are maintained, and this integrity and accuracy are maintained in real-time, achieving highly reliable and efficient data processing to provide users with stable and fast-responding trading services.

[0113] The following is in conjunction with the appendix Figure 3 Taking the application of the data recovery method provided in this specification in a securities trading scenario as an example, the data recovery method will be further explained. Among other things, Figure 3 The present specification shows a flowchart of a data recovery method according to an embodiment, which includes the following steps.

[0114] Step S302: Read the target business data packet containing the associated transaction market data from the target cache.

[0115] Step S304: If it is determined that there is an anomaly in the transaction market data based on the target business data packet read, stop the data reading operation and execute the data packet recovery task.

[0116] Specifically, the process involves parsing the read target business data packet to obtain its sequence identifier; detecting any anomalies in the market data based on the sequence identifier; if no anomalies are found, performing a market data distribution task based on the read target business data packet; and if so, stopping the data reading operation and performing a data packet recovery task.

[0117] Step S306: By executing the data packet recovery task, determine the abnormal identification information, abnormal channel information, and abnormal length information associated with the trading market data.

[0118] Step S308: Construct a data supplementation request based on the anomaly identifier information, anomaly channel information, and anomaly length information, and send the data supplementation request to the transaction data source node.

[0119] Step S310: Receive supplementary market data sent by the market data source node in response to the data supplementation request, construct a supplementary business data packet based on the supplementary market data, and use the supplementary business data packet as the target business data packet.

[0120] Step S312: While the data packet recovery task is in execution state, obtain multiple business data packets associated with the transaction market data, assign packet numbers to the multiple business data packets, and write the business data packets to the target cache in the order of the packet numbers.

[0121] Step S314: If the target cache reaches the storage limit threshold, switch to the target memory and continue writing business data packets to the target memory in the order of packet number.

[0122] Step S316: During the process of writing business data packets to the target memory, the available cache space of the target cache is detected.

[0123] Step S318: If the available cache space is greater than the preset space threshold, switch to the target cache and continue writing business data packets to the target cache in the order of packet number.

[0124] Step S320: When the data packet recovery task is in the completed state, determine the number of the packet to be read in the current polling period by polling the packet number, and detect the service data packets at the target location in the target cache and target memory according to the number of the packet to be read.

[0125] Step S322: If the target service data packet is determined to be associated with the target cache based on the detection result, the target service data packet is read from the target location in the target cache.

[0126] Step S324: If the target memory is determined based on the detection result and the packet number to be read is associated with the target memory, the target service data packet is read from the target location in the target memory.

[0127] Step S326: Determine the number of the packet to be read in the next polling cycle based on the number of the packet to be read, and perform the step of detecting the service data packets of the target location in the target cache and the target memory according to the number of the packet to be read, until multiple target service data packets that meet the task execution conditions are obtained, and then execute the market data distribution task based on the read target service data packets.

[0128] The data recovery method provided in this embodiment is applied to a programmable logic component (PLC), which includes a target cache and a target memory. To maximize the performance advantages of the PLC and enable rapid data recovery in the event of data anomalies without interrupting upstream services due to packet loss recovery, specifically, when processing market data using the PLC, after writing the real-time market data to the target cache, the associated target business data packets can be sequentially read from the target cache for subsequent market data distribution. During this process, if the read target business data packets indicate an anomaly in the market data, it suggests potential packet loss. To ensure business continuity and prevent upstream service interruptions, only the data reading operation from the target cache can be stopped, while the market data decoding and data packet recovery tasks will continue simultaneously. Furthermore, while the data packet recovery task is in progress, upstream data writing to the target cache is not restricted. This means that upstream data writing to the programmable logic component will continue, allowing data packets to be written to the cache for waiting. However, the cache's storage space is limited, and the packet loss recovery time is not fixed. Before packet loss recovery, the cache may become unable to continue writing data. Therefore, to ensure uninterrupted upstream data processing, multiple business data packets associated with market data can be written to the target cache and target memory in packet number order, with the write priority to the target cache being higher than that to the target memory. During the packet loss recovery phase, business data packets are prioritized for writing to the target cache, and only when the target cache reaches its storage limit are they written to the target memory. This ensures that upstream data processing and the data writing process remain continuous. Lost data is replenished by executing a data packet recovery task. When the data packet recovery task is complete, it indicates that packet replenishment is finished, allowing data reading from the target cache or target memory to resume. This enables the subsequent distribution of market data packets based on the retrieved target business data packets. In financial trading scenarios, even in the event of packet loss, data integrity and market accuracy are maintained, and this integrity and accuracy are maintained in real-time, achieving highly reliable and efficient data processing to provide users with stable and fast-responding trading services.

[0129] Corresponding to the above method embodiments, this specification also provides data recovery system embodiments. Figure 4 A schematic diagram of the structure of a data recovery system provided in one embodiment of this specification is shown. Figure 4As shown, the data recovery system 400 includes a server 410 and a terminal 420. The server is configured with a programmable logic component, which includes a target cache and target memory, comprising:

[0130] The server 410 is configured to read target business data packets associated with market data from the target cache via a programmable logic component; if it is determined that the market data is abnormal based on the read target business data packets, it stops the data reading operation and executes a data packet recovery task; while the data packet recovery task is in the execution state, it writes multiple business data packets associated with the market data to the target cache and the target memory in order of packet number, wherein the write operation priority of the target cache is higher than the write operation priority of the target memory; when the data packet recovery task is in the execution completion state, it resumes the data reading operation and sends the read target business data packets to the terminal.

[0131] The terminal 420 is used to perform market data processing tasks based on the target business data packets sent by the server.

[0132] In an optional embodiment, writing multiple business data packets associated with the transaction market data to the target cache and the target memory in order of packet number includes:

[0133] The system acquires multiple business data packets associated with the transaction market data, assigns packet numbers to the multiple business data packets, and writes the business data packets to the target cache in the order of the packet numbers. If the target cache reaches its storage limit threshold, the system switches to the target memory and continues to write business data packets to the target memory in the order of the packet numbers. During the process of writing business data packets to the target memory, the system detects the available cache space of the target cache. If the available cache space is greater than a preset space threshold, the system switches to the target cache and continues to write business data packets to the target cache in the order of the packet numbers.

[0134] In an optional embodiment, after the data recovery read operation step is performed, the following steps are included:

[0135] The process involves: determining the packet number to be read in the current polling period by polling the packet number; detecting service data packets at the target location in the target cache and the target memory according to the packet number to be read; if the detection result indicates that the packet number to be read is associated with the target cache, reading the target service data packet at the target location in the target cache; if the detection result indicates that the packet number to be read is associated with the target memory, reading the target service data packet at the target location in the target memory; determining the packet number to be read in the next polling period based on the packet number to be read, and performing the step of detecting service data packets at the target location in the target cache and the target memory according to the packet number to be read; and executing a market data distribution task based on the read target service data packet.

[0136] In an optional embodiment, the step of stopping the data reading operation and executing a data packet recovery task when it is determined that the transaction market data is abnormal based on the read target business data packet includes:

[0137] The process involves parsing the read target business data packet to obtain its sequence identifier; detecting any anomalies in the market data based on the sequence identifier; if no anomalies are found, executing a market data distribution task based on the read target business data packet; and if so, stopping the data reading operation and performing a data packet recovery task.

[0138] In an optional embodiment, performing the data packet recovery task includes:

[0139] By executing the data packet recovery task, a request is made to the transaction data source node to send supplementary business data packets; upon receiving the supplementary business data packets, it is determined that the data packet recovery task is in a completed state, and the steps of restoring data reading operation are performed.

[0140] In an optional embodiment, the step of requesting the transaction data source node to send supplementary business data packets by executing the data packet recovery task includes:

[0141] By executing the data packet recovery task, the abnormal identification information, abnormal channel information, and abnormal length information associated with the trading market data are determined; a data supplementation request is constructed based on the abnormal identification information, abnormal channel information, and abnormal length information, and the data supplementation request is sent to the trading market source node; supplementary trading market data is received from the trading market source node in response to the data supplementation request, and a supplementary business data packet is constructed based on the supplementary trading market data.

[0142] In an optional embodiment, before the step of reading the target business data packet containing related transaction market data from the target cache is executed, the method further includes:

[0143] Acquire multi-source heterogeneous data and preprocess the multi-source heterogeneous data to obtain transaction market data; wherein, acquiring multiple business data packets associated with the transaction market data includes: determining the current transaction market data in the transaction market data, and decoding the current transaction market data according to a binary protocol or information exchange protocol to obtain transaction-by-transaction data and / or snapshot data; reconstructing the transaction-by-transaction data and / or the snapshot data based on a preset format, and generating multiple business data packets based on the data reconstruction results.

[0144] In an optional embodiment, if the determination result based on the sequence identifier to detect whether the trading market data is abnormal is negative, the method further includes:

[0145] The packet volume information of the target service data packet is determined, and data expansion detection is performed based on the packet volume information. If the detection passes, the step of executing the market data distribution task based on the read target service data packet is executed. If the detection fails, a backpressure signal is transmitted to the target cache. If the target cache reaches the storage upper limit threshold, the process is switched to the target memory, which is used as the secondary cache of the target cache. The secondary cache is used to replace the target cache in processing data.

[0146] In summary, to maximize the performance advantages of programmable logic components (PLCs) and enable rapid data recovery in the event of data anomalies without upstream service interruption due to packet loss recovery, the following steps are taken: Specifically, when processing market data using PLCs, after writing real-time market data to the target cache, the associated target business data packets can be sequentially read from the target cache for subsequent market data distribution. During this process, if the read target business data packets indicate an anomaly in the market data, suggesting potential packet loss, to ensure business continuity and avoid upstream service interruption, only the data reading operation from the target cache can be stopped. However, the market data decoding and data packet recovery tasks will continue simultaneously. Furthermore, while the data packet recovery task is in progress, the upstream data writing operation to the target cache is not restricted. This means that the upstream data writing operation to the PLC will continue to execute, allowing data packets to be written to the cache for waiting. However, the cache's storage space is limited, and the packet loss recovery time is not fixed. Before packet loss recovery, the cache may become unable to continue writing data. Therefore, to ensure uninterrupted upstream data processing, multiple business data packets associated with market data can be written to the target cache and target memory in packet number order, with the write priority to the target cache being higher than that to the target memory. During the packet loss recovery phase, business data packets are prioritized for writing to the target cache, and only when the target cache reaches its storage limit are they written to the target memory. This ensures that upstream data processing and the data writing process remain continuous. Lost data is replenished by executing a data packet recovery task. When the data packet recovery task is complete, it indicates that packet replenishment is finished, allowing data reading from the target cache or target memory to resume. This enables the subsequent distribution of market data packets based on the retrieved target business data packets. In financial trading scenarios, even in the event of packet loss, data integrity and market accuracy are maintained, and this integrity and accuracy are maintained in real-time, achieving highly reliable and efficient data processing to provide users with stable and fast-responding trading services.

[0147] The above is an illustrative scheme of a data recovery system according to this embodiment. It should be noted that the technical solution of this data recovery system and the technical solution of the data recovery method described above belong to the same concept. For details not described in detail in the technical solution of the data recovery system, please refer to the description of the technical solution of the data recovery method described above.

[0148] Corresponding to the above method embodiments, this specification also provides embodiments of a data recovery device. Figure 5A schematic diagram of a data recovery apparatus according to one embodiment of this specification is shown. Figure 5 As shown, this device is applied to a programmable logic component, wherein the programmable logic component includes a target cache and a target memory, comprising:

[0149] The reading module 502 is configured to read target business data packets containing associated transaction market data from the target cache;

[0150] The stop module 504 is configured to stop the data reading operation and execute a data packet recovery task if it is determined that there is an anomaly in the transaction market data based on the target business data packet read.

[0151] The writing module 506 is configured to write multiple business data packets associated with the transaction market data to the target cache and the target memory in order of packet number when the data packet recovery task is in the execution state, wherein the writing operation priority of the target cache is greater than the writing operation priority of the target memory.

[0152] Recovery module 508 is configured to resume data reading operations when the data packet recovery task is in the completed state.

[0153] In an optional embodiment, writing multiple business data packets associated with the transaction market data to the target cache and the target memory in order of packet number includes:

[0154] The system acquires multiple business data packets associated with the transaction market data, assigns packet numbers to the multiple business data packets, and writes the business data packets to the target cache in the order of the packet numbers. If the target cache reaches its storage limit threshold, the system switches to the target memory and continues to write business data packets to the target memory in the order of the packet numbers. During the process of writing business data packets to the target memory, the system detects the available cache space of the target cache. If the available cache space is greater than a preset space threshold, the system switches to the target cache and continues to write business data packets to the target cache in the order of the packet numbers.

[0155] In an optional embodiment, after the data recovery read operation step is performed, the following steps are included:

[0156] The process involves: determining the packet number to be read in the current polling period by polling the packet number; detecting service data packets at the target location in the target cache and the target memory according to the packet number to be read; if the detection result indicates that the packet number to be read is associated with the target cache, reading the target service data packet at the target location in the target cache; if the detection result indicates that the packet number to be read is associated with the target memory, reading the target service data packet at the target location in the target memory; determining the packet number to be read in the next polling period based on the packet number to be read, and performing the step of detecting service data packets at the target location in the target cache and the target memory according to the packet number to be read; and executing a market data distribution task based on the read target service data packet.

[0157] In an optional embodiment, the step of stopping the data reading operation and executing a data packet recovery task when it is determined that the transaction market data is abnormal based on the read target business data packet includes:

[0158] The process involves parsing the read target business data packet to obtain its sequence identifier; detecting any anomalies in the market data based on the sequence identifier; if no anomalies are found, executing a market data distribution task based on the read target business data packet; and if so, stopping the data reading operation and performing a data packet recovery task.

[0159] In an optional embodiment, performing the data packet recovery task includes:

[0160] By executing the data packet recovery task, a request is made to the transaction data source node to send supplementary business data packets; upon receiving the supplementary business data packets, it is determined that the data packet recovery task is in a completed state, and the steps of restoring data reading operation are performed.

[0161] In an optional embodiment, the step of requesting the transaction data source node to send supplementary business data packets by executing the data packet recovery task includes:

[0162] By executing the data packet recovery task, the abnormal identification information, abnormal channel information, and abnormal length information associated with the trading market data are determined; a data supplementation request is constructed based on the abnormal identification information, abnormal channel information, and abnormal length information, and the data supplementation request is sent to the trading market source node; supplementary trading market data is received from the trading market source node in response to the data supplementation request, and a supplementary business data packet is constructed based on the supplementary trading market data.

[0163] In an optional embodiment, before the step of reading the target business data packet containing related transaction market data from the target cache is executed, the method further includes:

[0164] Acquire multi-source heterogeneous data and preprocess the multi-source heterogeneous data to obtain transaction market data; wherein, acquiring multiple business data packets associated with the transaction market data includes: determining the current transaction market data in the transaction market data, and decoding the current transaction market data according to a binary protocol or information exchange protocol to obtain transaction-by-transaction data and / or snapshot data; reconstructing the transaction-by-transaction data and / or the snapshot data based on a preset format, and generating multiple business data packets based on the data reconstruction results.

[0165] In an optional embodiment, if the determination result based on the sequence identifier to detect whether the trading market data is abnormal is negative, the method further includes:

[0166] The packet volume information of the target service data packet is determined, and data expansion detection is performed based on the packet volume information. If the detection passes, the step of executing the market data distribution task based on the read target service data packet is executed. If the detection fails, a backpressure signal is transmitted to the target cache. If the target cache reaches the storage upper limit threshold, the process is switched to the target memory, which is used as the secondary cache of the target cache. The secondary cache is used to replace the target cache in processing data.

[0167] In summary, to maximize the performance advantages of programmable logic components (PLCs) and enable rapid data recovery in the event of data anomalies without upstream service interruption due to packet loss recovery, the following steps are taken: Specifically, when processing market data using PLCs, after writing real-time market data to the target cache, the associated target business data packets can be sequentially read from the target cache for subsequent market data distribution. During this process, if the read target business data packets indicate an anomaly in the market data, suggesting potential packet loss, to ensure business continuity and avoid upstream service interruption, only the data reading operation from the target cache can be stopped. However, the market data decoding and data packet recovery tasks will continue simultaneously. Furthermore, while the data packet recovery task is in progress, the upstream data writing operation to the target cache is not restricted. This means that the upstream data writing operation to the PLC will continue to execute, allowing data packets to be written to the cache for waiting. However, the cache's storage space is limited, and the packet loss recovery time is not fixed. Before packet loss recovery, the cache may become unable to continue writing data. Therefore, to ensure uninterrupted upstream data processing, multiple business data packets associated with market data can be written to the target cache and target memory in packet number order, with the write priority to the target cache being higher than that to the target memory. During the packet loss recovery phase, business data packets are prioritized for writing to the target cache, and only when the target cache reaches its storage limit are they written to the target memory. This ensures that upstream data processing and the data writing process remain continuous. Lost data is replenished by executing a data packet recovery task. When the data packet recovery task is complete, it indicates that packet replenishment is finished, allowing data reading from the target cache or target memory to resume. This enables the subsequent distribution of market data packets based on the retrieved target business data packets. In financial trading scenarios, even in the event of packet loss, data integrity and market accuracy are maintained, and this integrity and accuracy are maintained in real-time, achieving highly reliable and efficient data processing to provide users with stable and fast-responding trading services.

[0168] The above is an illustrative scheme of a data recovery device according to this embodiment. It should be noted that the technical solution of this data recovery device and the technical solution of the data recovery method described above belong to the same concept. For details not described in detail in the technical solution of the data recovery device, please refer to the description of the technical solution of the data recovery method described above.

[0169] One embodiment of this specification provides an electronic device, which is a programmable logic component, used to execute the steps of the above-described data recovery method.

[0170] The above is an illustrative scheme of an electronic device according to this embodiment. It should be noted that the technical solution of this electronic device and the technical solution of the data recovery method described above belong to the same concept. For details not described in detail in the technical solution of the electronic device, please refer to the description of the technical solution of the data recovery method described above.

[0171] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0172] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0173] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0174] The preferred embodiments disclosed above are merely illustrative of this specification. Optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described in this specification. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification.

Claims

1. A data recovery method, characterized by, The application is applied to a programmable logic component, wherein the programmable logic component comprises a target cache and a target memory, and comprises: reading a target service data packet associated with transaction market data in the target cache; stopping data reading operation and executing a data packet recovery task if it is determined that the transaction market data is abnormal according to the read target service data packet; writing a plurality of service data packets associated with the transaction market data into the target cache and the target memory in the order of packet number if the data packet recovery task is in an executing state, wherein the writing priority of the target cache is higher than that of the target memory; resuming data reading operation if the data packet recovery task is in an executing completion state.

2. The data recovery method of claim 1, wherein, The writing of the plurality of service data packets associated with the transaction market data into the target cache and the target memory in the order of packet number comprises: obtaining the plurality of service data packets associated with the transaction market data, assigning packet numbers to the plurality of service data packets, and writing service data packets into the target cache in the order of packet number; switching to the target memory and continuing to write service data packets into the target memory in the order of packet number if the target cache reaches a storage upper threshold; detecting the available cache space of the target cache during the writing of service data packets into the target memory; switching to the target cache and continuing to write service data packets into the target cache in the order of packet number if the available cache space is greater than a preset space threshold.

3. The data recovery method of claim 1, wherein, After the step of resuming data reading operation, it comprises: determining the to-be-read packet number of the current polling cycle by polling the packet numbers, and detecting the target position of the target cache and the target memory according to the to-be-read packet number; reading a target service data packet from the target position in the target cache if the to-be-read packet number is associated with the target cache according to the detection result; reading a target service data packet from the target position in the target memory if the to-be-read packet number is associated with the target memory according to the detection result; determining the to-be-read packet number of the next polling cycle based on the to-be-read packet number, and executing the step of detecting the target position of the target cache and the target memory according to the to-be-read packet number, and executing a market distribution task based on the read target service data packet.

4. The data recovery method of claim 1, wherein, The step of stopping data reading operation and executing a data packet recovery task if it is determined that the transaction market data is abnormal according to the read target service data packet comprises: analyzing the read target service data packet to obtain the sequence identifier of the target service data packet; detecting whether the transaction market data is abnormal based on the sequence identifier; if not, executing a market distribution task based on the read target service data packet; if yes, executing the steps of stopping data reading operation and executing a data packet recovery task.

5. The data recovery method of claim 1, wherein, The step of executing a data packet recovery task comprises: By executing the data packet recovery task, the transaction market source node is requested to issue a supplementary business data packet; In the case of receiving the supplementary business data packet, it is determined that the data packet recovery task is in the execution completion state, and the step of executing the recovery data reading operation is executed.

6. The data recovery method of claim 5, wherein, The data packet recovery task is executed to request the transaction market source node to issue a supplementary business data packet, comprising: By executing the data packet recovery task, determine the abnormal identification information, abnormal channel information and abnormal length information associated with the transaction market data; According to the abnormal identification information, the abnormal channel information and the abnormal length information, a data supplement request is constructed, and the data supplement request is sent to the transaction market source node; Receive the supplementary transaction market data issued by the transaction market source node for the data supplement request, and construct the supplementary business data packet according to the supplementary transaction market data.

7. The data recovery method of claim 2, wherein, Before the step of reading the target business data packet associated with the target cache, it further comprises: Obtain multi-source heterogeneous data and preprocess the multi-source heterogeneous data to obtain transaction market data; Wherein, the acquisition of the plurality of business data packets associated with the transaction market data comprises: Determine the current transaction market data in the transaction market data, and decode the current transaction market data according to the binary protocol or information exchange protocol to obtain the per-bill data and / or snapshot data; Based on the preset format, the per-bill data and / or the snapshot data are reorganized, and a plurality of business data packets are generated according to the data reorganization result.

8. The data recovery method of claim 4, wherein, In the case that the judgment result of whether the transaction market data exists based on the sequence identification is no, it further comprises: Determine the packet volume information of the target business data packet, and perform data inflation detection based on the packet volume information; In the case of passing the detection, the step of executing the market distribution task based on the read target business data packet is executed; In the case of not passing the detection, the back pressure signal is transmitted to the target cache, and if the target cache reaches the storage upper limit threshold, the target memory is switched to the target cache, and the target memory is used as the secondary cache of the target cache, wherein the secondary cache is used to replace the target cache to process data.

9. A data recovery system, characterized by, The service end and the terminal are included, wherein the service end is configured with a programmable logic component, and the programmable logic component includes a target cache and a target memory, comprising: The server is configured to read target business data packets associated with transaction market data in the target cache through a programmable logic component; stop data reading operation and perform a data packet recovery task when it is determined that the transaction market data is abnormal according to the read target business data packets; write a plurality of business data packets associated with the transaction market data to the target cache and the target memory in order of packet number when the data packet recovery task is in an execution state, wherein the write operation priority of the target cache is higher than that of the target memory; and resume data reading operation and send the read target business data packets to the terminal when the data packet recovery task is in an execution completion state. The terminal is configured to perform a market processing task based on the target business data packets sent by the server. 10.An electronic device, comprising: The electronic device is a programmable logic component configured to perform the steps of the method of any one of claims 1 to 8.

Images