A method and system for building a distributed steady-state trading environment

Abstract

This application relates to the technical field of distributed transactions, and particularly to a method and system for building a distributed steady-state transaction environment. The method for building a distributed steady-state transaction environment includes the following steps: creating an independent shard for high-frequency targets according to the real-time request frequency of transaction targets; merging low-frequency targets into a shared shard; based on the shard type, using chain synchronization for the independent shard and global broadcast synchronization for the shared shard; preloading the intermediate state for nodes within the backup node group, and when the primary node fails, the backup node group directly generates a complete state and takes over; verifying the hash chain layer by layer according to the shard topology structure, where local verification is used for the independent shard and global arbitration is triggered for the shared shard; recycling the backup nodes of low-frequency targets and reassigning them to the backup group or synchronization path of high-frequency targets. This application can achieve deep coupling of sharding, synchronization, and fault tolerance mechanisms, improving resource utilization and fault recovery speed.

Description

Technical Field

[0001] The present application relates to the technical field of distributed transactions, and in particular to a method and system for building a distributed steady-state transaction environment. Background Art

[0002] As an important part of the modern financial market, the core competitiveness of high-frequency trading depends on the low-latency response and continuous high availability of the trading system. With the exponential growth of trading frequency and data throughput, the traditional centralized architecture can no longer meet market demand due to the risk of single point failure and scalability limitations. Therefore, distributed systems have become the inevitable choice for high-frequency trading infrastructure. However, the existing distributed architecture still faces fundamental technical bottlenecks in high-concurrency scenarios, and its vulnerability is particularly significant during extreme market fluctuations or sudden traffic shocks.

[0003] In the design of distributed trading architecture, the sharding mechanism is the core strategy for dealing with massive transaction data. The currently widely used static sharding solution allocates transaction targets (such as stock codes and futures contracts) to specific nodes through preset rules. Each sharding node needs to maintain a complete order book status, position data and transaction flow records. This strategy can achieve a basic level of load balancing and fault tolerance in a stable market environment, but it causes two major structural defects due to the lack of dynamic adaptability.

[0004] First, the redundant backup mechanism under the solid-state sharding architecture cannot guarantee transaction continuity. In order to achieve redundancy for fault recovery, existing systems usually replicate data between the master node and the backup node through full data synchronization. When the master node fails, the backup node needs to load the complete state snapshot of the current shard from the data cache or storage layer. This process involves reading a large amount of data such as order book depth, unfinished order queue, position change log, etc. from persistent storage. Studies have shown that the full synchronization time of mainstream systems usually reaches 2-5 seconds, resulting in a time window of several seconds to tens of seconds of interruption of trading services. Although this delay can be partially masked by the fault tolerance mechanism in conventional trading scenarios, it will directly lead to serious consequences such as failure of continuous order execution and failure of price discovery mechanism in high-frequency trading: for example, in a flash crash scenario with drastic price fluctuations, millisecond-level order processing interruptions may trigger an avalanche effect of a large number of unexecuted orders, destroying the market liquidity balance and fairness of participants.

[0005] Secondly, fixed sharding leads to a serious disconnect between resource allocation and business needs. Due to the significant dynamic differences in trading heat among underlying assets, cold underlying assets (such as stocks with low liquidity) remain idle for a long time, while hot underlying assets (such as highly volatile futures contracts) need to handle hundreds of thousands or even millions of TPS requests during specific periods. The static sharding architecture evenly distributes cold and hot underlying assets on a fixed number of nodes, resulting in the coexistence of resource waste and computing power bottlenecks: on the one hand, the computing, storage, and network resources of low-load shards are in a sub-healthy state for a long time; on the other hand, under the impact of sudden traffic, hot shards may experience chain reactions such as thread pool overflow and queue blocking, ultimately leading to a sharp drop in the performance of local nodes. Industry surveys show that in scenarios where the single-day amplitude of constituent stocks of the CSI 300 index exceeds 10%, the average response delay of shards of key underlying assets may increase by 3 to 5 times, directly restricting the execution efficiency of high-frequency algorithmic strategies.

[0006] To alleviate the above contradictions, the technical community has proposed several targeted improvement solutions, but none of them have broken through the underlying limitations of the existing architecture. Some solutions attempt to shorten the data synchronization time by optimizing the synchronization protocol (such as introducing lightweight log compression and adding dedicated network transmission channels). Measured data shows that even if the synchronization delay is optimized to the millisecond level, full synchronization still needs to first complete a transmission volume that is linearly related to the size of the dataset. When a shard contains millions of pending orders, the synchronization time will still exceed the acceptable threshold. Another type of solution focuses on expanding the node scale and redundancy level. For example, adopting a multi-active architecture or server cluster to enhance disaster tolerance capabilities, but thus falls into the "resource dilution" dilemma: although new nodes can share the computing pressure, the static allocation logic of fixed sharding leads to a lack of pertinence in resource expansion, instead increasing the overall burden of system management and redundant data traffic.

[0007] A deeper contradiction stems from the mismatch between architecture design and business dynamics: the static sharding mechanism cannot perceive the real-time change law of trading heat. When the heat of an underlying asset undergoes a sudden change in a short period (such as a sharp increase in the trading volume of ETFs triggered by the release of major policies), the corresponding fixed nodes may be overwhelmed by the traffic peak within a few minutes, thereby triggering a chain reaction in the data synchronization mechanism. The trade-off between full synchronization solutions and incremental synchronization technologies constitutes another core paradox - although a complete data copy can ensure business continuity, the strong positive correlation between synchronization time and data scale makes it unable to meet the millisecond-level fault recovery requirements of high-frequency trading scenarios; while solutions that rely entirely on incremental synchronization will pose a risk of data integrity due to network packet loss or inconsistent node states, which poses a fundamental challenge to the "exact matching" and "irreversible" business characteristics in high-frequency trading.

[0008] In summary, the inherent defects of the existing distributed transaction architecture have formed a technical closed-loop: the static allocation of shards and the full-volume mechanism of synchronization not only cause resource waste, but also exacerbate the performance bottleneck due to the inability to dynamically respond to business requirements. To achieve a breakthrough, it is urgent to build a distributed architecture that can autonomously sense load changes and dynamically adjust resource allocation and data synchronization strategies, while ensuring millisecond-level fault recovery capabilities and achieving elastic expansion of computing resources. Summary of the Invention

[0009] To solve the above problems, the present application provides a method and system for building a distributed steady-state transaction environment.

[0010] In a first aspect, the present application provides a method for building a distributed steady-state transaction environment, adopting the following technical solution:

[0011] A method for building a distributed steady-state transaction environment includes the following steps:

[0012] According to the real-time request frequency of the transaction target, create independent shards for high-frequency targets, and bind a synchronization path and a backup node group to the independent shards; merge low-frequency targets into shared shards, and bind a backup node group to each shard;

[0013] Based on the shard type, adopt chained synchronization for independent shards and global broadcast synchronization for shared shards;

[0014] Nodes in the backup node group pre-load the intermediate state, and when the primary node fails, the backup node group directly generates the complete state and takes over;

[0015] Verify the hash chain layer by layer according to the shard topology structure. Among them, local verification is adopted for independent shards, and global arbitration is triggered for shared shards;

[0016] Recycle the backup nodes of low-frequency targets and re-allocate them to the backup group or synchronization path of high-frequency targets.

[0017] In one embodiment: the step of creating independent shards for high-frequency targets according to the real-time request frequency of the transaction target specifically includes:

[0018] Track the number of requests per unit time of each transaction target;

[0019] When the number of requests of the transaction target is greater than the preset frequency, allocate several nodes from the global resource pool to form an independent shard.

[0020] In one embodiment: the synchronization path of the chained synchronization is successively the primary node, the synchronization path node, and the backup node group, and the incremental event is transmitted hop by hop along the preset path;

[0021] The method of global broadcast synchronization is to send incremental events to the global resource pool, and the idle nodes in the pool dynamically process the events.

[0022] In one embodiment: the number of synchronization path nodes of the independent shard is positively correlated with the real-time request frequency of the shard, and the nodes of the synchronization path are selected according to physical location adjacency, the difference in the length of the unprocessed event queue between path nodes, and the node delay situation.

[0023] In one embodiment: the number of nodes in the backup node group is dynamically allocated according to the real-time request frequency of the shard.

[0024] In one embodiment: the steps of dynamically allocating the number of nodes in the backup node group according to the real-time request frequency of the shard specifically include:

[0025] The number of nodes in the backup node group of the independent shard is determined based on the basic backup number and the real-time request frequency;

[0026] The number of nodes in the backup node group of the shared shard is determined based on the remaining nodes in the global resource pool and the shard weight.

[0027] In one embodiment: the verification steps of the independent shard specifically include:

[0028] After the master node processes the transaction, it generates an event and calculates a hash chain;

[0029] The master node sends the generated event and hash chain to the synchronization path nodes;

[0030] After each synchronization path node receives the event, it verifies the continuity of the hash chain. All synchronization path nodes form a local verification group, and when more than half of the synchronization path nodes confirm that the hash chain is valid, it means the verification passes;

[0031] When the verification fails, the independent shard rolls back the data from the synchronization path nodes.

[0032] In one embodiment: the verification steps of the shared shard specifically include:

[0033] Generate an event and a hash chain, and broadcast them to all nodes in the resource pool;

[0034] Idle nodes preempt to process the event, calculate the local hash chain and return the result;

[0035] Statistically calculate the total voting weight of all nodes for the current hash chain. When the total weight exceeds the preset weight threshold, it means the verification passes; among them, the voting weight of each node is determined based on historical behavior;

[0036] When the verification fails, select the node with the highest historical voting weight as the adjudicator, and its decision overrides other nodes.

[0037] In one embodiment, the backup nodes of the recovered low-frequency targets are allocated according to priorities, which are, in sequence, high-frequency targets with a current load rate exceeding the load threshold, independent shards with an average synchronization path node latency exceeding the latency threshold, and shared shards with the number of available nodes in the global resource pool lower than the preset threshold.

[0038] Second, this application provides a method and system for building a distributed steady-state trading environment, adopting the following technical solutions:

[0039] A system for building a distributed steady-state trading environment, characterized by including:

[0040] A dynamic sharding module, which creates independent shards for high-frequency targets according to the real-time request frequency of trading targets, and binds a synchronization path and a backup node group to the independent shards; merges low-frequency targets into shared shards, and binds a backup node group to each shard;

[0041] A data synchronization module, which adopts chain synchronization for independent shards and global broadcast synchronization for shared shards based on the shard type;

[0042] A pre-activated backup module, which pre-loads the intermediate state of the nodes in the backup node group, and when the primary node fails, the backup node group directly generates a complete state and takes over;

[0043] A consistency verification module, which hierarchically verifies the hash chain according to the shard topology structure. Among them, local verification is adopted for independent shards, and global arbitration is triggered for shared shards;

[0044] A resource recovery module, which recovers the backup nodes of low-frequency targets and reallocates them to the backup group or synchronization path of high-frequency targets.

[0045] In summary, this application has the following beneficial effects:

[0046] Through the dynamic sharding method formed by classifying high- and low-frequency targets, it can autonomously sense load changes, dynamically adjust resource allocation and data synchronization strategies, achieve deep coupling of sharding, synchronization, and fault tolerance mechanisms, and improve resource utilization and fault recovery speed. Description of the Drawings

[0047] Figure 1 is a flowchart of the method for building a distributed steady-state trading environment in this embodiment;

[0048] Figure 2 is a flowchart of the allocation method of the backup node group in this embodiment;

[0049] Figure 3 is a flowchart of the verification method of independent shards in this embodiment;

[0050] Figure 4 is a flowchart of the verification step method for shared shards in this embodiment;

[0051] Figure 5 is a framework diagram of a distributed steady-state transaction environment system in this embodiment. Detailed implementation manners

[0052] The present application will be further described in detail below with reference to the accompanying drawings.

[0053] To more clearly understand the purpose, technical solution and advantages of the present application, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments. However, those of ordinary skill in the art should understand that the present application can be implemented without these details. In some cases, well-known methods, processes, systems, components and / or circuits that have been described at a higher level will not be described in excessive detail in order to avoid obscuring various aspects of the present application with unnecessary descriptions. For those of ordinary skill in the art, it is obvious that various changes can be made to the disclosed embodiments of the present application, and without departing from the principles and scope of the present application, the general principles defined in the present application can be applied to other embodiments and application scenarios. Therefore, the present application is not limited to the illustrated embodiments, but conforms to the broadest scope consistent with the scope claimed in the present application.

[0054] It should be noted here that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation to the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0055] In the description of the present application, the meaning of several is one or more, the meaning of multiple is more than two, and understandings such as greater than, less than, exceeding, etc. do not include the present number, and understandings such as above, below, within, etc. include the present number. If there is a description of first and second, it is only for the purpose of distinguishing technical features and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features or implicitly indicating the sequence relationship of the indicated technical features.

[0056] In the description of the present application, the descriptions with reference to terms such as "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a combined manner.

[0057] An embodiment of this application discloses a method for building a distributed steady-state trading environment.

[0058] As Figure 1 shown, a method for building a distributed steady-state trading environment includes the following steps:

[0059] S100. According to the real-time request frequency of trading targets, create independent shards for high-frequency targets, and bind a synchronization path and a backup node group to the independent shards; merge low-frequency targets into shared shards, and bind a backup node group to each shard.

[0060] Among them, the step of creating independent shards for high-frequency targets according to the real-time request frequency of trading targets specifically includes:

[0061] Track the number of requests per unit time of each trading target, such as the order volume per second, etc.

[0062] When the number of requests of a trading target is greater than the preset frequency, allocate several nodes from the global resource pool to form an independent shard.

[0063] The core idea of this sharding strategy is to dynamically allocate resources according to the real-time request frequency of trading targets, separating high-frequency trading targets (such as popular stocks) from low-frequency targets (such as unpopular stocks). Its core objectives are threefold. First, to improve system throughput. The trading volume of high-frequency targets is concentrated, and dedicated shards are required to provide low-latency and high-concurrency processing capabilities. Second, to optimize resource utilization. Low-frequency targets do not need to monopolize resources, and resource waste is avoided by sharing shards. Third, to enhance system elasticity. Dynamically adjust the sharding strategy so that the system can flexibly respond to load changes.

[0064] Among them, the efficiency advantages of using independent shards for high-frequency targets: First, there is dedicated resource guarantee. High-frequency trading requires quick response. Independent shards allocate exclusive computing, network, and storage resources for high-frequency targets, avoiding competition for resources with other targets (especially low-frequency targets). Second, it can reduce cross-shard latency. The trading link within an independent shard is short, and the latency of the synchronization path (such as chain synchronization) is lower, thus improving trading throughput and response speed.

[0065] The cost-effectiveness of sharing shards for low-frequency targets: First, the resource pools can be merged. Low-frequency targets share the same set of resources through shared shards, avoiding allocating independent resources for targets that are frequently idle and reducing waste of servers, bandwidth, and storage. Second, it has dynamic expansion elasticity. When a low-frequency target suddenly becomes high-frequency (such as in the case of a market hot event), the system can quickly transfer the trading of this target from the shared shard to a newly created independent shard.

[0066] S200. Based on the shard type, use chain synchronization for independent shards and global broadcast synchronization for shared shards.

[0067] Chain synchronization transmits data through the interaction between nodes, rather than all nodes requesting data from a single master node. This distributed transmission method can significantly reduce the bandwidth pressure of the master node and improve data distribution efficiency. For example, independent shards of high-frequency targets require real-time synchronization of a large amount of transaction data. The parallel transmission capability of chain synchronization can avoid resource competition and delay accumulation between nodes.

[0068] The nodes in the chain-synchronized shards form a direct transmission path, avoiding the redundant transmission and waiting time of global broadcasts. The real-time requirements of high-frequency transactions (such as high-frequency quantitative transactions) can be quickly completed through short paths and local synchronization, ensuring that the transaction confirmation delay is controlled at the millisecond level, achieving the effect of low latency and high throughput.

[0069] Low-frequency targets in shared shards (such as unpopular assets or low-liquidity transactions) have low transaction frequencies, but their consistency requirements are extremely high (for example, cross-asset settlements must be globally confirmed). Global broadcast ensures that all replica nodes obtain the latest data, avoiding the risk of "forks" or "double spending" caused by some nodes not being synchronized.

[0070] This design takes into account the low latency and high throughput requirements of high-frequency scenarios and the global consistency of low-frequency scenarios, while combining the randomness and dynamics of sharding technology to achieve resource optimization, security enhancement and scalability breakthroughs.

[0071] Specifically, the synchronization path of chain synchronization is the master node, synchronization path node and backup node group in turn, and transmits incremental events hop by hop along the preset path. Among them, the number of synchronization path nodes of independent shards is positively correlated with the real-time request frequency of the shards, and the nodes of the synchronization path are selected based on the physical location proximity, the difference in the length of the unprocessed event queue between the path nodes, and the node delay.

[0072] Among them, physical location proximity means giving priority to nodes in the same computer room or low-latency network area. When the difference in the length of the unprocessed event queue between path nodes exceeds the preset threshold, it means that the load is large, so it is not given priority. And when the delay of a node exceeds the preset threshold, the link is automatically bypassed to rebuild the node and realize dynamic path switching. The nodes of the synchronous path are selected by the above three conditions to ensure low latency while avoiding path node overload.

[0073] In high-frequency trading scenarios, independent shards may face tens of thousands of order requests per second, far exceeding the processing capacity of ordinary shards. Increasing the number of nodes in the shard synchronization path can increase the number of relay nodes, which can ensure the efficiency and stability of data transmission under high load.

[0074] For example, the original path is primary node → node 1 → node 2 → backup node, and the single-hop delay is 2ms (total delay is 6ms).

[0075] After adding a relay node, the path becomes:

[0076] Master node → Node A (1ms) → Node B (1ms) → Backup node (total delay 3ms);

[0077] Master node → Node C (1ms) → Node D (1ms) → Backup node (total delay 3ms).

[0078] The data is transmitted in two parallel paths, and the actual effective delay is reduced to 3ms. Moreover, the single-hop delay is reduced to 1ms due to the reduced load.

[0079] The method of global broadcast synchronization is to send incremental events to the global resource pool, and the idle nodes in the pool dynamically process the events. Its core goal is to allow idle nodes to dynamically process events through a decentralized broadcast mechanism and maximize resource utilization through a dynamic order-grabbing method.

[0080] S300. Nodes in the backup node group preload the intermediate state. When the master node fails, the backup node group directly generates the complete state and takes over.

[0081] This step realizes zero-perception failover. In the traditional mode, according to the master-slave mode, the standby node needs to wait for the master node to fail before starting to synchronize data, resulting in a switching delay of several seconds to several minutes. During this period, the system may suspend services.

[0082] In this embodiment, the backup node group preloads the intermediate state of the master node (such as the position and order book, etc.), but does not submit it to the database for the time being, ensuring that there is no need to resynchronize data when the master node fails, and the service can be taken over immediately, realizing sub-second switching. In addition, the intermediate state is only stored in memory, and the computing overhead is reduced by generating snapshots regularly.

[0083] For example, when the master node processes the event "buy 100 shares", after preloading, the backup node marks the position +100 shares in memory, but does not actually deduct the money.

[0084] When the master node fails, the backup node can directly generate the final state based on the preloaded intermediate state and the latest events, and then submit the final state to the database to complete the transaction takeover.

[0085] S400. Verify the hash chain layer by layer according to the sharding topology structure. Among them, independent shards adopt local verification, and shared shards trigger global arbitration.

[0086] Adopt different verification methods for different shards to achieve the balance of delay and throughput, optimize resource utilization, enhance security, and improve scalability.

[0087] The local verification of independent shards has the advantages of quickly confirming transactions, supporting high throughput, and ensuring consistency within the shard.

[0088] Fast transaction confirmation: Only the nodes within the group need to verify the hash chain (such as 3 - 5 nodes), avoiding coordinating all the nodes in the network and significantly reducing latency. For example, in high-frequency trading scenarios, local verification can compress the confirmation time to within 10 ms, which is applicable to scenarios such as stock trading.

[0089] High throughput support: Local verification reduces network bandwidth occupancy, enabling independent shards to handle millions of TPS.

[0090] Guarantee of consistency within shards: Through local hash chain verification, it ensures that the data within the shards cannot be tampered with, and at the same time eliminates the waiting time for cross-shard synchronization.

[0091] The global arbitration of shared shards has the advantages of strong consistency for cross-shard transactions and guarantee of data eventual consistency.

[0092] Strong consistency for cross-shard transactions: When a transaction involves multiple shards, consensus needs to be reached through global arbitration.

[0093] Guarantee of data eventual consistency: Through majority voting across the network or segmented verification of the hash chain, it prevents global data inconsistency caused by local failures within shared shards.

[0094] S500, recycle the backup nodes of low-frequency targets and reallocate them to the backup groups or synchronization paths of high-frequency targets.

[0095] Recycling the backup nodes of low-frequency targets and reallocating them to the backup groups or synchronization paths of high-frequency targets is an effective dynamic resource management strategy. It provides an efficient resource management solution for distributed transaction systems by improving resource utilization, enhancing system elasticity, reducing resource waste, improving system stability, and optimizing system performance. This design can significantly improve the efficiency and stability of the system in practical applications, especially in high-concurrency and high-throughput transaction scenarios, providing strong guarantees for the high performance and reliability of the system.

[0096] In this embodiment, in order to dynamically optimize resource utilization, ensure service quality, and improve overall reliability, the recycled backup nodes of low-frequency targets are allocated according to priorities.

[0097] The priorities are divided into three levels. The requirements of the first-level priority are satisfied first, followed by the second-level priority, and finally the third-level priority. Among them, the first-level priority is the high-frequency targets with the current load rate exceeding the load threshold, the second-level priority is the independent shards with the average latency of synchronization path nodes exceeding the latency threshold, and the third-level priority is the shared shards with the number of available nodes in the global resource pool lower than the preset threshold.

[0098] In one of the embodiments, the number of nodes in the backup node group is dynamically allocated according to the real-time request frequency of the shards.

[0099] Specifically, Figure 2 As shown, the steps of dynamically allocating the number of nodes in the backup node group according to the real-time request frequency of the shards specifically include:

[0100] S201. The number of nodes in the backup node group of the independent shard is determined based on the number of basic backups and the real-time request frequency.

[0101] In this step, the value of the real-time request frequency can be directly used for calculation, but in order to reduce the load, in this embodiment, the real-time request frequency needs to be converted into a heat level before calculation.

[0102] The real-time request frequency can be pre-designed into multiple heat levels, and then a corresponding basic value is assigned to each heat level. For example, a real-time request frequency less than 1000 is low heat, greater than 1000 and less than 5000 is medium heat, and greater than 5000 is high heat. The basic value corresponding to low heat is 0, the basic value corresponding to medium heat is 1, and the basic value corresponding to high heat is 2.

[0103] The calculation formula for the number of nodes in the backup node group of an independent shard is:

[0104] The number of nodes in the backup node group = the number of basic backups + the shard heat level × elasticity coefficient

[0105] Elasticity coefficient = basic value × 1 + current total system load - average load in the past 24 hours Average load in the past 24 hours

[0106] Among them, the elastic coefficient ranges from 0.5 to 1.5, and the basic value can be preset to 1.

[0107] S202. The number of nodes in the backup node group of the shared shard is determined based on the number of remaining nodes in the global resource pool and the shard weight.

[0108] In this step, the number of remaining nodes in the global resource pool is obtained by real-time monitoring and broadcasting of the number of available nodes by the centralized resource management module. The shard weight is set by the business priority, such as 0.3 for core stocks, 0.1 for ordinary stocks, and 0.05 for unpopular derivatives. Important business shards need higher priority to ensure that their backup resources are sufficient.

[0109] In another embodiment, Figure 3 As shown in the figure, the verification steps of independent shards specifically include:

[0110] S411. The master node generates an event after processing the transaction and calculates the hash chain.

[0111] S412. The master node sends the generated event and hash chain to the synchronization path node.

[0112] S413. After each synchronization path node receives an event, it verifies the continuity of the hash chain. All synchronization path nodes form a local verification group, and when more than half of the synchronization path nodes confirm that the hash chain is valid, it indicates that the verification passes.

[0113] It should be noted here that the setting for passing the verification is not necessarily that more than half of the synchronization path nodes confirm that the hash chain is valid. It can also be set to a higher requirement, such as more than two-thirds of the synchronization path nodes confirm that the hash chain is valid.

[0114] S414. When the verification fails, the independent shard rolls back the data from the synchronization path node.

[0115] In another embodiment, as Figure 4 shown, the verification steps of the shared shard specifically include:

[0116] S421. Generate an event and a hash chain, and broadcast them to all nodes in the resource pool.

[0117] S422. Idle nodes preempt to process the event, calculate the local hash chain and return the result.

[0118] S423. Statistically calculate the total voting weight of all nodes for the current hash chain. When the total weight exceeds the preset weight threshold, it indicates that the verification passes; among them, the voting weight of each node is determined based on historical behavior.

[0119] Among them, the voting weight = (the number of correct responses - the number of errors) / the total number of participations.

[0120] S424. When the verification fails, select the node with the highest historical voting weight as the arbiter, and its decision overrides other nodes.

[0121] It should be noted that the node with the highest historical voting weight is not limited to only selecting one highest node. It can also be the total voting weight of all nodes higher than the threshold.

[0122] In one of the implementation manners, as Figure 5 shown, a system for building a distributed steady-state trading environment includes:

[0123] A dynamic sharding module, according to the real-time request frequency of the trading target, creates independent shards for high-frequency targets, and binds synchronization paths and backup node groups to the independent shards; merges low-frequency targets into shared shards, and binds backup node groups to each shard.

[0124] A data synchronization module, based on the shard type, adopts chained synchronization for independent shards and global broadcast synchronization for shared shards.

[0125] Pre-activation backup module, which backs up the pre-loaded intermediate state of nodes within the backup node group. When the primary node fails, the backup node group directly generates a complete state and takes over.

[0126] Consistency verification module, which verifies the hash chain layer by layer according to the sharding topology. Among them, independent shards adopt local verification, and shared shards trigger global arbitration.

[0127] Resource recycling module, which recycles the backup nodes of low-frequency targets and reallocates them to the backup group or synchronization path of high-frequency targets.

[0128] The embodiments of this specific implementation manner are all preferred embodiments of this application, and do not limit the protection scope of this application accordingly. Therefore, all equivalent changes made according to the structure, shape, and principle of this application shall be covered within the protection scope of this application.

Claims

1. A method for building a distributed steady-state trading environment, characterized in that It includes the following steps: According to the real-time request frequency of the trading target, create an independent shard for the high-frequency target, and bind a synchronization path and a backup node group to the independent shard; Merge the low-frequency targets into the shared shard, and bind a backup node group to each shard; Based on the shard type, adopt chain synchronization for the independent shard and global broadcast synchronization for the shared shard; Nodes in the backup node group preload the intermediate state. When the primary node fails, the backup node group directly generates the complete state and takes over; Verify the hash chain layer by layer according to the shard topology structure. Among them, local verification is adopted for the independent shard, and global arbitration is triggered for the shared shard; Recycle the backup nodes of the low-frequency targets and reallocate them to the backup group or synchronization path of the high-frequency targets.

2. The method for building a distributed steady-state trading environment according to claim 1, wherein The step of creating an independent shard for the high-frequency target according to the real-time request frequency of the trading target specifically includes: Track the number of requests per unit time for each trading target; When the number of requests of the trading target is greater than the preset frequency, allocate several nodes from the global resource pool to form an independent shard.

3. The method for building a distributed steady-state trading environment according to claim 1, characterized in that: The synchronization path of the chain synchronization is the primary node, the synchronization path node, and the backup node group in sequence, and the incremental event is transmitted hop by hop along the preset path; The method of global broadcast synchronization is to send the incremental event to the global resource pool, and the idle nodes in the pool dynamically process the event.

4. A method for building a distributed steady-state trading environment according to claim 1, characterized in that: The number of synchronization path nodes of the independent shard is positively correlated with the real-time request frequency of the shard, and the nodes of the synchronization path are selected according to the physical location adjacency, the difference in the length of the unprocessed event queue between the path nodes, and the node latency.

5. The method for building a distributed steady-state trading environment according to claim 1, wherein: The number of nodes in the backup node group is dynamically allocated according to the real-time request frequency of the shard.

6. The method for building a distributed steady-state trading environment according to claim 5, characterized in that, The step of dynamically allocating the number of nodes in the backup node group according to the real-time request frequency of the shard specifically includes: The number of nodes in the backup node group of the independent shard is determined based on the basic backup number and the real-time request frequency; The number of nodes in the backup node group of the shared shard is determined based on the remaining nodes in the global resource pool and the shard weight.

7. The method for building a distributed steady-state trading environment according to claim 1, wherein, The verification step of the independent shard specifically includes: The primary node generates an event after processing the transaction and calculates the hash chain; The primary node sends the generated event and the hash chain to the synchronization path node; After each synchronization path node receives the event, it verifies the continuity of the hash chain. All synchronization path nodes form a local verification group, and when more than half of the synchronization path nodes confirm that the hash chain is valid, it means the verification passes; When the verification fails, the independent shard rolls back the data from the synchronization path node.

8. The method for building a distributed steady-state trading environment according to claim 1, wherein The verification step of the shared shard specifically includes: Generate an event and a hash chain, and broadcast them to all nodes in the resource pool; Idle nodes preemptively process the event, calculate the local hash chain and return the result; Statistically calculate the total voting weight of all nodes for the current hash chain. When the total weight exceeds the preset weight threshold, it means the verification passes; among them, the voting weight of each node is determined based on historical behavior; When the verification fails, select the node with the highest historical voting weight as the adjudicator, and its decision overrides other nodes.

9. The method for building a distributed steady-state trading environment according to claim 1, characterized in that The backup nodes of the recovered low-frequency targets are allocated according to priorities, which are high-frequency targets with the current load ratio exceeding the load threshold, independent shards with the average delay of the synchronization path nodes exceeding the delay threshold, and shared shards with the number of available nodes in the global resource pool lower than the preset threshold in sequence.

10. A system for building a distributed steady-state trading environment, characterized in that, It includes: A dynamic sharding module that creates independent shards for high-frequency targets according to the real-time request frequency of trading targets, and binds synchronization paths and backup node groups to the independent shards; Merges low-frequency targets into shared shards and binds a backup node group to each shard; A data synchronization module that, based on the shard type, adopts chained synchronization for independent shards and global broadcast synchronization for shared shards; A pre-activated backup module that pre-loads the intermediate state of the nodes in the backup node group, and when the primary node fails, the backup node group directly generates the complete state and takes over; A consistency verification module that hierarchically verifies the hash chain according to the shard topology structure. Among them, local verification is adopted for independent shards, and global arbitration is triggered for shared shards; A resource recovery module that recovers the backup nodes of low-frequency targets and reallocates them to the backup group or synchronization path of high-frequency targets.

Images