Dynamic sharding processing method, device and storage medium based on alliance chain architecture

Abstract

This application relates to the field of blockchain, and more particularly to a dynamic sharding processing method, apparatus, consortium chain architecture, computer equipment, and storage medium based on a consortium chain architecture. The method comprises: modeling and generating a transaction graph based on historical consortium chain transaction data; performing cluster analysis on points in the transaction graph to obtain an initial classification result; matching points in the transaction graph based on the initial classification result to obtain an initial matching result; optimizing the initial allocation result with the number of target shards as a constraint and maximizing the theoretical throughput of all target shards as the goal, and outputting a sharding strategy for the transaction graph. The present invention dynamically adjusts the maintenance scope of each target shard to reduce the cross-shard transaction rate of the consortium chain architecture.

Description

Technical Field

[0001] The present application relates to the field of blockchain, and in particular to a dynamic sharding processing method, device, alliance chain architecture, computer equipment and storage medium based on an alliance chain architecture. Background Art

[0002] Blockchain technology distributes trust through multi-party participation, incentivizing or requiring participants to take corrective actions to maintain a distributed ledger, and is capable of auditing any misconduct. Blockchain technology is widely used in Web3 applications, non-fungible tokens, and decentralized finance.

[0003] Blockchain systems can be divided into three categories based on the range of nodes that can participate: (1) public chains, in which any node can participate; (2) consortium chains, in which nodes within multiple organizations can participate; and (3) private chains, in which nodes within a single organization can participate. Public chain systems are more popular abroad, but due to the lack of effective supervision, they are not suitable for domestic application scenarios. In other words, the vast majority of domestic blockchain systems are deployed in the form of consortium chains.

[0004] In the existing technology, since blockchain accounts in blockchain smart contract transactions have no fixed correlation trend, static sharding based on blockchain address space or other information often leads to a higher cross-shard transaction rate. Summary of the Invention

[0005] Based on this, it is necessary to provide a dynamic sharding processing method, device, alliance chain architecture, computer equipment and storage medium based on the alliance chain architecture to address the above technical problems.

[0006] In a first aspect, an embodiment of the present invention provides a dynamic sharding processing method based on a consortium chain architecture, the method comprising:

[0007] Based on the historical transaction data of the input alliance chain, model and generate the transaction graph;

[0008] Performing cluster analysis on the points in the transaction graph to obtain an initial classification result;

[0009] Based on the initial classification result, matching the points in the transaction graph to obtain an initial matching result;

[0010] Taking the number of target shards as a constraint and maximizing the theoretical throughput of all target shards as a goal, the initial allocation result is optimized to output a sharding strategy for the transaction graph.

[0011] In one embodiment, modeling and generating a transaction graph based on historical transaction data of the input consortium chain includes:

[0012] The accounts associated with each transaction are treated as points in the transaction graph;

[0013] Edges are added between the points corresponding to the two accounts associated with each transaction, and weights of the edges are configured to generate the transaction graph.

[0014] In one embodiment, performing cluster analysis on the points in the transaction graph to obtain an initial classification result includes:

[0015] Converting the transaction graph into a sparse matrix representation, wherein the weight between two points represents the sum of the weights of multiple edges between the two points in the transaction graph;

[0016] Pre-analyze the transaction graph using a deterministic community discovery algorithm to obtain a label of the community to which each point on the transaction graph belongs;

[0017] Sort the communities obtained from the pre-analysis by workload from large to small, add the largest k communities to the default community set, and add the remaining communities to the set to be optimized.

[0018] In one embodiment, matching points in the transaction graph based on the initial classification result to obtain an initial matching result includes:

[0019] Traverse the edges on the transaction graph and add the weight of the edge to the first weight sum; if the community to which the second point of the edge belongs exists in the default community set, add the weight of the edge to the second weight sum; if the edge is a self-loop edge, add the weight of the edge to the third weight sum; the first weight sum is the weight sum of the first point of the edge and all points, the second weight sum is the weight sum of the first point of the edge and all points in the k communities in the default community set, and the third weight sum is the weight sum of the self-loop edges of the first point of the edge;

[0020] If the community to which the first point of the edge belongs does not belong to the default community set, then if the point is associated with the default community, the default community is added to the candidate community set of the point; if the point is not associated with the default community, the candidate community set is set to the default community set; the community with the largest theoretical throughput increment in the candidate community set to which the point belongs is selected as the optimal community, and the point is assigned to the optimal community, the workload and limit throughput of the optimal community are updated, and the fourth weight sum is updated for all edges of the point; the fourth weight sum is the weight sum of all points in the optimal community where the second point of the edge and the first point are located; wherein the theoretical throughput increment is determined based on the first weight sum, the second weight sum, and the third weight sum.

[0021] In one embodiment, optimizing the initial allocation result with the number of target shards as a constraint and with the goal of maximizing the theoretical throughput of all target shards includes:

[0022] During one round of optimization, the node is traversed for all points in the candidate community set that are not empty. The community with the largest theoretical throughput increase in the candidate community set to which the node belongs is selected as the optimal community. If the sum of the theoretical throughput increase caused by the node leaving the original community and the theoretical throughput increase caused by the node joining the optimal community is greater than 0, the node is assigned to the optimal community.

[0023] Update the workload and maximum throughput of the original community and the optimal community where the point is located, and update the fourth weight sum and fifth weight sum of all edges of the point. The fifth weight sum is the weight sum of all points in the original community where the second point of the edge and the first point are located.

[0024] In one embodiment, optimizing the initial allocation result with the number of target shards as a constraint and with the goal of maximizing the theoretical throughput of all target shards further includes:

[0025] After a round of optimization, if the cumulative sum of throughput increments is greater than or equal to the optimization accuracy, the next round of optimization will be continued; otherwise, the optimization will be terminated.

[0026] In a second aspect, an embodiment of the present invention provides a dynamic sharding processing device based on a consortium chain architecture, the device comprising:

[0027] The transaction graph generation module is used to model and generate a transaction graph based on the historical transaction data of the input alliance chain;

[0028] A cluster analysis module, configured to perform cluster analysis on the points in the transaction graph to obtain an initial classification result;

[0029] An initial matching module, configured to match points in the transaction graph based on the initial classification result to obtain an initial matching result;

[0030] An optimization module is configured to optimize the initial allocation result with the number of target shards as a constraint and with the goal of maximizing the theoretical throughput of all target shards, and output a sharding strategy for the transaction graph.

[0031] In a third aspect, an embodiment of the present invention proposes a consortium chain architecture, including:

[0032] The consensus layer includes multiple parallel consensus groups, each of which includes multiple Worker nodes and a single Primary node;

[0033] The execution layer includes multiple parallel execution nodes, each of which is used to execute transactions distributed by the consensus layer;

[0034] The storage layer includes multiple databases, each of which is used to store historical transaction data and provides read and write access interfaces for each execution node.

[0035] A fragmentor is used to execute the method described in the first aspect.

[0036] In a fourth aspect, an embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps described in the first aspect.

[0037] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the processor executes the computer program, the steps described in the first aspect are implemented.

[0038] Compared to existing technologies, the above-described method, device, system, and storage medium model and generate a transaction graph based on historical transaction data input from a consortium chain; perform cluster analysis on points in the transaction graph to obtain an initial classification result; match points in the transaction graph based on the initial classification result to obtain an initial matching result; and optimize the initial allocation result with the number of target shards as a constraint and the goal of maximizing the theoretical throughput of all target shards, outputting a sharding strategy for the transaction graph. The present invention dynamically adjusts the maintenance scope of each target shard to reduce the cross-shard transaction rate of the consortium chain architecture. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a schematic diagram of the structure of the alliance chain architecture in one embodiment;

[0040] Figure 2 1 is a flow chart of a dynamic sharding processing method based on a consortium chain architecture in one embodiment;

[0041] Figure 3 Schematic diagram of a process for generating a transaction graph in one embodiment;

[0042] Figure 4 Schematic diagram of a cluster analysis method in one embodiment;

[0043] Figure 5 1 is a flow chart of an initial matching method in one embodiment;

[0044] Figure 6 Schematic diagram of the process of an optimization method in one embodiment;

[0045] Figure 7 Schematic diagram of the structure of a dynamic sharding processing device based on a consortium chain architecture in one embodiment;

[0046] Figure 8 Schematic diagram of the structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly describes the drawings required for describing the embodiments. Obviously, the drawings described below are merely examples or embodiments of the present invention. Those skilled in the art can apply the present invention to other similar scenarios based on these drawings without inventive effort. Unless otherwise apparent from the context or otherwise noted, the same reference numerals in the figures represent the same structure or operation.

[0048] As used herein and in the claims, unless the context clearly indicates otherwise, the terms "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0049] Although the present invention makes various references to certain modules in the apparatus according to embodiments of the present invention, any number of different modules can be used and run on a computing device and / or processor. The modules are only illustrative, and different aspects of the apparatus and method can use different modules.

[0050] It should be understood that when a unit or module is described as being "connected" or "coupled" to other units, modules, or blocks, it can refer to being directly connected or coupled, or communicating with other units, modules, or blocks, or there can be intervening units, modules, or blocks, unless the context clearly indicates otherwise. As used herein, the term "and / or" may include any and all combinations of one or more of the associated listed items.

[0051] like Figure 1 As shown, an embodiment of the present invention provides a consortium chain architecture, including: a consensus layer, including multiple parallel consensus groups, each of which includes multiple Worker nodes and a single Primary node; an execution layer, including multiple parallel execution nodes, each of which is used to execute transactions distributed by the consensus layer; a storage layer, including multiple databases, each of which is used to store historical transaction data and provide a read and write access interface for each of the execution nodes; a sharder, used to generate a sharding strategy and provide an interface for querying the sharding strategy.

[0052] Worker nodes are responsible for temporarily storing transactions submitted by clients, forming a network with the corresponding Worker nodes in the consensus groups of other consortium chain participants, propagating received client transactions, and submitting completed client transactions to their respective Primary nodes. Primary nodes are responsible for determining the final consensus order of client transactions within the consensus group using a sorting algorithm, forming a network with the Primary nodes in the consensus groups of other consortium chain participants, running a Byzantine fault-tolerant consensus protocol based on a directed acyclic graph, and then distributing the client transactions in the determined order to the execution layer according to the sharding strategy.

[0053] In this consortium chain architecture, transactions submitted by clients go through the following processing flow:

[0054] a. The client submits a transaction to a single consortium chain, which is then sent to a Worker node in a consensus group of the consortium chain through load balancing.

[0055] b. The Worker node that receives the transaction temporarily stores it. When the Worker node receives a sufficient number of transactions, it includes the transaction in a transaction block and propagates the transaction block to the corresponding Worker nodes of the corresponding consensus groups of other consortium chains in the network. After the transaction block is propagated, it notifies the Primary node to which it belongs.

[0056] c. The primary node of this consensus group communicates with the corresponding primary nodes of the corresponding consensus groups of other consortium chains in the network, runs a Byzantine fault-tolerant consensus protocol based on a directed acyclic graph, queries the sharder's current sharding strategy, and outputs the consensus result of the consensus group. The consensus result is output in the form of ordered transaction blocks, with transactions within the transaction block arranged in the order of intra-shard transactions first, followed by cross-shard transactions. Transactions within the transaction block need to be attached with the consensus group number and then distributed to the execution nodes corresponding to different shards in the execution layer. Intra-shard transactions are distributed to the corresponding shards, and all cross-shard transactions are first distributed to a dedicated cross-shard execution node (which needs to be grouped again). In addition, the consensus results of each consensus group need to be persisted, that is, stored in the storage layer.

[0057] d. After collecting the current consensus results of all consensus groups, the execution layer performs two rounds of execution. In the first round of execution, all execution nodes process transactions within the shards they are responsible for. Transactions from different consensus groups are fully ordered according to the consensus group number. The execution nodes use deterministic parallel execution technology to read and write to the storage layer, and execute in parallel between the execution nodes. In the second round of execution, the consensus group number is used to first determine the full order relationship of all cross-shard transactions, and then all cross-shard transactions are grouped according to the account usage and query involved. Each group of cross-shard transactions does not affect each other. Each group of cross-shard transactions is then handed over to a single execution node for execution. The execution node uses deterministic parallel execution technology to read and write to the storage layer, and executes in parallel between the execution nodes.

[0058] e. The client sends another request to query the access interface provided by the storage layer to confirm whether the transaction has been processed.

[0059] This consortium chain architecture adopts a layered design, abstracting the consensus layer, execution layer, and storage layer into distinct hierarchical structures. Within this architecture, each abstraction layer utilizes a parallel design. The consensus layer consists of multiple consensus groups, each running an independent Byzantine fault-tolerant consensus protocol based on a directed acyclic graph. Within each consensus group, multiple worker nodes are used for parallel processing. The execution layer consists of multiple execution nodes, each utilizing deterministic parallel execution technology to maximize execution efficiency. The storage layer consists of multiple database instances to distribute the read and write load on the storage layer. Furthermore, this architecture utilizes a dynamic sharding protocol designed specifically for consortium chain sharding, with a low cross-shard transaction rate, to regularly adjust and maintain the sharding strategy.

[0060] Because this consortium chain architecture also utilizes a dynamic sharding protocol with a low cross-shard transaction rate, the first round executes far more transactions than the second, meaning that cross-shard transactions account for a smaller proportion. Therefore, in the first round, multiple execution nodes process transactions in parallel without interfering with each other, improving execution layer throughput. While the second round processes cross-shard transactions with minimal impact on overall throughput, it also maximizes throughput by utilizing secure multi-node parallelism. If all transactions were executed in a single round, intra-shard and cross-shard transactions would be intertwined, making efficient processing difficult.

[0061] like Figure 2 As shown, the embodiment of the present invention provides a dynamic sharding processing method based on the alliance chain architecture, which is applied to Figure 1 The sharder in , includes the following steps:

[0062] S202: Modeling and generating a transaction graph based on historical transaction data of the input consortium chain;

[0063] S204: Perform cluster analysis on the points in the transaction graph to obtain an initial classification result;

[0064] S206: Matching points in the transaction graph based on the initial classification result to obtain an initial matching result;

[0065] S208: Taking the number of target shards as a constraint and maximizing the theoretical throughput of all target shards as a goal, the initial allocation result is optimized, and a sharding strategy for the transaction graph is output.

[0066] Based on steps S202-S208 above, a transaction graph is modeled and generated based on the historical transaction data of the consortium chain. Cluster analysis is performed on the points in the transaction graph to obtain an initial classification result. Based on the initial classification result, the points in the transaction graph are matched to obtain an initial matching result. The initial allocation result is optimized with the number of target shards as a constraint and the goal of maximizing the theoretical throughput of all target shards, and a sharding strategy for the transaction graph is output. This embodiment dynamically adjusts the maintenance scope of each consensus group to reduce the cross-shard transaction rate of the consortium chain architecture.

[0067] The transaction graph is a graph data structure abstracted from the historical transaction data processed by a blockchain system. It is an undirected, weighted graph in which nodes represent blockchain accounts and edges represent blockchain transactions. An edge connecting two nodes in the transaction graph indicates a historical transaction involving the blockchain accounts represented by those two nodes.

[0068] In step S202, Figure 3 As shown, the modeling and generation of the transaction graph based on the historical transaction data of the input alliance chain includes:

[0069] S302: The accounts associated with each transaction are used as points in the transaction graph.

[0070] S304: Add edges between the points corresponding to the two accounts associated with each transaction, and configure the weights of the edges to generate the transaction graph.

[0071] In an example embodiment, historical transaction data of the blockchain system is preprocessed to extract a set of accounts A involved in each transaction. If an account does not yet exist in the transaction graph, a point representing the account is added to the transaction graph.

[0072] For each transaction account set A, take two different accounts from all accounts at a time and add an edge between the two points represented by these two accounts. The weight of the edge is That is, the reciprocal of the number of combinations of two different accounts taken out from all accounts each time.

[0073] Before step S204, it is necessary to define parameters such as the workload and throughput of each intra-shard transaction and the workload and throughput of each cross-shard transaction.

[0074] Specifically, for a single community, the workload of each intra-shard transaction in the community is defined as 1, the throughput is defined as 1, the workload of each cross-shard transaction is defined as η, the throughput is the inverse of the number of shards involved, the workload of the entire shard is the sum of the workloads of all transactions within the shard, and the maximum throughput of the entire shard is the sum of the throughputs of all transactions within the shard.

[0075] For a single community, the processing capacity of the community is defined as λ. When the workload of the shard is less than or equal to the processing capacity of the shard, the theoretical throughput of the shard is the limit throughput. When the workload of the shard is greater than the processing capacity of the shard, the theoretical throughput of the shard is a part of the limit throughput, as shown in formula (1). Indicates the maximum throughput of the shard, workload indicates the workload of the shard, and Throughput indicates the theoretical throughput of the shard;

[0076]

[0077] The theoretical throughput of a sharding strategy is the sum of the theoretical throughputs of all communities within the sharding strategy.

[0078] In step S204, Figure 4 As shown, the cluster analysis of the points in the transaction graph is performed to obtain the initial classification results including:

[0079] S402: Convert the transaction graph into a sparse matrix representation, wherein the weight between two points represents the sum of the weights of multiple edges between the two points in the transaction graph;

[0080] S404: Pre-analyze the transaction graph using a deterministic community discovery algorithm to obtain a label of the community to which each point on the transaction graph belongs;

[0081] S406: Sort the communities obtained from the pre-analysis by workload from large to small, add the largest k communities to the default community set, and add the remaining communities to the set to be optimized.

[0082] Traverse each edge of each point on the transaction graph and count the transaction represented by the edge into the workload and maximum throughput of the community to which it belongs. If two points belong to the same community, the workload is calculated as half of the edge weight to avoid duplicate counting.

[0083] All communities obtained from the pre-analysis are sorted from large to small according to workload, the largest k communities are added to the default community set, and the remaining communities are added to the set to be optimized. If the number of communities obtained from the pre-analysis is less than k, empty sets are added to the default community set until the size of the default community set is equal to k, where k is also the number of target shards.

[0084] In step S206, Figure 5 As shown, the points in the transaction graph are matched based on the initial classification result to obtain the initial matching result, which includes:

[0085] S502: Traverse the edges on the transaction graph and add the weight of the edge to the first weight sum; if the community to which the second point of the edge belongs exists in the default community set, add the weight of the edge to the second weight sum; if the edge is a self-loop edge, add the weight of the edge to the third weight sum.

[0086] The first weight sum is the weight sum of the first point of the edge and all points, the second weight sum is the weight sum of the first point of the edge and all points in the k communities in the default community set, and the third weight sum is the weight sum of the self-loop edges of the first point of the edge.

[0087] S504: If the community to which the first point of the edge belongs does not belong to the default community set, then if the point is associated with the default community, the default community is added to the candidate community set of the point; if the point is not associated with the default community, the candidate community set is set as the default community set; the community with the largest theoretical throughput increment in the candidate community set to which the point belongs is selected as the optimal community, the point is assigned to the optimal community, the workload and limit throughput of the optimal community are updated, and the fourth weight sum of all edges of the point is updated.

[0088] The fourth weighted sum is the weighted sum of all points in the optimal community where the second point of the edge and the first point are located; and the theoretical throughput increment is determined based on the first weighted sum, the second weighted sum, and the third weighted sum.

[0089] In an example embodiment, each edge (u, v) of each point on the transaction graph is traversed, and the weight w of each edge is added to the first weight and (the sum of the weights of all points connected to point u), if the community q to which point v belongs exists in the default community set, add the weight w to the second weight sum (the sum of the weights of point u and all points in a community (q)), if the edge is a self-loop edge, then the weight w is added to the third weight sum midpoint (the sum of the weight of point u and itself (the self-loop edge)).

[0090] If the community p to which point u belongs is the default community set, the point allocation phase is skipped, otherwise the point allocation phase is entered. In the allocation phase of each point, the candidate community set needs to be calculated first. The candidate community set is determined by traversing the default community set. For the default candidate community q, if the second weight and Then add community q to the candidate community set of point u. If the candidate community set of point u is an empty set after traversing the default community set, then set the candidate community set of point u as the default community set.

[0091] Traverse each community q in the candidate community set of point u and calculate the theoretical throughput increase brought by point u joining community q. Formula (2) calculates the number of cross-shard transactions that will become community q among all transactions involving point u. Formula (3) calculates the number of transactions in community q that will be converted from cross-shard transactions to intra-shard transactions. Formula (4) calculates the new workload of community q: workload' q , formula (5) calculates the new limit throughput of community q Formula (6) and (7) use formula (1) to calculate the new and old theoretical throughput of community q, respectively: q 、throughput q , formula (8) calculates the community q with the largest theoretical throughput increment best

[0092]

[0093]

[0094]

[0095]

[0096]

[0097]

[0098]

[0099] Assign point u to community q best , update community q best The workload and limit throughput of the node u are calculated, and the fourth weight and

[0100] In step S208, Figure 6 As shown, the optimization of the initial allocation result with the number of target shards as a constraint and the theoretical throughput of all target shards as a goal includes:

[0101] S602: During one round of optimization, the points whose candidate community sets are not empty are traversed. The community with the largest theoretical throughput increase in the candidate community set to which the point belongs is selected as the optimal community. If the sum of the theoretical throughput increase caused by the point leaving the original community and the theoretical throughput increase caused by the point joining the optimal community is greater than 0, the point is assigned to the optimal community.

[0102] S604: Update the workload and maximum throughput of the original community and the optimal community where the point is located, and update the fourth weight sum and the fifth weight sum of all edges of the point. The fifth weight sum is the weight sum of all points in the original community where the second point of the edge and the first point are located.

[0103] Specifically, after completing the initial assignment of all points, the algorithm enters the optimization phase. The optimization phase is a round-by-round convergence process. In each round of the optimization phase, the algorithm traverses each point u on the transaction graph and first calculates its candidate community set. If the candidate community set is empty, the optimization of that point is skipped.

[0104] If the candidate community set of point u is not empty, first calculate the theoretical throughput increase caused by point u leaving the original community p. Formula (9) calculates the number of cross-shard transactions that will no longer belong to community p among all transactions involving point u. Formula (10) calculates the number of transactions that will be converted from intra-shard transactions of community p to cross-shard transactions. Formula (11) calculates the new workload of community p: workload' p , formula (12) calculates the new limit throughput of community p Formulas (13) and (14) calculate the new and old theoretical throughputs of community p respectively: p 、thoroughput p , formula (15) calculates the theoretical throughput increment Δ caused by point u leaving the original community p p,leave .

[0105]

[0106]

[0107]

[0108]

[0109]

[0110]

[0111] △p，leaue =throughput' p -throughput p (15)

[0112] Traverse each community q in the candidate community set of point u and calculate the community q with the largest theoretical throughput increment according to formulas (2) to (8) best , and record the increment as like Then assign point u to community q best ,Will The cumulative throughput increment of this round is used to update the community p, q best The workload and limit throughput of the node u are calculated, and the fourth weight and and the fifth weight

[0113] After a round of optimization, if the cumulative sum of throughput increments is greater than or equal to the optimization accuracy ∈, the next round of optimization will continue; otherwise, the optimization will end.

[0114] At this time, the label of the community to which each point on the transaction graph belongs is one of the k default candidate communities. Based on the label of the community to which each point on the transaction graph belongs, it is divided into k target shards, and the shard label to which each point belongs is output, that is, the output sharding strategy.

[0115] It should be understood that, although the various steps in the above flow chart are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the above flow chart may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0116] In one embodiment, if Figure 7 As shown, the present invention provides a dynamic sharding processing device based on the alliance chain architecture, the device comprising:

[0117] The transaction graph generation module 702 is used to model and generate a transaction graph based on the historical transaction data of the input alliance chain;

[0118] Cluster analysis module 704, configured to perform cluster analysis on the points in the transaction graph to obtain an initial classification result;

[0119] An initial matching module 706 is configured to match points in the transaction graph based on the initial classification result to obtain an initial matching result;

[0120] The optimization module 708 is configured to optimize the initial allocation result with the number of target shards as a constraint and the theoretical throughput maximization of all target shards as a goal, and output a sharding strategy for the transaction graph.

[0121] The specific limitations of the dynamic sharding processing device based on the consortium chain architecture can be found in the limitations of the dynamic sharding processing method based on the consortium chain architecture above and will not be repeated here. The various modules in the dynamic sharding processing device based on the consortium chain architecture can be implemented in whole or in part through software, hardware, or a combination thereof. The aforementioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of the aforementioned modules.

[0122] In one embodiment, the present invention provides a computer device, which may be a server, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory and a network interface connected via a device bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating device, a computer program and a database. The internal memory provides an environment for the operation of the operating device and the computer program in the non-volatile storage medium. The database of the computer device is used to store motion detection data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements the steps in any of the above-mentioned embodiments of the distributed storage method for multimodal document data.

[0123] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0124] In one embodiment, an embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned embodiments of the dynamic sharding processing method based on the alliance chain architecture are implemented.

[0125] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0126] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0127] The above embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A dynamic sharding processing method based on alliance chain architecture, characterized in that: The method comprises: Based on the historical transaction data of the input alliance chain, model and generate the transaction graph; Performing cluster analysis on the points in the transaction graph to obtain an initial classification result; Based on the initial classification result, matching the points in the transaction graph to obtain an initial matching result; Taking the number of target shards as a constraint and maximizing the theoretical throughput of all target shards as a goal, the initial matching result is optimized to output a sharding strategy for the transaction graph; The cluster analysis of the points in the transaction graph to obtain the initial classification results includes: Converting the transaction graph into a sparse matrix representation, wherein the weight between two points represents the sum of the weights of multiple edges between the two points in the transaction graph; Pre-analyze the transaction graph using a deterministic community discovery algorithm to obtain a label of the community to which each point on the transaction graph belongs; Sort the communities obtained from the pre-analysis by workload from large to small, add the largest k communities to the default community set, and add the remaining communities to the set to be optimized; Matching points in the transaction graph based on the initial classification result to obtain an initial matching result includes: Traverse the edges on the transaction graph and add the weight of the edge to the first weight sum; if the community to which the second point of the edge belongs exists in the default community set, add the weight of the edge to the second weight sum; if the edge is a self-loop edge, add the weight of the edge to the third weight sum; the first weight sum is the weight sum of the first point of the edge and all points, the second weight sum is the weight sum of the first point of the edge and all points in the k communities in the default community set, and the third weight sum is the weight sum of the self-loop edges of the first point of the edge; If the community to which the first point of the edge belongs does not belong to the default community set, then if the point is associated with the default community, the default community is added to the candidate community set of the point; if the point is not associated with the default community, the candidate community set is set to the default community set; the community with the largest theoretical throughput increment in the candidate community set to which the point belongs is selected as the optimal community, and the point is assigned to the optimal community, the workload and limit throughput of the optimal community are updated, and the fourth weight sum is updated for all edges of the point; the fourth weight sum is the weight sum of all points in the optimal community where the second point of the edge and the first point are located; wherein the theoretical throughput increment is determined based on the first weight sum, the second weight sum, and the third weight sum.

2. The method according to claim 1, characterized in that The modeling and generating of the transaction graph based on the historical transaction data of the input alliance chain includes: The accounts associated with each transaction are treated as points in the transaction graph; Edges are added between the points corresponding to the two accounts associated with each transaction, and weights of the edges are configured to generate the transaction graph.

3. The method according to claim 1, characterized in that The optimizing of the initial matching result with the number of target shards as a constraint and with the goal of maximizing the theoretical throughput of all target shards includes: During one round of optimization, the node is traversed for all points in the candidate community set that are not empty. The community with the largest theoretical throughput increase in the candidate community set to which the node belongs is selected as the optimal community. If the sum of the theoretical throughput increase caused by the node leaving the original community and the theoretical throughput increase caused by the node joining the optimal community is greater than 0, the node is assigned to the optimal community. Update the workload and maximum throughput of the original community and the optimal community where the point is located, and update the fourth weight sum and fifth weight sum of all edges of the point. The fifth weight sum is the weight sum of all points in the original community where the second point of the edge and the first point are located.

4. The method according to claim 1, wherein The optimizing of the initial matching result with the number of target shards as a constraint and with the goal of maximizing the theoretical throughput of all target shards further includes: After a round of optimization, if the cumulative sum of throughput increments is greater than or equal to the optimization accuracy, the next round of optimization will be continued; otherwise, the optimization will be terminated.

5. A dynamic sharding processing device based on alliance chain architecture, characterized in that: The device comprises: The transaction graph generation module is used to model and generate a transaction graph based on the historical transaction data of the input alliance chain; A cluster analysis module, configured to perform cluster analysis on the points in the transaction graph to obtain an initial classification result; An initial matching module, configured to match points in the transaction graph based on the initial classification result to obtain an initial matching result; An optimization module, configured to optimize the initial matching result with the number of target shards as a constraint and with the goal of maximizing the theoretical throughput of all target shards, and output a sharding strategy for the transaction graph; The cluster analysis of the points in the transaction graph to obtain the initial classification results includes: Converting the transaction graph into a sparse matrix representation, wherein the weight between two points represents the sum of the weights of multiple edges between the two points in the transaction graph; Pre-analyze the transaction graph using a deterministic community discovery algorithm to obtain a label of the community to which each point on the transaction graph belongs; Sort the communities obtained from the pre-analysis by workload from large to small, add the largest k communities to the default community set, and add the remaining communities to the set to be optimized; Matching points in the transaction graph based on the initial classification result to obtain an initial matching result includes: Traverse the edges on the transaction graph and add the weight of the edge to the first weight sum; if the community to which the second point of the edge belongs exists in the default community set, add the weight of the edge to the second weight sum; if the edge is a self-loop edge, add the weight of the edge to the third weight sum; the first weight sum is the weight sum of the first point of the edge and all points, the second weight sum is the weight sum of the first point of the edge and all points in the k communities in the default community set, and the third weight sum is the weight sum of the self-loop edges of the first point of the edge; If the community to which the first point of the edge belongs does not belong to the default community set, then if the point is associated with the default community, the default community is added to the candidate community set of the point; if the point is not associated with the default community, the candidate community set is set to the default community set; the community with the largest theoretical throughput increment in the candidate community set to which the point belongs is selected as the optimal community, and the point is assigned to the optimal community, the workload and limit throughput of the optimal community are updated, and the fourth weight sum is updated for all edges of the point; the fourth weight sum is the weight sum of all points in the optimal community where the second point of the edge and the first point are located; wherein the theoretical throughput increment is determined based on the first weight sum, the second weight sum, and the third weight sum.

6. A consortium chain architecture, characterized in that: include: The consensus layer includes multiple parallel consensus groups, each of which includes multiple Worker nodes and a single Primary node; The execution layer includes multiple parallel execution nodes, each of which is used to execute transactions distributed by the consensus layer; The storage layer includes multiple databases, each of which is used to store historical transaction data and provides read and write access interfaces for each execution node. A sharder, configured to execute the method according to any one of claims 1 to 4.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images