A blockchain system hierarchical cooperative scheduling optimization method based on transaction volume prediction

Abstract

The application discloses a blockchain system layered cooperative scheduling optimization method based on transaction volume prediction and relates to the technical field of blockchains.The application increases the performance index, transaction volume index and resource usage index of the blockchain system to provide data support for subsequent optimization, monitors and visually observes the operation of the system in real time, and uses a dynamic adjustment method to cope with sudden frequent transactions and the hysteresis of dynamic adjustment.According to a power formula for measuring the overall performance of the system, compared with single parameter optimization, the overall performance of the system is improved to a certain extent.A multi-objective scheduling optimization method based on an ant colony algorithm is used to cope with the dynamic characteristics of container resource usage, to adjust the target node of the task with the change of the load expectation, to ensure the relative balance of the load, to meet the task demand of the blockchain system, to rationally allocate the existing resources, and to maximize the resource utilization rate as much as possible.

Description

Technical Field

[0001] This invention relates to the field of blockchain technology, and in particular to a hierarchical collaborative scheduling optimization method for blockchain systems based on transaction volume prediction. Background Technology

[0002] Because blockchain systems aim for consensus in a trustless environment, they suffer from lower throughput and higher latency compared to traditional centralized systems. The low performance of blockchain has always been a point of criticism, making research to improve its performance essential. Improving the throughput and response speed of blockchain systems has always been a challenging task. With blockchain systems increasingly deployed in containers, how to manage these containers within a cluster and rationally allocate cluster resources to improve the performance of the blockchain system is a problem that needs to be studied.

[0003] There is considerable research in the field of blockchain system performance optimization, but this research focuses on consensus mechanisms and smart contracts, typically concentrating on only one aspect, such as the reconstruction of transaction execution models (e.g., FabricSharp and Fabric++ resolve inter-transaction read conflicts by reordering transactions), control parameter optimization, and smart contract optimization. However, strategies for integrating collaborative optimization schemes at various levels are still lacking.

[0004] Existing research on container resource management includes Kubernetes' default scheduling strategy, threshold-based scheduling strategy, and multi-objective optimization-based scheduling strategy. Kubernetes is currently a widely used container management tool. The Kubernetes default scheduling strategy selects the most suitable node for the current service deployment through node evaluation. The scheduler uses Kubernetes' watch mechanism to discover newly created Pods in the cluster that have not yet been scheduled to a node. The scheduler will schedule each discovered unscheduled Pod to a suitable node to run. The scheduler makes scheduling selections based on the scheduling principles described below, but it does not consider the cluster resource status in multi-service scenarios.

[0005] Common scheduling strategies include threshold-based scheduling and multi-objective optimization-based scheduling. Threshold-based scheduling involves manually setting relevant threshold parameters based on the actual deployment scenario and task requirements of the cluster, performing simple strategy optimization on the original scheduling strategy. The most important advantage of this method is its simplicity and convenience. Although this strategy can optimize and improve cluster resource allocation to a certain extent, it fails to achieve the expected results for transactions with sudden and frequent changes in resource demand. It cannot respond promptly or in advance to transactions with response time sensitivity, which has a certain impact on the performance of the blockchain system. Furthermore, the diverse types of resources and the large number of thresholds to be determined increase the workload of operation and maintenance.

[0006] Multi-objective optimization-based scheduling strategies consider multiple objectives or criteria when deciding how to allocate resources and schedule containers across nodes in a cluster. This approach is particularly useful in complex distributed systems where multiple competing objectives need to be balanced to achieve optimal overall performance. In multi-objective optimization scheduling methods, the scheduler considers multiple objectives simultaneously, such as minimizing response time, maximizing resource utilization, and reducing energy consumption. The scheduler uses optimization algorithms to find the optimal solution that balances these objectives. Commonly used multi-objective optimization algorithms include genetic algorithms, particle swarm optimization (PSO), and ant colony optimization (ACO). PSO and genetic algorithms often encounter local solutions during the scheduling process, which can lead to significant fluctuations in cluster load when solving scheduling problems, typically requiring combination with other algorithms for optimization.

[0007] In summary, the existing technical solutions mainly have the following drawbacks:

[0008] (1) Existing research on blockchain performance optimization usually focuses on only one level: transaction execution model optimization, smart contract optimization, and single parameter optimization. An optimization strategy that integrates all levels is missing.

[0009] (2) The adjustment strategy based on simple thresholds is not suitable for situations where transaction volume changes suddenly and frequently in the blockchain system. When encountering transactions that are sensitive to response time, it cannot respond in a timely or early manner, which has the characteristic of lag and has a certain impact on the overall blockchain system.

[0010] (3) The default scoring function in the native Kubernetes scheduling strategy only uses the difference between CPU and memory utilization, ignoring the impact of other resource performance indicators in the cluster on the scheduling task. Furthermore, when a resource object requests resources, it only determines the priority order of scheduling based on the amount of resource requests, without considering the actual resource situation in the current cluster environment, and lacks the flexibility to allocate resources reasonably.

[0011] (4) The scheduling strategy based on simple thresholds failed to achieve the expected results for tasks with sudden and frequent changes in resource demand. Moreover, as the number of task types increases, the number of thresholds to be determined also increases, which increases the workload of operation and maintenance.

[0012] (5) Resource scheduling is an NP (Non-deterministic Polynomial) hard problem. Existing multi-objective optimization methods often get stuck in local optimization and find it difficult to achieve global optimization, resulting in poor optimization performance. Summary of the Invention

[0013] To address the shortcomings of existing technologies, this invention provides a hierarchical collaborative scheduling optimization method for blockchain systems based on transaction volume prediction. This method aims to solve the problem that, since blockchain is a decentralized distributed database that aims for consensus in a trustless environment, the relatively low speed and throughput of the entire blockchain system when processing transactions and data may pose some limitations and challenges to the practical application of blockchain systems.

[0014] The technical solution of this invention is as follows:

[0015] A hierarchical collaborative scheduling optimization method for blockchain systems based on transaction volume prediction includes the following steps:

[0016] Step 1: Add performance metrics, transaction information metrics, and resource usage metrics of the blockchain system deployed in the container to the blockchain system. Collect the performance metrics and transaction information metrics of the blockchain system in real time, store the transaction information metrics in the time-series database InfluxDB, and use front-end visualization technology to visualize the number of transactions submitted and the block height of the blockchain in the transaction information metrics.

[0017] The performance metrics of the blockchain system include blockchain throughput and latency; the transaction information metrics include blockchain transaction sending rate, blockchain transaction receiving rate, number of submitted transactions, and blockchain block height; the resource metrics used by the blockchain system deployed in the container include CPU usage, memory usage, and network transmission bandwidth.

[0018] Step 2: Use a load testing tool to perform a load test on the blockchain system, and then collect the transaction sending volume data of the blockchain system and store it in the time-series database InfluxDB; the transaction sending volume data refers to the amount of data of transactions sent by the client to the blockchain system.

[0019] Step 3: Use a time series prediction model to predict the transaction volume data to obtain the predicted transaction volume data of the blockchain system; the time series prediction model is an LSTM (Long Short-Term Memory) network model.

[0020] Step 3.1: Clean the transaction volume data in the time series database InfluxDB to obtain cleaned transaction volume data;

[0021] Step 3.2: Divide the cleaned transaction volume data into training sequences and test sequences according to time order; specifically, for each transaction volume data, set a division time point, use the transaction volume data of a set length before the division time point as the training sequence, and use the transaction volume data of a set length after the division time point as the test sequence.

[0022] Step 3.3: Normalize the training and test sequences to obtain normalized training and test sequences;

[0023] Step 3.4: Train the LSTM long short-term memory network model using the normalized training sequence. During the training process, the parameters of the model are continuously adjusted through the backpropagation algorithm to obtain the trained LSTM long short-term memory network model.

[0024] Step 3.5: Validate the trained LSTM (Long Short-Term Memory) network model using normalized test sequences. Input the transaction volume data of a set length before the split time point in the normalized test sequences into the LSTM network model to obtain the prediction results for the transaction volume data of a set length after the split time point. Use the evaluation metrics MSE and R2 score to evaluate the prediction performance of the LSTM network model. Adjust the model parameters according to the evaluation results to obtain the final LSTM network model.

[0025] Step 3.6: Obtain the transaction volume data of a set length before the current time and use the final LSTM Long Short-Term Memory network model to predict the transaction volume data of a set length after the current time;

[0026] Step 4: Conduct experimental tests on the blockchain system by controlling variables to determine the threshold for block size and the threshold for blockchain transaction reception rate, and adjust the transaction reception rate and block size according to the predicted transaction sending volume data to optimize the overall performance of the blockchain system.

[0027] Step 4.1: Change one variable at a time, and conduct multiple rounds of experimental tests on the blockchain system by controlling the variables to observe the performance of the variable after it is applied to the blockchain system. Configure the block size and blockchain transaction reception rate corresponding to the optimal performance as thresholds for block size and transaction reception rate, respectively. The variables include block size and blockchain transaction reception rate. The standard for the overall performance of the blockchain system to reach the optimal level is: when the power indicator, which measures the system performance, is at its maximum, the overall performance of the blockchain system reaches the optimal level.

[0028] Based on the performance metrics of a blockchain system, the method for calculating and measuring system performance, specifically the metric of "power," is as follows:

[0029]

[0030] Where Throughput is the throughput of the blockchain system, a is any constant greater than 1, ResponseTime is the average latency of a transaction, and power is a metric for measuring system performance.

[0031] Step 4.2: Adjust the block size and blockchain transaction reception rate based on the predicted transaction sending volume data, the configured block size threshold, and the blockchain transaction reception rate threshold to optimize the overall performance of the blockchain system.

[0032] The specific methods for adjusting the block size and blockchain transaction acceptance rate are as follows: when formula (2) is met, the transaction acceptance rate is increased; when formula (3) is met, the block size is adjusted.

[0033] if(Trd≥Rt1) (2)

[0034]

[0035] Where Trd is the transaction sending rate within a specific time interval, Tr is the average rate at which the client sends transactions; Tx max Rt1 is the maximum size of a single transaction that the blockchain system can accept; Rt1 is the threshold for the configured blockchain transaction acceptance rate; B size It is the threshold for the configured block size;

[0036] Step 5: Deploy the blockchain system with the best overall performance into a container. Each blockchain node corresponds to one container. Build a Docker container image of the blockchain system and use Kubernetes for container orchestration and container resource scheduling.

[0037] Step 5.1: Deploy the blockchain system into a container;

[0038] Step 5.2: Collect resource usage metrics of the blockchain system deployed in each container, use Heapster to summarize the collected resource usage metrics of the blockchain system deployed in the container, and then import them into a third-party tool to visualize the resource usage metrics of the blockchain system deployed in the container using front-end visualization technology.

[0039] Step 5.3: The resource requirements of the blockchain system are diverse and dynamic. Based on the monitored resource usage indicators of the blockchain system deployed in the container, the ant colony algorithm in the multi-objective optimization algorithm is used to optimize resource scheduling. Different scheduling arrangements are made for the different CPU, memory and network resource requirements of each node in the blockchain system.

[0040] Step 5.3.1: Based on the constraint information between nodes in the blockchain system and the available resource information of hosts in the Kubernetes cluster, construct a solution model for the scheduling problem; the Kubernetes cluster includes several hosts;

[0041] The solution model for the scheduling problem is as follows:

[0042] aik∈{0,1} (4)

[0043]

[0044]

[0045]

[0046]

[0047]

[0048] Where a is an assignment matrix with binary values, a ik To allocate matrix elements, if the i-th container is allocated to the k-th cluster host, then a ik =1, otherwise a ik =0, as shown in formula (4); i is the container number, k represents the host number in the cluster; M is the number of hosts in the cluster; N is the number of containers; j represents the container number that is not equal to i; For the resource requirements of a container, h represents a specific resource type, including CPU, memory, and network, and H represents a set of resource types; Indicates the resources available to the host;

[0049] Step 5.3.2: Input the resource requirements of the blockchain system container, i.e., the resource indicators used by the blockchain system deployed in the container, the available resources of the host in the Kubernetes cluster, the scheduling problem solution model and constraints, into the ant colony algorithm solver to obtain the solution;

[0050] Step 5.3.3: Input the solution obtained by the ant colony algorithm solver into the container deployer to generate the Kubernetes deployment file;

[0051] Step 5.3.4 Unload the running container and reschedule the blockchain system container to the node according to the Kubernetes deployment file. Observe the status of the container and the node through the Kubernetes cluster management tool to ensure that it has been successfully scheduled to the node and is running normally. If it is running normally, end the scheduling. Otherwise, repeat step 5.2.4 until it is successfully scheduled to the node and is running normally.

[0052] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0053] (1) This invention adds performance indicators, transaction volume indicators, and resource usage indicators to the blockchain system to provide data support for subsequent optimization. Real-time monitoring and visualization of key indicators and data allow for real-time observation of the system's operation and the making of necessary adjustments.

[0054] (2) The present invention adopts a dynamic optimization method based on transaction volume prediction to deal with the sudden and frequent changes in transactions and the lag of dynamic adjustment. After using the dynamic adjustment method, according to the power formula for measuring the overall performance of the system, it can be concluded that the overall performance of the system is improved to a certain extent compared with single parameter optimization.

[0055] (3) This invention employs a multi-objective scheduling optimization method based on ant colony optimization to address the dynamic characteristics of container resource usage. Compared to particle swarm optimization and genetic algorithms in multi-objective optimization, ant colony optimization adjusts the target nodes of tasks according to changes in load expectations. Although the algorithm has higher complexity, it can ensure a relatively balanced load. Since blockchain systems are deployed in containers, and the host resources for deploying blockchain systems are limited, this method can meet the task requirements of the blockchain system while also rationally allocating existing resources to maximize resource utilization.

[0056] In summary, the hierarchical collaborative scheduling optimization method for blockchain systems based on transaction volume prediction proposed in this invention can improve the overall performance of blockchain systems, expand the application scenarios of blockchain in production and daily life, and meet more application needs. Attached Figure Description

[0057] Figure 1 This is a diagram illustrating the hierarchical collaborative scheduling optimization steps of the blockchain system in this embodiment of the invention.

[0058] Figure 2 This is a flowchart illustrating the indicator monitoring and visualization process of the blockchain system in this embodiment of the invention.

[0059] Figure 3 This is a flowchart of the dynamic optimization process based on transaction volume prediction in an embodiment of the present invention;

[0060] Figure 4 This is a flowchart illustrating the containerized deployment process of the blockchain system in this embodiment of the invention.

[0061] Figure 5 This is a resource scheduling graph based on the ant colony algorithm in an embodiment of the present invention. Detailed Implementation

[0062] This invention divides the blockchain system into three layers: the user layer, the system core layer, and the operation and maintenance management layer. The user layer handles user transaction requests, while the system core layer includes transaction processing, consensus algorithms, query tracing, storage, and network transmission. The operation and maintenance management layer provides the blockchain system with an execution engine, resource monitoring, and visualized system management, integrating the various layers for collaborative optimization. For the user layer and the system core layer, a layered collaborative optimization scheme based on transaction volume prediction is proposed. In the operation and maintenance management layer, the multi-objective optimization algorithm ant colony optimization is used for resource scheduling optimization. The transaction volume data of the blockchain system is stored in a time-series database, and a time-series prediction model is used to predict the transaction volume of the blockchain system. Based on the predicted transaction volume, the parameters of each layer in the blockchain system are dynamically adjusted to improve system throughput and latency. The multi-objective optimization algorithm ant colony optimization is used to schedule resources in the blockchain system deployment environment, rationally allocating existing cluster resources to maximize cluster resource utilization and meet the performance requirements of the blockchain system, thereby achieving optimal overall performance of the blockchain system.

[0063] A hierarchical collaborative scheduling optimization method for blockchain systems based on transaction volume prediction, such as Figure 1 As shown, it includes the following steps:

[0064] Step 1: Add performance metrics, transaction information metrics, and resource usage metrics of the blockchain system deployed in the container to the blockchain system. Use the Prometheus tool to collect the performance metrics and transaction information metrics of the blockchain system in real time. Store the transaction information metrics in the time-series database InfluxDB. Use front-end visualization technologies (Vue and Echarts) to visualize the number of transactions submitted and the block height of the blockchain in the transaction information metrics.

[0065] The performance metrics of the blockchain system include blockchain throughput and latency; the transaction information metrics include blockchain transaction sending rate, blockchain transaction receiving rate, number of submitted transactions, and blockchain block height; the resource metrics used by the blockchain system deployed in the container include CPU usage, memory usage, and network transmission bandwidth.

[0066] In this embodiment, as follows Figure 2 The diagram shows the monitoring and visualization of blockchain system performance metrics. It adds performance metrics (tendermint metrics), transaction information metrics (tm_load_test metrics), and resource usage metrics of the blockchain system deployed in containers (cadvisor metrics) to provide data support for the hierarchical collaborative scheduling optimization of the blockchain system.

[0067] The Prometheus Client collects these key metrics in real time and pushes them to the Prometheus Server. The Prometheus Server pulls these data and stores them locally. The time-series database InfluxDB reads the data from the Prometheus Server through the storage engine's remote API and stores it in the database.

[0068] To monitor the system's operation in real time and make necessary adjustments, the metric data is visualized. Metric values ​​pushed to the Prometheus server are queried via the Prometheus query API, and the block height and number of submitted transactions in the blockchain system are loaded into the front-end interface. The front-end interface uses the Vue framework, and the back-end uses the Gin framework in Go, with the front-end service deployed in a container. Data at one-minute intervals is retrieved via the time range query in the Prometheus query API, and the data is refreshed every minute. These metrics are visualized using Vue and Echarts technologies.

[0069] Step 2: Use the tm-load-test load testing tool to perform a load test on the blockchain system, and then use the Prometheus tool to collect the transaction sending volume data of the blockchain system and store it in the time-series database InfluxDB; the transaction sending volume data refers to the amount of transaction data sent by the client to the blockchain system.

[0070] Step 3: As Figure 3 As shown, a time series prediction model is used to predict transaction volume data to obtain the predicted transaction volume data of the blockchain system; the time series prediction model is an LSTM (Long Short-Term Memory) network model.

[0071] Step 3.1: Clean the transaction volume data in the time series database InfluxDB to obtain cleaned transaction volume data;

[0072] Step 3.2: Divide the cleaned transaction volume data into training sequences and test sequences according to time order; specifically, for each transaction volume data, the transaction volume data of the first 50 minutes is used as the training sequence, and the transaction volume data of the last 10 minutes is used as the test sequence.

[0073] Step 3.3: Normalize the training and test sequences to obtain normalized training and test sequences;

[0074] Step 3.4: Train the LSTM long short-term memory network model using the normalized training sequence. During the training process, continuously adjust the model parameters through the backpropagation algorithm to enable it to more accurately predict the transaction volume data in the next 10 minutes, and obtain the trained LSTM long short-term memory network model.

[0075] Step 3.5: Validate the trained LSTM (Long Short-Term Memory) network model using the normalized test sequence. Input the transaction volume data of the first 50 minutes in the normalized test sequence into the LSTM network model to obtain the prediction results of the transaction volume data of the last 10 minutes. Use the evaluation metrics MSE (Mean Squared Error) and R2 score to evaluate the prediction performance of the LSTM network model. Adjust the model parameters according to the evaluation results to obtain the final LSTM network model.

[0076] Step 3.6: Obtain the transaction volume data for the 50 minutes preceding the current time and use the final LSTM (Long Short-Term Memory) network model to predict the transaction volume data for the next 10 minutes;

[0077] Step 4: Conduct experimental tests on the blockchain system by controlling variables to determine the threshold for block size and the threshold for blockchain transaction reception rate. Adjust the transaction reception rate and block size based on the predicted transaction sending volume data to optimize the overall performance of the blockchain system. The standard for optimal overall performance of the blockchain system is: when the power indicator, which measures system performance, is at its maximum, the overall performance of the blockchain system is optimal.

[0078] In this implementation, the blockchain system is tested using a controlled variable approach. One controlled variable is changed each time, and the performance of the blockchain system after applying the controlled variable is observed. Through multiple rounds of testing, thresholds for block size and blockchain transaction acceptance rate are configured based on the block size and blockchain transaction acceptance rate corresponding to the optimal performance observed in the tests. Based on the predicted transaction volume and the configured thresholds, when the adjustment conditions are met, the transaction acceptance rate and block size are adjusted to achieve optimal overall performance. The controlled variables include block size and transaction acceptance rate.

[0079] Step 4.1: Change one variable at a time, and conduct multiple rounds of experimental tests on the blockchain system by controlling the variables to observe the performance of the variable after it is applied to the blockchain system. Configure the block size and blockchain transaction reception rate corresponding to the optimal performance as thresholds for block size and transaction reception rate, respectively. The variables include block size and blockchain transaction reception rate. The standard for the overall performance of the blockchain system to reach the optimal level is: when the power indicator, which measures the system performance, is at its maximum, the overall performance of the blockchain system reaches the optimal level.

[0080] Based on the performance metrics of a blockchain system, the method for calculating and measuring system performance, specifically the metric of "power," is as follows:

[0081]

[0082] Throughput is the throughput of the blockchain system, 'a' is any constant greater than 1, ResponseTime is the average latency of transactions, and power is a metric for measuring system performance; the larger the value, the better the system performance.

[0083] Step 4.2: Adjust the block size and blockchain transaction reception rate based on the predicted transaction sending volume data, the configured block size threshold, and the blockchain transaction reception rate threshold to optimize the overall performance of the blockchain system.

[0084] Throughput and latency are two crucial performance metrics for blockchain systems. They are conflicting objectives; optimizing one often comes at the expense of the other. Adjusting a parameter in a blockchain system frequently results in either increased throughput and increased latency, or decreased throughput and increased latency. To optimize the overall performance, a threshold-based dynamic adjustment algorithm is proposed based on predicted transaction volume.

[0085] Specifically: when the system's transaction sending volume increases significantly when formula (2) is satisfied, the transaction receiving rate can be appropriately increased; when the predicted transaction volume is large when formula (3) is satisfied, the block size should be adjusted.

[0086] if(Trd≥Rt1) (2)

[0087]

[0088] Where Trd is the transaction sending rate within a specific time interval, Tr is the average rate at which the client sends transactions; Tx max Rt1 is the maximum size of a single transaction that the blockchain system can accept; Rt1 is the threshold for the configured blockchain transaction acceptance rate; B size It is the threshold for the configured block size;

[0089] Step 5: Deploy the blockchain system into containers, with each blockchain node corresponding to one container. Build a Docker container image of the blockchain system and use Kubernetes for container orchestration and container resource scheduling.

[0090] In this embodiment, in order to provide a running environment for the blockchain system, the blockchain system adopts cloud containerization deployment. The multi-node deployment of containers is implemented based on Docker containers, and Kubernetes is used for container orchestration and container resource scheduling.

[0091] Step 5.1: Deploy the blockchain system into a container;

[0092] Step 5.2: Use the cAdvisor tool to collect resource usage metrics of the blockchain system deployed in each container. Use Heapster to summarize the resource usage metrics of the blockchain system deployed in each container collected by the cAdvisor tool, and then import them into a third-party tool (such as InfluxDB) to provide data support for subsequent optimization. Use front-end visualization technology to visualize the resource usage metrics of the blockchain system deployed in the container.

[0093] like Figure 4 The diagram shows a container deployment of a blockchain system. In the hierarchical collaborative scheduling optimization of a blockchain system, the deployment, resource monitoring, and visualization of the blockchain system are prerequisites for hierarchical collaborative optimization and resource scheduling. Therefore, the first step is to deploy a multi-node blockchain network in containers to provide a blockchain system platform. Container multi-node deployment is implemented using Docker containers. Deploying a multi-node blockchain system in containers requires functional tests such as message sending and querying to ensure the blockchain system is functioning correctly. Resource usage metrics for the blockchain system are added to provide data support for subsequent optimization. Metrics for blockchain system resource monitoring include CPU utilization, memory usage, and network traffic transmission. The Prometheus monitoring tool is used to add these metrics. The Prometheus monitoring service is started when the Tendermint node of the blockchain system is running, and the monitoring metric values ​​are pushed to the Prometheus server; the Prometheus service is shut down when the node is shut down. Container resource monitoring reads resource usage information through the container engine's built-in API.

[0094] Step 5.3: The resource requirements of the blockchain system are diverse and dynamic. Based on the resource usage indicators of the blockchain system deployed in the container, the ant colony algorithm in the multi-objective optimization algorithm is used to optimize resource scheduling. Different scheduling arrangements are made for the different CPU, memory and network resource requirements of each node in the blockchain system.

[0095] like Figure 5 The diagram shown is a dynamic scheduling flowchart based on the ant colony algorithm, which mainly includes the following steps:

[0096] Step 5.3.1: Based on the constraint information between nodes in the blockchain system and the available resource information of hosts in the Kubernetes cluster, construct a solution model for the scheduling problem, as shown. Figure 5 The problem is defined in the text; the Kubernetes cluster includes several hosts;

[0097] To formally describe the scheduling problem, consider a blockchain system with N node containers and a Kubernetes cluster with M hosts. Each container has a set of resource requirements, as well as constraints describing the relationships between containers. The problem is to find a solution to an allocation matrix.

[0098] The solution model for the scheduling problem is as follows:

[0099] a ik ∈{0,1} (4)

[0100]

[0101]

[0102]

[0103]

[0104]

[0105] Where a is an assignment matrix with binary values, a ik To allocate matrix elements, if the i-th container is allocated to the k-th cluster host, then a ik =1, otherwise a ik =0, as shown in formula (4); i is the container number, k represents the host number in the cluster; M is the number of hosts in the cluster; N is the number of containers; j represents the container number that is not equal to i; For the resource requirements of a container, h represents a specific resource type, including CPU, memory, and network, and H represents a set of resource types; Indicates the resources available to the host;

[0106] Formula (5) indicates that a container must be assigned to a cluster; Formula (6) is a conflict constraint: if the i-th container and the j-th container have a conflict constraint, then there is no situation where they are deployed on the same host; Formula (7) is a co-location constraint: if the i-th container and the j-th container have a co-location constraint, this means that containers in the co-location relationship should be deployed on the same host. The constraint can be described as a set of equations, i.e., a ik =a jk Formula (8) represents the exclusive constraint: if the i-th container has an exclusive constraint, the host that deploys the container cannot deploy other components. Because only one container is deployed, the sum of the element values ​​in column k of the allocation matrix is ​​1. Formula (9) represents the container constraint, which specifies the capacity limit of the cluster host. The total amount of a specific resource type h required by the container allocated to a specific host does not exceed the available resources of that host.

[0107] In this implementation, the co-location constraint means that blockchain nodes in a co-location relationship should be deployed on a single host within the Kubernetes cluster; the exclusive deployment constraint means that the host where this blockchain node is deployed cannot deploy other blockchain nodes; and the conflict constraint corresponds to the situation where there are blockchain nodes that cannot be deployed on the same host. The information of the scheduling problem solution model is encoded in JSON format and stored in the model.json file.

[0108] Step 5.3.2: Input the resource requirements of the blockchain system container, i.e., the resource indicators used by the blockchain system deployed in the container, the available resources of the host in the Kubernetes cluster, the scheduling problem solution model and constraints, into the ant colony algorithm solver to obtain the solution;

[0109] In this implementation, the resource requirement information and the available host resources in the Kubernetes cluster are obtained through resource monitoring and stored in the InfluxDB database. This data is read from the InfluxDB database, stored in the data.json file, and the solution to the problem is obtained by calling the built-in ant colony algorithm library in Python.

[0110] Step 5.3.3: Input the solution obtained by the ant colony algorithm solver into the container deployer to generate the Kubernetes deployment file;

[0111] Specifically, the container deployer, through its converter, first extracts the solution obtained by the ant colony algorithm solver into an allocation matrix. This allocation matrix records the relationship between hosts and blockchain nodes in the Kubernetes cluster. If the blockchain node is scheduled to a host in the corresponding Kubernetes cluster, the element in the corresponding matrix has a value of 1; if it is not scheduled to a host in the cluster, the element in the corresponding matrix has a value of 0. Then, based on the resource information required by the blockchain node container, it determines the name of the host to be allocated, the container name, the image and port information, and generates a Kubernetes deployment file in YAML format.

[0112] Step 5.3.4 Unload the running container and reschedule the blockchain system container to the node according to the Kubernetes deployment file. Since the information in the blockchain system is stored in the database, the container can be smoothly scheduled to ensure the high availability and stability of the blockchain system. Observe the status of the container and the node through the Kubernetes cluster management tool to ensure that it has been successfully scheduled to the node and is running normally. If it is running normally, end the scheduling; otherwise, repeat step 5.2.4 until it is successfully scheduled to the node and running normally.

Claims

1. A hierarchical collaborative scheduling optimization method for a blockchain system based on transaction volume prediction, characterized in that, Includes the following steps: Step 1: Add performance metrics, transaction information metrics, and resource usage metrics of the blockchain system deployed in the container to the blockchain system. Collect the performance metrics and transaction information metrics of the blockchain system in real time, store the transaction information metrics in the time-series database InfluxDB, and use front-end visualization technology to visualize the number of transactions submitted and the block height of the blockchain in the transaction information metrics. Step 2: Use a load testing tool to perform a load test on the blockchain system, and then collect the transaction sending volume data of the blockchain system and store it in the time-series database InfluxDB; the transaction sending volume data refers to the amount of data of transactions sent by the client to the blockchain system. Step 3: Use a time series prediction model to predict the transaction volume data to obtain the predicted transaction volume data of the blockchain system; the time series prediction model is an LSTM (Long Short-Term Memory) network model. Step 4: Conduct experimental tests on the blockchain system by controlling variables to determine the threshold for block size and the threshold for blockchain transaction reception rate, and adjust the transaction reception rate and block size according to the predicted transaction sending volume data to optimize the overall performance of the blockchain system. Step 4.1: Change one variable at a time, and conduct multiple rounds of experimental tests on the blockchain system by controlling the variables to observe the performance of the variable after it is applied to the blockchain system. Configure the block size and blockchain transaction reception rate corresponding to the optimal performance as thresholds for block size and transaction reception rate, respectively. The variables include block size and blockchain transaction reception rate. The standard for the overall performance of the blockchain system to reach the optimal level is: when the power indicator, which measures the system performance, is at its maximum, the overall performance of the blockchain system reaches the optimal level. The method for calculating power, an indicator for measuring system performance, is as follows: (1)； in, It is the throughput of the blockchain system. Let be any constant greater than 1. It is the average latency of the transaction; Indicators for measuring system performance; Step 4.2: Adjust the block size and blockchain transaction reception rate based on the predicted transaction sending volume data, the configured block size threshold, and the blockchain transaction reception rate threshold to optimize the overall performance of the blockchain system. The specific methods for adjusting the block size and blockchain transaction acceptance rate are as follows: when formula (2) is met, the transaction acceptance rate is increased; when formula (3) is met, the block size is adjusted. （2）； ) （3）； in, This refers to the transaction sending rate within a specific time interval. It is the average rate at which clients send transactions; It is the maximum size of a single transaction that a blockchain system can accept; It is the threshold for the configured blockchain transaction acceptance rate; It is the threshold for the configured block size; Step 5: Deploy the blockchain system with the best overall performance into a container. Each blockchain node corresponds to one container. Build a Docker container image of the blockchain system and use Kubernetes for container orchestration and container resource scheduling.

2. The hierarchical collaborative scheduling optimization method for a blockchain system based on transaction volume prediction according to claim 1, characterized in that, The performance metrics of the blockchain system in step 1 include the blockchain throughput and latency; the transaction information metrics include the blockchain transaction sending rate, the blockchain transaction receiving rate, the number of submitted transactions, and the blockchain block height; the resource metrics used by the blockchain system deployed in the container include CPU usage, memory usage, and network transmission bandwidth.

3. The hierarchical collaborative scheduling optimization method for a blockchain system based on transaction volume prediction according to claim 1, characterized in that, Step 3 specifically includes: Step 3.1: Clean the transaction volume data in the time series database InfluxDB to obtain cleaned transaction volume data; Step 3.2: Divide the cleaned transaction volume data into training sequences and test sequences according to time order; specifically, for each transaction volume data, set a division time point, use the transaction volume data of a set length before the division time point as the training sequence, and use the transaction volume data of a set length after the division time point as the test sequence. Step 3.3: Normalize the training and test sequences to obtain normalized training and test sequences; Step 3.4: Train the LSTM long short-term memory network model using the normalized training sequence. During the training process, the parameters of the model are continuously adjusted through the backpropagation algorithm to obtain the trained LSTM long short-term memory network model. Step 3.5: Validate the trained LSTM (Long Short-Term Memory) network model using normalized test sequences. Input the transaction volume data of a set length before the split time point in the normalized test sequences into the LSTM network model to obtain the prediction results for the transaction volume data of a set length after the split time point. Use the evaluation metrics MSE and R2 score to evaluate the prediction performance of the LSTM network model. Adjust the model parameters according to the evaluation results to obtain the final LSTM network model. Step 3.6: Obtain the transaction volume data of a set length before the current time and use the final LSTM Long Short-Term Memory network model to predict the transaction volume data of a set length after the current time.

4. The hierarchical collaborative scheduling optimization method for a blockchain system based on transaction volume prediction according to claim 1, characterized in that, Step 5 specifically includes: Step 5.1: Deploy the blockchain system into a container; Step 5.2: Collect resource usage metrics of the blockchain system deployed in each container, use Heapster to summarize the collected resource usage metrics of the blockchain system deployed in the container, and then import them into a third-party tool to visualize the resource usage metrics of the blockchain system deployed in the container using front-end visualization technology. Step 5.3: The resource requirements of the blockchain system are diverse and dynamic. Based on the resource usage indicators of the blockchain system deployed in the container, the ant colony algorithm in the multi-objective optimization algorithm is used to optimize resource scheduling. Different scheduling arrangements are made for the different CPU, memory and network resource requirements of each node in the blockchain system.

5. The hierarchical collaborative scheduling optimization method for a blockchain system based on transaction volume prediction according to claim 4, characterized in that, Step 5.3 specifically includes: Step 5.3.1: Based on the constraint information between nodes in the blockchain system and the available resource information of hosts in the Kubernetes cluster, construct a solution model for the scheduling problem; the Kubernetes cluster includes several hosts; The solution model for the scheduling problem is as follows: （4）； （5）； （6）； （7）； （8）； （9）； in, For an assignment matrix with binary values, To assign matrix elements in the matrix, if the first element... The container was assigned to the first... On each cluster host, then ,otherwise As shown in formula (4); For the container number, Indicates the host number in the cluster; This represents the number of hosts in the cluster. The number of containers; Not equal to The container number; For the resource requirements of containers, The table specifies the resource types, including CPU, memory, and network. A collection representing resource types; Indicates the resources available to the host; Step 5.3.2: Input the resource requirements of the blockchain system container, i.e., the resource indicators used by the blockchain system deployed in the container, the available resources of the host in the Kubernetes cluster, the scheduling problem solution model and constraints, into the ant colony algorithm solver to obtain the solution; Step 5.3.3: Input the solution obtained by the ant colony algorithm solver into the container deployer to generate the Kubernetes deployment file; Step 5.3.4 Unload the running container and reschedule the blockchain system container to the node according to the Kubernetes deployment file. Observe the status of the container and the node through the Kubernetes cluster management tool to ensure that it has been successfully scheduled to the node and is running normally. If it is running normally, end the scheduling. Otherwise, repeat step 5.2.4 until it is successfully scheduled to the node and running normally.

Images