Distributed computing synchronization method and system for data centers

Abstract

The application provides a data center-oriented distributed computing synchronization method and system, comprising the following steps: S1: a scheduling node generates and encrypts a signal packet tuple and sends it to all computing nodes; S2: after receiving the encrypted signal packet tuple, the computing node performs a signature verification operation, iteratively calculates according to the parameters in the signal packet tuple that passes the signature verification, and returns the result to the scheduling node; S3: the scheduling node estimates the computing capacity of each computing node according to the result returned by all computing nodes; S4: the scheduling node evaluates the comprehensive computing coordination capacity of each computing node according to the computing capacity of each computing node; S5: the scheduling node allocates computing tasks to each computing node according to the computing coordination capacity of each computing node for different tasks, and dynamically adjusts the computing coordination capacity. The method and system provided by the application are suitable for providing accurate and stable evaluation results for the actual computing capacity of the distributed computing nodes of the data center, helping to enhance the adaptability of the system and improve the allocation efficiency of the operation resources of the system.

Description

Technical Field

[0001] This invention belongs to the field of distributed computing and data processing, and specifically relates to a distributed computing synchronization method and system for data centers. Background Technology

[0002] With the increasing maturity of information technologies such as artificial intelligence, big data, and 5G, the Internet of Things (IoT) has moved from conception to marketization, bringing richer experiences and unprecedented economic development opportunities. One of the core underlying technologies supporting the development of IoT is relying on large data centers to provide real-time model calculation results for various terminal devices, enabling intelligent scheduling of massive numbers of devices. To maximize the computing power of data centers, it is necessary to research a distributed data computing and synchronization algorithm to ensure that the computing power and data processing capabilities of each node in the data center can be effectively utilized, thereby saving hardware investment costs and achieving energy conservation and emission reduction.

[0003] The commonly used distributed data computing method in the industry involves a scheduling node performing weighted calculations based on the hardware parameters of each computing node to evaluate a performance index. Computational tasks are then allocated according to this performance index, thus scheduling computation across nodes. The advantage of this method is its simple system topology, resulting in strong system stability. However, its disadvantages include poor dynamic adjustment capability. Hardware parameters do not fully reflect the actual computing power of a node, and it cannot be adjusted in real-time according to changes in the data center topology. This means that the computing power of some nodes cannot be fully utilized, especially when the data center's computing nodes contain various hardware devices and architectures with different performance levels, and when the system topology changes—a situation that frequently occurs during data center construction and operation. Therefore, there is a need to research a distributed computing and data synchronization method that can adjust the workload in real-time. Summary of the Invention

[0004] The purpose of this invention is to overcome the problem that the existing technology lacks dynamic adjustment capabilities and is difficult to fully utilize the computing power of some data centers, and to provide a distributed computing synchronization method for data centers.

[0005] To achieve the above-mentioned objectives, this method adopts the following technical solution:

[0006] A distributed computing synchronization method for data centers includes the following steps:

[0007] Step S1: The scheduling node generates a signal packet containing a time stamp locally, signs the signal packet and its additional information using its private key, generates an encrypted signal packet tuple, and sends the encrypted signal packet tuple to all computing nodes.

[0008] Step S2: After receiving the encrypted signal packet tuple, the computing node performs a signature verification operation, discards the signal packet tuples that fail the signature verification, performs iterative calculations on the signal packet tuples that pass the signature verification based on the parameters provided therein, and sends the calculation result tuple back to the scheduling node.

[0009] Step S3: The scheduling node waits for all computing nodes to return the computation result tuples and performs hash verification and signature verification operations on the data in each tuple. If the verification fails, the corresponding computing node is required to re-provide the computation result tuple until the verification passes. Then, the scheduling node evaluates the computing capabilities of each computing node based on the computation result tuples returned by each computing node, including the volatility of computing capabilities and the average computing time.

[0010] Step S4: The scheduling node evaluates the overall computing coordination capability of each computing node based on the average computing time of each computing node and the average computing time of all computing nodes.

[0011] Step S5: Upon receiving a computing task, the scheduling node allocates tasks with different computing volumes based on the computing coordination capabilities of each computing node, and updates its overall computing coordination capability based on the time taken for each node to complete its computing volume.

[0012] Further, in step S1, the encrypted signal packet tuple is:

[0013] (S,T,M,N,PUB,SIG)

[0014] Where S is the signal packet generated by the scheduling node, T is the timestamp of the signal packet, M is the initial computation difficulty value (default is a positive integer), N is the number of difficulty values ​​that each computing node needs to solve (default is a positive integer), PUB is the public key of the scheduling node, and SIG is the signature data generated by the scheduling node using the scheduling node's private key PRI.

[0015] Furthermore, in step S2, the iterative calculations performed by the computing nodes include the following steps:

[0016] Step S21: Randomly generate auxiliary data a, and calculate the tuple (a, p, j, t) i,j The hash value h j until a hash value h that meets specific conditions is found. j For example, h j ≤M / j, then the tuple (a,p,j,t) i,j ,h j Store the j-th element C[j] of array C, where i represents the node number (i∈[1,V]), V is the number of nodes (j∈[1,N-1]), j is the iteration variable (initialized to 1), p is the position variable (initialized to the signature SIG of the signal packet tuple), and ti,j To compute node i, find a h that meets the requirements j The local time is used, and each element of the result array C is initialized to empty;

[0017] Step S22: Assign the value h to p j Then increment the iteration variable j by 1 and repeat step S21 until all N hash values ​​that meet the requirements are calculated;

[0018] Step S23: Compute node i uses its private key pri i For tuples (C, pub) i ,sig i Perform a signature operation to generate signature data sig i And the calculated result tuple (C, pub) i ,sig i It is sent to the scheduling node.

[0019] Furthermore, in step S3, the time consumption between the (j+1)th calculation and the jth calculation of node i is u. i,j =t i,j+1 -t i,j The fluctuation of the computing power of node i is shown in the following equation:

[0020]

[0021] Among them, e i Let e ​​be the average computation time for node i. i =(t i,N -t i,1 ) / (N-1).

[0022] Furthermore, in step S4, the comprehensive computing coordination capability of computing node i is determined by the computing performance index k. i The judgment is as shown in the following formula:

[0023] k i =α / e i +β / w i +γ / |e i -B|

[0024] in, This represents the average computation time of all computing nodes, with α, β, γ > 0 being default constants.

[0025] Furthermore, in step S5, whenever a computing task is received, the scheduling node coordinates the computing resources k of each computing node. i Given the total computational workload X, dynamically allocate different computational workloads x to each computing node i. i :

[0026]

[0027] After each computing node i completes its task, the scheduling node determines the time d taken for that node to complete the task. i Average time across all nodes Update the computational coordination capability of node i, and update the computational performance index k of node i. i :

[0028]

[0029] in δ>0 is the default constant.

[0030] According to a second aspect of the present invention, a distributed computing synchronization system for data centers is provided, comprising:

[0031] Signal packet generation and transmission module: controls the scheduling node to generate signal packets containing time stamps, signs the signal packets and their additional information using the scheduling node's private key, generates encrypted signal packet tuples, and sends them to all computing nodes;

[0032] Signal packet receiving and processing module: controls the computing node to receive encrypted signal packet tuples, perform signature verification, and discard signal packet tuples that fail the signature verification. For signal packet tuples that pass the signature verification, the control computing node performs iterative calculations based on the parameters in the tuples, and then sends the calculation result tuples back to the scheduling node.

[0033] The result verification and capability assessment module controls the scheduling node to receive the computation result tuples returned by all computing nodes, performs hash verification and signature verification on these tuples, and controls the scheduling node to require computing nodes whose computation results have failed verification to resubmit the computation results until verification is successful. Then, the scheduling node assesses the computing capabilities of each computing node based on the returned computation result tuples, including the volatility of computing capabilities and the average computation time.

[0034] Computational Coordination Capability Evaluation Module: Controls the scheduling node to use the average computation time of each computing node and the average computation time of all computing nodes to evaluate and update the comprehensive computational coordination capability of each computing node.

[0035] Dynamic task allocation module: Controls the scheduling node to dynamically allocate tasks with different computational loads based on the computing coordination capabilities of each computing node, and updates its overall computing coordination capabilities based on the time taken for each computing node to complete the task after the task is completed.

[0036] This invention employs a randomized computation task to evaluate the actual computing power of distributed computing nodes in a data center, providing accurate and stable evaluation results. By dynamically adjusting the computing power, the strategy ensures that the computing power value converges to a stable state over time. This method not only enhances the system's adaptability but also improves the efficiency of resource allocation. Attached Figure Description

[0037] Figure 1 This is an overall flowchart of the distributed computing synchronization method and system for data centers provided by the present invention. Detailed Implementation

[0038] The present invention will be further described and illustrated below with reference to the accompanying drawings and specific embodiments.

[0039] This invention primarily constructs a distributed computing synchronization method and system for data centers. It provides accurate and stable evaluation results of the actual computing capabilities of distributed computing nodes in data centers, helping to enhance system adaptability and improve the efficiency of system computing resource allocation. The method steps are as follows: Figure 1 As shown. The specific steps are as follows:

[0040] Step S1: The scheduling node generates a signal packet S locally, records the current time T, and signs the tuple (S, T, M, N, PUB) using its private key PRI to generate signature data SIG. Finally, it sends the signal packet tuple to all computing nodes.

[0041] (S,T,M,N,PUB,SIG)

[0042] Where PUB is the public key of the scheduling node, M is the initial computation difficulty value, which is a positive integer by default, and N is the number of difficulty values ​​that each computing node needs to solve, which is a positive integer by default.

[0043] Step S2: When computation node i receives the signal packet tuple, i∈[1,V], where V is the number of computation nodes, it first performs a signature verification operation on the signal packet tuple. If the signature verification fails, the signal packet tuple is discarded; otherwise, the following iterative operation is performed: (the initial value of the iteration variable j=1, the initial value of the position variable p=SIG, and each element of the computation result array C is initialized to empty)

[0044] Step S21: Randomly generate auxiliary data a, and calculate the tuple (a, p, j, t) i,j The hash value h j until a value satisfying h is found j Hash value h ≤ M / j j Until then. Then process the tuple (a, p, j, t) i,j ,h j) is placed into the j-th element C[j] of array C. Where, t i,j To compute node i, find a h that meets the requirements j The local time.

[0045] Step S22: Assign the value h to p j Then increment by 1 and repeat step S21 until all N hash values ​​that meet the requirements are found.

[0046] Step S23: Compute node i uses its private key pri i For tuples (C, pub) i Perform a signature operation to generate signature data sig i And the calculated result tuple (C, pub) i ,sig i Send it to the scheduling node.

[0047] Step S3: The scheduling node calculates the single-machine computing performance of each computing node. The scheduling node waits for all computing nodes to return the computation result tuples and performs hash verification and signature verification operations on the data in each tuple. If the verification fails, the corresponding computing node is required to re-provide the computation result tuple until the verification passes. Then, the computing power of each computing node is calculated, including the fluctuation of computing power and the computation time. The time between the (j+1)th computation and the jth computation of computing node i is...

[0048] u i,j =t i,j+1 -t i,j j∈[1,N-1]

[0049] The fluctuation of the computing power of node i is

[0050]

[0051] Among them, e i Let e ​​be the average computation time for node i. i =(t i,N -t i,1 ) / (N-1).

[0052] Step S4: The scheduling node evaluates the overall computing coordination capability of each computing node. The computing performance index of computing node i is...

[0053] k i =α / e i +β / w i +γ / |e i -B|

[0054] in, This represents the average computation time across all computing nodes. α, β, γ > 0 are default constants.

[0055] Step S5: The scheduling node dynamically adjusts the computational coordination index of each computing node. Upon receiving a computational task, let its total computational load be X, then the computational load allocated to computing node i is...

[0056]

[0057] Let computing node i complete the computation amount x i The calculation time spent is d. i Then its updated computational performance index is

[0058]

[0059] in This represents the average computation time of all computing nodes in this computation task, with δ>0 being a default constant.

[0060] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A distributed computing synchronization method for data centers, characterized in that, Includes the following steps: Step S1: The scheduling node generates a signal packet containing a time stamp locally, signs the signal packet and its additional information using its private key, generates an encrypted signal packet tuple, and sends the encrypted signal packet tuple to all computing nodes. Step S2: After receiving the encrypted signal packet tuple, the computing node performs a signature verification operation, discards the signal packet tuples that fail the signature verification, performs iterative calculations on the signal packet tuples that pass the signature verification based on the parameters provided therein, and sends the calculation result tuple back to the scheduling node. Step S3: The scheduling node waits for all computing nodes to return the computation result tuples and performs hash verification and signature verification operations on the data in each tuple. If the verification fails, the corresponding computing node is required to re-provide the computation result tuple until the verification passes. Then, the scheduling node evaluates the computing capabilities of each computing node based on the computation result tuples returned by each computing node, including the volatility of computing capabilities and the average computing time. Step S4: The scheduling node evaluates the overall computing coordination capability of each computing node based on the average computing time of each computing node and the average computing time of all computing nodes. Step S5: Upon receiving a computing task, the scheduling node allocates tasks with varying computational loads based on the computing coordination capabilities of each computing node, and updates its overall computing coordination capability based on the time taken for each node to complete its computational load; In step S2, the iterative computation performed by the computing node includes the following steps: Step S21: Randomly generate auxiliary data Calculate tuples hash value until a hash value that meets specific conditions is found. Then the tuple Store in array The element ,in, Indicates the number of the compute node. , To calculate the number of nodes, , This is the iteration variable, initialized to 1. For the position variable, the initial value is set to the signature of the signal packet tuple. , For computing nodes After finding a suitable candidate The local time of the calculation result array Each element is initialized to empty; Step S22: Assigned value and the iteration variable Increment by 1, repeat step S21, until all the required values ​​are calculated. One hash value; Step S23: Calculate the node Use its private key pairs of tuples Perform a signature operation to generate signature data. and the calculated tuple Send to the scheduling node.

2. The distributed computing synchronization method for data centers according to claim 1, characterized in that, In step S1, the encrypted signal packet tuple is: in, Signal packets generated for the scheduling node, For the time stamp of the signal packet, This is the initial difficulty value for calculation. This represents the number of difficulty values ​​that each computing node needs to solve. The public key for the scheduling node. The scheduling node uses the scheduling node's private key. The generated signature data.

3. The distributed computing synchronization method for data centers according to claim 1, characterized in that, In step S3, the nodes are computed. The +1st calculation and the first The time consumed between calculations is ,node The fluctuation of computing power is shown in the following formula: in, For computing nodes The average computation time, .

4. The distributed computing synchronization method for data centers according to claim 3, characterized in that, In step S4, the computation node is calculated. The comprehensive computing coordination capability is expressed through the computing performance index. The judgment is as shown in the following formula: in, This represents the average computation time across all computing nodes. This is the default constant.

5. The distributed computing synchronization method for data centers according to claim 1, characterized in that, In step S5, whenever a computing task is received, the scheduling node coordinates the computing capabilities of each computing node. Total computational load of the task For each computing node Dynamically allocate different computational loads : Each compute node After completing the task, the scheduling node determines the time taken to complete the task. Average time across all nodes Update the computational coordination capabilities of the nodes, update the nodes Computational performance index : in , This is the default constant.

6. A distributed computing synchronization system for data centers, comprising: Signal packet generation and transmission module: controls the scheduling node to generate signal packets containing time stamps, signs the signal packets and their additional information using the scheduling node's private key, generates encrypted signal packet tuples, and sends them to all computing nodes; Signal packet receiving and processing module: controls the computing node to receive encrypted signal packet tuples, perform signature verification, and discard signal packet tuples that fail the signature verification. For signal packet tuples that pass the signature verification, the control computing node performs iterative calculations based on the parameters in the tuples, and then sends the calculation result tuples back to the scheduling node. The result verification and capability assessment module controls the scheduling node to receive the computation result tuples returned by all computing nodes, performs hash verification and signature verification on these tuples, and controls the scheduling node to require computing nodes whose computation results have failed verification to resubmit the computation results until verification is successful. Then, the scheduling node assesses the computing capabilities of each computing node based on the returned computation result tuples, including the volatility of computing capabilities and the average computation time. Computational Coordination Capability Evaluation Module: Controls the scheduling node to use the average computation time of each computing node and the average computation time of all computing nodes to evaluate and update the comprehensive computational coordination capability of each computing node. Dynamic task allocation module: Controls the scheduling node to dynamically allocate tasks with different computational loads based on the computational coordination capabilities of each computing node, and updates the overall computational coordination capability of each computing node based on the time taken to complete the task after the task is completed. The iterative calculations performed by the control computing node of the signal packet receiving and processing module include the following steps: Step S21: Randomly generate auxiliary data Calculate tuples hash value until a hash value that meets specific conditions is found. Then the tuple Store in array The element ,in, Indicates the number of the compute node. , To calculate the number of nodes, , This is the iteration variable, initialized to 1. For the position variable, the initial value is set to the signature of the signal packet tuple. , For computing nodes After finding a suitable candidate The local time of the calculation result array Each element is initialized to empty; Step S22: Assigned value and the iteration variable Increment by 1, repeat step S21, until all the required values ​​are calculated. One hash value; Step S23: Calculate the node Use its private key pairs of tuples Perform a signature operation to generate signature data. and the calculated tuple Send to the scheduling node.

Images