A method and system for load balancing of coding computation in heterogeneous multi-hop network topologies

Abstract

The present invention provides a coding calculation load balancing distribution method and system under a heterogeneous multi-hop network topology, the coding calculation load balancing distribution method comprising the following steps: placement step: matrix multiplication task A is divided into N sub-matrix tasks A1, A2, ..., A according to the coding calculation strategy N , assigned to N working nodes; calculation steps: for matrix-vector multiplication, a subtask is specified by its corresponding submatrix, and the main working node collects the subtask results to obtain Ax. The completion efficiency of the calculation task is directly converted into the tolerance for the impact of any downtime working node, the distance between the working node and the main node, and the reduction of the calculation delay; transmission step: the working node transmits the calculation data to the next working node, and the last working node transmits the calculation data to the main node. The beneficial effects of the present invention: the coding calculation load balancing distribution method realizes the minimization of the total processing delay of the multi-hop network system for the matrix-vector multiplication task.

Description

Technical Field

[0001] The present invention relates to the field of big data, and in particular to a method and system for balancing and allocating coding calculation loads under a heterogeneous multi-hop network topology. Background Art

[0002] With the generation of continuous data streams, a single computing node is no longer able to handle large-scale computing tasks alone. In recent years, distributed computing has been used for large-scale computing because it has several advantages over centralized computing. Distributed computing refers to a group of computing nodes that jointly solve computing tasks as a single system by sharing their network and storage resources. First, distributed computing is able to provide computing services with high reliability and fault tolerance. Second, distributed computing has a high computing speed because the computing load is shared among different computing nodes. Third, distributed computing systems are scalable because computing nodes can be easily added. Fourth, distributed computing is economical to use computing nodes with low-cost hardware. Similarly, distributed computing has been adopted in cloud computing and other emerging services. Given the above advantages, distributed computing has been applied to many real-world applications, such as telecommunication networks (e.g., telephone networks and wireless sensor networks), network applications (e.g., global networks, massively multiplayer online games and virtual reality communities, distributed database management systems and network file systems), real-time process control (e.g., aircraft control systems), and parallel computing (e.g., cluster computing, grid computing, and computer graphics).

[0003] Distributed computing has become mainstream in many computing platforms (e.g., Hadoop) to support today's growing demand for processing large-scale data and computing workloads. In addition, its inherent scalability, which means that users can easily add worker nodes to the system as computing needs grow, has led to distributed computing becoming an emerging de facto standard in many modern computing architectures. More specifically, in distributed computing, a large task is divided into several subtasks, which are then assigned to machines that may be physically separated. However, some downtime machines (stragglers) from various sources of latency, including transient and permanent failures, can potentially limit the performance of distributed computing systems, as commodity computing nodes are often deployed to improve cost-effectiveness. To address this issue, the concept of Coded Computation is introduced, which leverages redundant sub-computations to speed up distributed matrix-vector multiplication.

[0004] Existing coding computation work mainly focuses on wired networks without considering the topology. Therefore, the communication delay caused by the computation results and the topology and returned by the network is ignored, which seriously affects the total delay of the system. For wireless multi-hop network topology, due to the time-varying characteristics of the wireless link, its topology and communication delay cannot be ignored.

[0005] However, due to the changes in the topology and the differences in the computing power of each working node, the performance of the system will encounter the bottleneck of topology and computing power limitations. Specifically, there are two main reasons: on the one hand, the working nodes are far away and the data flow is transmitted through multiple layers, resulting in huge transmission energy consumption. On the other hand, the heterogeneity of the entire topology system and unreasonable resource deployment will lead to unnecessary delay overhead. Summary of the invention

[0006] The present invention provides a method for balancing and allocating coding calculation loads in a heterogeneous multi-hop network topology, comprising the following steps:

[0007] Placement step: Matrix multiplication task A is divided into N subtasks A1, A2, …, A according to the encoding calculation strategy. N , assigned to N working nodes.

[0008] Computational steps: For matrix-vector multiplication, a subtask is specified by its corresponding submatrix; the master worker node collects the subtask results to obtain Ax. The efficiency of completing the computational task directly translates into the tolerance to the impact of a down worker node, the distance between the worker node and the master node, and the reduction of computational latency.

[0009] Transmission step: The worker node transmits the computation data to the next worker node, and the last worker node transmits the computation data to the master node.

[0010] As a further improvement of the present invention, in the coding calculation load balancing distribution method, A is divided into k sub-matrices of equal size, and an (N, k) MDS code is applied to these sub-matrices to generate A1, A2, ..., A N subtasks, and then perform the following steps:

[0011] Step 1: For each K, calculate represents the average total delay of node n based on (N,k)MDS.

[0012] Step 2: Sort and get the average delay after sorting

[0013] Step 3: Minimize and get the optimal number of sub-matrices k * .

[0014] As a further improvement of the present invention, in step 1,

[0015] Among them, a n is greater than 0, and is the shift parameter in the shifted exponential distribution; μ n is greater than 0, which is the downtime parameter of the nth working node; l n and l i is the number of rows of the calculation submatrix assigned to the nth and ith working nodes; v is the speed of light for electromagnetic wave transmission. If it is a terrestrial network, the second term in the formula can be ignored, but it cannot be ignored for a satellite network; d is the distance between two adjacent nodes; b is the number of bits contained in each row of data in the calculation matrix, R i is the data transmission rate of the i-th working node.

[0016] In step 3, the optimal number of matrix blocks is obtained.

[0017] Among them, K th represents the Kth value of the matrix block, Represents the total delay of the MDS code calculation of the nth computing node.

[0018] As a further improvement of the present invention, the coding calculation load balancing distribution method adopts a delay optimization algorithm LA-RC based on a rateless code, and the delay optimization algorithm LA-RC performs the following steps:

[0019] Step A: The problem of minimizing the total delay Decomposed into two sub-problems: Obtaining the optimal load problem with a fixed delay and the minimum delay for successful decoding

[0020] Step B: For a fixed feasible time T RC , solve the problem

[0021] Step C: Obtaining the Optimal Load That is, the number of rows assigned to the nth worker node.

[0022] Step D: Solve the problem get That is the optimal feasible time.

[0023] As a further improvement of the present invention,

[0024] in is the expected number of sub-matrix-vector multiplication equations that can be aggregated at the master node r is the minimum number of rows required for successful decoding;

[0025] The expected number of equations aggregated at the master node can be expressed as:

[0026]

[0027] Since there is no constraint on load distribution, It can be decomposed into N decoupled optimization problems, namely: For all n∈[N] working nodes, the solution satisfies the following optimality conditions:

[0028]

[0029] This formula obeys: λ n is a positive solution to the following equation:

[0030] use express The solution of LA-RC load sharing is defined as: The optimal load is obtained at The expected total return at is:

[0031]

[0032] The present invention also discloses a coding calculation load balancing distribution system under a heterogeneous multi-hop network topology, comprising:

[0033] Placement module: used to divide the matrix multiplication task A into N subtasks A1, A2, …, A according to the coding calculation strategy N , assigned to N working nodes.

[0034] Computation module: A subtask is specified by its associated message. For matrix-vector multiplication, a subtask is specified by its corresponding submatrix. The master node collects enough subtask results to obtain Ax. The efficiency of completing the computing task is directly converted into the tolerance to the impact of a downtime worker node, the distance between the worker node and the master node, and the reduction of computing latency.

[0035] Transmission module: The working node transmits the calculated data to the next working node, and the last working node transmits the calculated data to the master node.

[0036] As a further improvement of the present invention, in the coding calculation load balancing distribution system, A is divided into k sub-matrices of equal size, and a (N, k) MDS code is applied to these sub-matrices, and then the following modules are run:

[0037] Calculation module: For each K, calculate represents the average total delay of node n based on (N,k)MDS.

[0038] Sorting module: Sort and get the average delay after sorting

[0039] Processing module: Minimize and get the optimal number of sub-matrices k * .

[0040] As a further improvement of the present invention, in the calculation module,

[0041] Among them, a n is greater than 0, and is the shift parameter in the shifted exponential distribution; μ n is greater than 0, which is the downtime parameter of the nth working node; l n and l i is the number of rows of the calculation submatrix assigned to the nth and ith working nodes; v is the speed of light for electromagnetic wave transmission. If it is a terrestrial network, the second term in the formula can be ignored, but it cannot be ignored for a satellite network; d is the distance between two adjacent nodes; b is the number of bits contained in each row of data in the calculation matrix, R i is the data transmission rate of the i-th working node;

[0042] In the processing module, obtain the optimal number of matrix blocks

[0043] Among them, K th represents the Kth value of the matrix block, Represents the total delay of the MDS code calculation of the nth computing node.

[0044] As a further improvement of the present invention, the coding calculation load balancing distribution system adopts the delay optimization algorithm LA-RC based on the rateless code, and the delay optimization algorithm LA-RC runs the following units:

[0045] Decomposition unit: for the problem of minimizing the total delay Decomposed into two sub-problems: Obtaining the optimal load problem with a fixed delay and the minimum delay for successful decoding

[0046] The first processing unit: For a fixed feasible time T RC , solve the problem

[0047] Second processing unit: Get the optimal load That is, the number of rows assigned to the nth worker node.

[0048] The third processing unit: problem solving get That is the optimal feasible time.

[0049] As a further improvement of the present invention, in is the expected number of sub-matrix-vector multiplication equations that can be aggregated at the master node; r is the minimum number of rows required for successful decoding;

[0050] The expected number of equations aggregated at the master node can be expressed as:

[0051]

[0052] Since there is no constraint on load distribution, It can be decomposed into N decoupled optimization problems, namely: For all n∈[N] working nodes, the solution satisfies the following optimality conditions:

[0053]

[0054] This formula obeys: λ n is a positive solution to the following equation:

[0055] use express The solution of LA-RC load sharing is defined as: The optimal load is obtained at The expected total return at is:

[0056]

[0057] The beneficial effects of the present invention are: 1. The coding calculation load balancing distribution method of the present invention takes into account the heterogeneous characteristics of the system, combines the coding calculation technology, and jointly optimizes the load and coding parameters of the computing nodes to achieve minimization of the total processing delay of the multi-hop network system for matrix-vector multiplication tasks; 2. The coding calculation load balancing distribution method of the present invention better improves the computing speed of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a coding calculation system model diagram of matrix-vector multiplication under heterogeneous multi-hop network topology of the present invention;

[0059] Figure 2 The invention determines k based on MDS allocation * Algorithm flow chart;

[0060] Figure 3is a flow chart of the rate-free coding load distribution algorithm of the present invention;

[0061] Figure 4 is a simulation diagram of the average total delay of the present invention where k varies with n;

[0062] Figure 5 is the delay performance gain graph of MDS code load distribution and rateless code load distribution on two exponential delay model benchmark test schemes;

[0063] Figure 6 This is a graph showing the variation of the average total delay with the number of nodes under the four schemes. DETAILED DESCRIPTION

[0064] The present invention discloses a method for balancing the coding calculation load distribution under a heterogeneous multi-hop network topology. First, a system model in which the topology and the coding scheme are integrated is established. Compared with the time phase and transmission phase proposed in the previous classical model, this model also includes a propagation phase. Then, based on this model, the total delay formula for task completion is derived and an optimization problem is proposed. Subsequently, a load distribution algorithm based on MDS code and rateless code is proposed to minimize the total delay of system tasks.

[0065] The present invention considers a coding computing system based on coding distributed computing (CDC), including a master worker and N coding computing systems. Figure 1 As shown in Figure 1, all worker nodes use one-way transport and are connected to each other via wireless links. The set of worker nodes is represented by The propagation distance and speed between working nodes are d and v respectively.

[0066] The purpose of the master task is to complete the task with the help of N worker nodes, but the master task needs to obtain the final result. For example, multiplying a matrix With vector The system workflow is as follows: 1) In the placement phase, task A is first divided into N subtasks A1, A2, …, A according to a certain coding calculation strategy. N , distributed to N worker nodes. 2) In the calculation phase, a subtask is specified by its associated message. For matrix-vector multiplication, a subtask is specified by its corresponding submatrix. The master node collects enough subtask results to obtain Ax. The completion efficiency of the calculation task directly translates into the tolerance to the impact of a down worker node, the distance between the worker node and the master node, and the reduction of the calculation delay. 3) In the transmission phase, the worker node transmits the calculation data to the next worker node, and the last worker node transmits the calculation data to the master node.

[0067] The coding calculation load balancing distribution method of the present invention is used represents the random variable representing the computation time of the task at worker n, and assumes that the computation time are independent of each other. The coding calculation load balancing distribution method of the present invention considers that the distribution of the calculation time random variable is exponentially distributed. More specifically, consider a 2-parameter shift exponential distribution of the execution time of each working node, that is, the CDF of the execution time of working node n, for and n∈[N], where a n >0 is the transfer parameter, μ n >0 indicates the distributed parameter related to the working node n represents the number of rows assigned to a worker node. The shift exponential model of computation time is the sum of a constant (deterministic) term and a variable (random) term, and is driven by the distribution model of data file query latency in cloud storage systems proposed by Liang and Kozat.

[0068] Once each worker completes its computation assignment, it will immediately propagate the computation result to the master server through the multi-hop network. represents the task propagation time and propagation time at node n The variable is The latency of job n to offload its computational tasks The calculation is as follows: Where b represents the data size of the inner product of size m, R1, R2, …, R N represents the achievable data rate associated with worker node n.

[0069] From the above two models, we can conclude that the total latency of a single worker includes computation time, propagation time, and transmission time:

[0070] The coding calculation load balancing distribution method of the present invention considers the problem of using N working clusters to distribute the calculation of matrix-vector multiplication Ax, where A is a matrix of size r×m. is the load distribution vector. Let T TOT is a random variable, representing the waiting time for receiving a set of decodable results, that is, there are at least r inner products. The goal of the coding calculation load balancing distribution method of the present invention is to find the optimal load distribution vector to minimize the average waiting time by solving the following optimization problem:

[0071]

[0072] In a heterogeneous multi-hop topology, due to the different service ranges of computing nodes, the node computing resources have uneven busy and idle state characteristics, resulting in unpredictable Straggler phenomena in onboard computing devices. The coding computing load balancing distribution method of the present invention intends to use the method of providing redundant information by coding computing to reduce the impact of slow nodes on the computing performance of the system. The coding computing principle will not be repeated here. Common coding computing schemes (such as replication and MDS codes) all provide computing redundancy through error correction codes to reduce the impact of slow nodes, but the generation of coding redundancy also means an increase in computing load. Rateless fountain codes solve this problem well. LT codes are a type of rateless fountain code that is easy to implement. Not only does it have a relatively small additional computing load, it also has low decoding complexity and a computing delay close to an ideal load balancing state.

[0073] For heterogeneous clusters, in order to implement a coding solution, A can be divided into k sub-matrices of equal size and a (N, k) MDS code is applied to these sub-matrices. Then, the master node can obtain the final result from any k responses. Lee et al. found the optimal k for the average running time of the average running time of the shift exponential runtime model. In addition, it is difficult to directly find the optimal k for the coding calculation load balancing distribution method model of the present invention. Let T MDS =T TOT The goal of the coding calculation load balancing distribution method of the present invention is to find the optimal k by solving the following optimization problem: First, for a fixed feasible value k, we can identify the rows of the encoded data matrix assigned to worker node n. At the same time, load The row vector of worker nodes n, The CDF of the execution time is as follows:

[0074]

[0075] The average total delay of node n based on (N,k) MDS is:

[0076]

[0077] The problems that need to be solved are as follows:

[0078] For multi-hop topology networks of wireless heterogeneous computing, the traversal method can be used to determine the minimum average total delay and the optimal recovery threshold. Figure 2 shows the determination of k based on MDS allocation *The sorting algorithm of LA-MDS is quick sort, and its average time complexity is O(nlog2(n)). It can be seen that LA-MDS is different from the traditional optimal load distribution method of coded distributed computing using heterogeneous worker nodes, which does not consider transmission capacity and hop count.

[0079] In order to derive the delay optimization algorithm based on rateless code (LA-RC), we first reformulate the definition of It is shown that the substitution formula can be solved efficiently, requiring an exhaustive search over all possible recovery thresholds. The solution to the substitution problem yields Figure 3 LA-RC shown. Let T RC =T TOT The goal of the coding calculation load balancing distribution method of the present invention is to find the optimal load balance by solving the following optimization problem:

[0080] The total reward of the master node at this moment is: in Since LA-RC utilizes the rateless characteristic of LT codes, LA-RC does not consider the transmission time.

[0081] For non-systematic LT codes, decoding uses an iterative peeling decoder, that is, in each iteration, the decoder finds a coded symbol, overwrites the corresponding source symbol, and then removes the symbol from all other coded symbols connected to the source symbol, and repeats the operation.

[0082] The coding calculation load balancing distribution method of the present invention proposes a definition The following two-step substitution formula of the main variable. First, for a fixed feasible time T RC , maximize the total reward of different load allocations, that is, solve Then, given from The load distribution obtained

[0083] match Find the minimum time T RC , so that with a higher probability, at the main time T CMP There is enough total reward, that is, to solve:

[0084]

[0085] Considering the exponential distribution of worker node runtime, we first continue to solve The expected number of equations aggregated at the master node is:

[0086]

[0087] Since there is no constraint on load distribution, It can be decomposed into N decoupled optimization problems, namely: For all n∈[N] working nodes, the solution satisfies the following optimality conditions:

[0088]

[0089] This formula obeys: λ n is a positive solution to the following equation:

[0090] In addition, express Now, the LA-RC load sharing is defined as: The optimal load is obtained at The expected total return at is:

[0091]

[0092] First, we analyze the regularity of the optimal recovery threshold for different hop counts, and then compare the transmission parameter R n and calculation parameters (a n ,μ n ) on the average total delay, and finally evaluate the performance of the proposed LA-MDS and LA-RC with other strategies. The parameter settings used in the simulation part are shown in Table I.

[0093] Table I: Simulation parameters

[0094]

[0095]

[0096] exist Figure 4 In the simulation, the parameters are set to N = (N1, N2, N3, N4, N5) = (20, 40, 60, 80, 100), and it is found that k = (18, 36, 53, 69, 85) under LA-MDS achieves the lowest expected delay. In order to obtain better experimental results, the optimal k in the following simulation parameters is selected. The symbol ○ represents the optimal k value.

[0097] Numerical results are given to evaluate the performance of LA-MDS and LA-RC. In the simulation of the coding computation load balancing distribution method of the present invention, a shift exponential model is considered, assuming units of seconds per row (seconds / row) for a and 1 / μ. The underlying computational task is to compute r=10000 inner products using a heterogeneous cluster of N=100 workers, where different scenarios of heterogeneity are considered. For each scenario considered, the following load distribution scheme is implemented:

[0098] 1) Uniform Uncoded Allocation: Each worker node is allocated an equal number of rows, that is, for each worker node n, the load l n =r / N.

[0099] 2) Non-Uniform Uncoded Allocation: The load allocated to each worker node is proportional to the expected time to calculate an inner product, that is, for each worker node n, l n ∝nμ n R n / (a n μ n R n +R n +bμ n ). In addition, set

[0100] 3) LA-MDS: Assign the same number of coding rows to each worker node, i.e., l n = r / k. The optimal k can be found by minimizing the average computation time of receiving at least r inner product results on the master node.

[0101] 4) LA-RC: Assign asymptotically optimal load to each working node, i.e.

[0102] For simulations under the shifted exponential model, two cases (random heterogeneity) in Table I are considered. Figure 2-5 The performance comparison of the four schemes for the exponential runtime model is described. The coding calculation load balancing distribution method of the present invention draws the following conclusions from the research results:

[0103] LA-RC significantly outperforms both baseline load sharing schemes. For scenario 1, LA-RC is 61% faster than uncoded and 56% faster than load-balanced uncoded. In particular, for scenario 2, LA-RC achieves an 88% gain, a 78% increase over uncoded and load-balanced uncoded.

[0104] For scenario 1, when the computation time accounts for a higher proportion, LA-MDS outperforms load balancing without coding. Therefore, LA-MDS can effectively reduce the latency caused by computation. However, for scenario 2, when the transmission time accounts for a larger proportion, load balancing without coding outperforms LA-MDS.

[0105] Both load-balanced uncoded and LA-MDS improve the performance of the uncoded scheme. In the load-balanced uncoded scheme, distributing a larger load to faster machines leads to better performance, while for the LA-MDS scheme, (N,k)MDS leads to better performance because the master does not need to wait for all the results. LA-RC provides the best expected execution time among the four schemes because it combines the advantages of load-balanced uncoded and uniform coding by using effective load balancing and a minimum number of redundant computations.

[0106] The coding computation load balancing distribution method of the present invention will give numerical results for evaluating the impact of hop count (number of nodes). For convenience, the parameters of scenario 1 in the previous section will be considered.

[0107] like Figure 6 As shown in Figure 2, the average total time of the four schemes decreases as the number of nodes increases. Figure 6 There are two ○s worth noting. For the first ○, when the number of working nodes is less than about 18 people, the performance of uncoded is slightly better than LA-MDS. For the second ○, when the number of working nodes is greater than 74, load balancing uncoded is not better than uncoded. As the number of working nodes increases, the average total time of the four schemes tends to a fixed value. Therefore, continuously increasing the number of workers does not reduce the system latency too much. This is of great significance for the encoding calculation load balancing distribution method of the present invention to select an appropriate number of nodes in practice to avoid wasting resources.

[0108] The coding calculation load balancing distribution method of the present invention takes into account the heterogeneous characteristics of the system, combines the coding calculation technology, jointly optimizes the load and coding parameters of the computing nodes, and minimizes the total delay of the system:

[0109] 1. By studying matrix-vector multiplication on heterogeneous multi-hop topology and combining it with the development of coding computing technology, a joint optimization method for computing node load and coding parameters is proposed to better improve the computing speed of the system.

[0110] 2. Using MDS code and rateless code, the total delay formula under heterogeneous multi-hop topology is established, and the concept of optimal recovery threshold is proposed to represent the optimal load of computing nodes in heterogeneous multi-hop topology. A load optimization algorithm based on MDS code and rateless code is proposed, which verifies the effectiveness of these two algorithms in increasing the system computing speed in a system composed of various computing nodes in heterogeneous multi-hop topology.

[0111] The beneficial effects of the present invention are: 1. The coding calculation load balancing distribution method of the present invention takes into account the heterogeneous characteristics of the system, combines the coding calculation technology, and jointly optimizes the load and coding parameters of the computing nodes to achieve minimization of the total processing delay of the multi-hop network system for matrix-vector multiplication tasks; 2. The coding calculation load balancing distribution method of the present invention better improves the computing speed of the system.

[0112] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for balancing the coding computation load under a heterogeneous multi-hop network topology, characterized in that: The steps include: Placement step: Matrix multiplication task A is divided into N subtasks A1, A2, …, A according to the encoding calculation strategy. N , assigned to N working nodes; Computation steps: For matrix-vector multiplication, a subtask is specified by its corresponding submatrix; the master node collects the subtask results to obtain Ax, which is a matrix With vector ) Transmission step: The working node transmits the calculated data to the next working node, and the last working node transmits the calculated data to the master node; The coding calculation load balancing distribution method adopts the delay optimization algorithm LA-RC based on the rateless code, and the delay optimization algorithm LA-RC performs the following steps: Step A: The problem of minimizing the total delay Decomposed into two sub-problems: Obtaining the optimal load problem with a fixed delay and the minimum delay for successful decoding Step B: For a fixed feasible time T RC , solve the problem Step C: Obtain the optimal load vector That is, the number of rows assigned to the nth working node; Step D: Solve the problem get That is, the optimal feasible time; in is the expected number of sub-matrix-vector multiplication equations that can be aggregated at the master node; r is the minimum number of matrix rows required for successful decoding, also called the return; The expected number of equations aggregated at the master node is expressed as: Among them, μ n is greater than 0, which is the downtime parameter of the nth working node; l n is the number of rows of the submatrix that the nth worker node is assigned to calculate; a n is greater than 0, and is the shift parameter in the shifted exponential distribution; A random variable representing the computation time of the task at worker n; Since there is no constraint on load distribution, Decomposed into N decoupled optimization problems, namely: For all n∈[N] working nodes, the solution satisfies the following optimality conditions: This formula obeys: λ n is a positive solution to the following equation: use express The solution of LA-RC load sharing is defined as: The optimal load is obtained at The expected total return at is: in A variable representing the task propagation time at node n.

2. A coding computing load balancing distribution system under heterogeneous multi-hop network topology, characterized in that: include: Placement module: used for matrix multiplication to divide task A into N subtasks A1, A2, ..., A according to the coding calculation strategy N , assigned to N working nodes; Computation module: For matrix-vector multiplication, a subtask is specified by its corresponding submatrix; the master node collects the subtask results to obtain Ax, which is a matrix With vector ) Transmission module: The working node transmits the calculated data to the next working node, and the last working node transmits the calculated data to the master node; The coding calculation load balancing distribution system adopts the delay optimization algorithm LA-RC based on rateless code. The delay optimization algorithm LA-RC runs the following units: Decomposition unit: for the problem of minimizing the total delay Decomposed into two sub-problems: Obtaining the optimal load problem with a fixed delay and the minimum delay for successful decoding The first processing unit: For a fixed feasible time T RC , solve the problem Second processing unit: Get the optimal load That is, the number of rows assigned to the nth working node; The third processing unit: problem solving get That is, the optimal feasible time; in is the expected number of sub-matrix-vector multiplication equations that can be aggregated at the master node; r is the minimum number of matrix rows required for successful decoding, also called the return; The expected number of equations aggregated at the master node is expressed as: Among them, μ n is greater than 0, which is the downtime parameter of the nth working node; l n It is nth The number of rows of the submatrix that are assigned to the worker nodes; a n is greater than 0, and is the shift parameter in the shifted exponential distribution; A random variable representing the computation time of the task at worker n; Since there is no constraint on load distribution, Decomposed into N decoupled optimization problems, namely: For all n∈[N] working nodes, the solution satisfies the following optimality conditions: This formula obeys: λ n is a positive solution to the following equation: use express The solution of LA-RC load sharing is defined as: The optimal load is obtained at The expected total return at is: in A variable representing the task propagation time at node n.

Images