Deadline sensing load distribution method and system for heterogeneous coding distributed computing

By using MDS encoding and Newton iterative method to optimize load allocation in heterogeneous nodes in distributed computing systems for heterogeneous nodes, the load allocation problem in heterogeneous nodes is solved, and the task success rate and system performance optimization are achieved.

CN119045996APending Publication Date: 2024-11-29HOHAI UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411091468.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

In large-scale distributed computing systems composed of heterogeneous nodes, existing load allocation schemes are difficult to effectively allocate workloads within the task completion time period, resulting in high probability of task failure and high system tail delay and encoding redundancy, especially It is in real-time tasks with strict deadline requirements.

Method used

The calculation matrix is ​​precoding using maximum distance-dividable MDS encoding, and a period-aware load allocation problem under a typical running time model is established. By approximating the success probability of the task based on the central limit theorem, the original load allocation problem is transformed into non-convex multivariable optimization. Problem, the iterative algorithm of Newton's iterative method is used to solve the optimal load allocation scheme.

Benefits of technology

Differentiated load allocation is realized based on node computing performance, maximizing the probability of success of computing tasks, while reducing the system tail delay and encoding redundancy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119045996A_ABST
    Figure CN119045996A_ABST
Patent Text Reader

Abstract

The invention discloses a deadline sensing load distribution method and system for heterogeneous coding distributed computing, and the method comprises the steps: carrying out the precoding and deployment of a calculation matrix through MDS coding for a matrix-vector multiplication task of a distributed computing system composed of a main node and a group of n working nodes, establishing a deadline perception load distribution problem under the typical operation time model to minimize the task failure probability; a failure probability is approximated by utilizing a simple and effective normal approximation method through a central limit theorem, and an original problem is converted into a non-convex multivariate optimization problem; and solving an optimization problem through an efficient iterative algorithm based on a Newton iteration method to obtain an optimal load distribution scheme. According to the method, differentiated load distribution can be carried out according to the calculation performance of the nodes, so that the node potential can be exerted to a greater extent, and the requirement that the calculation task can be completed within a certain time with a large probability is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of distributed computing, and in particular to a deadline-aware load distribution method and system for heterogeneous coding distributed computing. Background Art

[0002] Distributed computing, which leverages the concurrent processing capabilities of multiple computing nodes, has become a mainstream approach for large-scale machine learning and big data analysis. However, the performance of these large-scale distributed systems is significantly affected by the lag of individual computing nodes. Specifically, the running time of computing nodes tends to fluctuate significantly due to various factors such as changes in the network environment, resource sharing and competition, software and hardware failures, and uneven workloads. This variation can significantly extend the overall computing time of distributed computing tasks.

[0003] To mitigate the impact of stragglers, encoded computation has become an effective approach, where the encoded subtasks are executed in parallel on different nodes and the desired result can be recovered by decoding only from the results of some of the subtasks. Many efficient encoding schemes have been proposed for various distributed computing tasks, such as matrix multiplication, gradient descent, data shuffling, secure and private computing, etc.

[0004] For optimizing the performance of coded distributed computing, such as average latency and task failure probability, an important issue is how to properly distribute the workload of each node before computing. In the case of homogeneous computing nodes (worker nodes), a uniform load distribution scheme, i.e., assigning the same workload to each node, is generally considered to be an effective load distribution strategy. The researchers analyzed the optimal workload of each node based on the characteristics of different computing tasks and the runtime model of computing nodes.

[0005] However, large-scale distributed computing systems are usually composed of heterogeneous nodes with different computing capabilities. For this situation, uniform load distribution schemes are far from optimal. In recent years, a variety of effective load distribution schemes have been proposed for coding computations on heterogeneous clusters. One approach to load distribution is to assign enough workloads to each computing node, which makes it difficult for it to return the computation results within the task completion time limit and consumes too much storage resources. On the other hand, if too few workloads are deployed on the computing nodes, they may complete the computational tasks too early and remain idle, affecting the overall task completion time. However, this type of work mainly focuses on minimizing the expected delay. For real-time tasks with strict deadline requirements, such as distributed path planning, entertainment, and Internet of Things (IoT) applications, these schemes often lead to a high probability of task failure. Summary of the invention

[0006] Purpose of the invention: In order to address the deficiencies in existing solutions, the present invention provides a deadline-aware load distribution method and system for heterogeneous coding distributed computing, which can perform differentiated load distribution according to the computing performance of the nodes, so that the potential of the nodes can be brought into play to a greater extent and ensure that the computing tasks can be completed within a certain time with a high probability.

[0007] Technical solution: In order to achieve the above invention objectives, the technical solution of the present invention is as follows:

[0008] A deadline-aware load distribution method for heterogeneous coding distributed computing includes the following steps:

[0009] For the matrix-vector multiplication task of a distributed computing system consisting of a master node and a group of n worker nodes, the maximum distance separable MDS coding is used to pre-code and deploy the computation matrix, and the deadline-aware load distribution problem under a typical runtime model is established, where p i (l i ) represents the probability that the worker node i returns the local result to the master node within the task deadline. i (l i ) establishes a task success probability model P(l), and the load distribution problem is solved by optimizing the load distribution scheme l = {l1,l2,…,l n} to minimize the overall task failure probability 1-P(l), where l i represents the workload assigned to worker node i;

[0010] Based on the central limit theorem, the task success probability is approximated and the original load distribution problem is transformed into a non-convex multivariable optimization problem.

[0011] An iterative algorithm based on Newton iteration method is used to solve the optimization problem and obtain the optimal load distribution plan. Tasks are assigned to working nodes according to the optimal load distribution plan.

[0012] The present invention also provides a deadline-aware load distribution system for heterogeneous coding distributed computing, comprising:

[0013] The load distribution problem establishment module is used to pre-code and deploy the calculation matrix for the matrix-vector multiplication task of a distributed computing system consisting of a master node and a group of n worker nodes using maximum distance separable MDS coding, and establish the deadline-aware load distribution problem under the typical running time model, where p i (l i ) represents the probability that the worker node i returns the local result to the master node within the task deadline. i (l i) establishes a task success probability model P(l), and the load distribution problem is solved by optimizing the load distribution scheme l = {l1,l2,…,l n} to minimize the overall task failure probability 1-P(l), where l i represents the workload assigned to worker node i;

[0014] The problem conversion module is used to approximate the task success probability based on the central limit theorem and convert the original load distribution problem into a non-convex multivariable optimization problem;

[0015] The problem solving and application module is used to solve the optimization problem using an iterative algorithm based on the Newton iteration method, obtain the optimal load distribution plan, and allocate tasks to the working nodes according to the optimal load distribution plan.

[0016] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the deadline-aware load distribution method for heterogeneous coding distributed computing as described above are implemented.

[0017] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the deadline-aware load distribution method for heterogeneous coding distributed computing as described above are implemented.

[0018] Compared with the prior art, the present invention has the following advantages and beneficial effects: providing a deadline-aware load distribution method for heterogeneous coding distributed computing, transforming the optimization problem into a non-convex multivariable optimization problem by establishing a deadline-aware load distribution problem under a typical runtime model and an approximate processing scheme based on the central limit theorem, and proposing an efficient iterative algorithm to solve the optimization problem, and finally realizing load distribution. The present invention can perform differentiated load distribution according to the computing performance of the node, so that the node potential can be exerted to a greater extent, the probability of computing success of the computing task can be maximized, and the system tail delay and coding redundancy can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of the deadline-aware load distribution method for heterogeneous coding distributed computing of the present invention;

[0020] Figure 2 is a task execution flow chart of the coding distributed computing system of the present invention;

[0021] Figure 3 It is a schematic diagram that proves that the P1 problem is a non-convex multivariable optimization problem. DETAILED DESCRIPTION

[0022] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0023] like Figure 1 As shown, the present invention discloses a deadline-aware load distribution method for heterogeneous coding distributed computing, comprising the following steps:

[0024] Step 1: Focus on the basic matrix-vector multiplication task, use MDS coding to pre-code and deploy the computational matrix, and establish deadline-aware load distribution problems under three typical runtime models to minimize the probability of task failure.

[0025] Step 2, a simple and effective normal approximation method is proposed to approximate the failure probability through the central limit theorem, and the original problem is transformed into a non-convex multivariable optimization problem;

[0026] Step three, use an efficient iterative algorithm based on Newton iteration method to solve the optimization problem and obtain the optimal load distribution solution.

[0027] The following is a detailed description of each step of the method.

[0028] In step one, the computation matrix is ​​pre-coded using a coding scheme and a deadline-aware load distribution problem is established.

[0029] like Figure 2 As shown, consider a typical distributed computing system consisting of a master node and a group of n worker nodes, represented as Focusing on the matrix-vector multiplication problem, given a matrix r represents the total number of rows in the matrix, m represents the number of columns in the matrix, and we want to calculate any input vector The corresponding output is y = Ax. In order to alleviate the impact of the dropout problem, a method based on MDS coding is adopted, in which each working node is assigned a local data block When the master node receives input x, it broadcasts x to all worker nodes. Each worker node computes subtask A after receiving x. i x, and then returns the result to the master node. This encoding scheme distributes encoding subtasks among different nodes and executes tasks in parallel, so that the required result can be recovered from the results of some subtasks. The specific encoding process is as follows:

[0030] Step 1: First, encode the calculation matrix A into a larger matrix It contains an extra row, q>r, by applying the following formula

[0031]

[0032] in is a coding matrix, whose property is that the matrix composed of any r rows of G is full rank;

[0033] Step 2: Decompose the encoding matrix of step 1 into n non-intersecting sub-matrices To each working node, Among them A i Each row is a random linear encoding combination of the rows in the encoding matrix obtained in step 1, l i Yes A i number of rows;

[0034] Step 3, the master node will wait for the first r inner products and use them to decode and recover the final result y.

[0035] The present invention considers a timely calculation task of matrix-vector multiplication, that is, for any given input vector x, it is required to complete the calculation of y=Ax within a given deadline d. However, due to the randomness of the calculation time of each working node, some working nodes may not be able to return their local results within the deadline, which may cause the master node to be unable to obtain sufficient results from these working nodes to restore the result of Ax.

[0036] In order to describe the task failure probability, which is a key performance indicator of distributed computing systems, we first introduce the following notations. Let T i (l i ) indicates that worker node i calculates A i x and returns the result to the master node for the total running time, defining p i (l i ) is the probability that worker node i returns the local result to the master node within the deadline d:

[0037] p i (l i )=Pr(T i (l i )≤d)

[0038] Assume that the running time of each working node is T i (l i ) are independent of each other, and the decoding time of the master node can be ignored. Then, the task success probability P(l) can be expressed as:

[0039]

[0040] where l={l1,l2,…,l n} indicates load distribution, l i represents the workload assigned to worker node i. Here, the workload refers to the matrix data block A assigned to each node. ir represents the number of inner products that need to be completed for decoding, l represents the set of worker nodes that successfully return the local subtask results to the master node, is an indicator variable, when This variable is 1 when the sum of all successfully calculated inner products is greater than or equal to the decoding number r, otherwise it is 0.

[0041] Then by optimizing the load distribution scheme l={l1,l2,…,l n The deadline-aware task load allocation optimization problem (denoted as problem LAP) to minimize the task failure probability 1-P(l) is modeled as:

[0042]

[0043] This constraint indicates that l i is a positive integer not exceeding r.

[0044] In order to solve the above load allocation problem (LAP), it is necessary to determine the running time T i (l i ), mainly considering three typical T i (l i ) probability model.

[0045] Consider that the execution time of each working node follows a 2-parameter shift exponential distribution, that is, load l i The cumulative distribution function of the running time of the row vector worker nodes can be expressed as:

[0046]

[0047] Among them, β i >0 is the shift parameter, μ i >0 is a discrete parameter associated with the worker node. This model has been shown to be a good setting to simulate the runtime of computing nodes in a cloud network, achieving a good balance between accuracy and tractability of the analysis.

[0048] The analysis is further extended to the shifted Weibull distribution, where the running time of each working node is considered to follow a 3-parameter shifted Weibull distribution, and the cumulative distribution function can be expressed as:

[0049]

[0050] Among them, β i >0 is the shift parameter, μ i >0 is a discrete parameter, α i >0 is the shape parameter associated with the worker node. It is a broader model that can describe the risk of failure on various computer platforms.

[0051] Considering that the execution time of each working node follows a 2-parameter Pareto distribution, the cumulative distribution function can be expressed as:

[0052]

[0053] Among them, m i >0 indicates the scale parameter, τ i >1 indicates a shape parameter, which is used to describe the weight of the tail. Pareto distribution is a common heavy-tailed distribution model, often used to describe data distributions with power-law tails.

[0054] The above load distribution problem involves the running time T of each node i (e i ), so three typical running time models are considered. Given a time limit d, the success probability p of each node i completing the submatrix within the time limit can be obtained according to the running time model. i (l i ), and then obtain the expression of the overall task failure probability 1-P(l). The ultimate goal is to ensure that the task failure probability is minimized within the time limit.

[0055] In step 2, the present invention utilizes the central limit theorem to approximate the failure probability and transforms the original problem into a non-convex multivariable optimization problem.

[0056] Specifically, the mission success probability P(l) is a multivariate polynomial with an exponential number of terms, which makes the result quite complicated. LAP contains multiple decision variables and the objective function is nonlinear, which makes it a nonlinear programming optimization problem that is more difficult to solve.

[0057] To solve this problem, the present invention adopts a solution idea based on the central limit theorem and provides an approximate expression of P(l). For any working node i, set I i (l i ) represents the indicator of whether the worker node i can successfully transmit the subtask calculation result to the master node within the deadline d, and is set to a 0-1 random variable. If the worker node i completes the subtask result before the deadline d, then I i (l i )=1; otherwise, set I i (l i )=0.

[0058] For three typical runtime models, the present invention approximates the task success probability based on the central limit theorem, and the processing process is as follows:

[0059]

[0060] The original load allocation problem LAP can be transformed into the following, which is defined as P1:

[0061]

[0062] In actual scenarios, i is fairly large, e.g. on the order of 100 row vectors. Therefore, the effect of rounding on the load distribution should be negligible.

[0063] Figure 3 It is proved that the P1 problem is a non-convex multivariable optimization problem. In the experimental simulation, the present invention takes the above three typical running time models as examples, assuming that n = 20 working nodes are used to calculate r = 1000 inner products, and the remaining workloads are fixed. i =75. Figure 3 Shows a single l i The influence of the size of f(l), it is obvious that the objective function f(l) is a non-convex function.

[0064] In step three, the present invention solves the problem using a heuristic iterative algorithm based on Newton's iteration method.

[0065] Specifically, the load distribution amounts are independent of each other, so each l i can be processed separately. When n-1 variables are fixed, the above multivariate optimization problem can be transformed into a single variable optimization problem, that is, the multivariate function f(l) about n variables can be transformed into a single variable function g(l i ).

[0066] assumed The univariate optimization problem (defined as P2) can be expressed as follows:

[0067]

[0068] P2 can be viewed as a one-dimensional function extremum problem, and the optimal solution can be obtained by using Newton's method.

[0069] Under the shift exponential running time model, the node success probability p i (l i ) into g(l i ) and taking its partial derivative, we can get:

[0070]

[0071] Under the shifted Weibull runtime model, the node success probability p i (l i ) into g(l i ) and taking its partial derivative we can get:

[0072]

[0073] Under the Pareto runtime model, the node success probability p i (l i ) into g(l i ) and taking its partial derivative, we can get:

[0074]

[0075] The specific execution steps of the algorithm are as follows:

[0076] Step C1, determine the input of the algorithm: the heterogeneous parameters of the working nodes {α i ,β i ,μ i ,m i ,τ i}, task deadline d, maximum number of iterations M, and allowable error ε;

[0077] Step C2, determine the output of the algorithm: the maximum mission success probability P(e*) and the optimal load distribution plan

[0078] Step C3, initialization of the algorithm, the load distribution scheme l under the HCMM scheme is l={l1,l2,…,l n} as the initial value to reduce the number of iterations;

[0079] Step C4: fix n-1 variables in each iteration process and optimize a single variable to find the optimal value The main operation of P2 involves how to find the extreme points of a single variable function. Obviously, g(l i ) is either at the endpoint or at the extreme point. i ) is a strictly monotonic function, and the optimal value will be at the endpoint. On the other hand, suppose g(l i ) is zero, and Newton's iteration method can be used to find the extreme point s i =g ′-1 (0). Therefore, the optimal value It is worth noting that in each iteration, A -i and B -i will be updated to find the optimal value for each node In this way, after each iteration, a new set of n-element load distribution vectors will be obtained. Continue iterating until the difference in the probability of task success between the two rounds |f(l)-f(l)| is less than the allowable error ε.

[0080] The present invention also provides a deadline-aware load distribution system for heterogeneous coding distributed computing, comprising:

[0081] The load distribution problem establishment module is used to pre-code and deploy the calculation matrix for the matrix-vector multiplication task of a distributed computing system consisting of a master node and a group of n worker nodes using maximum distance separable MDS coding, and establish the deadline-aware load distribution problem under the typical running time model, where p i (l i ) represents the probability that the worker node i returns the local result to the master node within the task deadline. i (l i ) establishes a task success probability model P(l), and the load distribution problem is solved by optimizing the load distribution scheme l = {l1,l2,…,l n} to minimize the overall task failure probability 1-P(l), where l i represents the workload assigned to worker node i;

[0082] The problem conversion module is used to approximate the task success probability based on the central limit theorem and convert the original load distribution problem into a non-convex multivariable optimization problem;

[0083] The problem solving and application module is used to solve the optimization problem using an iterative algorithm based on the Newton iteration method, obtain the optimal load distribution plan, and allocate tasks to the working nodes according to the optimal load distribution plan.

[0084] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the deadline-aware load distribution method for heterogeneous coding distributed computing as described above are implemented.

[0085] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the deadline-aware load distribution method for heterogeneous coding distributed computing as described above are implemented.

[0086] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art should understand that the specific implementation modes of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A Deadline-Aware Load Distribution Method for Heterogeneous Coding Distributed Computing, It is characterized in that The following steps are involved: For the matrix-vector multiplication task of a distributed computing system consisting of a master node and a group of n worker nodes, the maximum distance separable MDS coding is used to pre-code and deploy the computation matrix, and the deadline-aware load distribution problem under a typical runtime model is established, where p i (l i ) represents the probability that the worker node i returns the local result to the master node within the task deadline. i (l i ) establishes a task success probability model P(l), and the load distribution problem is solved by optimizing the load distribution scheme l = {l 1 ,l 2 ,…,l n } to minimize the overall task failure probability 1-P(l), where l i represents the workload assigned to worker node i; Based on the central limit theorem, the task success probability is approximated and the original load distribution problem is transformed into a non-convex multivariable optimization problem. An iterative algorithm based on Newton iteration method is used to solve the optimization problem and obtain the optimal load distribution plan. Tasks are assigned to working nodes according to the optimal load distribution plan.

2. The method according to claim 1, It is characterized in that According to p i (l i ) Establish a mission success probability model P(l), including: Assume T i (l i ) indicates that worker node i calculates A i x and returns the result to the master node, the total running time, A i is the local data block assigned to worker node i, x is the input vector, and the probability that worker node i returns the local result to the master node within the deadline d is p i (l i )=Pr(T i (l i )≤d), the running time of each working node is T i (l i ) are independent of each other, and the decoding time of the master node is ignored, then the task success probability P(l) is expressed as: Where l = {l 1 ,l 2 ,…,l n } represents the load distribution scheme, r represents the number of inner products that need to be completed for decoding, Indicates that the local subtask results are successfully returned to the set of worker nodes of the master node. is an indicator variable, when This variable is 1 when the sum of all successfully calculated inner products is greater than or equal to the decoding number r, otherwise it is 0.

3. The method according to claim 1, It is characterized in that The typical runtime model includes: Consider that the execution time of each working node follows a 2-parameter shift exponential distribution, that is, load l i The cumulative distribution function of the running time of the row vector worker nodes is expressed as: where β i >0 is the shift parameter, μ i >0 is a discrete parameter associated with a worker node; Considering that the running time of each working node follows a 3-parameter shifted Weibull distribution, the cumulative distribution function is expressed as: where α i >0 is the shape parameter associated with the working node; Considering that the execution time of each working node follows a 2-parameter Pareto distribution, the cumulative distribution function is expressed as: Among them, m i >0 indicates the scale parameter, τ i >1 indicates a shape parameter, which describes the heaviness of the tail.

4. The method according to claim 3, It is characterized in that Based on the central limit theorem to approximate the probability of task success, the original load distribution problem is transformed into the following problem: This problem is a non-convex multivariable optimization problem.

5. The method according to claim 4, It is characterized in that The iterative algorithm based on Newton iteration method is used to solve the optimization problem, and the optimal load distribution solution is obtained, including: According to the independent load distribution, the multivariable optimization problem is transformed into a single variable optimization problem: Among them, A -i =∑ j≠i l j p j (l j )-r, The input of the algorithm is: the heterogeneous parameters of the working nodes {α i ,β i ,μ i ,m i ,τ i }, task deadline d, maximum number of iterations M, and allowable error ε; The output of the algorithm is: maximum task success probability P(l * ) and optimal load distribution scheme After the algorithm is initialized, it enters the iteration process. In each round of iteration, n-1 variables are fixed, and optimizing a single variable is to find the optimal value of the working node i. After each iteration, a new set of n-element load distribution vectors is obtained. Continue iterating until the difference in the success probability of the two rounds of tasks |f(l * )-f(l)| is less than the allowable error ε.

6. The method according to claim 5, It is characterized in that The initialization of the algorithm includes: setting the load distribution scheme l under the HCMM scheme to {l 1 ,l 2 ,…,l n } as the initial value.

7. The method according to claim 5, It is characterized in that The optimal value of worker node i The calculation is as follows: where s i is the extreme point s found using Newton's iteration method i =g ′-1 (0).

8. A deadline-aware load distribution system for heterogeneous coding distributed computing. It is characterized in that include: The load distribution problem establishment module is used to pre-code and deploy the calculation matrix for the matrix-vector multiplication task of a distributed computing system consisting of a master node and a group of n worker nodes using maximum distance separable MDS coding, and establish the deadline-aware load distribution problem under the typical running time model, where p i (l i ) represents the probability that the worker node i returns the local result to the master node within the task deadline. i (l i ) establishes a task success probability model P(l), and the load distribution problem is solved by optimizing the load distribution scheme l = {l 1 ,l 2 ,…,l n } to minimize the overall task failure probability 1-P(l), where l i represents the workload assigned to worker node i; The problem conversion module is used to approximate the task success probability based on the central limit theorem and convert the original load distribution problem into a non-convex multivariable optimization problem; The problem solving and application module is used to solve the optimization problem using an iterative algorithm based on the Newton iteration method, obtain the optimal load distribution plan, and allocate tasks to the working nodes according to the optimal load distribution plan.

9. A computer device, It is characterized in that include: one or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the deadline-aware load distribution method for heterogeneous coding distributed computing as described in any one of claims 1-7 are implemented.

10. A computer storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the deadline-aware load distribution method for heterogeneous coding distributed computing as described in any one of claims 1 to 7 are implemented.