Method for distributed matrix multiplication using task entanglement-based coding

Abstract

The present invention is a method for processing large-scale matrix operations in a distributed computing environment, wherein a main server encodes information that needs to be transmitted to multiple edge devices for distributed matrix operations based on Chebyshev polynomials, and reduces the information that needs to be transmitted by coding based on task entanglement, and immediately performs decoding when the operation result received from each edge device reaches a recovery threshold to derive the matrix operation result.

Description

Technology Field

[0001] The present invention relates to parallel computing technology, and more specifically, to a technology for performing high-dimensional matrix operations in a distributed computing environment. Background Technology

[0002] Matrix multiplication is the most fundamental operation in various fields such as big data analysis, machine learning, and image processing. In particular, since most machine learning network structures, such as CNNs and FCNs, require massive matrix multiplication, performing matrix multiplication faster and more efficiently is a critical issue that determines the performance of machine learning.

[0003] In particular, in machine learning where the size of the data used is constantly increasing, processing large amounts of data on a single computer requires an enormous amount of time due to memory limitations. To address this problem, distributed computing and distributed machine learning, which involve dividing data, allocating it to multiple edge devices for computation, and then receiving the results back for data processing, have recently been gaining attention.

[0004] Conventional distributed computing was primarily based on synchronization systems. That is, it was implemented by assigning only one task per edge device from the main server, followed by sequential communication starting from the edge device that completed the task. This approach suffers from a problem where overall computation speed is degraded due to stragglers.

[0005] The lag problem was addressed by using methods such as duplicating computation tasks and assigning them to multiple edge devices, or by redundantly assigning coded tasks based on coding theory; however, these methods have the problem that the lag's unfinished work is completely ignored.

[0006] Furthermore, in distributed computing, there is a possibility that information may be leaked by eavesdroppers during the communication process between the main server and edge devices, or from edge devices that have been compromised by attacks. Prior art literature

[0007] Yu, Q., Maddah-Ali, MA, and Avestimehr, AS, "Polynomial codes: an optimal design for high-dimensional coded matrix multiplication", Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 4406-4416, 2017.Yu, Q., Maddah-Ali, MA, and Avestimehr, AS, "Straggler mitigation in distributed matrix multiplication: Fundamental limits and optimal coding", IEEE Transactions on Information Theory, vol. 66, no. 3, pp. 1920-1933, 2020. The problem to be solved

[0008] The present invention aims to provide a distributed matrix multiplication method that increases the speed of overall matrix operations without the lag problem using a work encoding method based on Chebyshev Polynomial Codes.

[0009] Additionally, another objective of the present invention is to provide a distributed matrix multiplication method that can utilize the partial calculation results of a delay without ignoring the calculation results of the delay.

[0010] Additionally, another objective of the present invention is to provide a distributed matrix multiplication method that can reduce the amount of information that a main server must transmit to an edge device by using a task entanglement-based coding method. means of solving the problem

[0011] A distributed matrix operation method using work entanglement-based coding according to the first embodiment of the present invention is a method in which a main server performs a distributed matrix operation using a plurality of edge devices, and includes a partitioning step, an encoding step, a transmission step, a reception step, and a decoding step.

[0012] The partitioning step is a step in which the main server partitions the first matrix and the second matrix to be processed into m first submatrices and n second submatrices, respectively.

[0013] The encoding step is a step in which the main server encodes m first submatrices and n second submatrices for each edge device into an encoding matrix based on task entanglement-based coding using Chebyshev polynomials.

[0014] The transmission phase is the stage where the main server transmits the matrix encoded for each edge device to the corresponding edge device.

[0015] The receiving phase is the phase where the main server receives the results of matrix operations from the edge devices.

[0016] The decoding step is a step in which the main server decodes the received matrix operation results to recover the operation results of the first matrix and the second matrix when the number of received matrix operation results reaches the first recovery threshold.

[0017] According to the first embodiment of the present invention, the encoding step may include a first encoding step and a second encoding step.

[0018] The first encoding step is a step in which the main server encodes m first submatrices for each edge device into L2 encoding matrices based on task entanglement-based coding using a first Chebyshev polynomial of degree L1, according to the number of tasks L determined (L=L1L2, L1 and L2 are coprime).

[0019] The second encoding step is a step in which the main server encodes n second submatrices for each edge device into L1 encoding matrices based on work entanglement-based coding using a second Chebyshev polynomial of degree L2.

[0020] According to the first embodiment of the present invention, the encoding step may further include an operation coordinate selection step.

[0021] The operation coordinate selection step is a step in which the main server selects L2 operation coordinates to be used in the first encoding step and L1 operation coordinates to be used in the second encoding step for each edge device.

[0022] According to the first embodiment of the present invention, the main server can encode an encoding matrix into L2 encoding matrices by adding a matrix encoded based on work entanglement using a first Chebyshev polynomial of degree L1 to the encoding matrix in the first encoding step, and encode an encoding matrix into L1 encoding matrices by adding a matrix encoded based on work entanglement using a second Chebyshev polynomial of degree L2 to the encoding matrix in the second encoding step.

[0023] In this case, when the number of matrix operation results received during the decoding phase reaches a second recovery threshold, the main server can decode the received matrix operation results to recover the operation results of the first matrix and the second matrix.

[0024] A distributed matrix operation method using work entanglement-based coding according to a second embodiment of the present invention is a method for performing distributed matrix operations using a main server and a plurality of edge devices in a distributed computing environment in which a plurality of edge devices have a first matrix data set and a second matrix data set to be operated on, and includes a one-hot encoding step, a first encoding step, a second encoding step and a transmission step, a first matrix encoding step, a second matrix encoding step, a matrix operation step, a transmission step of the operation result, a reception step, and a decoding step.

[0025] The one-hot encoding step is the step in which the main server one-hot encodes the indices within the corresponding data sets of the first matrix and the second matrix to be processed, respectively.

[0026] The first encoding step is a step in which the main server encodes the one-hot encoded matrix for the first matrix for each edge device into the first encoding matrix based on work entanglement-based coding using Chebyshev polynomials.

[0027] The second encoding step is a step in which the main server encodes the one-hot encoded matrix for the second matrix for each edge device into a second encoding matrix based on work entanglement-based coding using Chebyshev polynomials.

[0028] The transmission phase is the stage where the main server transmits the matrix encoded for each edge device to the corresponding edge device.

[0029] The first matrix encoding step is a step in which each edge device encodes the first matrix by matrix multiplying the entire matrix of the first matrix data set with the first encoding matrix.

[0030] The second matrix encoding step is a step in which each edge device encodes the second matrix by performing matrix multiplication of the entire matrix of the second matrix data set and the second encoding matrix.

[0031] The matrix operation step is a step in which each edge device performs matrix operation on the encoded first matrix and the encoded second matrix.

[0032] The computation result transmission step is the step in which each edge device transmits the computation result to the main server.

[0033] The receiving phase is the phase where the main server receives the results of matrix operations from the edge devices.

[0034] The decoding step is a step in which the main server decodes the received matrix operation results to recover the operation results of the first matrix and the second matrix when the number of received matrix operation results reaches the first recovery threshold.

[0035] According to a second embodiment of the present invention, the first encoding step is a step of encoding a matrix that is one-hot encoded for a first matrix for each edge device according to a determined number of tasks L (L=L1L2, where L1 and L2 are coprime) into L2 encoding matrices based on task entanglement-based coding using a first Chebyshev polynomial of degree L1, and the second encoding step is a step of encoding a matrix that is one-hot encoded for a second matrix for each edge device into L1 encoding matrices based on task entanglement-based coding using a second Chebyshev polynomial of degree L2.

[0036] A distributed matrix operation method using work entanglement-based coding according to a second embodiment of the present invention may further include an operation coordinate selection step.

[0037] The operation coordinate selection step is the step in which the main server selects L2 operation coordinates to be used in the first encoding step and L1 operation coordinates to be used in the second encoding step.

[0038] According to a second embodiment of the present invention, the main server can encode an encoding matrix into L2 encoding matrices by adding a matrix encoded based on work entanglement using a first Chebyshev polynomial of degree L1 to the encoding matrix in a first encoding step, and encode an encoding matrix into L1 encoding matrices by adding a matrix encoded based on work entanglement using a second Chebyshev polynomial of degree L2 to the encoding matrix in a second encoding step.

[0039] In this case, when the number of matrix operation results received during the decoding phase reaches a second recovery threshold, the main server can decode the received matrix operation results to recover the operation results of the first matrix and the second matrix. Effects of the invention

[0040] The present invention can increase the speed of overall matrix operations without the lag problem using a work encoding method based on Chebyshev Polynomial Codes.

[0041] In addition, the present invention can utilize the partial calculation results of the delay without ignoring the calculation results of the delay.

[0042] In addition, the present invention can reduce the amount of information that a main server must transmit to an edge device by using a task entanglement-based coding method. Brief explanation of the drawing

[0043] Figure 1 compares examples of processing distributed matrix multiplication in a distributed computing environment. Figure 2 compares the coding concept for a task of the prior art with the coding concept for a task of the present invention. FIG. 3 is a flowchart of a distributed matrix multiplication according to a first embodiment of the present invention. Figure 4 illustrates the concept of variance matrix multiplication according to a second embodiment of the present invention. FIG. 5 is a flowchart of a distributed matrix multiplication according to a second embodiment of the present invention. Specific details for implementing the invention

[0044] The foregoing and additional aspects are embodied in the embodiments described with reference to the attached drawings. It is understood that the components of each embodiment may be combined in various ways within the embodiment unless otherwise stated or contradictory. Each block in the block diagram may represent a physical part in some cases, but in others, it may be a logical representation of a part of the function of a single physical part or a function spanning multiple physical parts. Sometimes, the entity of a block or part thereof may be a set of program instructions. These blocks may be implemented in whole or in part by hardware, software, or a combination thereof.

[0045] As used in this specification, A and B are matrices, respectively and It is defined as.

[0046] Figure 1 compares examples of processing distributed matrix multiplication in a distributed computing environment. Figure 1 illustrates various examples of processing matrix multiplication, one of the most fundamental building blocks in machine learning and deep learning, in a distributed manner, specifically an example of distributed matrix multiplication for C = A × B (where A and B are matrices). Figure 1 illustrates various examples of assigning tasks to resolve the straggler problem, which processes a given task much slower than other edge devices.

[0047] Figure 1 shows one main server and four workers, i.e., edge devices, in common.

[0048] Figure 1 (a) shows a method of duplicating computation tasks and assigning them to multiple workers. The main server divides matrix A into two submatrices of equal size, A1 and A2, and divides the A × B task into A1 × B and A2 × B tasks, assigning each task to two workers so that the result matrix C can be obtained without delay even if one worker is delayed per task. However, as in the example of Figure 1 (a), if two workers (W3 and W4) responsible for the same task both calculate slowly, the main server cannot obtain the final result until it receives the results from these two workers, and thus problems caused by delays still occur.

[0049] Figure 1(b) is an example where the main server assigns tasks using MDS (Maximum Distance Separable) code, and the main server can decode the final result C when it receives task results from the two fastest workers. Therefore, Figure 1(b) can allow two delayers.

[0050] Figure 1 (c) is an example in which input matrices A and B are both encoded using a polynomial function and assigned to a worker as a single task, and the final result C is decoded by utilizing the work result of the fastest worker.

[0051] In both (b) and (c) of Figure 1, the operation results of the delayed person whose work has not been completed are not used, resulting in a waste of resources.

[0052] Meanwhile, FIG. 1(d) is an example of the task assignment used by the present invention, in which the main server encodes and assigns two smaller tasks to each worker, each half the size of the original task. In this case, when each worker completes one of the two assigned tasks, they immediately transmit the result to the main server, and the main server can decode the final result using the two task results of worker W1 and the result of one of the two tasks of workers W2, W3, and W4. Consequently, this approach can reduce the overall processing time in a distributed computing environment by fully utilizing computing resources.

[0053] A distributed matrix operation method using work entanglement-based coding according to the first embodiment of the present invention is a method in which a main server performs a distributed matrix operation using a plurality of edge devices, and includes a partitioning step, an encoding step, a transmission step, a reception step, and a decoding step.

[0054] Main servers and edge devices are computing devices that include software concepts in addition to hardware. A computing device is a device that includes a processor and memory connected to the processor and containing program instructions executable by the processor. In addition to the processor and memory, such a computing device may be a computer device that additionally includes a GPU (Graphic Processor Unit), storage devices, displays, input devices, etc. The processor is a processor that executes program instructions, and memory is connected to the processor and stores program instructions executable by the processor, data that the processor uses for computation, and data processed by the processor.

[0055] The main server and edge devices are connected via a network.

[0056] The partitioning step is the step in which the first matrix and the second matrix to be processed by the main server are partitioned into m first submatrices and n second submatrices, respectively. If the first matrix is ​​denoted as A and the second matrix as B, it can be expressed as in <Equation 1>.

[0057]

[0058] A1 is the first submatrix of A, and A m is the m-th submatrix of A, B1 is the first submatrix of B, and B n is the nth submatrix of B.

[0059] A w If we define (w ∈ [1:m]) as a submatrix of A and, B z If we define (Z ∈ [1:n]) as a submatrix of B am.

[0060] m and n are values ​​determined in advance by considering the sizes of input matrices A and B and the distributed computing environment.

[0061] The encoding phase is the step in which the main server encodes m first submatrices and n second submatrices for each edge device into an encoding matrix based on task entanglement-based coding using Chebyshev polynomials. That is, the main server encodes matrices A and B to be processed in the encoding phase by encoding them using Chebyshev polynomials. The encoded matrix , If written as And, The main server performs encoding of matrices A and B to be processed for each edge device, and encodes even a single device into multiple encoding matrices.

[0062] According to the first embodiment of the present invention, the encoding step may include a first encoding step and a second encoding step.

[0063] The first encoding step is a step in which the main server encodes m first submatrices for each edge device into L2 encoding matrices based on task entanglement-based coding using a first Chebyshev polynomial of degree L1, according to the number of tasks L determined (L=L1L2, L1 and L2 are coprime).

[0064] The task assigned by the main server to the edge device for distributed matrix multiplication is the multiplication of encoded matrices. One task requires two encoded matrices. Therefore, if it is determined that L tasks are required for the multiplication of two input matrices A and B, in the prior art, the main server must transmit two encoded matrices, i.e., 2L encoded matrices, to the edge device for each task.

[0065] The present invention enables the performance of L tasks by providing only L1+L2 encoded matrices, by using a method that allows a single encoded matrix to be repeatedly used across multiple tasks based on task entanglement for L tasks (L=L1L2, where L1 and L2 are coprime). Task entanglement refers to a situation where individual tasks are entangled so that multiple tasks are performed using the same encoded matrix. Equation 2 below represents the task entanglement condition of the matrix encoding matrices A and B.

[0066]

[0067] W is the number of edge devices.

[0068] FIG. 2 compares the coding concept for a task of the prior art with the coding concept for a task of the present invention. FIG. 2(a) shows the conventional coding scheme for a task of the prior art, which includes six tasks performed on the i-th edge device ( For ) 6 encoded matrices for matrix A ( 6 encoded matrices for ) and matrix B( It can be seen that ) is required. In contrast, the task entanglement-based coding scheme of the present invention uses three encoded matrices for matrix A for six tasks ( For ) and matrix B, two encoded matrices( It can be seen that ) is required. In this case, the work entanglement condition is am.

[0069] The main server, in the first encoding step, uses the encoding function of <Equation 3> for the first submatrix of matrix A using the Chebyshev polynomial. p A Encode by encoding through (x).

[0070]

[0071] f(x) is a Chebyshev polynomial of degree L1, and g(x) is a Chebyshev polynomial of degree L2.

[0072] The main server has L evaluation points for each edge device Select (j∈[1:L]), and find L2 operation coordinates for the first matrix (m first submatrices) according to the work entanglement condition of <Equation 2>, and the encoding function It is encoded into L2 encoding matrices through.

[0073] The second encoding step is a step in which the main server encodes n second submatrices for each edge device into L1 encoding matrices based on work entanglement-based coding using a second Chebyshev polynomial of degree L2.

[0074] The main server has L evaluation points for each edge device that are identical to those used in the first encoding step. For (j∈[1:L]), find L1 operation coordinates for the second matrix (n second submatrices) according to the work entanglement condition of <Equation 2> and the encoding function It is encoded into L1 encoding matrices through.

[0075] According to the first embodiment of the present invention, the encoding step may further include an operation coordinate selection step.

[0076] The operation coordinate selection step is performed first before encoding, in which the main server selects L2 operation coordinates to be used in the first encoding step and L1 operation coordinates to be used in the second encoding step for each edge device.

[0077] The Chebyshev polynomials used in task entanglement-based coding are commutative polynomials, and a Chebyshev polynomial of degree d has d distinct real roots at (-1, 1). Utilizing these characteristics, the main server obtains L operation coordinates for L tasks during the operation coordinate selection phase, and then selects L1+L2 operation coordinates based on the task entanglement condition. The procedure for obtaining operation coordinates is as follows.

[0078] f(x) is a Chebyshev polynomial of degree L1, and g(x) is a Chebyshev polynomial of degree L2.

[0079] i) For the i-th edge device, any value t between (-1, 1). i Select . At this time t i It selects a value different from the value selected on other edge devices. That is, The other side am.

[0080] ii) root of Find .

[0081] iii) root of Find .

[0082] iv) For j such that it satisfies Rearrange it. That is, rearrange it so that the work entanglement condition is satisfied.

[0083] The process from i) to iv) is repeated for each edge device.

[0084] The main server performs encoding during the encoding phase by assigning the x value of the operation coordinates to the variable x of the encoding function. Accordingly, the main server generates L2 encoded matrices for the first matrix A and L1 encoded matrices for the second matrix B.

[0085] The transmission step is a step in which the main server transmits an encoded matrix for each edge device to the corresponding edge device. Accordingly, the main server transmits L1+L2 encoded matrices to each edge device. Compared to the prior art which requires the transmission of 2L encoded matrices for L tasks, the transmission data is reduced to L1+L2 according to the present invention.

[0086] Each edge device performs matrix multiplication using the encoded matrix received from the main server. That is, the i-th edge device uses the encoded matrix received from the main server and L tasks by multiplying (j∈[1:L2], k∈[1:L1]). It performs calculations. At this time, the edge device immediately transmits the calculation results to the main server once each task is completed.

[0087] The receiving phase is the phase where the main server receives the results of matrix operations from the edge devices.

[0088] The decoding step is a step in which the main server decodes the received matrix operation results to recover the operation results of the first and second matrices when the number of received matrix operation results reaches the first recovery threshold. That is, the main server performs decoding immediately when it receives operation results from all edge devices up to the first recovery threshold. Since the original matrix was encoded into an encoded matrix with a size of 1 / m and 1 / n, the amount of operation performed by a single operation at the edge device is 1 / mn of the original operation. Therefore, since the main server can recover the final target value by having mn operation values ​​of the encoded matrix, the first recovery threshold can be calculated as the product of m × n.

[0089] The final operation result can be expressed using the product of submatrices as shown in <Equation 4>.

[0090]

[0091] Therefore, the final operation result is obtained using <Equation 5>, and each coefficient matrix can be extracted using repeated division of f(x) and g(x). Specifically, at If you divide it It can be found as the quotient, and the remainder in this case If you divide by The quotient can be obtained. Each coefficient matrix is ​​extracted by utilizing repeated division in this way.

[0092]

[0093] According to the first embodiment of the present invention, the main server can encode an encoding matrix into L2 encoding matrices by adding a matrix encoded based on work entanglement using a first Chebyshev polynomial of degree L1 to the encoding matrix as in <Equation 6> in the first encoding step, and encode an encoding matrix into L1 encoding matrices by adding a matrix encoded based on work entanglement using a second Chebyshev polynomial of degree L2 to the encoding matrix as in <Equation 6> in the second encoding step. That is, the main server can apply security constraints in the first encoding step and the second encoding step.

[0094]

[0095] Zw and Zz are random matrices, respectively, and E is the number of conspiring edge devices.

[0096] The encoding function in this case uses the first matrices A and B and matrix Z, each of which has a random value, as coefficients. Since matrix Z is the matrix with the largest entropy in information theory because each element value is extracted from a random distribution, the encoding function of <Equation 6> guarantees complete data security in distributed computing and distributed machine learning. This can be proven in information theory through the zero-knowledge proof proposed by Shannon. This means that each edge device cannot obtain any information from the encoded matrices assigned to perform the task, and this can be formulated as in <Equation 7>.

[0097]

[0098] Equation 7 is the encoded matrix assigned to all edge devices for L tasks. , This means that the mutual information between the set and the original matrices A and B is zero. In other words, it means that even if all colluding E edge devices share their assigned matrices to conspire to infer the original matrix, they can never determine what value the original matrix is ​​from an information-theoretic perspective. This is possible because the entropy of the random matrix Z used in the encoding function is greater than the entropy of the input matrices A and B.

[0099] In this case, when the number of matrix operation results received during the decoding phase reaches a second recovery threshold, the main server can decode the received matrix operation results to recover the operation results of the first matrix and the second matrix. The first recovery threshold corresponds to the number of inner product results of the same number as the original matrix multiplication, but in the case of encoding by adding a random matrix, the second recovery threshold is the number of inner product results that is approximately linearly proportional to the number of assigned operations as in <Equation 8>, and the final result can be recovered from the operation result corresponding to the second recovery threshold.

[0100]

[0101] E is the number of edge servers that conspire to leak matrix information.

[0102] In contrast, in the case of the encoding method using polynomials of the prior art, when security constraints are applied, for decoding Dog internal results are required.

[0103] FIG. 3 is a flowchart of distributed matrix multiplication according to a first embodiment of the present invention. The main server receives a first matrix and a second matrix to be operated on and divides them into m first submatrices and n second submatrices, respectively (S1000). For each edge device, the main server selects L2 operation coordinates to be used for encoding the first matrix and L1 operation coordinates to be used in the second encoding step for L operations (S1010). At this time, the operation coordinates must satisfy the operation entanglement condition of <Equation 2>. For each edge device, the main server encodes the first matrix into L2 secure encoding matrices based on operation entanglement-based coding using a first Chebyshev polynomial of degree L1 as in <Equation 6> (S1020), and encodes the second matrix into L1 secure encoding matrices based on operation entanglement-based coding using a first Chebyshev polynomial of degree L2 as in <Equation 6> (S1030). The main server transmits L1+L2 encoded matrices to each edge device (S1040). Each edge device performs L operations using the received encoded matrices. At this time, each edge device immediately transmits the operation results to the main server as soon as the individual operations are completed (S1050). The main server receives the matrix operation results from the edge devices (S1060), and when the number of received matrix operation results reaches a second recovery threshold, it decodes the received matrix operation results to recover the operation results of the first matrix and the second matrix (S1070).

[0104] FIG. 4 illustrates the concept of distributed matrix multiplication according to a second embodiment of the present invention. The example in FIG. 4 is a distributed matrix multiplication method applicable when edge devices can access a matrix data set, i.e., a matrix library, in which matrices to be computed are stored, or when they store a matrix data set themselves. That is, in the second embodiment, for matrix computation, the main server does not send data to be computed to each edge device; instead, the edge devices can already access the matrix data set and transmit an index within the matrix data set to be computed to perform distributed matrix computation.

[0105] When edge devices can access the matrix library, distributed computing proceeds in a manner where the main server sends a query to the edge device to perform matrix operations, and the edge device executes the matrix operations accordingly. In this process, the edge devices can infer what data the main server intends to process through the received request. If the main server requests a matrix operation by simply transmitting the index within the dataset of the matrix data to be used for the operation, the edge devices can very easily infer the information required by the main server. Since edge devices can identify the main server's computational preferences, personal information may be leaked. Therefore, to protect personal information, the main server's personal information can be protected by requesting the desired matrix operation by encoding the index information within the desired matrix dataset. The privacy protection constraint is expressed by <Equation 9> below.

[0106]

[0107] A and B are matrix libraries, respectively, and q and r are the indices of matrix library A and matrix library B, respectively. Here, I(x;y) is a function representing the amount of information about x that can be known when y is known, and the above equation This means that even if you know, you cannot obtain any information about q and r.

[0108] A distributed matrix operation method using work entanglement-based coding according to a second embodiment of the present invention is a method for performing distributed matrix operations using a main server and a plurality of edge devices in a distributed computing environment in which a plurality of edge devices have a first matrix data set and a second matrix data set to be operated on, and includes a one-hot encoding step, a first encoding step, a second encoding step, a transmission step, a first matrix encoding step, a second matrix encoding step, a matrix operation step, a transmission of operation results step, a reception step, and a decoding step.

[0109] The one-hot encoding step is the stage where the main server one-hot encodes the indices within the corresponding data sets of the first and second matrices to be processed. Since indices are scalar values, the main server one-hot encodes them to make them suitable for use in matrix operations.

[0110] The first encoding step is a step in which the main server encodes the one-hot encoded matrix for the first matrix for each edge device into the first encoded matrix based on task entanglement-based coding using Chebyshev polynomials. That is, the main server one-hot encoded the matrix of the indices within the first matrix data set of the first matrix to be computed as shown in <Equation 10> below Encode as.

[0111] The second encoding step is a step in which the main server encodes the one-hot encoded matrix for the second matrix for each edge device into a second encoded matrix based on task entanglement-based coding using Chebyshev polynomials. That is, the main server one-hot encoded the matrix of the indices within the second matrix data set of the second matrix to be computed as shown in <Equation 10> below Encode as.

[0112]

[0113] c1 and c2 are arbitrary constants set according to the situation, m and n are the number of partitions for the first and second matrices, respectively, and θ A and θ A are each one-hot encoded matrices. f(x) is a Chebyshev polynomial of degree L1, and g(x) is a Chebyshev polynomial of degree L2.

[0114] According to a second embodiment of the present invention, the first encoding step is a step of encoding a matrix one-hot encoded for a first matrix for each edge device according to a determined number of tasks L (L = L1L2, where L1 and L2 are coprime) into L2 encoding matrices based on task entanglement-based coding using a first Chebyshev polynomial of degree L1, and the second encoding step is a step of encoding a matrix one-hot encoded for a second matrix for each edge device into L1 encoding matrices based on task entanglement-based coding using a second Chebyshev polynomial of degree L2. The following <Equation 11> is the task entanglement condition of the matrix encoded by one-hot encoded matrices of indices within the matrix library of matrices A and B.

[0115]

[0116]

[0117] W is the number of edge devices.

[0118] The main server has L evaluation points for each edge device Select (j∈[1:L]), find the operation coordinates of the one-hot encoded matrix L2 for the first matrix according to the work entanglement condition, as shown in <Equation 10> Encode as, and L identical evaluation points Find the operation coordinates of L1 one-hot encoded matrices for the second matrix according to the work entanglement condition at (j∈[1:L]) as in <Equation 10> Encode as.

[0119] According to a second embodiment of the present invention, the encoding step may further include an operation coordinate selection step.

[0120] The operation coordinate selection step is performed first before encoding, in which the main server selects L2 operation coordinates to be used in the first encoding step and L1 operation coordinates to be used in the second encoding step for each edge device.

[0121] The transmission step is a step in which the main server transmits an encoded matrix for each edge device to the corresponding edge device. Accordingly, the main server transmits L1+L2 encoded matrices to each edge device. Compared to the prior art which requires the transmission of 2L encoded matrices for L tasks, the transmission data is reduced to L1+L2 according to the present invention.

[0122] The first matrix encoding step is a step in which each edge device encodes the first matrix by performing matrix multiplication of the entire matrix of the first matrix data set and the first encoding matrix as in <Equation 12>. At this time, each edge device does not know which matrix in the first matrix data set is used for the operation. In addition, similar to the first embodiment, the first matrix is ​​divided into m parts in the second embodiment as well. That is, the entire matrix within the first matrix data set is divided into m parts and then encoding is performed.

[0123]

[0124] α is the number of A matrices in the first matrix data set. is the k-th submatrix of the entire matrix within the first matrix data set, and Is It is the k-th submatrix of.

[0125] The second matrix encoding step is a step in which each edge device encodes the second matrix by performing matrix multiplication of the entire matrix of the second matrix data set and the second encoding matrix as in <Equation 13>. Similarly, each edge device does not know which matrix in the second matrix data set is used for the operation. In addition, just as in the first embodiment, the second matrix is ​​divided into n parts in the second embodiment as well. That is, the entire matrix within the second matrix data set is divided into n parts and then encoding is performed.

[0126]

[0127] β is the number of B matrices in the second matrix data set. is the k-th submatrix of the entire matrix within the second matrix data set, and Is It is the k-th submatrix of.

[0128] The matrix operation step is a step in which each edge device performs matrix operation on the encoded first matrix and the encoded second matrix.

[0129] The computation result transmission step is the step in which each edge device transmits the computation result to the main server.

[0130] The receiving phase is the phase where the main server receives the results of matrix operations from the edge devices.

[0131] The decoding stage is a step in which the main server decodes the received matrix operation results to recover the operation results of the first and second matrices when the number of received matrix operation results reaches the first recovery threshold. That is, the main server performs decoding immediately when it receives operation results from all edge devices up to the first recovery threshold. Since the original matrix was encoded into an encoded matrix with a size of 1 / m and 1 / n, the amount of computation for a single operation performed at the edge is 1 / mn of the original operation. Therefore, the main server can recover the final target value by having mn operation values ​​of the encoded matrix. Thus, the first recovery threshold can be calculated as the product of m × n.

[0132] The final operation result can be expressed using the product of submatrices as shown in <Equation 4>.

[0133] Therefore, the final operation result is obtained using <Equation 14>, and each coefficient matrix can be extracted using repeated division of f(x) and g(x). Specifically, at If you divide it The quotient can be obtained, and each coefficient matrix can be extracted by utilizing repeated division in this way.

[0134]

[0135]

[0136] ,

[0137] At this time ,

[0138] am.

[0139] According to a second embodiment of the present invention, the main server can encode an encoding matrix into L2 encoding matrices by adding a matrix encoded based on work entanglement using a first Chebyshev polynomial of degree L1 to the encoding matrix as in <Equation 15> in the first encoding step, and encode an encoding matrix into L1 encoding matrices by adding a matrix encoded based on work entanglement using a second Chebyshev polynomial of degree L2 to the encoding matrix as in <Equation 15> in the second encoding step. That is, the main server can apply security constraints in the first encoding step and the second encoding step.

[0140]

[0141]

[0142] Z is a random matrix.

[0143] The encoding function in this case uses a matrix Z, in which each element has a random value, as a coefficient. Since the values ​​of each element of matrix Z are drawn from a random distribution, it is the matrix with the largest entropy according to information theory. The encoding function of <Equation 15> guarantees complete data security in distributed computing and distributed machine learning. This can be proven information-theoretically through the zero-knowledge proof proposed by Shannon. This means that each edge device cannot obtain any information from the encoded matrices assigned to perform the task, and this can be formulated as in <Equation 7>.

[0144] In this case, when the number of matrix operation results received during the decoding phase reaches a second recovery threshold, the main server can decode the received matrix operation results to recover the operation results of the first matrix and the second matrix. The first recovery threshold corresponds to the number of inner product results of the same number as the original matrix multiplication, but in the case of encoding by adding a random matrix, the second recovery threshold is the number of inner product results that is approximately linearly proportional to the number of assigned operations as in <Equation 16>, and the final result can be recovered from the operation result corresponding to the second recovery threshold.

[0145]

[0146] In contrast, in the case of the encoding method using polynomials of the prior art, when security constraints are applied, for decoding Dog internal results are required.

[0147] FIG. 5 is a flowchart of distributed matrix multiplication according to a second embodiment of the present invention. The main server determines the indices of the first matrix and the second matrix to be computed in the first matrix data set and the second matrix data set (S2000). In addition, the main server determines the values ​​of m and n to divide the first matrix and the second matrix to be computed into how many submatrices, respectively. The main server one-hot encodes the determined indices (S2010). For each edge device, the main server selects L2 operation coordinates to be used for encoding the matrix one-hot encoded for the first matrix and L1 operation coordinates to be used for encoding the matrix one-hot encoded for the second matrix for L operations (S2020). At this time, the operation coordinates must satisfy the operation entanglement condition. The main server encodes the first matrix, which is one-hot encoded for each edge device, into L2 encoded matrices with security applied based on work entanglement-based coding using a first Chebyshev polynomial of degree L1 as in <Equation 13> (S2030), and encodes the second matrix into L1 encoded matrices with security applied based on work entanglement-based coding using a first Chebyshev polynomial of degree L2 as in <Equation 13> (S2040). The main server transmits L1+L2 encoded matrices to the corresponding edge device for each edge device (S2050). Each edge device encodes the first matrix by performing matrix multiplication of the entire matrix of the first matrix data set and the first encoding matrix as in <Equation 11> (S2060), and encodes the second matrix by performing matrix multiplication of the entire matrix of the second matrix data set and the second encoding matrix as in <Equation 12> (S2070), and then performs matrix operations on the encoded first matrix and the encoded second matrix. At this time, each edge device immediately transmits the operation results to the main server as soon as the individual operations are completed (S2080).The main server receives matrix operation results from edge devices (S2090), and when the number of received matrix operation results reaches a second recovery threshold, it decodes the received matrix operation results to recover the operation results of the first matrix and the second matrix (S2100).

[0148] Although the present invention has been described above with reference to embodiments with reference to the accompanying drawings, it is not limited thereto and should be interpreted to encompass various modifications that can be obviously derived from them by those skilled in the art. The claims are intended to encompass such modifications.

Claims

Claim 1 A method for a main server to perform distributed matrix operations using multiple edge devices comprises: a partitioning step of partitioning a first matrix and a second matrix to be operated on into m first submatrices and n second submatrices, respectively; an encoding step of encoding the m first submatrices and n second submatrices into an encoding matrix based on work entanglement-based coding using Chebyshev polynomials for each edge device; a transmission step of transmitting the encoded matrix for each edge device to the corresponding edge device; and a reception step of receiving matrix operation results from the edge devices. A distributed matrix operation method using work entanglement-based coding, comprising: a decoding step for decoding the received matrix operation results to recover the operation results of a first matrix and a second matrix when the number of received matrix operation results reaches a first recovery threshold; wherein the encoding step comprises a first encoding step for encoding m first submatrices for each edge device into L2 encoding matrices based on work entanglement-based coding using a first Chebyshev polynomial of degree L1 according to a determined number of operations L (L=L1L2, L1 and L2 are coprime), and a second encoding step for encoding n second submatrices for each edge device into L1 encoding matrices based on work entanglement-based coding using a second Chebyshev polynomial of degree L2. Claim 2 delete Claim 3 In claim 1, the encoding step further comprises: an operation coordinate selection step for selecting L2 operation coordinates to be used in the first encoding step and L1 operation coordinates to be used in the second encoding step; wherein the operation coordinates are obtained by performing steps i) through iv) below for each edge device, i) any value t between (-1, 1). i select ( ii) the number of edge devices root of Find (f(x) is the first Chebyshev polynomial of degree L1)iii) root of Seeking ( g(x) iv)the second Chebyshev polynomial of degree L2 For j satisfying Rearrange the operation coordinates so that the work entanglement condition of <Equation 2> is satisfied. A distributed matrix operation method using rearrangement and task entanglement-based coding.[Equation 2] W is the number of edge devices. Claim 4 A distributed matrix operation method using work entanglement-based coding according to claim 3, wherein in the first encoding step, a matrix encoded by adding a random matrix based on work entanglement-based coding using a first Chebyshev polynomial of degree L1 to an encoding matrix to encode it into L2 encoding matrices, and in the second encoding step, a matrix encoded by adding a random matrix based on work entanglement-based coding using a second Chebyshev polynomial of degree L2 to an encoding matrix to encode it into L1 encoding matrices. Claim 5 A distributed matrix operation method using work entanglement-based coding according to claim 4, wherein the decoding step decodes the received matrix operation results to recover the operation results of the first matrix and the second matrix when the number of received matrix operation results reaches a second recovery threshold. Claim 6 A method for performing distributed matrix operations using a main server and multiple edge devices in a distributed computing environment having a first matrix data set and a second matrix data set to be operated on by multiple edge devices, comprising: a one-hot encoding step in which the main server one-hot encodes the indices within the corresponding data sets of the first matrix and the second matrix to be operated on, respectively; a first encoding step in which the main server, for each edge device, encodes the one-hot encoded matrix for the first matrix into a first encoded matrix based on work entanglement-based coding using Chebyshev polynomials; a second encoding step in which the main server, for each edge device, encodes the one-hot encoded matrix for the second matrix into a second encoded matrix based on work entanglement-based coding using Chebyshev polynomials; a transmission step in which the main server transmits the encoded matrix for each edge device to the corresponding edge device; a first matrix encoding step in which each edge device encodes the first matrix by matrix multiplying the entire matrix of the first matrix data set with the first encoded matrix; and a second matrix encoding step in which each edge device encodes the second matrix by matrix multiplying the entire matrix of the second matrix data set with the second encoded matrix. A second matrix encoding step; a matrix operation step in which each edge device performs a matrix operation on an encoded first matrix and an encoded second matrix; an operation result transmission step in which each edge device transmits the operation result to a main server; a reception step in which the main server receives matrix operation results from the edge devices; and a decoding step in which the main server decodes the received matrix operation results to recover the operation results of the first matrix and the second matrix when the number of received matrix operation results reaches a first recovery threshold.A distributed matrix operation method using task entanglement-based coding, comprising: a first encoding step is a step of encoding a matrix one-hot encoded for a first matrix for each edge device according to a determined number of tasks L (L=L1L2, where L1 and L2 are coprime) into L2 encoding matrices based on task entanglement-based coding using a first Chebyshev polynomial of degree L1; and a second encoding step is a step of encoding a matrix one-hot encoded for a second matrix for each edge device into L1 encoding matrices based on task entanglement-based coding using a second Chebyshev polynomial of degree L2. Claim 7 delete Claim 8 In claim 6, the method further comprises: a computational coordinate selection step in which a main server selects L2 computational coordinates to be used in a first encoding step and L1 computational coordinates to be used in a second encoding step; wherein the computational coordinates are obtained by performing steps i) through iv) below for each edge device, i) any value t between (-1, 1). i select ( ii) the number of edge devices root of Find (f(x) is the first Chebyshev polynomial of degree L1)iii) root of Seeking ( g(x) iv)the second Chebyshev polynomial of degree L2 For j satisfying Rearrange the operation coordinates so that the work entanglement condition of <Equation 2> is satisfied. A distributed matrix operation method using rearrangement and task entanglement-based coding. Claim 9 A distributed matrix operation method using work entanglement-based coding according to claim 8, wherein in the first encoding step, the main server encodes the encoding matrix into L2 encoding matrices by adding a matrix encoded based on work entanglement-based coding using a first Chebyshev polynomial of degree L1 to the encoding matrix, and in the second encoding step, the main server encodes the encoding matrix into L1 encoding matrices by adding a matrix encoded based on work entanglement-based coding using a second Chebyshev polynomial of degree L2 to the encoding matrix. Claim 10 In claim 9, the decoding step decodes the received matrix operation results when the number of matrix operation results received by the main server reaches a second recovery threshold, thereby recovering the operation results of the first matrix and the second matrix, a distributed matrix operation method using work entanglement-based coding.

Images