Method for distributed matrix multiplication using the network structure of distributed computing

Abstract

A distributed matrix multiplication method in a distributed computing system configured with a master node and a plurality of worker nodes according to an embodiment of the present invention comprises: a step in which a master node generates a first partitioning function and a second partitioning function based on the number of groups according to the partitioning order (pm) of a first input matrix and the partitioning order (pn) of a second input matrix, and broadcasts them through a network having a multi-access channel structure; a step in which at least pmn worker nodes each encode different first submatrices and second submatrices forming parts of the first input matrix and the second input matrix based on the first partitioning function and the second partitioning function received through the network, thereby generating a first encoding matrix and a second encoding matrix; and a step in which each worker node performs an encoding matrix multiplication operation using the first encoding matrix and the second encoding matrix generated by each. and the master node includes the step of collecting the results of the operation from each of the worker nodes and restoring the result of the multiplication operation for the first input matrix and the second input matrix.

Description

Technology Field

[0001] The present invention relates to a distributed matrix multiplication method and a distributed computing system for performing the same, and more specifically, to a method and system for improving the accuracy of distributed matrix multiplication based on the structural characteristics of a network. Background Technology

[0002] Recently, services accompanied by programs or applications called high intelligence, such as big data analysis, augmented reality, and haptic communication, are being introduced, and high-dimensional matrix operations are essential for their actual commercialization.

[0003] Since performing such heavy computations on a single device takes a long time, research is being conducted on distributed computing that distributes communication across multiple edge devices.

[0004] Notably, coding computing emerged to resolve congestion issues regarding computational accuracy, speed, and load; this is a method that enables the restoration of the original matrix multiplication result using the coded matrix multiplication result.

[0005] However, despite the fact that communication channels, noise, and network structures directly affect the computational accuracy of coding computing, existing coding computing methods did not consider these circumstances, and their network structures simply utilized P2P communication.

[0006] Specifically, due to computing power or communication constraints of edge devices, the time it takes for parallel processing operations to be transmitted to the main device can be significantly delayed; therefore, even if all operation results are received from the remaining edge devices, the result of the entire matrix multiplication operation cannot be recovered if results are not received from one or more edge devices.

[0007] In other words, to address situations where results are inefficiently obtained due to such stragglers, techniques have been proposed to recover the operation value of the entire matrix product based on submatrix encoding, even if operation results are not received from at least some of the edge devices.

[0008] However, most existing studies were conducted under the assumption that the communication link between the main device and the edge device is error-free; this failed to account for communication fading or noise, presenting a problem that made it difficult to extend distributed computing techniques to wireless communication environments. Prior art literature

[0009] Paper Q. Yu, MA Maddah-Ali, and AS Avestimehr, “Straggler mitigation in distributed matrix multiplication: Fundamental limits and optimal coding,” IEEE Transactions on Information Theory, vol. 66, no. 3, pp. 1920-1933, 2020. The problem to be solved

[0010] The present invention aims to solve the problems of the aforementioned prior art by providing a system and a method that perform distributed product operations by utilizing structural characteristics in a wireless network environment based on a group algebra-based coding computing technique.

[0011] However, the technical problems that this embodiment aims to solve are not limited to the technical problems described above, and other technical problems may exist. means of solving the problem

[0012] A distributed computing system for performing a distributed matrix multiplication operation according to one embodiment of the present invention includes a master node that processes a multiplication operation of a first input matrix and a second input matrix, and a plurality of worker nodes configured to process a portion of the multiplication operation of the first input matrix and the second input matrix distributed by the master node in parallel and transmit it to the master node.

[0013] According to one embodiment of the present invention, the distributed matrix multiplication method comprises: a step in which the master node generates a first partitioning function based on the number of groups according to the partitioning order (pm) of the first input matrix and a second partitioning function based on the number of groups according to the partitioning order (pn) of the second input matrix, and broadcasts them through a network having a multiple access channel structure; a step in which at least pmn worker nodes generate a first encoding matrix and a second encoding matrix by encoding different first submatrixes and second submatrixes forming parts of the first input matrix and the second input matrix, respectively, based on the first partitioning function and the second partitioning function received through the network; a step in which each worker node performs an encoding matrix multiplication operation using the first encoding matrix and the second encoding matrix generated by each; and a step in which the master node collects the results of the operation from each worker node and restores the result of the multiplication operation for the first input matrix and the second input matrix. Effects of the invention

[0014] According to one embodiment of the present invention, the speed, accuracy, and load reduction of high-dimensional matrix multiplication operations are improved.

[0015] According to one embodiment of the present invention, the complexity of high-dimensional matrix operations is reduced compared to conventional research.

[0016] According to one embodiment of the present invention, it can be applied even in a noisy wireless communication environment, and calculation errors can be minimized.

[0017] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below. Brief explanation of the drawing

[0018] FIG. 1 is a structural diagram of a distributed computing system according to one embodiment of the present invention. FIG. 2 is a block diagram of the configuration of a master node according to one embodiment of the present invention. FIG. 3 is a conceptual diagram illustrating a distributed matrix multiplication operation method that forms the basis of the present invention. FIG. 4 is a conceptual diagram illustrating a first embodiment of the distributed matrix multiplication operation method of the present invention. FIG. 5 is a conceptual diagram illustrating a second embodiment of the distributed matrix multiplication operation method of the present invention. FIG. 6 is a flowchart of the operation of a distributed matrix multiplication method according to one embodiment of the present invention. FIGS. 7(a) and FIGS. 7(b) are drawings for explaining the effects derived from an embodiment of the present invention. Specific details for implementing the invention

[0019] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0020] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other components interposed between them. Furthermore, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0021] In the following description, terms such as first, second, etc. may be used to describe various components, but said components should not be limited by said terms, and said terms are used only for the purpose of distinguishing one component from another. Additionally, in the following description, singular expressions may include plural expressions unless the context clearly indicates otherwise.

[0022] The "user terminal" mentioned below may be implemented as a computer or portable terminal capable of connecting to a server or other terminal via a network. Here, the computer may include, for example, a notebook, desktop, or laptop equipped with a web browser. The portable terminal may include, for example, a wireless communication device that ensures portability and mobility, such as a smartphone, tablet PC, wearable device, and various VR HMD devices, as well as various devices equipped with communication modules such as Bluetooth Low Energy (BLE), NFC, RFID, Ultrasonic, Infrared, WiFi, and LiFi. Furthermore, the "network" refers to a connection structure capable of exchanging information between respective nodes, such as terminals and servers, and includes a Local Area Network (LAN), a Wide Area Network (WAN), the World Wide Web (WWW), wired and wireless data communication networks, telephone networks, and wired and wireless television communication networks. Examples of wireless data communication networks include, but are not limited to, 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Bluetooth communication, infrared communication, ultrasonic communication, Visible Light Communication (VLC), and LiFi.

[0023] Hereinafter, an embodiment of the present invention will be described in detail with reference to the attached drawings.

[0024] FIG. 1 is a structural diagram of a distributed computing system according to an embodiment of the present invention. Referring to FIG. 1, the distributed computing system (10) includes a master node (100) and a plurality of worker nodes (200) connected to the master node (100) through a network.

[0025] A master node (100) according to one embodiment refers to a main device or server within a distributed computing system (10) and can process a high-intelligence task requested from a user terminal (not shown) and provide the result. Here, the high-intelligence task is implemented as a program or application, such as big data analysis, haptic communication, augmented reality, mixed reality, virtual reality, metaverse, artificial intelligence model, etc., but is not necessarily limited to examples, and any task requiring high-dimensional matrix operations should be interpreted as being included in the present invention.

[0026] Typically, the master node (100) can process the multiplication of the first input matrix and the second input matrix to derive a desired result within a task requested by a user terminal, and this is merely an exemplary expression for illustrative purposes, and the number of matrices subject to the multiplication operation is not actually limited.

[0027] FIG. 2 is a block diagram of the configuration of a master node (100) according to an embodiment of the present invention. Referring to FIG. 2, the master node (100) may include a processor (110), a memory (120), and a communication module (130).

[0028] The memory (120) can store commands, etc., for the master node (100) to perform a multiplication operation of a first input matrix and a second input matrix. The memory (120) can store an executable program that generates and executes one or more commands to implement the operation.

[0029] The memory (120) may include internal memory and / or external memory, and may include volatile memory such as DRAM, SRAM, or SDRAM, non-volatile memory such as OTPROM (one time programmable ROM), PROM, EPROM, EEPROM, mask ROM, flash ROM, NAND flash memory, or NOR flash memory, flash drives such as SSD, CF (compact flash) card, SD card, Micro-SD card, Mini-SD card, Xd card, or memory stick, or storage devices such as HDD. The memory (120) may include magnetic storage media or flash storage media, but is not limited thereto.

[0030] The processor (110) can execute its own process included in the distributed matrix multiplication operation method according to the following embodiment based on programs and instructions stored in memory (120). The processor (110) may include all kinds of devices capable of processing operations on data.

[0031] A processor (110) may refer to a data processing device embedded in hardware having a physically structured circuit to perform functions expressed by code or instructions included in a program, for example. Examples of such data processing devices embedded in hardware may include, but are not limited to, processing devices such as a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), and a graphics processing unit (GPU).

[0032] The communication module (130) may provide a communication interface that provides transmission and reception signals as packet data to external devices, including user terminals and worker nodes (200), using wireless communication technology. Additionally, the communication module (130) may be a device including hardware and software necessary to transmit and receive control signals or data signals, etc., through wired or wireless connections with other network devices.

[0033] According to one embodiment, a plurality of worker nodes (200) are included in a distributed computing system (10), and below, an environment is exemplified in which a total of W worker nodes, from the first worker node (WN1) to the second worker node (WNW), are configured.

[0034] A worker node (200) refers to a type of edge device or server that processes a portion of the multiplication operation of the first input matrix and the second input matrix distributed by the master node (100) in parallel for the distributed matrix multiplication operation of the first input matrix and the second input matrix, and transmits the result to the master node (100).

[0035] For example, a master node (100) may receive a task processing request signal from a user terminal. The master node (100) may request distributed processing from at least one worker node (200) for the task received from the user terminal. Each worker node (200) processes different parts of the multiplication operation of the first input matrix and the second input matrix in parallel and transmits them to the master node (100) via wireless communication, and the master node (100) may aggregate the distributed processing results to generate a task processing result signal and transmit it to the user terminal.

[0036] Hereinafter, the process of a distributed matrix multiplication operation according to the present invention will be specifically explained using FIGS. 3 to 5.

[0037] First, let's organize the terms used in the formulas. and represents real and complex numbers, respectively. is defined as a finite field of size N. is defined as the set of integers of the modulo N operation of integer N. is defined as the remainder of R divided by 1. am. It is defined as the inner product of the vector v and the vector w. is defined as the expectation value of the random variable X. Is It is defined as the size (cardinality) of the set. is a finite field It is defined as the multiplicative group (cyclic group). is of matrix M The nth row and Represents the nth column. represents the Moore-Penrose inverse of matrix M. represents the conjugate transpose of matrix M. represents the Frobenius norm of matrix M. class represents the Kronecker product and Hadamard product of vectors M and N, respectively.

[0038] In addition, the input matrix and its multiplication operation performed according to the user's request are defined as follows. However, this is merely an example for illustrative purposes, and the scope of the present invention is not limited thereto.

[0039] In this example, two matrices and Consider a distributed computing system (10) for a distributed multiplication operation. The master node (100) has a first input matrix ( ) and the second input matrix ( The product of ) We intend to calculate [something], and to do so, we request partial operations in parallel from at least some of the first worker node (WN1) to the W worker node (WNW).

[0040] At this time, the first input matrix ( ) and the second input matrix ( ) can be expressed as a first submatrix according to the first partitioning degree (pm) and a second submatrix according to the second partitioning degree (pn), as shown in Equation 1 below.

[0041] [Formula 1]

[0042]

[0043] At least pmn of the W worker nodes (200) have a first encoding matrix in which different first and second submatrices are encoded ( ) and the second encoding matrix ( Encoded matrix multiplication operations can be performed on each part using ). (w is in the range of 1 to W)

[0044] Based on the above assumptions, first, an embodiment that forms the basis of the encoding-based distributed matrix multiplication method will be explained using FIG. 3. FIG. 3 is a conceptual diagram for explaining the distributed matrix multiplication method that forms the basis of the present invention.

[0045] The master node (100) encodes the first submatrix and the second submatrix as shown in Equation 2 below for distribution to the w-th worker node (Wnw) to obtain the first encoded matrix ( ) and the second encoding matrix ( Creates ).

[0046] [Equation 2]

[0047]

[0048] Here, and Each is an encoding parameter that is set differently for each worker node (200) by the master node (100) so that the original result C can be restored.

[0049] The master node (100) has a preset transmission power P based on the following Equation 3. A and P B and transmission time According to, the first encoding matrix ( ) and the second encoding matrix ( ) respectively and Quantize the vector.

[0050] [Equation 3]

[0051]

[0052] The master node (100) during and It is transmitted to the w-th worker node (Wnw). Accordingly, the signal received by the worker node (Wnw) is set as shown in Equation 4 below.

[0053] [Equation 4]

[0054]

[0055] Here, and are respectively, and It is the channel coefficient of the wireless network from the master node (100) to the w-th worker node (Wnw). Also, and and respectively represent the channel noise of the network. Meanwhile, represents a composite Gaussian noise process, and its components are composite Gaussian random variables with a mean of 0 and a variance of 1.

[0056] After receiving the signal encoded as above, the w-th worker node (Wnw) and each and Decode it and perform the encoding matrix multiplication operation as shown in Equation 5 below.

[0057] [Formula 5]

[0058]

[0059] Afterwards, the w-th worker node (Wnw) transmission time Transmitted power P during w Depending on The vector is quantized. Accordingly, the signal received by the master node (100) from the worker node (Wnw) is given as Equation 6 below.

[0060] [Equation 6]

[0061]

[0062] Here, Is It is the channel coefficient of the network from the w-th worker node (Wnw) to the master node (100), and represents channel noise.

[0063] The master node (100) Decoding is performed based on, and the first input matrix ( ) and the second input matrix ( The multiplication operation of ) Restore to. That is, and It can be expressed as having mn submatrices, and and It can be defined as the (i, j)th partitioned submatrix.

[0064] As described above, the embodiment forming the basis of the present invention illustrated in FIG. 3 is a method in which a master node (100) communicates individually with each worker node (200) to transmit an encoding matrix for the distribution of operations, and each worker node (200) also communicates in parallel with the master node (100) to transmit the result of the encoding matrix multiplication operation. In the present invention, this is defined as OMA-OMA (orthogonal multiple access-orthogonal multiple access).

[0065] Hereinafter, the first and second embodiments of the present invention, devised based on OMA-OMA, will be described. FIG. 4 is a conceptual diagram illustrating the first embodiment of the distributed matrix multiplication method of the present invention. FIG. 5 is a conceptual diagram illustrating the second embodiment of the distributed matrix multiplication method of the present invention.

[0066] First, the present invention aims to present a method that is improved compared to OMA due to the structural characteristics of the wireless network environment. Specifically, the present invention presents a transmission technique that minimizes the probability of computational errors by utilizing the characteristics of broadcasting (BC) and multiple access channels (MAC), which is defined in this specification as BnC (Broadcast-and-Compute).

[0067] Two established theories were applied to establish BnC. The first theory explains matrix multiplication using a group theory framework by mapping each submatrix of the main matrix to elements of various types of groups, based on group algebra. (H. Cohn and C. Umans, "A group-theoretic approach to fast matrix multiplication," in Proc. 44th Annu. IEEE Symp. Found. Comput. Sci., Cambridge, MA, USA, 2003, pp. 438-449.)

[0068] Another second theory aims to reconstruct the linear combination of a given message using a MAC structure in a distributed computing environment. This theory proposes lattice-based encoders and decoders to reconstruct messages at improved transmission speeds, and this framework was defined as Compute-and-Forward (CF). ("Compute-and-forward: Harnessing interference through structured codes," IEEE Trans. Inf. Theory, vol. 57, no. 10, pp. 6463-6486, Oct. 2011.)

[0069] To further explain the second theory, k transmitted to relay d for each transmitter indexed as w = 1, ..., W w When there is a message vector of length, this is a finite field It is defined uniformly and independently of . The goal of the r-th relay is to reliably restore the linear combination of messages, which can be expressed by Equation 7 below.

[0070] [Equation 7]

[0071]

[0072] The second theory introduced CF to restore the equation of the linear combination at the desired transmission speed, and one embodiment of the present invention derived the following algorithm based on this.

[0073] Based on the second theory, all channel vectors and coefficient vector About Nested grid code with a rate ...may exist. In this case, the r-th relay is, according to the following Equation 8( Limited to a part of), the grid equation as in Equation 9 below (grid points ) average error probability It can be decoded.

[0074] [Equation 8]

[0075]

[0076] [Formula 9]

[0077]

[0078] Furthermore, based on the second theory, minimizing the noise upper limit of Equation 8 It can be calculated using the following formula 10.

[0079] [Formula 10]

[0080]

[0081] As a result, the computation speed can be derived from Equation 11 below.

[0082] [Equation 11]

[0083]

[0084] Based on these two theories, BnC adopts a method of requesting encoding matrix multiplication operations from each worker node (200) as the same information is broadcast through a wireless network. Additionally, a method of linearly combining the results of encoding matrix multiplication operations performed by each worker node (200) based on MAC and transmitting them collectively to the master node (100) can be additionally introduced.

[0085] Referring to FIG. 4, the first embodiment of the present invention can be defined as BnC-OMA, as a method of applying BnC when a master node (100) requests the distribution of matrix multiplication operations to each worker node. Briefly, unlike OMA, in which the master node (100) encodes each submatrix and sends it to the worker nodes (200), the first embodiment is a technique in which, when the master node (100) broadcasts the same information based on the number of units, each worker node (200) utilizes this to encode each submatrix differently.

[0086] Specifically, the first embodiment proceeds as follows. First, the master node (100) has a first input matrix ( based on the number of military units) The first partition function according to the partitioning degree (pm) of ) ) and the second input matrix ( The second partition function according to the partition degree (pn) of ) Generate ) as in Formula 12 below.

[0087] [Equation 12]

[0088]

[0089] That is, the first partition function ( ) and second partition function( ) is the first input matrix ( ) and the second input matrix ( Corresponding to the different first and second submatrices of ), each worker node (200) is responsible for the first encoding matrix ( ) and the second encoding matrix ( It can be understood as a frame that is commonly applied to generating ).

[0090] First partition function( ) and second partition function( A more specific example of generating ) is as follows. For the sake of the main explanation, group theory and representation theory, which form the basis of the first theory, are already known and will be omitted, and the triple product property applied to encoding matrix multiplication into group numbers will be introduced.

[0091] The triple product rule states that a given group subset for The right quotient set of It is denoted as . (i.e., ). If three subsets For each element According to Equation 13 below, S, T, and U can be seen as satisfying the triple product law.

[0092] [Equation 13]

[0093]

[0094] Here, the method of encoding matrix products by partitioning them based on the number of units satisfies the triple product rule Consider . Also, two complex matrices, A with the size of and Consider B with the size of . Accordingly, the army number-based encoding for A and B can be defined as shown in Equation 14 below.

[0095] [Equation 14]

[0096]

[0097] According to Formula 14, group elements The result of the AB matrix multiplication operation can be obtained from the coefficients of . Intuitively, this can be viewed as a generalization of the cyclic convolution concept of the DFT (discrete Fourier transform) to an arbitrary group.

[0098] Therefore, based on the aforementioned concept of the number of groups, when p, m, and n corresponding to the partitioning order are disjoint, a distribution of matrix multiplication operations utilizing a cyclic group can be introduced. That is, the master node (100) has a first input matrix ( ) and the second input matrix ( A cyclic group in which the elements g of the group are linearly combined and broadcast corresponding to each submatrix, based on the partition degree of ). Constructs. Here, is an arbitrary integer and is a relatively very small number compared to p, m, and n according to the known theory of military numbers.

[0099] Meanwhile, the circulating group As shown in Equation 15 below, three subsets are considered, and these satisfy the triple product rule.

[0100] [Formula 15]

[0101]

[0102] Therefore, based on the theory of military numbers, the master node (100) is a cyclic group Based on, the first input matrix ( ) and the second input matrix ( As ) maps each partitioned submatrix to an arbitrary matrix structure, the first partition function ( ) and second partition function( ) can be generated. That is, the master node (100) The submatrices divided using the element g of can be embedded, which can be expressed by the above Equation 12.

[0103] so, and It is represented as an embedded army number-based frame, but the encoding matrices processed by each worker node (200) are the same army number and From the perspective of the points derived from and can be understood as a type of partitioning function. Therefore, the master node (100) is the first partitioning function ( ) and second partition function( By generating and broadcasting through a network of MAC structures, the distribution of matrix multiplication operations to each worker node (200) can be carried out.

[0104] OMA is the first input matrix ( at the master node (100) ) and the second input matrix ( Encode each submatrix of ) to obtain the first encoding matrix ( ) and the second encoding matrix ( A method of generating ) and distributing to each worker node (200) via individual communication, or BnC is a first partition function ( ) and second partition function( ) are transmitted in batches, and at the worker node (200) each first encoding matrix ( ) and the second encoding matrix ( There is a fundamental difference in generating ). Accordingly, the difference in transmission methods between BnC and OMA is the pre-configured transmission time In this case, the master node (100) has a first encoding matrix ( ) and the second encoding matrix ( Instead of transmitting ), the first partition function( ) and second partition function( It can be interpreted as a quantization step size for transmitting ).

[0105] Specifically, in BnC, the first partition function ( ) and second partition function( ) is the number of troops and Since it is an element of, the first partition function( ) and second partition function( ) are each and A group of cyclic groups with coefficients in the form of a complex matrix ( It can be expressed as a linear combination of the elements (g) of ).

[0106] Based on this, the master node (100) has a first partition function ( ) and second partition function( Total transmission time of ) Matrix multiplication operations can be distributed to each worker node (200) by dividing pmn intervals with equal spacing and transmitting different coefficients for each interval. That is, the master node (100) in the i-th interval the first partition function ( g, the i-th group element of ) i The coefficients of can be transmitted through the network.

[0107] Here, the coefficient of each group is Since it is a complex matrix with dimensions, the master node (100) is the first partition function ( About ) It can be quantized into bits. Second partition function ( ) is also quantized in the same way.

[0108] Therefore, the master node (100) has a first partition function ( ) and second partition function( The quantization unit for broadcasting ) can be given by Equation 16 below.

[0109] [Equation 16]

[0110]

[0111] That is, the quantization step size for transmitting each element of a cyclic group in BnC is the first partition function ( ) and second partition function( ) transmission time and channel capacity, first input matrix ( ) and the second input matrix ( It can be determined based on the division order and dimension of ).

[0112] When broadcasting by the master node (100) proceeds, at least pmn (W > pmn) worker nodes (200) receive the first partition function ( ) and second partition function( Based on ), the first input matrix ( ) and the second input matrix ( By encoding the distinct first and second submatrices that form part of ) respectively, the first encoding matrix ( ) and the second encoding matrix ( Creates ).

[0113] As an example, in BnC, the first encoding matrix generated by each worker node (200) ) and the second encoding matrix ( ) is the first partition function according to Equation 12( ) and second partition function( As derived by ), Equation 2 in OMA can be equivalently modeled as shown in Equation 17 below.

[0114] [Equation 17]

[0115]

[0116] To explain this, each worker node (200) has a first encoding matrix ( ) and the second encoding matrix ( Since matrix multiplication operations must be performed on ), at this time, the first partition function ( ) and second partition function( The coefficients of the cyclic group elements g corresponding to the first and second submatrices in ) are complex values ​​( It can be converted into ). Based on the aforementioned first theory, the cyclic group ( The irreducible representation (irreducible representation) of ) is a total of as a function It is a dog, It can be expressed as shown in Equation 18 below.

[0117] [Equation 18]

[0118]

[0119] Here, am.

[0120] In addition, of the above Equation 18 According to, is established, is a cyclic group for any integer ( )cast It is the set of remainders divided by. That is, as shown in Equation 19 below, the remainder of dividing the degree applied to the coefficients converted to complex values ​​by pmn varies for each worker node (200), and accordingly, each worker node (200) has a different first encoding matrix ( based on Equation 17 above). ) and the second encoding matrix ( ) can be derived.

[0121] [Equation 19]

[0122]

[0123] First encoding matrix ( ) and the second encoding matrix ( When ) is generated, each worker node (200) uses it to perform an encoding matrix multiplication operation.

[0124] As an example, the transmission signal in BnC can be equivalently modeled by Equation 20 below in relation to Equations 4 and 6 described in OMA, as the effects of the encoder and decoder on the network are removed.

[0125] [Formula 20]

[0126]

[0127] Here, ( )Is It is quantization noise that is uniformly distributed across the interval.

[0128] Accordingly, each worker node (200) is the result of the operation Acquire the quantization and transmit it to the master node (100).

[0129] As an example, each worker node (200) may quantize the results of the encoding matrix multiplication operations performed by each node for sequentially set transmission intervals and transmit them to the master node (100). Here, a transmission interval may refer to the total transmission time taken to transmit the results of all encoding matrix multiplication operations divided into pmn equal intervals.

[0130] Specifically, the w-th worker node (Wnw) in the w-th transmission section With ergodic channel capacity, maximum It can transmit bits. Also, The size of Therefore, each element is It can be quantized as. Therefore, The quantization step size of is It can be expressed as.

[0131] And, after removing the influence of the encoder and decoder of the network channel, the received signal of the master node (100) is It can be represented as. Here, Is It refers to quantization noise that follows a uniform distribution in the interval (a=C).

[0132] Subsequently, the master node (100) collects the results of the encoding matrix multiplication operation from each worker node (200) and the first input matrix ( ) and the second input matrix ( Restores the multiplication operation of ).

[0133] As a specific embodiment, the master node (100) from all worker nodes (200) If you collect, the one you originally intended to obtain It can be restored in the following way. First, the result of the encoding matrix multiplication operation ( The index set of the worker node (200) that transmitted ) If defined as such, the result of the encoded matrix multiplication operation ( )for The nth row and Connecting the nth column can be represented by the following formula 21.

[0134] [Equation 21]

[0135]

[0136] In addition, a column vector can be defined as shown in Equation 22 below.

[0137] [Equation 22]

[0138]

[0139] Is Since it can be expressed as a linear combination of, also It can be expressed as a linear combination of, and accordingly and The linear equation of can be expressed as Equation 23 below.

[0140] [Equation 23]

[0141]

[0142] Here, since P is a value already set in the master node (100), the master node (100) By restoring, It can derive.

[0143] In the first embodiment described above, as illustrated in FIG. 4, BnC is applied during the process of distributing matrix multiplication from the master node (100) to the worker node (200), and the process of transmitting the result of the encoding matrix multiplication operation from the worker node (200) to the master node (100) follows OMA, and is defined as BnC-OMA in this specification.

[0144] Next, referring to FIG. 5, the second embodiment of the present invention is a method in which a master node (100) utilizes Compute-and-Forward (CF) according to the second theory in the process of aggregating the encoding matrix multiplication operations of each worker node (200). Accordingly, the second embodiment is defined as BnC-CF in this specification, and the process up to the stage where each worker node (200) performs the encoding matrix multiplication operation overlaps with the first embodiment and is replaced by the aforementioned method.

[0145] According to the second embodiment, the result of each encoding matrix multiplication operation performed by each worker node (200) ( ) is linearly combined during transmission over the network depending on the characteristics of the MAC structure ( It can be reached to the master node (100).

[0146] The difference between OMA and CF is the result of the distributed matrix multiplication operation performed by the w-th worker node (Wnw) It can be viewed as the encoding and decoding process of ). That is, by utilizing the grid-based CF encoder and decoder introduced in the second theory A new speed can be derived for transmitting to the master node (100).

[0147] To this end, a function for encoding matrix multiplication of the worker node (200) is identified, and the result of the operation ( The computational speed at which ) can be transmitted based on CF must be calculated.

[0148] Specifically, the first encoding matrix ( ) and the second encoding matrix ( According to Equation 17 defining ), the first encoding matrix ( ) and the second encoding matrix ( The product can be derived using the following formula 24.

[0149] [Equation 24]

[0150]

[0151] According to Equation 24, if we calculate w=1, From the linear combination of the operation results for, the matrix to be obtained ( The (i, j)th submatrix of ) can be represented by the following Equation 25.

[0152] [Formula 25]

[0153]

[0154] Here, And, am.

[0155] Accordingly, based on Formula 7 of CF, the function is set as in Formula 26 below, and at each worker node (200) When transmitted in this way, the master node (100) uses the matrix ( The (i, j)th submatrix of ) ) can be restored.

[0156] [Equation 26]

[0157]

[0158] In particular, according to Equation 11 based on the second theory, the following Equation 27 can be derived, and the master node (100) decodes Equation 26 at a computational speed satisfying this and a matrix ( Each submatrix of ) can be restored.

[0159] [Equation 27]

[0160]

[0161] That is, the result of each encoding matrix multiplication operation ( ) is the first input matrix ( ) and the second input matrix ( It can be seen that the lattice-based complex values ​​according to the division order of ) are linearly combined into an equation (Equation 26) in which coefficients are applied. Accordingly, the master node (100) can decode the equation (Equation 26) at a computational speed that satisfies a preset equation (Equation 27) based on noise on the network, and restore the result of the multiplication operation for each first submatrix and second submatrix.

[0162] Meanwhile, the result of the encoding matrix multiplication operation of each worker node (200) ( In ) transmission, the CF decoder without any distortion It is assumed that the field size of the CF is sufficiently large to obtain. Based on this, each worker node (200) has a transmission speed ( ) and operation speed ( The result of each encoded matrix multiplication operation ( ) can be quantized and transmitted.

[0163] That is, each worker node (200) is a grid equation ( Each element of ) It can be uniformly quantized into bits, and the quantization step size can be expressed by the following Equation 28.

[0164] [Equation 28]

[0165]

[0166] Therefore, in the first embodiment, the received signal according to the OMA ( ) can be modeled equivalently as in Equation 29 below.

[0167] [Equation 29]

[0168]

[0169] Here, is the section It is quantization noise with a uniform distribution over. )

[0170] With respect to the first and second embodiments of the present invention described above, the effects supported by FIG. 7 are presented. FIG. 7(a) and FIG. 7(b) are drawings for explaining the effects derived from the embodiments of the present invention.

[0171] Specifically, FIG. 7 is experimental data showing the average computation error according to the signal-to-noise ratio (SNR). p=6 and m=n=1 were set, and FIG. 7(a) shows the graph of the experiment results with 12 worker nodes (200) and FIG. 7(b) shows the graph with 90. Here, UN stands for uncoded scheme, which means that encoding for the distributed matrix multiplication operation was not performed. GA stands for encoding performed based on the first theory.

[0172] First, referring to FIGS. 7(a) and FIGS. 7(b), it can be seen that the GA, in which distributed matrix multiplication operations are performed based on the first theory (army number), shows better performance than the UN, in which they are not. Additionally, it can be seen that when transmission is performed using BnC (first embodiment), the performance is superior to when using the existing OMA. Furthermore, as shown in FIG. 7(b), when the number of worker nodes (200) is large, the use of CF (second embodiment) is more effective than when using the existing OMA, thus confirming that it is effective for introduction into high-intelligence operations.

[0173] Hereinafter, the main contents of the present invention will be summarized using FIG. 6. FIG. 6 is a flowchart of the operation of a distributed matrix multiplication method according to one embodiment of the present invention, and overlapping contents are replaced by the foregoing.

[0174] First, the master node (100) has a first input matrix ( ) and the second input matrix ( The product of ) We intend to calculate [something], and to do so, we request partial operations in parallel from at least some of the first worker node (WN1) to the W worker node (WNW).

[0175] In step S610, the master node (100) has a first input matrix ( The partitioning order (pm) of ) and the second input matrix ( The first partition function based on the number of groups according to the partition degree (pn) of ) ) and second partition function( Creates ).

[0176] As an example, the master node (100) is a first input matrix ( ) and the second input matrix ( A cyclic group () composed of elements (g) corresponding to each submatrix partitioned by ) Construct ) and, based on this, linearly combine each submatrix by mapping it to an arbitrary matrix structure to obtain the first partition function ( ) and second partition function( Can generate ).

[0177] In step S620, the master node (100) is the first partition function ( ) and second partition function( Broadcasts ) through a network with a multi-access channel structure.

[0178] As an example, the master node (100) is a first partition function ( ) and second partition function( The pre-set transmission time of ) is divided into pmn intervals with equal spacing, and for each divided interval, coefficients that are set differently are transmitted corresponding to each first submatrix and second submatrix.

[0179] As an example, the master node (100) has a first division function (Quantization Step Size) with a preset quantization unit (Quantization Step Size). ) and second partition function( ) can be quantized. Here, the quantization unit ( , ) is the first partition function( ) and second partition function( ) transmission time and channel capacity, first input matrix ( ) and the second input matrix ( It can be set based on the division order and dimension of ).

[0180] In step S630, at least pmn worker nodes (200) receive a first partition function ( ) and second partition function( Based on ), the first input matrix ( ) and the second input matrix ( By encoding the distinct first and second submatrices that form part of ) respectively, the first encoding matrix ( ) and the second encoding matrix ( Creates ).

[0181] As an example, each worker node (200) is a first partition function ( ) and second partition function( The coefficients of the first and second submatrices, each responsible for the distributed product operation in ), are complex values ​​( It can be converted into ). Here, the remainder obtained by dividing the degree applied to the complex value by the above pmn is different for each worker node (200).

[0182] In step S640, each worker node (200) has a first encoding matrix (each generated by itself) ) and the second encoding matrix ( Encoded matrix multiplication operation using ) Performs ).

[0183] As an optional first embodiment, each worker node (200) has the result of the encoding matrix multiplication operation performed by each sequentially set transmission interval ( ) can be quantized and transmitted to the master node (100). Here, the transmission interval may be the total transmission time taken to transmit the result of all encoding matrix multiplication operations divided into pmn equal intervals.

[0184] As an optional second embodiment, the result of each encoding matrix multiplication operation performed by each worker node (200) ( ) is linearly combined during transmission over the network ( ) can be reached to the master node (100). For example, the result of each encoding matrix multiplication operation ( ) is the first input matrix ( ) and the second input matrix ( Equation in which grid-based complex values ​​according to the partitioning order of ) are applied as coefficients ( It may be a linear combination of ).

[0185] In step S650, the master node (100) receives the result of the operation from each worker node (200) By combining ) the first input matrix ( ) and the second input matrix ( The result of the multiplication operation on ) Restores ).

[0186] As an embodiment extending from the second embodiment above, the master node (100) has a computational speed that satisfies a preset formula based on noise on the network ( ) lattice equation( Decrypting ) and the result of the multiplication operation for each first submatrix and second submatrix ( , , ) can be restored.

[0187] One embodiment of the present invention may also be implemented in the form of a non-transient recording medium comprising computer-executable instructions, such as program modules executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media. Additionally, a computer-readable medium may include all computer storage media. A computer storage medium includes both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data.

[0188] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0189] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0190] 10: Distributed Computing Systems 100: Master Node 200: Worker Node

Claims

Claim 1 A distributed matrix multiplication method in a distributed computing system comprising a master node and a plurality of worker nodes, comprising: a) a step in which a master node generates a first partitioning function and a second partitioning function based on the number of groups according to the partitioning order (pm) of a first input matrix and the partitioning order (pn) of a second input matrix, and broadcasts them to the plurality of worker nodes through a network having a multi-access channel structure; b) a step in which at least pmn worker nodes each encode different first submatrices and second submatrices forming parts of the first input matrix and the second input matrix based on the first partitioning function and the second partitioning function received through the network, thereby generating a first encoding matrix and a second encoding matrix; c) a step in which each worker node performs an encoding matrix multiplication operation using the first encoding matrix and the second encoding matrix generated by each; and d) a distributed matrix multiplication method comprising the step of the master node collecting the results of the operation from each worker node and restoring the result of the multiplication operation for the first input matrix and the second input matrix. Claim 2 A distributed matrix multiplication method according to claim 1, wherein step a) comprises the step of the master node constructing a cyclic group composed of elements corresponding to each submatrix into which the first input matrix and the second input matrix are divided, and generating the first partition function and the second partition function by linearly combining the submatrixes based on the cyclic group by mapping each submatrix to an arbitrary matrix structure. Claim 3 A distributed matrix multiplication method according to claim 1, wherein step a) comprises the step of the master node dividing the preset transmission time of the first partition function and the second partition function into pmn intervals having equal intervals, and transmitting coefficients that are set differently corresponding to each first submatrix and second submatrix for each partitioned interval. Claim 4 A distributed matrix multiplication method according to claim 1, wherein step a) comprises the step of the master node quantizing the first partition function and the second partition function into a pre-set quantization unit (Quantization Step Size) and broadcasting, wherein the quantization unit is set based on the transmission time and channel capacity of the first partition function and the second partition function, and the partition order and dimension of the first input matrix and the second input matrix. Claim 5 A method for performing a distributed matrix multiplication operation according to claim 1, wherein step b) includes the step of each worker node converting the coefficients of a first submatrix and a second submatrix, each responsible for performing a distributed product operation in the first partition function and the second partition function, into complex values, wherein the remainder obtained by dividing the degree applied to the complex values ​​by pmn is different for each worker node. Claim 6 A distributed matrix multiplication method according to claim 1, wherein step c) includes the step of each worker node quantizing the result of the encoding matrix multiplication operation performed by each of the sequentially set transmission intervals and transmitting it to the master node, wherein the transmission interval is a division of the total transmission time taken to transmit the result of all encoding matrix multiplication operations into pmn equal intervals. Claim 7 A distributed matrix multiplication method according to claim 1, wherein step c) comprises the step of linearly combining the results of each encoding matrix multiplication operation performed by each worker node while being transmitted through the network and reaching the master node. Claim 8 A distributed matrix multiplication method according to claim 7, wherein step d) comprises the step of the master node decoding the equation at a computational speed satisfying a preset formula based on noise on the network to restore the result of the multiplication operation for each first submatrix and second submatrix, wherein the result of each encoded matrix multiplication operation is linearly combined into an equation in which a lattice-based complex value according to the division order of the first input matrix and the second input matrix is ​​applied as a coefficient. Claim 9 In a distributed computing system that performs a distributed matrix multiplication operation method, a master node that processes the multiplication operation of a first input matrix and a second input matrix; The method comprises a plurality of worker nodes configured to process in parallel a portion of the multiplication operation of the first input matrix and the second input matrix distributed by the master node and transmit it to the master node, and the process of performing the distributed matrix multiplication operation method comprises: the master node generating a first partitioning function and a second partitioning function based on the number of groups according to the partitioning order (pm) of the first input matrix and the partitioning order (pn) of the second input matrix, and broadcasting them to the plurality of worker nodes through a network of a multi-access channel structure; at least pmn worker nodes generating a first encoding matrix and a second encoding matrix by encoding different first submatrices and second submatrices forming parts of the first input matrix and the second input matrix, respectively, based on the first partitioning function and the second partitioning function received through the network; each worker node performing an encoding matrix multiplication operation using the first encoding matrix and the second encoding matrix generated by each; and the master node A distributed computing system that collects the results of the operation from each of the above worker nodes and restores the result of the multiplication operation for the first input matrix and the second input matrix. Claim 10 A distributed computing system according to claim 9, wherein the master node constructs a cyclic group composed of elements corresponding to each submatrix into which the first input matrix and the second input matrix are divided, and generates the first partitioning function and the second partitioning function by linearly combining the submatrixes based on the cyclic group by mapping them to an arbitrary matrix structure. Claim 11 A distributed computing system according to claim 9, wherein the master node divides the preset transmission time of the first partition function and the second partition function into pmn intervals having equal intervals, and transmits coefficients that are set differently for each of the divided intervals, corresponding to the first submatrix and the second submatrix. Claim 12 A distributed computing system according to claim 9, wherein the master node quantizes and broadcasts the first partition function and the second partition function into a preset quantization unit, wherein the quantization unit is set based on the transmission time and channel capacity of the first partition function and the second partition function, and the partition order and dimension of the first input matrix and the second input matrix. Claim 13 A distributed computing system according to claim 9, wherein each worker node converts the coefficients of a first submatrix and a second submatrix, each responsible for a distributed product operation in the first partition function and the second partition function, into complex values, and the remainder obtained by dividing the degree applied to the complex values ​​by pmn is different for each worker node. Claim 14 A distributed computing system according to claim 9, wherein each worker node quantizes the result of an encoding matrix multiplication operation performed by each of the sequentially set transmission intervals and transmits it to the master node, wherein the transmission interval is a division of the total transmission time taken to transmit the result of all encoding matrix multiplication operations into pmn equal intervals. Claim 15 A distributed computing system according to claim 9, wherein the result of each encoding matrix multiplication operation performed by each worker node is linearly combined while being transmitted through the network and reaches the master node. Claim 16 A distributed computing system according to claim 15, wherein the result of each encoding matrix multiplication operation is linearly combined into an equation in which grid-based complex values ​​according to the division order of the first input matrix and the second input matrix are applied as coefficients, and the master node decodes the equation at a computational speed satisfying a preset formula based on noise on the network to restore the result of the multiplication operation for each first submatrix and second submatrix.

Images