Ensemble communication method and device based on network coding, equipment and medium

Abstract

The embodiment of the invention provides a set communication method and device based on network coding, equipment and a medium, and the method comprises the steps: responding to a network coding coefficient obtaining request, and constructing an initial coding coefficient matrix; calculating a condition number of the initial coding coefficient matrix to obtain an initial condition number; based on the initial condition number, optimizing the initial coding coefficient matrix to obtain a target coding coefficient matrix; sending the target coding coefficient matrix to a target computing node, so that the target computing node executes a set communication operation on the target data block based on the target coding coefficient matrix; wherein the target data block comprises a floating-point number, and the set communication operation comprises floating-point operation. According to the embodiment of the invention, the data transmission precision can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network coding, in particular to a set communication method and device based on network coding, equipment and medium. BACKGROUND

[0002] Set communication refers to the process of information exchange between multiple computing nodes in a computer cluster. Common set communication operations include broadcast, multicast, reduction, scattering, etc. Network coding is a technology designed to improve network transmission efficiency. By encoding original data packets at the source node, intermediate nodes re-encode the encoded data packets from different sources to obtain new encoded data packets, and the destination node decodes the received encoded data packets to recover the original data packets from different sources or obtain their calculation results (such as summation), thereby optimizing data flow and maximizing network throughput.

[0003] In traditional network coding technology, source nodes, intermediate nodes and destination nodes mainly rely on algebraic operations on finite fields to implement encoding and decoding. Because finite fields provide rigorous and closed algebraic structures, they theoretically ensure the mathematical reversibility and error-free nature of the encoding and decoding process. However, in many practical applications such as scientific computing and artificial intelligence, the data processed is often real numbers, while computers use floating-point numbers to approximate real numbers. Floating-point operations are a kind of finite-precision operations. When network coding is implemented based on floating-point operations, errors in the multi-level operation process will accumulate, leading to a deviation between the actual data decoded by the destination node and the ideal data, reducing the precision of data transmission. Therefore, a set communication method is needed that can effectively control error accumulation in the floating-point number domain, thereby improving data transmission and calculation precision. SUMMARY

[0004] The main purpose of the embodiments of the present application is to provide a set communication method and device based on network coding, equipment and medium, aiming to improve data transmission and calculation precision.

[0005] To achieve the above purpose, the first aspect of the embodiments of the present application provides a set communication method based on network coding, which comprises: In response to a network coding coefficient acquisition request, an initial coding coefficient matrix is constructed; The condition number of the initial coding coefficient matrix is calculated to obtain an initial condition number; Based on the initial condition number, the initial coding coefficient matrix is optimized to obtain a target coding coefficient matrix; The target encoding coefficient matrix is sent to the target computing node, so that the target computing node performs a collective communication operation on a target data block based on the target encoding coefficient matrix; wherein the target data block includes floating-point numbers, and the collective communication operation includes floating-point operation.

[0006] To achieve the above object, a second aspect of the embodiment of the present application provides a collective communication device based on network coding, the device comprising: The request response module is configured to construct an initial encoding coefficient matrix in response to a network coding coefficient acquisition request. The condition number calculation module is configured to calculate a condition number of the initial encoding coefficient matrix to obtain an initial condition number. The matrix optimization module is configured to optimize the initial encoding coefficient matrix based on the initial condition number to obtain a target encoding coefficient matrix. The matrix sending module is configured to send the target encoding coefficient matrix to the target computing node, so that the target computing node performs a collective communication operation on a target data block based on the target encoding coefficient matrix; wherein the target data block includes floating-point numbers, and the collective communication operation includes floating-point operation.

[0007] To achieve the above object, a third aspect of the embodiment of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the collective communication method based on network coding of the first aspect when executing the computer program.

[0008] To achieve the above object, a fourth aspect of the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the collective communication method based on network coding of the first aspect.

[0009] The collective communication method and device based on network coding, the equipment and the medium provided by the present application, in response to the network coding coefficient acquisition request, construct an initial encoding coefficient matrix; calculate the condition number of the initial encoding coefficient matrix to obtain an initial condition number; based on the initial condition number, the initial encoding coefficient matrix is optimized, since the initial condition number can represent the upper bound of the distortion rate of the initial encoding coefficient matrix, the initial condition number is used as the optimization basis of the initial encoding coefficient matrix, which can effectively constrain the distortion generated by the network coding on floating-point numbers, improve the precision and reliability of floating-point operation in collective communication; using the optimized target encoding coefficient matrix to perform collective communication operation can ensure the accuracy of data block in the transmission and processing process, and reduce the performance loss caused by floating-point number representation and operation error. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 is the flowchart of the collective communication method based on network coding provided by the embodiment of the present application; Figure 2 is Figure 1 a flowchart of step S103 in Figure 3 is another flowchart of the set communication method based on network coding provided by the embodiments of the present application; Figure 4 is a schematic diagram of set communication provided by the embodiments of the present application; Figure 5 is a schematic diagram of a target topology provided by the embodiments of the present application; Figure 6 is a schematic diagram of a data routing path provided by the embodiments of the present application; Figure 7 is still another flowchart of the set communication method based on network coding provided by the embodiments of the present application; Figure 8 is a structural schematic diagram of the set communication apparatus based on network coding provided by the embodiments of the present application; Figure 9 is a hardware structural schematic diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0011] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not intended to limit the present application.

[0012] It should be noted that although the functional modules are divided in the apparatus schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the apparatus or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0013] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0014] Modern computing node clusters rely on collective communication mechanisms to achieve cross-node data collaboration, including various collective communication operations such as broadcast, gather, reduce, all-reduce, reduce-scatter, and all-gather. Among them, multicast supports efficient data distribution from a single source node to multiple destination nodes, while summation reduction is used to aggregate data from different nodes. Other complex collective communication operations, such as all-reduce, reduce-scatter, and all-gather, can be implemented through the combination of multicast and reduction. Network coding is a technology designed to improve network transmission efficiency. By encoding original data packets at the source node, intermediate nodes re-encode the encoded data packets from different sources to obtain new encoded data packets, and destination nodes decode the received encoded data packets to recover the original data packets from different sources or obtain their calculation results (such as summation), thereby optimizing data flow and maximizing network throughput.

[0015] In related technologies, source nodes, intermediate nodes, and destination nodes mainly rely on algebraic operations on finite fields to implement encoding and decoding, because finite fields provide rigorous and closed algebraic structures that theoretically ensure the mathematical reversibility and error-free nature of the encoding and decoding processes. However, in many practical applications such as scientific computing and artificial intelligence, the data processed is often real numbers, while computer systems use floating-point numbers to approximate real numbers. Floating-point operations are a kind of finite-precision operations, and when network coding is implemented based on floating-point operations, errors in the multi-level operation process will accumulate, leading to a deviation between the actual data decoded by the destination node and the ideal data, reducing the precision of data transmission. Therefore, a collective communication method is needed that can effectively control error accumulation in the floating-point number domain, thereby improving data transmission and calculation precision.

[0016] For example, in a machine learning scenario, source nodes, intermediate nodes, and destination nodes are selected in a computing node cluster; each source node transmits gradient slices to the destination node through the intermediate node via collective communication. The data in the gradient slices, such as gradients, is essentially real data, and the encoding and decoding operations in the collective communication process involve floating-point operations, which will accumulate floating-point errors, affecting the precision of gradient data transmission and aggregation, and further affecting the convergence of the machine learning model.

[0017] Based on this, the embodiments of the present application provide a network coding-based collective communication method and device, equipment, and medium, aiming to improve data transmission and calculation precision.

[0018] The method and device for set communication based on network coding provided by the embodiments of the present application are described in detail as follows. First, a method for set communication based on network coding is described.

[0019] The method for set communication based on network coding provided by the embodiments of the present application can be applied to a terminal, a server, or software running in the terminal or the server. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc. The server can be configured as a physical server, a server cluster composed of multiple physical servers, or a distributed system. The server can also be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform. The software can be an application program for implementing the method for set communication based on network coding, but is not limited to the above forms.

[0020] The present application can be used in many general or special computer system environments or configurations. For example, personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as a program module. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0021] Figure 1 is an optional flowchart of the method for set communication based on network coding provided by the embodiments of the present application, Figure 1 The method in can include, but is not limited to, steps S101 to S104.

[0022] Step S101, in response to a network coding coefficient acquisition request, constructing an initial coding coefficient matrix; Step S102, calculating the condition number of the initial coding coefficient matrix to obtain an initial condition number; Step S103, based on the initial condition number, optimizing the initial coding coefficient matrix to obtain a target coding coefficient matrix; In step S104, the target encoding coefficient matrix is sent to the target computing node, so that the target computing node performs a collective communication operation on the target data block based on the target encoding coefficient matrix; wherein the target data block includes floating-point numbers, and the collective communication operation includes floating-point operations.

[0023] As can be easily understood, the network encoding coefficient is a calculation parameter used when encoding or decoding a data packet. The network encoding coefficient acquisition request is an instruction for triggering the construction of the encoding coefficient matrix, which can include the number of nodes participating in the collective communication, the network topology between the nodes, the data dimension of the data packet, and the like, but is not limited thereto. Based on the number of nodes, the network topology, the data dimension, and the like in the network encoding coefficient acquisition request, the dimension of the encoding coefficient matrix can be determined to create the encoding coefficient matrix.

[0024] As can be easily understood, the target computing node refers to a computing node participating in the collective communication in the computer cluster, in particular, a source node in the collective communication operation. For example, please refer to Figure 4 , Figure 4 The all-reduce operation process driven by encoding is illustrated. Nodes 1-6 encode the local symbols before sending data, generating two parallel data, intermediate nodes 7-10 aggregate encoded data from multiple paths through addition operations, and forward the aggregated results to the upstream nodes, and the final node 11 distributes back to all nodes after recovering the aggregated symbols through decoding. Nodes 1-10 can be regarded as target computing nodes, and the target encoding coefficient matrix can be obtained by the above method to determine the encoding coefficients of each target computing node. Correspondingly, the decoding operation is the inverse operation of the encoding operation, and the decoding coefficients of node 11 can be determined based on the target encoding coefficient matrix.

[0025] As can be understood, when optimizing the initial encoding coefficient matrix, all matrix elements in the initial encoding coefficient matrix can be optimized as variables, or only part of the matrix elements in the initial encoding coefficient matrix can be selected as variables for optimization. For example, when the data routing path of the data packet is determined, all matrix elements in the initial encoding coefficient matrix are optimized as variables; when a computing node is added to the data routing path, the matrix element corresponding to the added computing node is optimized as a variable.

[0026] It can be understood that the rank of the encoding coefficient matrix determines the decoding ability of the computing node to the data packet: if the coefficient matrix is full rank, the computing node can recover the original data by Gaussian elimination algorithm; otherwise, only part of the data can be recovered. In the real number field, due to the continuity and infinite density of the value space of the encoding coefficient, the full rank of the encoding coefficient matrix can be strictly guaranteed by the probability measure theory. Specifically, the encoding coefficient combination that makes the rank of the matrix reduce dimension constitutes a low-dimensional algebraic variety in the parameter space, and its Lebesgue measure on the real number field is zero. Therefore, when optimizing the initial encoding coefficient matrix, multiple initial encoding coefficient matrices of full rank can be obtained by multiple independent sampling from a continuous probability distribution, and the one with the smallest condition number is selected as the target encoding coefficient matrix; the initial encoding coefficient matrix can also be iteratively optimized by the minimum condition number objective function, and the iteration stopping conditions include that the encoding coefficient matrix is full rank, the absolute value of the difference between the objective function values of the adjacent two iterations is less than a threshold, etc. without being limited thereto.

[0027] Taking the machine learning scene as an example, at least one data routing path is selected in the computer cluster to complete the reduction sum. The computing nodes in the data routing path can be divided into: a source node generating gradient slice data, a destination node requiring a reduction sum result of the gradient slice data, and an intermediate node connecting the source node and the destination node. The data processing process is as follows: the source node and the intermediate nodes on the path encode the gradient data according to the predetermined encoding strategy and corresponding encoding coefficients, and forward along the path; the destination node receives the encoded data packet, and performs a decoding operation using a decoding coefficient to recover the reduction sum result of the gradient slice data of the source node.

[0028] Definition Let S be a set of source nodes, Let D be a destination node. Let denote the original data of the source node with dimension , denote the encoding coefficient matrix of the source node . The source node forwards the encoded data along the path. The intermediate nodes re-encode and forward the received data, and the encoding coefficients of all the intermediate nodes are uniformly represented by a matrix with dimension , where is the total number of links on the data routing path. At the destination node, the decoding coefficient matrix is represented as . Specifically: First, construct the dual multicast problem of the reduction sum, that is, regard as the source node, as the destination node, and reverse the transmission direction of each edge on the data routing path. In the multicast problem, node needs to send data transmitted to each node . Node The encoding coefficient matrix of node is denoted as The encoding coefficient matrix of intermediate node is denoted as Because the encoding is linear, the encoding data transmitted on each link is essentially a linear transformation of the original data , for example, the th link, is denoted as The transmitted data can be equivalently denoted as , where is the th column of , and is a zero matrix, denotes the horizontal concatenation of and , and is the dimensional identity matrix. Thus, the data received by node can be denoted as , where = is the set of incoming links to node . The construction of and satisfies that for all , is full rank. Without loss of generality, is a square matrix, exists. A specific construction is a sequential construction process, starting from node , one link is added at a time, and a series of sub-data routing paths are constructed. For the initial routing path, we choose such that the corresponding is full rank. One link is added, and is chosen such that the corresponding is full rank. One link is added, and is chosen such that the corresponding is full rank. This process is repeated until all paths are included, and is obtained.

[0029] Second, for the reduction sum problem, the encoding coefficient matrix of source node is set as , the encoding coefficient matrix of intermediate node is set as , and the decoding coefficient matrix of destination node is set as .

[0030] It is easy to understand that the limited precision of floating-point numbers cannot accurately represent real numbers, and floating-point operations performed by computing nodes also have operation errors. Considering that the representation error of floating-point numbers in the target data block is difficult to avoid, the disclosure embodiments define that there are floating-point representation errors and floating-point operation errors in the process of the target computing node performing the set communication operation on the target data block based on the target encoding coefficient matrix. The floating-point representation error is the precision loss introduced when converting the encoding coefficient matrix in the real number field to floating-point representation, and the floating-point operation error is the floating-point operation error introduced when the encoding coefficient matrix performs matrix operation. Based on the floating-point representation error and the floating-point operation error, the distortion rate of network coding is defined, and it is verified that the distortion rate is related to the condition number of the encoding coefficient matrix, so that the encoding coefficient matrix is optimized based on the condition number of the encoding coefficient matrix, which can reduce the distortion rate of network coding, and further improve the set communication precision.

[0031] For example, based on the encoding coefficient matrix Through matrix multiplication operation Encoding the data block , the floating-point representation error is expressed as:

[0032] In the formula, float function, used to convert the data block into a floating-point number.

[0033] It can be understood that the floating-point operation error depends on the numerical characteristics of the data block and the encoding coefficient matrix, so the floating-point operation error is expressed as:

[0034] Therefore, the data block transmitted on the communication link can be expressed as

[0035] wherein, .

[0036] Therefore, the distortion rate caused by floating-point number coding is defined as the relative error between the actual output (data block ) and the ideal output ( ) as follows:

[0037] The distortion produced by network coding on floating-point numbers can be regarded as a matrix disturbance to the linear transformation of the data block . It is known that:

[0038] In the formula, y = xA, y' = x(A + E), y is an output vector obtained by multiplying a row vector x with a matrix A, y' is an output vector obtained by multiplying the row vector x with the matrix A after the matrix A is disturbed by E, is a condition number of the matrix A. According to the formula, when the matrix A is disturbed by E, a relative error of the output vector y is amplified by the condition number of the matrix A.

[0039] Therefore, the distortion rate satisfies:

[0040] In the formula, represents a condition number of the matrix .

[0041] As can be seen from the above, the distortion rate is related to the condition number of the matrix . Therefore, the embodiments of the present application depict the upper bound of the distortion rate by the condition number of the matrix , and optimize the encoding coefficient matrix A, B by using the condition number of the matrix , so as to reduce the distortion generated by network coding on floating-point numbers.

[0042] The steps S101 to S104 shown in the embodiments of the present application, in response to a network coding coefficient acquisition request, construct an initial encoding coefficient matrix; calculate a condition number of the initial encoding coefficient matrix to obtain an initial condition number; and based on the initial condition number, optimize the initial encoding coefficient matrix. Since the initial condition number can represent the upper bound of the distortion rate of the initial encoding coefficient matrix, using the initial condition number as the optimization basis of the initial encoding coefficient matrix can effectively constrain the distortion generated by network coding on floating-point numbers, thereby improving the precision and reliability of floating-point operations in set communication; and using the optimized target encoding coefficient matrix to perform set communication operations can ensure the accuracy of data blocks in the transmission and processing process, and reduce the performance loss caused by floating-point number representation and operation errors.

[0043] Please refer to Figure 2 In some embodiments, the step S103 can include but is not limited to steps S201 to S203: Step S201, determining whether the initial condition number satisfies a preset iteration stopping condition; Step S202, if the initial condition number does not satisfy the preset iteration stopping condition, performing iterative updating on the initial encoding coefficient matrix until the iteration termination condition is satisfied to obtain a target encoding coefficient matrix; Step S203, if the initial condition number satisfies the preset iteration stopping condition, taking the initial encoding coefficient matrix as the target encoding coefficient matrix.

[0044] It can be understood that the iteration stopping condition can include a condition number threshold, or a difference threshold of target function values of adjacent two iterations, a minimum iteration number, and the like, which are not limited thereto.

[0045] In step S201 of some embodiments, an initial condition number of an initial encoding coefficient matrix is calculated, which is expressed in a formula as:

[0046] wherein, represents a matrix norm, which can be a column sum norm, a row sum norm, a spectral norm, etc.; represents the initial encoding coefficient matrix; represents the initial condition number. When , it is considered that the initial encoding coefficient matrix is well-conditioned, and the distortion caused by network coding of floating-point numbers based on the initial encoding coefficient matrix has little effect on the decoding result. When , it is considered that the initial encoding coefficient matrix is ill-conditioned, and the distortion caused by network coding of floating-point numbers based on the initial encoding coefficient matrix will cause significant errors in the output, and the initial encoding coefficient matrix needs to be optimized until .

[0047] In step S202 of some embodiments, the initial encoding coefficient matrix is iteratively updated by solving a first objective function. The first objective function aims to minimize the initial condition number, which is expressed in a formula as:

[0048] wherein, represents a set of target computing nodes.

[0049] The steps S201 to S203 shown in the embodiments of the present application determine whether to iteratively update the initial encoding coefficient matrix according to a preset iteration stopping condition, reduce numerical instability caused by high condition number, and obtain a target encoding coefficient matrix that can effectively suppress the amplification and accumulation of floating-point errors and improve the data transmission accuracy of the target data block containing floating-point numbers.

[0050] Referring to Figure 3 , in some embodiments, the network coding-based set communication method can further include, but is not limited to, steps S301 to S305 before step S104: Step S301, obtaining an initial computing node; Step S302, based on a preset time step, constructing a copy of the initial computing node at different time steps to obtain a copy computing node; Step S303, based on the topological relationship between the initial computing nodes, constructing a connection edge between the copy computing nodes to obtain a target topology graph; Step S304, performing path retrieval on the target topology graph to obtain a data routing path; Step S305: Select the target computing node from the replica computing nodes based on the data routing path.

[0051] In step S304 of some embodiments, the target topology graph can be searched using methods such as shortest path search and breadth / depth-first search to obtain the data routing path; however, this is not the only one.

[0052] In a straightforward manner, if there are communication links between the initial compute nodes, then connection edges are constructed between the corresponding replica compute nodes, and these connection edges point to replica compute nodes with larger time steps.

[0053] In a straightforward manner, the data routing path includes at least one replica compute node in the target topology graph. Since the replica compute node is a replica of the initial compute node, each replica compute node corresponds to an initial compute node. The initial compute node corresponding to each replica compute node in the data routing path can be used as the target compute node.

[0054] For example, please see Figure 5 and Figure 6 , Figure 5 This diagram shows the physical network, the target topology, and the set of nodes performing the full reduction. Node set A includes nodes 1, 2, 3, and 4. Based on the physical network, replicas of each node are constructed at T=0, 1, 2...N. Based on the initial computation of the topological relationships between nodes, it is determined whether communication links exist between nodes. If a communication link exists, a connection edge is constructed between the corresponding replicas. For replicas of the same node, it is assumed that there is a communication link with infinite bandwidth between them, thus obtaining the target topology graph. Time step... It is a positive integer, parameter The reduction capability of the target topology graph can be directly controlled; its optimal value corresponds to the reduction capability as... The critical point where growth enters the saturation range. In the target topology graph, [the following will be included]. The set of nodes at time points Treat it as the source node set, The set of corresponding nodes at time N is denoted as And take it as the destination node set. Then the total reduction problem is transformed into realizing the reduction of the source node set in the target topology graph. to the destination node set The communication process of aggregation and distribution. Selecting in the target topology map, such as... Figure 6 The data routing path shown is in At time 1, node 3 encodes the received gradient slice data 1 and the locally computed gradient slice data 3, and transmits the encoded data packet to node 4; At time t, node 4 decodes the data it has, recovers the reduction result of the gradient slice data of the source node, and multicasts the reduction result to each node .

[0055] It can be understood that in order to implement the full reduction, the computing nodes must have a mutual reachable routing path between each other. Therefore, the routing path of the full reduction is a ring. For a network with a ring topology, it is a great challenge to establish an effective and feasible network coding strategy. The cyclic flow of data in the ring introduces complex timing and dependency relationships, which destroys the independence of the data packets required for decoding, resulting in decoding failure or strategy failure to converge.

[0056] Based on this, the steps S301 to S305 shown in the embodiments of the present application are based on the preset time step, construct the copies of the initial computing nodes at different time steps, obtain the copy computing nodes, and discretize the continuous communication process into a series of time slices; based on the topological relationship between the initial computing nodes, construct the connection edges between the copy computing nodes, obtain the target topology graph, and the original ring network is equivalent to a logical directed acyclic network. Therefore, path retrieval and coding design are performed in the directed acyclic target topology graph, thereby improving the efficiency of the collective communication.

[0057] In some embodiments, after step S304, the network coding-based collective communication method can further include but is not limited to steps S401 to S404: Step S401, intersecting a plurality of data routing paths to obtain a multiplexing computing node; Step S402, obtaining an initial multiplexing coefficient, calculating the communication performance value of the data routing path where the multiplexing computing node is located based on the initial multiplexing coefficient, and obtaining an initial performance value; Step S403, based on the initial performance value, optimizing the initial multiplexing coefficient to obtain a target multiplexing coefficient; Step S404, sending the target multiplexing coefficient to the multiplexing computing node, so that the multiplexing computing node performs a collective communication operation on the target data block based on the target multiplexing coefficient.

[0058] It is easy to understand that when the data routing paths are multiple, the multiplexing computing node is provided with multiple encoding coefficient matrices corresponding to different data routing paths. For example, after path retrieval, a first path and a second path are obtained, and the intersection of the first path and the second path is the multiplexing computing node; then, a first initial computing node set is determined based on the first path, the first initial computing node set is taken as a target computing node, and a first encoding coefficient of each first initial computing node is determined. Similarly, a second initial computing node set is determined based on the second path, the second initial computing node set is taken as a target computing node, and a second encoding coefficient of each second initial computing node is determined. Therefore, the multiplexing computing node stores the first encoding coefficient corresponding to the first path and the second encoding coefficient corresponding to the second path.

[0059] It is easy to understand that the processing capability of the multiplexing computing node is limited, and performing the set communication operation on the target data block based on the target multiplexing coefficient can allocate more resources to the data routing path with better performance or to the data routing path requiring higher bandwidth / lower delay, so as to disperse the traffic to the relatively idle data routing path. Therefore, the target multiplexing coefficient can be used as a communication resource allocation weight, such as a bandwidth allocation weight, a transmission priority, etc., and the multiplexing computing node performs the set communication operation based on the communication resource allocation weight. For example, in the first path and the second path, node 2 needs to transmit encoding data packet 1 and encoding data packet 2 to node 1 and node 4 at T = 2, respectively, and then the target multiplexing coefficient is determined by the method of the embodiment of the present disclosure. Node 2 takes the target multiplexing coefficient as the bandwidth allocation weight to determine the bandwidth allocated to the encoding data packet 1 and the encoding data packet 2.

[0060] In some embodiments, after step S401, the method further comprises: calculating the floating point accumulated error of each multiplexing computing node, and deleting the multiplexing computing node with the floating point accumulated error greater than a preset error threshold.

[0061] It can be understood that in the related art, the optimal data routing path is generally selected from multiple data routing paths for set communication, but it is difficult to accurately evaluate the communication performance of each data routing path, and the set communication efficiency needs to be further improved.

[0062] Therefore, the steps S401 to S404 shown in the embodiments of the present disclosure aim to simultaneously utilize multiple qualified data routing paths to improve the set communication efficiency, obtain the multiplexing computing node by taking the intersection of multiple data routing paths, obtain the initial performance value by calculating the communication performance value of the data routing path where the multiplexing computing node is located based on the initial multiplexing coefficient, further optimize the initial multiplexing coefficient based on the initial performance value to obtain the target multiplexing coefficient, and finally send the target multiplexing coefficient to the multiplexing computing node to enable it to perform the set communication operation on the target data block based on the target multiplexing coefficient, so as to maximize the overall reduction capability, and realize the global optimal utilization of network resources.

[0063] In some embodiments, step S403 can include, but is not limited to, steps S501-S503: Step S501: determining whether the initial performance value is greater than a preset performance threshold value; Step S502: if the initial performance value is less than the performance threshold value, iteratively updating the initial multiplexing coefficient until the iteration termination condition is met to obtain the target multiplexing coefficient; Step S503: if the initial performance value is greater than or equal to the performance threshold value, taking the initial multiplexing coefficient as the target multiplexing coefficient.

[0064] In step S502 of some embodiments, the initial multiplexing coefficient is obtained, and a second objective function is constructed to iteratively update the initial multiplexing coefficient. The second objective function aims to maximize the global communication performance value. For example, in the summation reduction scenario, the second objective function can be set to maximize the summation reduction performance:

[0065] In the formula, represents the multiplexing coefficient of the i-th data routing path where the multiplexing computing node is located, represents the summation reduction performance of the i-th data routing path; N represents the set of data routing path numbers.

[0066] The constraint condition of setting the second objective function includes that the weighted sum of network traffic of all data routing paths does not exceed the available bandwidth of the network; the constraint condition is expressed by the formula:

[0067] In the formula, represents the network traffic demand of the i-th data routing path on each communication link; represents a vector whose elements correspond to the available bandwidth of each communication link in the computer cluster; represents the element-wise less than or equal to in the vector sense.

[0068] The steps S501-S503 shown in the embodiments of the present application determine whether the initial performance value is greater than the preset performance threshold value, and decide whether to iteratively update the initial multiplexing coefficient according to the determination result, to finally obtain the target multiplexing coefficient. Specifically, when the initial performance value is less than the performance threshold value, the initial multiplexing coefficient is iteratively updated until the iteration termination condition is met to optimize the performance, and when the initial performance value is greater than or equal to the performance threshold value, the initial multiplexing coefficient is directly taken as the target multiplexing coefficient. This process ensures the adaptive optimization of the multiplexing coefficient, so that when the multiplexing computing node performs the collective communication operation based on the target multiplexing coefficient, the collective communication efficiency can be effectively improved. ​

[0069] In some embodiments, based on the initial multiplexing coefficient, calculating the communication performance value of the data routing path where the multiplexing computing node is located to obtain the initial performance value can include but is not limited to steps S601-S603: Step S601, calculating the sum-reduction maximum flow of the multiplexing computing node to obtain a first traffic value; Step S602, calculating the multicast maximum flow of the multiplexing computing node to obtain a second traffic value; Step S603, based on the first traffic value and the first traffic value, calculating the target traffic value of the multiplexing computing node; Step S604, based on the target traffic value and the initial multiplexing coefficient, calculating the communication performance value of the data routing path where the multiplexing computing node is located to obtain the initial performance value The steps S601-S604 shown in the embodiments of the application: by calculating the sum-reduction maximum flow of the multiplexing computing node based on the data routing path to obtain the first traffic value, and calculating the multicast maximum flow of the multiplexing computing node based on the data routing path to obtain the second traffic value, and then calculating the target traffic value of the multiplexing computing node based on the first traffic value and the second traffic value, and finally calculating the initial performance value based on the target traffic value and the initial multiplexing coefficient, the initial performance value quantitatively evaluates the communication ability of the multiplexing computing node in the network, providing accurate performance basis for subsequent optimization of the multiplexing coefficient, so that network resources can be more effectively utilized in the collective communication operation.

[0070] In some embodiments, based on the initial multiplexing coefficient, calculating the communication performance value of the data routing path where the multiplexing computing node is located to obtain the initial performance value can include but is not limited to steps S701-S705: Step S701, based on the target encoding coefficient matrix, determining the encoding coefficient of the multiplexing computing node to obtain a local encoding coefficient matrix; Step S702, calculating the floating-point representation error of the local encoding coefficient matrix to obtain a first error; Step S703, calculating the floating-point operation error of the local encoding coefficient matrix to obtain a second error; Step S704, calculating the floating-point cumulative error according to the first error and the second error; Step S705, based on the floating-point cumulative error and the initial multiplexing coefficient, calculating the communication performance value of the data routing path where the multiplexing computing node is located to obtain the initial performance value.

[0071] It is easy to understand that the target encoding coefficient matrix includes encoding coefficients of a plurality of target computing nodes, and after the multiplexing computing node is determined, the encoding coefficients of the multiplexing computing node can be found in the target encoding coefficient matrix to obtain a local encoding coefficient matrix. The first error is a difference between a real value of the local encoding coefficient matrix and a floating point value, and the second error is a ratio of an encoding error of the original data to the original data, and the encoding error is a difference between an ideal encoding value of the original data and an actual encoding value, the ideal encoding value is a product of a real value of the original data and a floating point value of the local encoding coefficient matrix, and the actual encoding value is a floating point value of the ideal encoding value. According to the floating point cumulative error and the initial multiplexing coefficient, the multiplexing computing node can be calculated.

[0072] In step S704 of some embodiments, the floating point cumulative error can be obtained by weighted sum, average calculation and other methods on the first error and the second error.

[0073] It is easy to understand that for each multiplexing computing node, its communication performance value can be represented by the following three performance indicators: full reduction performance, floating point cumulative error and network flow. The full reduction performance is the smaller value of the sum reduction maximum flow from the source node set to the multiplexing computing node and the multicast maximum flow from the multiplexing computing node to the destination node set. The network flow is the network link bandwidth resource used by the multiplexing computing node. The communication performance of the multiplexing computing node to a certain extent represents and restricts the communication performance of the data routing path where it is located.

[0074] The steps S701 to S705 shown in the embodiments of the application determine the encoding coefficients of the multiplexing computing node based on the target encoding coefficient matrix to obtain the local encoding coefficient matrix, calculate the floating point representation error of the local encoding coefficient matrix to obtain the first error and the floating point operation error to obtain the second error, then calculate the floating point cumulative error according to the first error and the second error, and finally calculate the initial performance value based on the floating point cumulative error and the initial multiplexing coefficient, accurately quantify the cumulative effect of the floating point error in encoding and operation, provide an objective index based on error for evaluating communication performance, and thus can reduce the influence of floating point error on data transmission when optimizing the multiplexing coefficient in the subsequent, improve the precision and reliability of floating point number processing in the set communication operation.

[0075] In some embodiments, a computing node cluster for performing summation reduction is set up, including 1 root node, 10 relay nodes, and 210 source nodes, each of which is randomly connected to 4 relay nodes. Through network coding, the computing node cluster only drives one communication interaction to complete the aggregation of 840 data items. The encoding and decoding operations of all nodes in the computing node cluster are based on 32-bit floating-point numbers. The first group of experiments constructs the encoding coefficient matrix of the root node by the method disclosed in the embodiments of the present application, and iteratively updates it with the purpose of minimizing the condition number of the encoding coefficient matrix of the root node, to obtain the updated encoding coefficient matrix; and determines the encoding coefficient matrix of each node based on the encoding coefficient matrix. The second group of experiments calculates the encoding coefficient matrix of the computing node cluster by the method of random sampling. For the results of the two groups of experiments, please refer to Figure 7 . In the first group of experiments, the sum of the logarithmic condition numbers of the encoding matrices of the computing nodes is , the sum of the condition numbers of the encoding matrices is , the maximum value of the logarithmic condition numbers of the encoding matrices is , and the distortion of the aggregated data is Distortion = 3.9 x 10 -7 . In the second group of experiments, the sum of the logarithmic condition numbers of the encoding matrices of the computing nodes is , the sum of the condition numbers of the encoding matrices is , the maximum value of the logarithmic condition numbers of the encoding matrices is , and the distortion of the aggregated data is Distortion = 11 x 10 -7 . It can be seen that the above indicators of the first group of experiments are significantly lower than those of the second group of experiments, indicating that the encoding matrix calculated by the method disclosed in the embodiments of the present application is not only more stable as a whole, but also can maintain a low error sensitivity in the worst case, thereby achieving higher data aggregation accuracy in actual distributed computing.

[0076] For details, please refer to Figure 8 . The embodiments of the present application also provide a set communication device based on network coding, which can implement the set communication method based on network coding described above, and the device comprises: a request response module, configured to construct an initial encoding coefficient matrix in response to a network coding coefficient acquisition request; a condition number calculation module, configured to calculate the condition number of the initial encoding coefficient matrix to obtain an initial condition number; a matrix optimization module, configured to optimize the initial encoding coefficient matrix based on the initial condition number to obtain a target encoding coefficient matrix; a matrix sending module, configured to send the target encoding coefficient matrix to a target computing node, so that the target computing node performs a set communication operation on a target data block based on the target encoding coefficient matrix; wherein the target data block comprises floating-point numbers, and the set communication operation comprises floating-point operation.

[0077] The implementation of the network coding based set communication device is basically the same as the above-mentioned implementation of the network coding based set communication method, and will not be repeated here.

[0078] The embodiments of the present application also provide an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the above-mentioned network coding based set communication method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0079] Please refer to Figure 9 , Figure 9 The hardware structure of the electronic device of another embodiment is illustrated, which includes: The processor 901 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., to execute related programs to implement the technical solutions provided by the embodiments of the present application. The memory 902 can be implemented in the form of a ROM (ReadOnly Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 902 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 902 and called and executed by the processor 901 to implement the network coding based set communication method of the embodiments of the present application. The input / output interface 903 is used to realize information input and output. The communication interface 904 is used to realize the communication interaction between the device and other devices, which can realize communication through a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.). The bus 905 transmits information between various components (for example, the processor 901, the memory 902, the input / output interface 903, and the communication interface 904) of the device. The processor 901, the memory 902, the input / output interface 903, and the communication interface 904 realize the communication connection between each other in the device through the bus 905.

[0080] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the network coding based set communication method.

[0081] The memory, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged with respect to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0082] The network coding based set communication method, the network coding based set communication device, the electronic device and the storage medium provided by the embodiment of the present application, in response to a network coding coefficient acquisition request, construct an initial coding coefficient matrix; calculate the condition number of the initial coding coefficient matrix to obtain an initial condition number; based on the initial condition number, the initial coding coefficient matrix is optimized, since the initial condition number can represent the distortion rate upper bound of the initial coding coefficient matrix, the initial condition number is used as the optimization basis of the initial coding coefficient matrix, which can effectively constrain the distortion generated by network coding on floating point numbers, and improve the precision and reliability of floating point operation in set communication; using the optimized target coding coefficient matrix to perform set communication operation can ensure the accuracy of the data block in the transmission and processing process, and reduce the performance loss caused by floating point number representation and operation error.

[0083] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0084] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than the figures shown, or combine certain steps, or different steps.

[0085] The device embodiments described above are only schematic, and the units illustrated as separate components can or can not be physically separated, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0086] Those skilled in the art can understand that all or some of the steps in the network coding based set communication method disclosed above and the function modules / units in the system and device can be implemented as software, firmware, hardware and appropriate combinations thereof.

[0087] The terms "first", "second", "third", "fourth" and the like in the description of the application and in the claims hereof, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed herein is to be interpreted to allow that combinations of the recited elements will come under the scope of the application. Moreover, the terms "comprising", "having", "including", and "containing" are to be construed as open-ended terms (i.e., meaning "including, but not limited to,") unless otherwise noted. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0088] It should be understood that, in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases: only A, only B, and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be singular or plural.

[0089] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and network coding based set communication method can be implemented in other ways. For example, the above-described apparatus embodiments are only illustrative, for example, the division of the above-mentioned units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0090] The units described as separate components above can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0091] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0092] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the network coding based set communication method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0093] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and the scope of the rights of the embodiments of the present application is not limited thereto. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.

Claims

1. A collection communication method based on network coding, characterized in that, The method includes: In response to the request to obtain network coding coefficients, an initial coding coefficient matrix is ​​constructed; Calculate the condition number of the initial coding coefficient matrix to obtain the initial condition number; Based on the initial condition number, the initial coding coefficient matrix is ​​optimized to obtain the target coding coefficient matrix; The target encoding coefficient matrix is ​​sent to the target computing node so that the target computing node performs a set communication operation on the target data block based on the target encoding coefficient matrix; wherein the target data block includes floating-point numbers and the set communication operation includes floating-point operations.

2. The network coding-based aggregate communication method according to claim 1, characterized in that, The optimization of the initial coding coefficient matrix based on the initial condition number to obtain the target coding coefficient matrix includes: Determine whether the initial condition number satisfies the preset iteration stop condition; If the initial condition number does not meet the preset iteration stop condition, the initial coding coefficient matrix is ​​iteratively updated until the iteration termination condition is met, and the target coding coefficient matrix is ​​obtained. If the initial condition number satisfies the preset iteration stopping condition, then the initial coding coefficient matrix is ​​used as the target coding coefficient matrix.

3. The network coding-based aggregate communication method according to claim 1, characterized in that, Also includes: Obtain the initial computing node; Based on a preset time step, replicas of the initial computing node are constructed at different time steps to obtain replica computing nodes; Based on the topological relationship between the initial computing nodes, connecting edges are constructed between the replica computing nodes to obtain the target topology graph; Path retrieval is performed on the target topology map to obtain the data routing path; Based on the data routing path, a target computing node is selected from the replica computing nodes.

4. The network coding-based aggregate communication method according to claim 3, characterized in that, The number of data routing paths is multiple; the method further includes: The intersection of multiple data routing paths is used to obtain a reused computing node; Obtain the initial reuse factor, and based on the initial reuse factor, calculate the communication performance value of the data routing path where the reuse computing node is located to obtain the initial performance value; Based on the initial performance value, the initial reuse coefficient is optimized to obtain the target reuse coefficient; The target reuse factor is sent to the reuse computing node so that the reuse computing node performs a set communication operation on the target data block based on the target reuse factor.

5. The network coding-based aggregate communication method according to claim 4, characterized in that, The optimization of the initial reuse coefficient based on the initial performance value to obtain the target reuse coefficient includes: Determine whether the initial performance value is greater than a preset performance threshold; If the initial performance value is less than the performance threshold, the initial reuse coefficient is iteratively updated until the iteration termination condition is met to obtain the target reuse coefficient. If the initial performance value is greater than or equal to the performance threshold, then the initial reuse factor is used as the target reuse factor.

6. The network coding-based aggregate communication method according to claim 4, characterized in that, The step of calculating the communication performance value of the data routing path where the multiplexing computing node is located based on the initial multiplexing coefficient to obtain the initial performance value includes: Calculate the summation reduction maximum flow of the multiplexed computing node to obtain the first flow value; Calculate the multicast maximum flow of the multiplexed computing node to obtain the second flow value; Based on the first traffic value and the second traffic value, the target traffic value of the multiplexed computing node is calculated; Based on the target traffic value and the initial reuse coefficient, the communication performance value of the data routing path where the reuse computing node is located is calculated to obtain the initial performance value.

7. The network coding-based aggregate communication method according to claim 4, characterized in that, The step of calculating the communication performance value of the data routing path where the multiplexing computing node is located based on the initial multiplexing coefficient to obtain the initial performance value includes: Based on the target coding coefficient matrix, the coding coefficients of the multiplexing computing node are determined to obtain the local coding coefficient matrix; The floating-point representation error of the local coding coefficient matrix is ​​calculated to obtain the first error; The floating-point operation error of the local coding coefficient matrix is ​​calculated to obtain the second error; The floating-point cumulative error is calculated based on the first error and the second error; Based on the cumulative floating-point error and the initial reuse coefficient, the communication performance value of the data routing path where the reuse computing node is located is calculated to obtain the initial performance value.

8. A network coding-based aggregate communication device, characterized in that, The device includes: The request-response module is used to respond to network coding coefficient acquisition requests and construct the initial coding coefficient matrix; The condition number calculation module is used to calculate the condition number of the initial coding coefficient matrix to obtain the initial condition number; The matrix optimization module is used to optimize the initial coding coefficient matrix based on the initial condition number to obtain the target coding coefficient matrix; A matrix sending module is used to send the target encoding coefficient matrix to a target computing node, so that the target computing node performs a set communication operation on a target data block based on the target encoding coefficient matrix; wherein, the target data block includes floating-point numbers, and the set communication operation includes floating-point operations.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the network coding-based collective communication method of any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When a computer program is executed by a processor, it implements the network coding-based collection communication method of any one of claims 1 to 7.

Images