A communication system and method for big data distributed computing cluster

Abstract

The present invention provides a communication system and method for a big data distributed computing cluster. The communication system, method and related computer-readable storage medium for a big data distributed computing cluster provide efficient, reusable and low-overhead RDMA channel connections between cluster nodes during distributed computing, and provide efficient RDMA channels for sample data / parameters in a timely manner during distributed computing, so as to overcome communication bottlenecks and improve distributed computing efficiency.

Description

Technical Field

[0001] The present invention relates to the field of communication technology for big data distributed computing, and in particular, to a communication system and method applied to a big data distributed computing cluster. Background Art

[0002] Today's society is a rapidly developing society with advanced technology, information circulation, closer communication between people, and more convenient life. Big data is the product of this high-tech era. Based on this, some people even assert that the future will not be the IT era, but the DT era, DT is Data Technology. If data is likened to a coal mine, then, just as coal is divided into coking coal, fat coal, lean coal, and poor coal according to its combustion degree, and coal mines are divided into open-pit coal mines and underground coal mines according to their mining difficulty (obviously, the cost of raw coal mining in different types of mining areas is very different), big data is not only about its "large reserves", but more importantly, it is "useful". Therefore, in the field of big data, the value content of data and the attributes of the mining cost dimension are often more important than the simple amount of data; for most practitioners, how to efficiently use the large-scale data resources they have is the key to winning the competition.

[0003] Although the big data technology in a broad sense includes a series of complex technologies such as storage, computing and analysis, downstream computing and analysis have always been the key to realizing the "use" of big data, especially the distributed computing that realizes the deep utilization of big data has always been the core of big data technology. To understand the distributed computing technology in the field of big data, it is better to start with the MapReduce distributed computing model as an example. Since the MapReduce theoretical model was proposed, it has become the cornerstone of the current big data ecosystem after more than ten years of development. In short, MapReduce is mainly divided into two parts: Map (mapping) and Reduce (reduction); generally speaking, distributed computing can be realized through the two functions of map() and reduce(). For example, when submitting a computing job to the MapReduce framework, the MapReduce framework will first divide the computing job into several map tasks; then assign the map tasks to different computing nodes (mappings) for execution. Each map task processes a portion of the total input sample data corresponding to the current computing job. In the Map phase, the sample data corresponding to each task is input to the corresponding computing node (mapping); after the map task is calculated, some intermediate data objects, i.e., parameters, will be generated. These parameters will be used as input data for the reduce task; and the reduce task is to aggregate the parameters of the corresponding map tasks together and output them; in the Reduce phase, they can also be aggregated hierarchically / step by step by several computing nodes (reduction). Whether it is the Map phase or the Reduce phase, in the MapReduce framework, there is obviously a large amount of data communication demand. However, the existing cross-end communication modules of applications are all built around the TCP / IP protocol. In a low-speed network (the network protocol stack is usually the TCP / IP / Ethernet mentioned above), the data copy overhead is not significant compared to the network hardware transmission time. However, when network devices with a general bandwidth of 40Gb / s-300Gb / s are used in commercial clusters for big data processing, the TCP channel becomes the communication bottleneck for cluster data transmission. Summary of the invention

[0004] In view of this, the present invention provides a communication system and method applied to a big data distributed computing cluster, which is applied to the big data distributed computing cluster to improve the communication efficiency of big data transmission objects such as sample data and parameters by providing it with additional reliable and low-overhead channel connections and efficient channels with timely response.

[0005] On the one hand, an embodiment of the present invention provides a communication system for a big data distributed computing cluster, which is applied to a cluster built / deployed based on a big data distributed computing framework such as MapReduce. It provides efficient, reusable and low-overhead RDMA channel connections between related nodes in the cluster, and promptly responds to provide efficient RDMA channels for sample data / parameters during the distributed computing process, so as to overcome communication bottlenecks and improve distributed computing efficiency.

[0006] The above-mentioned communication system applied to big data distributed computing clusters includes: a TCP channel service unit and an RDMA channel service unit; wherein the TCP channel service unit is used to provide a TCP channel for cross-end transmission of data objects in the distributed computing process; the RDMA channel service unit is used to provide a reusable RDMA channel connection between nodes of the distributed computing cluster (especially for the transmission of several groups of data objects between nodes) and to more efficiently provide an RDMA channel for the cross-end transmission of the data objects by calling the RDMA channel connection in the distributed computing process; in the distributed computing process, the data objects transmitted across the RDMA channel mainly include the sample data / parameters mentioned above and the corresponding task identifiers; the data objects transmitted across the TCP channel mainly include various control messages in the distributed computing process, RDMA channel messenger meta-information, etc.; the RDMA channel service unit is based on its current The RDMA channel connection and RDMA channel meta-information provided before the data object request provide the RDMA channel; the RDMA channel meta-information mainly includes: registered memory addresses at both ends of the transmission and RDMA channel messenger meta-information; the RDMA channel messenger meta-information includes a remote memory identification flag; wherein the remote memory identification flag is provided to the channel adapter (HCA) of the remote host node to allow the remote process to access the local system memory during the RDMA operation; during the RDMA operation, the RDMA channel is formed by the registered memory address and the remote memory identification flag to realize the reading / writing of the remote memory; the task identifier is mainly used to identify the sample data / parameters being transmitted; for example, to identify the task to which the sample data / parameters belong in the distributed computing process, so that the sample data / parameters can be recognized by the distributed computing framework and participate in the corresponding calculation (including reduction calculation) as input.

[0007] In the various embodiments of the first aspect, some of which are provided as another possible implementation method, in a communication system applied to a big data distributed computing cluster, the RDMA channel connection therein can be used for communication of part or all sample data / parameters and their corresponding task identifiers of a task, multiple tasks or even all tasks between the nodes.

[0008] Considering the overhead of RDMA communication, for objects with small data volume, such as sample data and parameters with small length, when requesting and using RDMA channel for cross-end transmission, there is often a negative benefit. Therefore, in various embodiments of the first aspect, some of them are provided as a possible implementation method, in a communication system applied to a big data distributed computing cluster, if the sample data / parameter is smaller than the preset data length, it is transmitted through a TCP channel.

[0009] In the various embodiments of the first aspect, some of them are provided as another possible implementation method. In a communication system applied to a big data distributed computing cluster, cross-end transmission of RMDA channel messenger metadata and the like in a distributed computing process is performed through remote procedure calls (RPCs) to improve the response speed of both ends, thereby improving the overall communication efficiency.

[0010] In each embodiment of the first aspect, some of which are provided as another possible implementation method, in a communication system applied to a big data distributed computing cluster, the provision of the RDMA channel includes but is not limited to: between a task scheduling node and a task computing node, and between a task computing node and a task settlement node.

[0011] On the other hand, an embodiment of the present invention provides a communication method applied to a big data distributed computing cluster, which is applied to a cluster built / deployed based on a big data distributed computing framework such as MapReduce. It provides efficient, reusable and low-overhead RDMA channel connections between related nodes in the cluster, and promptly responds to provide efficient RDMA channels for sample data / parameters during the distributed computing process, so as to overcome communication bottlenecks and improve distributed computing efficiency.

[0012] In combination with the first aspect, the above-mentioned communication method applied to a big data distributed computing cluster includes: when a computing job is submitted to a distributed computing framework, during the execution of distributed computing, the cross-end transmission of sample data / parameters, such as between a task scheduling node and a task computing node, or between a task computing node and a task settlement node, is configured to be transmitted via RDMA; when the sample data / parameter requests a cross-end transmission process, RDMA channel metadata of the sample data or parameters is generated, and the RDMA channel messenger metadata in the RDMA channel metadata is transmitted across the TCP channel; based on the RDMA channel connection maintained at both ends of the communication and the RDMA channel metadata, an RDMA channel is provided for the sample data / parameters, and the sample data / parameters and their corresponding task identifiers are transmitted via the RDMA channel; wherein, the RDMA channel connection is initialized when the cluster node is deployed or when the task is requested before the current sample data / parameter communication is requested. The request is established to provide an RDMA channel for the sample data / parameters more efficiently; wherein the RDMA channel meta-information mainly includes: the registered memory addresses at both ends of the transmission and the RDMA channel messenger meta-information; wherein the RDMA channel messenger meta-information includes a remote memory identification flag corresponding to the current data object, i.e., the sample data / parameter; the remote memory identification flag is provided to the channel adapter (HCA) of the remote host node to allow the remote process to access the local system memory during the RDMA operation; during the RDMA operation, the RDMA channel is formed by the registered memory address and the remote memory identification flag to realize the reading / writing of the remote memory; the task identifier is mainly used to identify the sample data / parameters being transmitted; for example, to identify the task to which the sample data / parameters belong in the distributed computing process, so that the sample data / parameters can be recognized by the distributed computing framework and participate in the corresponding calculation (including reduction calculation) as input.

[0013] In the various embodiments of the second aspect, some of which are provided as a possible implementation method, in the communication method applied to a big data distributed computing cluster, the RDMA channel connection therein can be used for the communication of part or all sample data / parameters and their corresponding task identifiers of a task, multiple tasks or even all tasks between the nodes.

[0014] Considering the overhead of RDMA communication, for objects with small data volume, such as sample data and parameters with small length, when requesting and using RDMA channel for cross-end transmission, there is often a negative benefit. Therefore, in the embodiments of the second aspect, some of them are provided as a possible implementation method, in the communication method applied to a big data distributed computing cluster, if the sample data / parameter is less than the preset data length, it is transmitted through the TCP channel.

[0015] In the various embodiments of the second aspect, some of them are provided as a possible implementation method. In the communication method applied to a big data distributed computing cluster, cross-end transmission of RMDA channel messenger metadata and the like in the distributed computing process is performed through remote procedure calls (RPCs) to improve the response speed of both ends and thereby improve the overall communication efficiency.

[0016] In yet another aspect, an embodiment of the present invention provides a computer-readable storage medium.

[0017] In combination with the above aspects, the above-mentioned computer-readable storage medium stores codes for executing cross-end transmission of sample data or parameters in a distributed computing process, including: instructions for executing the communication method operations applied to a big data distributed computing cluster as described in the second aspect.

[0018] The communication system, method and related computer-readable storage medium provided in the above-mentioned embodiments for application in big data distributed computing clusters provide efficient, reusable and low-overhead RDMA channel connections between cluster nodes during the distributed computing process, and promptly respond to provide efficient RDMA channels for sample data / parameters during the distributed computing process, so as to overcome communication bottlenecks and improve distributed computing efficiency.

[0019] The technical solution of the present invention is further described below through the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings involved in some embodiments of the present invention or the description of the prior art are briefly introduced below.

[0021] Figure 1 A schematic diagram of cross-end transmission of Map phase data objects between a task scheduling engine and a task computing engine in a distributed computing process provided by a preferred embodiment of the present invention;

[0022] Figure 2 A schematic diagram of cross-end transmission of data objects in the Map phase between task computing engines in a distributed computing process provided by a preferred embodiment of the present invention;

[0023] Figure 3 A schematic diagram of a process for cross-end transmission of sample data / parameters in a big data distributed computing cluster built based on a MapReduce framework provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The technical scheme in the embodiment of the present invention is described clearly and completely below in conjunction with the accompanying drawings of the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] In the process of big data distributed computing, whether it is the Map stage or the Reduce stage, there will undoubtedly be a large amount of cross-end data communication needs. The communication module implementations in existing distributed computing applications are mostly built around the TCP / IP protocol. When network devices with a bandwidth of 40Gb / s-300Gb / s are used in commercial clusters for big data processing, the TCP channel becomes the communication bottleneck for cluster data transmission.

[0026] Therefore, the present invention provides a communication system and method applied to a big data distributed computing cluster, which is applied to the big data distributed computing cluster to improve the communication efficiency of big data transmission objects such as sample data and parameters by providing it with additional reliable and low-overhead channel connections and efficient channels with timely response.

[0027] The following are some preferred embodiments of the present invention.

[0028] Some preferred embodiments of the present invention provide a communication system applied to a big data distributed computing cluster, which is applied to a cluster built / deployed based on a big data distributed computing framework such as MapReduce. The communication system applied to a big data distributed computing cluster includes: a TCP channel service unit and an RDMA channel service unit; wherein the TCP channel service unit is used to provide a TCP channel for cross-end transmission of data objects in a distributed computing process; the RDMA channel service unit is used to provide a reusable RDMA channel connection between nodes of a distributed computing cluster (especially for transmission between nodes or a number of groups of data objects) and to more efficiently provide an RDMA channel for cross-end transmission of the data objects by calling the RDMA channel connection during the distributed computing process; during the distributed computing process, the data objects transmitted across the RDMA channel mainly include the sample data / parameters mentioned above and the corresponding task identifiers; the data objects transmitted across the TCP channel mainly include various control messages in the distributed computing process, RDMA channel messenger metadata, etc.; the RDMA channel service unit provides a TCP channel service unit based on its current data. The RDMA channel is provided according to the RDMA channel connection provided before the object request and the RDMA channel meta-information; the RDMA channel meta-information mainly includes: the registered memory addresses at both ends of the transmission and the RDMA channel messenger meta-information; the RDMA channel messenger meta-information includes a remote memory identification flag; wherein the remote memory identification flag is provided to the channel adapter (HCA) of the remote host node to allow the remote process to access the local system memory during the RDMA operation; during the RDMA operation, the RDMA channel is formed by the registered memory address and the remote memory identification flag to realize the reading / writing of the remote memory; the task identifier is mainly used to identify the sample data / parameters to be transmitted; for example, to identify the task to which the sample data / parameters belong in the distributed computing process, so that the sample data / parameters can be recognized by the distributed computing framework and participate in the corresponding calculation (including reduction calculation) as input.

[0029] In some of the above preferred embodiments, a preferred implementation is also provided, specifically including: the RDMA channel connection can be used for communication of part or all sample data / parameters and their corresponding task identifiers of one task, multiple tasks or even all tasks between the nodes. The RDMA channel connection can be destroyed after a period of time, and created again when there is a request for communication, and can also be maintained until the node is destroyed or shut down.

[0030] In some of the above preferred embodiments, a preferred implementation method is also provided, which specifically includes: considering the overhead of RDMA communication, if the sample data / parameter is smaller than the preset data length, it is transmitted through a TCP channel.

[0031] In some of the above-mentioned preferred embodiments, a preferred implementation method is also provided, specifically including: cross-end transmission of RMDA channel messenger meta-information, etc. in the process of performing distributed computing through remote procedure calls (RPCs), by timely obtaining the RMDA channel messenger meta-information and timely responding after obtaining the RMDA channel messenger meta-information, efficiently providing an RDMA channel for the sample data / parameter transmission, thereby improving the overall communication efficiency.

[0032] In some of the above-mentioned preferred embodiments, a preferred implementation method is also provided, specifically including: the RDMA channel not only provides cross-end transmission of sample data / parameters between the task scheduling node and the task computing node in the Map phase, but also provides cross-end transmission of sample data / parameters between the task computing node and the task settlement node in the distributed computing framework, such as parameter aggregation in the Reduce phase.

[0033] Some other preferred embodiments of the present invention provide a communication method applied to a big data distributed computing cluster, which is applied to a distributed computing process performed by a cluster built / deployed based on a big data distributed computing framework such as MapReduce. The communication method applied to a big data distributed computing cluster includes: when a computing job is submitted to a distributed computing framework, during the execution of distributed computing, the cross-end transmission of sample data / parameters, such as between a task scheduling node and a task computing node, or between a task computing node and a task settlement node, is configured to be transmitted via RDMA; when the sample data / parameter requests a cross-end transmission process, the RDMA channel meta-information of the sample data or parameter is generated, and the RDMA channel messenger meta-information in the RDMA channel meta-information is transmitted across the TCP channel, and an RDMA channel is provided for the sample data / parameter according to the RDMA channel connection maintained at both ends of the communication and the RDMA channel meta-information, and the sample data / parameter and its corresponding task identifier are transmitted via the RDMA channel; wherein the RDMA channel connection is established at the time of cluster node deployment initialization or upon request before the current sample data / parameter requests communication, It is capable of providing an RDMA channel for the sample data / parameters more efficiently; wherein the RDMA channel meta-information mainly includes: registered memory addresses at both ends of the transmission and the RDMA channel messenger meta-information; wherein the RDMA channel messenger meta-information includes a remote memory identification flag corresponding to the current data object, i.e., the sample data / parameter; the remote memory identification flag is provided to the channel adapter (HCA) of the remote host node to allow the remote process to access the local system memory during the RDMA operation; during the RDMA operation, the RDMA channel is formed by the registered memory address and the remote memory identification flag to realize the reading / writing of the remote memory; the task identifier is mainly used to identify the sample data / parameters being transmitted; for example, to identify the task to which the sample data / parameters belong in the distributed computing process, so that the sample data / parameters can be recognized by the distributed computing framework and participate in the corresponding calculation (including reduction calculation) as input.

[0034] In some of the above-mentioned preferred embodiments, a preferred implementation method is also provided, specifically including: the RDMA channel connection therein can be used for the communication of part or all sample data / parameters and their corresponding task identifiers of a task, multiple tasks or even all tasks between the nodes.

[0035] In some of the above preferred embodiments, a preferred implementation method is also provided, which specifically includes: considering the overhead of RDMA communication, if the sample data / parameter is smaller than the preset data length, it is transmitted through a TCP channel.

[0036] In some of the above-mentioned preferred embodiments, a preferred implementation method is also provided, specifically including: cross-end transmission of RMDA channel messenger meta-information, etc. in the process of performing distributed computing through remote procedure calls (RPCs), by timely obtaining the RMDA channel messenger meta-information and timely responding after obtaining the RMDA channel messenger meta-information, efficiently providing an RDMA channel for the sample data / parameter transmission, thereby improving the overall communication efficiency.

[0037] In combination with the above embodiments, the following describes the specific process of cross-end transmission of sample data / parameters in a distributed computing process performed on the communication system / method after it is applied to a MapReduce framework in combination with the accompanying drawings. This embodiment takes a distributed computing cluster system based on a MapReduce framework as an example. In the distributed computing process, the cross-end transmission of sample data / parameters can be not only the cross-end transmission between the task scheduling node and the task computing node in the distributed computing process (such as Figure 1 As shown, for example, in the Map phase), it can also be a cross-end transmission between the task computing node and the task settlement node (such as Figure 2 As shown, for example, in the Reduce phase), Figure 3 As shown, its specific implementation is as follows:

[0038] When a computing job (Job) is submitted to the distributed computing system, it enters the Map stage. The task scheduling engine in the distributed computing system will run the program, call the map() function, divide the computing job into several map tasks (Task), and assign them to different task computing engines (mappings) for execution. Of course, the sample data outside the node where the task computing engine is located needs to be input across the end; and each task computing engine (mapping) will generate its own parameters after processing the assigned sample data. These intermediate data objects need to be aggregated together, whether they are directly aggregated to one task computing engine (reduction) or hierarchical / step aggregation (that is, there are multiple task computing engines (reduction) and multi-step aggregation), cross-end communication is also required. In order to improve the cross-end transmission efficiency of the above-mentioned sample data / parameters, it is configured to be transmitted via RDMA;

[0039] In the distributed computing process, when the node is deployed and initialized or due to a request, before there is specific sample data / parameter communication, a reusable RDMA channel connection is established between the nodes. When there is sample data / parameter request communication between the nodes in this channel, the RDMA channel connection is called, and the memory is registered at both ends of the communication (the sample data / parameter memory is registered as RDMA memory and the corresponding RDMA memory is registered at the other end), and the RDMA channel meta-information (that is, the registered memory address at both ends of the transmission, the RDMA channel messenger meta-information) is obtained, and the RDMA channel messenger meta-information and the remote memory identification flag are sent to the other end. The two ends form an RDMA channel more efficiently between the memories at both ends through the RDMA channel connection and the RDMA channel meta-information, and the cross-end transmission of the sample data / parameters and the communication of the corresponding task identifier TaskID are performed by executing the RDMA transmission operation. Among them, the TaskID mentioned above is the identifier of the task, etc. of the data object identified by the MapReduce framework, and the remote memory identification flag is provided to the channel adapter (HCA) of the remote host node to allow the remote process to access the local system memory during the RDMA operation; during the RDMA operation, the RDMA channel is formed by registering the memory address and the remote memory identification flag to realize the reading / writing of the remote memory.

[0040] Still other preferred embodiments of the present invention provide a computer-readable storage medium storing codes for executing cross-end transmission of sample data or parameters in a distributed computing process, including instructions for executing the communication method operations applied to a big data distributed computing cluster as described in any of the above embodiments.

[0041] The above description is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto.

Claims

1. A communication system applied to a big data distributed computing cluster, characterized in that: The system comprises: A TCP channel service unit and an RDMA channel service unit; wherein the TCP channel service unit is used to provide a TCP channel for cross-end transmission of data objects in a distributed computing process; An RDMA channel service unit is used to provide a reusable RDMA channel connection between nodes of a distributed computing cluster, and to more efficiently provide an RDMA channel for cross-end transmission of data objects through the RDMA channel connection during distributed computing; In the distributed computing process, data objects transmitted across terminals through RDMA channels include sample data used for distributed computing, parameters generated during the distributed computing process, and their corresponding task identifiers; Data objects transmitted across the TCP channel include: RDMA channel messenger meta information; The RDMA channel service unit provides the RDMA channel according to the RDMA channel connection and RDMA channel meta information provided by it before the current data object request; The RDMA channel meta information includes: registered memory addresses at both ends of the transmission and the RDMA channel messenger meta information; The RDMA channel messenger meta-information includes a remote memory identification flag; the remote memory identification flag is provided to a channel adapter of a remote host node to allow a remote process to access local system memory during an RDMA operation; during an RDMA operation, an RDMA channel is formed by registering a memory address and a remote memory identification flag to realize reading / writing of remote memory.

2. The communication system for big data distributed computing cluster according to claim 1, characterized in that: The RDMA channel connection is used for communicating several sample data / parameters of several tasks between nodes, wherein the nodes include the distributed computing cluster nodes and / or the remote host nodes.

3. The communication system applied to a big data distributed computing cluster according to claim 2, characterized in that: If the sample data / parameter is smaller than the preset data length, it is transmitted through the TCP channel.

4. The communication system for big data distributed computing cluster according to claim 1, characterized in that: Cross-end transmission of RMDA channel messenger meta-information in distributed computing processes is performed via remote procedure calls.

5. The communication system for big data distributed computing cluster according to claim 1, characterized in that: The RDMA channel is provided for sample data / parameter transmission between task scheduling nodes and task computing nodes, and between task computing nodes and task settlement nodes.

6. A communication method applied to a big data distributed computing cluster, characterized in that: The method comprises: when a computing job is submitted to a distributed computing framework, in the distributed computing process, cross-end transmission of sample data / parameters therein is configured to be transmitted via RDMA; When the sample data / parameter request is in a cross-end transmission process, RDMA channel meta information of the sample data / parameter is generated; and transmitting the RDMA channel messenger meta-information in the RDMA channel meta-information across ends through a TCP channel; According to the RDMA channel connection maintained at both ends of the communication and the RDMA channel meta information, providing the RDMA channel for the sample data / parameters, and transmitting the sample data / parameters and their corresponding task identifiers through the RDMA channel; Wherein, the RDMA channel connection is established before the communication request, which can provide the RDMA channel for the sample data / parameters more efficiently; The RDMA channel meta information includes: registered memory addresses at both ends of the transmission and the RDMA channel messenger meta information; the RDMA channel messenger meta information includes a remote memory identification flag; the remote memory identification flag is provided to the channel adapter of the remote host node to allow the remote process to access the local system memory during the RDMA operation; During RDMA operation, the RDMA channel is formed by registering the memory address and the remote memory identification flag to realize the reading / writing of the remote memory.

7. The communication method applied to a big data distributed computing cluster according to claim 6, characterized in that: The RDMA channel connection is used for communicating several sample data / parameters of several tasks between nodes, wherein the nodes include the distributed computing cluster nodes and / or the remote host nodes.

8. The communication method applied to a big data distributed computing cluster according to claim 6, characterized in that: If the sample data / parameter is smaller than the preset data length, it is transmitted through the TCP channel.

9. The communication method applied to a big data distributed computing cluster according to claim 6, characterized in that: Cross-end transmission of RMDA channel messenger meta-information in distributed computing processes is performed via remote procedure calls.

10. A computer-readable storage medium, characterized in that: Stored are codes for executing cross-end transmission of sample data / parameters in a distributed computing process, including: instructions for executing the operation of the communication method applied to a big data distributed computing cluster as described in any one of claims 6-9.

Images