Heterogeneous communication method, device, equipment, storage medium and system

Abstract

The invention relates to a heterogeneous communication method and device, computer equipment, a computer readable storage medium and a system. The method comprises the following steps: receiving a task execution request; analyzing the task execution request to obtain a target task and a task demand; determining a target computing node and a target communication path according to the task demand and the state of the target environment; and forwarding the target task to the target computing node, wherein the target computing node cooperatively executes the target task according to the target communication path. By adopting the method, efficient communication can be carried out in a heterogeneous computing environment.

Description

Technical Field

[0001] The present application relates to the field of heterogeneous computing technology, and in particular to a heterogeneous communication method, apparatus, computer equipment, computer-readable storage medium and system. Background Art

[0002] With the advent of the information explosion era, the performance of computer components is approaching its limit. The homogeneous system constructed by continuously enhancing the performance of a certain hardware can no longer meet the growing computing needs. Therefore, a more flexible computing model, heterogeneous computing, came into being.

[0003] However, in heterogeneous computing environments, efficient communication and collaboration between devices become a major challenge due to differences in hardware types and architectures. Different hardware devices need to exchange data frequently, but existing communication libraries are often optimized for specific hardware or architectures, making it difficult for them to run efficiently in heterogeneous computing environments. These communication libraries can usually only perform optimization work between homogeneous hardware and cannot effectively manage communication between different types of hardware devices, resulting in communication bottlenecks and efficiency issues in data exchange between devices. Summary of the invention

[0004] Based on this, it is necessary to provide a heterogeneous communication method, apparatus, computer device, computer-readable storage medium and computer program product that can efficiently transmit in order to solve the above technical problems.

[0005] In a first aspect, the present application provides a heterogeneous communication method, the method comprising:

[0006] Receive task execution request;

[0007] Parsing the task execution request to obtain the target task and task requirements;

[0008] Determine the target computing node and the target communication path according to the task requirements and the state of the target environment;

[0009] The target task is forwarded to the target computing node, wherein the target computing node collaboratively executes the target task according to the target communication path.

[0010] In one embodiment, the target environment includes a plurality of initial computing nodes, and the state of the target environment includes a node state and a node topology; determining the target computing node and the target communication path according to the task requirements and the state of the target environment includes:

[0011] Detecting the target environment to obtain the node status and the node status; the target environment includes a plurality of initial computing nodes; wherein the target computing nodes include a plurality of computing devices;

[0012] Selecting the target computing node from the initial computing node according to the node status and the task requirement;

[0013] A target communication path is determined according to the target computing node and the node topology.

[0014] In one embodiment, each of the target computing nodes includes a plurality of computing devices; the target computing node includes a plurality of computing devices; the state of the target environment includes a network topology; and determining the target communication path according to the target computing node and the node topology includes:

[0015] Determine the target computing node to which each computing device belongs according to the node topology;

[0016] In the case where the computing devices belong to the same target computing node, using a physical link between the computing devices as the target communication path;

[0017] In the case that the computing devices do not belong to the same target computing node, a target logical link is selected from the logical links as the target communication path in combination with the network topology and the task requirement.

[0018] In one embodiment, the method further comprises:

[0019] Monitoring the communication status of each of the logical links;

[0020] If the target logical link does not meet the communication requirement corresponding to the task requirement, the target logical link is switched according to the communication status.

[0021] In one of the embodiments, switching the target logical link according to the communication state includes:

[0022] According to the communication status, obtaining an available logical link;

[0023] Calculating the load status of each of the available logical links;

[0024] The target logical link is switched according to the load status.

[0025] In one embodiment, selecting the target computing node from the initial computing node according to the node state and the task requirement includes:

[0026] Determining resource availability of each of the computing devices in each of the initial nodes according to the node status;

[0027] selecting a reference computing node from each of the initial nodes according to the resource availability;

[0028] The target computing node is obtained from the reference computing node in combination with a preset method.

[0029] In a second aspect, the present application provides a heterogeneous communication device, the device comprising:

[0030] A request receiving module, used for receiving a task execution request;

[0031] A parsing module, used to parse the task execution request to obtain the target task and task requirements;

[0032] A path determination module is used to determine the target computing node and the target communication path according to the task requirements and the state of the target environment; each of the target computing nodes includes different types of computing devices; and a task execution module is used to execute the target task on each of the target computing nodes based on the target communication protocol corresponding to the target communication path.

[0033] In a third aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0034] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0035] In a fifth aspect, the present application provides a heterogeneous communication system, including a computer program, a communication framework, and a target computing node;

[0036] The communication framework is used to receive a task execution request; parse the task execution request to obtain a target task and task requirements; determine a target computing node and a target communication path according to the task requirements and the state of the target environment; forward the target task to the target computing node; and receive a calculation result;

[0037] The target computing node is used to collaboratively execute the target task according to the target communication path and send the computing result to the communication framework.

[0038] The above-mentioned heterogeneous communication method, apparatus, computer equipment, computer-readable storage medium and system, after the communication framework receives and parses the task execution request and obtains a clear target task and corresponding task requirements, it automatically matches the target computing node and the target communication path according to the task requirements and the status of the target environment. This can shield the differences in the underlying communication media and protocols, greatly simplifying the integration and operation in the heterogeneous computing environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0040] FIG1 is a schematic diagram of the structure of a heterogeneous communication system in one embodiment;

[0041] FIG2 is a schematic diagram of the structure of a heterogeneous communication system in another embodiment;

[0042] FIG3 is a schematic diagram of the connection between the communication framework, the upper layer software and the underlying hardware in one embodiment;

[0043] FIG4 is a schematic diagram of a flow chart of a heterogeneous communication method in one embodiment;

[0044] FIG5 is a block diagram of a heterogeneous communication device according to an embodiment;

[0045] FIG. 6 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] With the rapid development of artificial intelligence technology, the demand for computing resources is increasing. In terms of large parameter model training, for example, the training process of GPT-4 (Generative Pre-trained Transformer 4) consumes about 2.15 times 1025 floating point operations and utilizes 25,000 A100 GPUs (Graphics Processing Units), which takes 90 to 100 days. The training of large models at a scale of tens of thousands of cards requires a highly adapted underlying hardware architecture and an optimized software ecosystem.

[0048] Due to the limited resources of a single type of GPU, building a heterogeneous cluster using multiple types of GPU accelerators has become an effective way to solve the problem of insufficient computing resources. This is because the same type of GPU accelerators have their own advantages when processing specific tasks. For example, some GPUs have better floating-point computing capabilities during deep learning reasoning or training, while other accelerators perform better in specific computing modes (such as matrix multiplication and convolution operations). By building a heterogeneous cluster, the most suitable hardware can be intelligently allocated according to the computing characteristics of the task, accelerating the execution of the task and further improving resource utilization.

[0049] Therefore, making full use of different types of GPU accelerators to train large-scale models is a good way to solve the demand for a large number of computing resources. However, current large-scale model distributed training systems usually only support homogeneous GPU accelerators, which means that they can only use the same type of GPU for training. In addition, in order to improve the efficiency of intra-node communication, existing chips have increased the intra-node communication bandwidth, but the existing collective communication library has failed to make full use of these dedicated communication links from different manufacturers, resulting in low intra-node communication efficiency. In addition, existing combined communication operations, such as all-reduce (All-Reduce Operation, full reduction operation), reduce (Reduce Operation, reduction operation), etc., not only rely on data transmission, but also require computing operations. However, since the communication operator is closely bound to the underlying hardware, especially the hardware optimization for specific chips, it cannot work effectively together in a heterogeneous environment.

[0050] In summary, the above problems lead to limited resource scheduling and communication efficiency in the existing heterogeneous computing environment, and the advantages of heterogeneous clusters cannot be fully utilized.

[0051] Therefore, this application proposes a heterogeneous communication method that can uniformly manage the communication between different types of computing chips, use their respective dedicated high-performance collective communication libraries for intra-node communication, and achieve efficient model parallel and data parallel communication.

[0052] This application is applied to distributed heterogeneous computing environments. Heterogeneous computing is a computing node composed of various types of computing devices, such as GPU, NPU (Neural Processing Unit) and CPU (Central Processing Unit), and these different devices are interconnected through physical links. In a distributed heterogeneous computing environment, computing nodes are distributed on multiple physical nodes, and the nodes are connected through logical links to exchange data and coordinate task processing, thereby achieving efficient execution of large-scale computing tasks.

[0053] Exemplarily, in conjunction with FIG. 1 , FIG. 1 is a schematic diagram of the structure of a heterogeneous communication system in an embodiment, including a communication framework 102 and a target computing node 104 .

[0054] Among them, the communication framework 102 is used to receive a task execution request; parse the task execution request to obtain the target task and task requirements; determine the target computing node and the target communication path according to the task requirements and the state of the target environment; forward the target task to the target computing node; and receive the calculation results.

[0055] The target computing node 104 is used to collaboratively execute the target task according to the target communication path and send the computing result to the communication framework.

[0056] Further, in conjunction with FIG. 2, FIG. 2 is a schematic diagram of the structure of a heterogeneous communication system in another embodiment. FIG. 2 includes two computing nodes, computing node 1 is an NPU cluster, and computing node 2 is a GPU cluster. Computing node 1 includes multiple NPU devices, and computing node 2 includes multiple GPU devices. In some large-scale distributed systems, different computing nodes are located in different geographical locations, so the computing devices in different computing nodes communicate through logical links. In other embodiments, computing node 2 may also be a hybrid cluster of GPUs and CPUs.

[0057] The heterogeneous communication method proposed in this application is applied to the communication framework in Figure 1, and is used as an abstract layer connecting the underlying hardware and the upper-level application software. It is responsible for transmitting data between the underlying hardware and the upper-level deep learning framework.

[0058] Continuing with FIG3, FIG3 is a schematic diagram of the connection between the communication framework and the upper-layer software and the underlying hardware in one embodiment. The upper-layer application software of the communication framework in FIG3 includes Pytorch, MindSpore, TensorFlow, etc. The communication framework supports centralized collective communication operations for synchronizing data between multiple computing nodes.

[0059] In an exemplary embodiment, as shown in FIG4 , a heterogeneous communication method is provided, which is described by taking the method applied to the communication framework in FIG1 as an example, and includes the following steps S402 to S408 . Among them:

[0060] Step S402: receiving a task execution request.

[0061] Among them, the task execution request is an instruction issued by the upper-level application software, requiring the distributed computing system to execute a specific computing task.

[0062] Optionally, the task execution request includes a target task and a task requirement. The target task refers to the core computing work that the heterogeneous system actually needs to complete, as well as the input data or model operations related to the task, such as computing, data processing, or model training. The task requirement refers to the set of resources and execution conditions required for the target task. Task requirements usually include computing resource requirements and communication requirements. Computing resources can be the number of GPUs or NPUs, etc.; task requirements refer to the requirements for data exchange in a distributed environment, such as communication bandwidth, the type of communication operation required, such as All-Reduce, P2P (Peer-to-Peer, point-to-point), etc., and the choice of communication protocols, such as InfiniBand (wireless broadband), RoCE (RDMA over Converged Ethernet, remote direct memory access based on converged Ethernet).

[0063] Step S404: parse the task execution request to obtain the target task and task requirements.

[0064] Optionally, if the task execution request is in a structured format, such as JSON (JavaScript Object Notation) and XML (eXtensible Markup Language), corresponding parsing tools, such as a JSON parser, can be used to convert the task request into a data structure that can be operated by a computer, such as a dictionary and an object. Afterwards, the fields in the data structure are parsed to obtain the target task and task requirements.

[0065] For example, the resource_requirements field may be extracted and parsed to obtain the task requirements, and the task_type field may be parsed to determine the target task.

[0066] Step S406, determining the target computing node and the target communication path according to the task requirements and the state of the target environment.

[0067] The target computing node refers to the computing resources allocated for executing the target task. For example, there are multiple computing nodes in the current target environment, but not every computing node is suitable for processing the target task. For example, if the target task is an image processing task, a GPU node is more suitable for processing the target task.

[0068] Among them, the target communication path refers to the communication link used when the task transmits data between different computing nodes. Communication links include physical links and logical links. Physical links include PCIe (Peripheral Component Interconnect Express), NVLink (NVIDIA Link), etc., and logical links include network protocols RoCE, InfiniBand, etc. Continuing with Figure 1, the computing devices inside the computing node are connected through physical links, and the computing devices across nodes communicate through logical links. Whether it is a physical link or a logical link, there are usually multiple available paths between two computing devices. Therefore, in this embodiment, the target communication path will be determined based on the task requirements and the state of the target environment.

[0069] Optionally, according to the parsed task requirements and in combination with the current state of the target environment, a suitable target computing node and target communication path are selected. The computing environment state includes the current state of each computing node, the communication state, and the node topology. The node state may refer to the current computing resource status of the computing node, such as the availability of hardware devices such as CPU, GPU, or NPU, and the workload (such as the number of tasks and computing density). These states can be used to determine whether a computing node can carry the target task. For example, if the task requires a large amount of GPU resources, the GPU availability and current load status of the computing node will determine whether it is the target computing node. The communication state refers to the network connection between the computing node and other nodes. The communication state may include network performance indicators such as bandwidth, delay, and packet loss rate. This information helps to select the target communication path. For example, the logical link between nodes may be InfiniBand, RoCE, or TCP / IP over Ethernet (Transmission Control Protocol / Internet Protocol over Ethernet), and different links have different bandwidth and delay characteristics. By analyzing the current load and performance of these links, the system can select the logical link that best suits the task requirements to ensure the speed and reliability of data transmission.

[0070] Step S408, forwarding the target task to the target computing node, and the target computing node collaboratively executes the target task according to the target communication path.

[0071] In this step, after the target task is distributed to multiple computing nodes, the target computing nodes work together according to the target communication path to execute the target task.

[0072] Optionally, each target computing node performs efficient data exchange and synchronization operations based on the communication path to which it is connected, such as PCIe, NVLink, or RoCE, to ensure that different types of computing devices can effectively collaborate and give full play to their respective performance advantages. In this way, parallel processing and communication coordination of tasks can be achieved between multiple target computing nodes, thereby optimizing computing efficiency and accelerating the completion of target tasks.

[0073] For example, during the calculation process, the target computing nodes need to frequently exchange intermediate results. For example, in the training of deep learning models, the gradient synchronization between nodes is completed through the All-Reduce operation, which requires efficient communication protocol support. At this stage, data is transmitted between nodes through the target communication path, and finally the calculation results are generated and sent to the upper-level software. According to the previously selected target communication path and communication protocol, the target task is assigned to the target computing node. Once the target task is forwarded, each target computing node will start to perform specific computing operations according to the task requirements.

[0074] In the above heterogeneous communication method, after receiving and parsing the task execution request and obtaining a clear target task and corresponding task requirements, the communication framework automatically matches the target computing node and the target communication path according to the task requirements and the status of the target environment. This can shield the differences in the underlying communication media and protocols, greatly simplifying the integration and operation in the heterogeneous computing environment.

[0075] Further, in one embodiment, the state of the target environment includes node state and node state; the target environment is detected to obtain node state and node state; the target environment includes several initial computing nodes; according to the node state and task requirements, the target computing node is selected from the initial computing nodes; according to the target computing node and node topology, the target communication path is determined.

[0076] First, the target environment is detected to obtain the node status of each initial computing node in the target environment. Then, based on the detected node status and task requirements, the target computing node that is most suitable for executing the target task is selected from the initial nodes to ensure that computing resources can be fully utilized.

[0077] Optionally, the task requirements clearly describe the resources and execution conditions required for the target task. Therefore, the task requirements and the node status of each initial computing node are combined to select a target computing node that matches the task requirements from each initial computing node to execute the target task.

[0078] Furthermore, the above-mentioned selecting the target computing node from the initial computing nodes according to the node status and task requirements includes: determining the resource availability of each computing device in each initial node according to the node status; selecting a reference computing node from each initial node according to the resource availability; and obtaining the target computing node from the reference computing node in combination with a preset method.

[0079] Among them, resource availability refers to the availability of various computing resources in the computing device, reflecting whether the computing of these resources can meet the task requirements.

[0080] The preset method refers to a predefined strategy for selecting the most suitable target computing node from the reference computing nodes. Exemplarily, the preset method may be a scheduling algorithm, a fault tolerance strategy, a cost control strategy, and the like.

[0081] Optionally, based on the node status, the resource availability of each computing device in each initial node is evaluated, such as the number of CPUs and the current workload. Then, the task requirements are matched with the resource availability of each computing device in each initial node to obtain reference computing nodes. Then, a preset method is used to select target computing nodes from the reference computing nodes. These reference computing nodes are computing nodes whose node status meets the task requirements.

[0082] For example, suppose the upper-layer software needs to perform an image processing task that requires at least 4 GPUs and requires these GPUs to have sufficient video memory to store intermediate results. After matching the node status, the system finds the following reference computing nodes that meet the conditions:

[0083] Node A: has 8 GPUs, 5 of which are currently idle.

[0084] Node B: has 4 GPUs, 4 of which are currently idle.

[0085] Node C: has 6 GPUs, 3 of which are currently idle, so it does not meet the task requirements.

[0086] Based on the initial screening, Node A and Node B are listed as reference computing nodes because they both have at least 4 idle GPUs.

[0087] Furthermore, if the preset method is a scheduling algorithm, the target computing node can be selected from the reference computing nodes in combination with the scheduling algorithm. Exemplarily, if the goal of the scheduling algorithm is resource optimization, then computing nodes with low load and sufficient resources will be preferentially selected. Exemplarily, if the goal of the scheduling algorithm is topology optimization, then nodes with high bandwidth and low latency connections to other nodes will be preferentially selected to ensure efficient data transmission.

[0088] Furthermore, if the preset method is a fault-tolerant strategy, the target computing node can be selected from the reference computing nodes in combination with the fault-tolerant strategy. Exemplarily, if the purpose of the fault-tolerant strategy is stability, then computing nodes supported by redundant resources or spare nodes will be preferred to ensure that tasks can be quickly switched when a failure occurs. Exemplarily, if the purpose of the fault-tolerant strategy is geographic redundancy, then nodes with similar geographical locations or in the same data center will be preferred to reduce the risk of communication interruption caused by network failures.

[0089] Furthermore, if the preset method is a cost control strategy, for example, if the purpose of the cost control strategy is to minimize computing costs, computing nodes with lower operating costs will be preferentially selected, such as nodes with low performance configuration, to reduce resource usage expenses.

[0090] Finally, the target communication path is determined based on the node topology relationship between the target computing node and the target computing node. This step is mainly to determine whether the computing device belongs to the same target computing node, so as to select a suitable communication path. Exemplarily, the communication within the same computing node can use physical links for direct communication. If the computing devices are distributed on different computing nodes, logical links need to be used for communication.

[0091] The topological relationship may be obtained by obtaining the connection information between various computing devices and constructing a complete topological map based on the connection information. The connection information includes intra-node connections and inter-node connections. Exemplarily, it may be determined as an intra-node connection based on whether there is a direct physical connection between devices, such as NVLink, PCIe, bus architecture, etc. Exemplarily, it may be determined as an inter-node connection based on the network connection method between nodes, such as InfiniBand, RoCE, Ethernet, etc.

[0092] In the above embodiment, by detecting the target environment, the communication framework can obtain the status of each initial computing node and the topological information between nodes. Subsequently, according to the task requirements and node status, the target computing node that is most suitable for executing the task is selected from the initial computing nodes, and the target communication path is determined according to the node topological relationship. In this way, it is ensured that the task is assigned to the most suitable computing node, and the optimal communication path is selected for data transmission. In a dynamic heterogeneous computing environment, this automated scheduling and path selection mechanism can effectively cope with the dynamic changes of resources and ensure the efficient operation of the system.

[0093] Further, in one embodiment, the state of the target environment includes the network topology; each target computing node includes a number of computing devices; the state of the target environment includes the network topology; the target computing node includes a number of computing devices; the state of the target environment includes the network topology; based on the target computing node and the node topology, the target communication path is determined, including: based on the node topology, determining the target computing node to which each computing device belongs; if the computing devices belong to the same target computing node, using the physical link between the computing devices as the target communication path; if the computing devices do not belong to the same target computing node, selecting the target communication path from the logical link in combination with the network topology and task requirements.

[0094] First, the target computing node to which the computing devices belong is determined based on the node topology, thereby determining whether these computing devices are located in the same target computing node. This can further determine whether they communicate through physical links or logical links.

[0095] Optionally, if the computing devices belong to the same target computing node, the communication framework will select the physical link between the computing devices as the target communication path. In this case, the communication between the devices will be carried out through the high-speed hardware connection within the node to ensure low latency and high bandwidth transmission.

[0096] Further, assuming that the physical links include NVLink and PCIe, a selection is made from the physical links according to a preset method. Exemplarily, the preset method may be a priority of the physical link, for example, NVLink has a higher priority than PCIe.

[0097] Optionally, if the computing devices do not belong to the same target computing node, the network topology is combined to select the best logical link from the logical link set, that is, the target logical link as the target communication path. Since two target computing nodes usually include multiple initial logical links for communication, in this embodiment, the target logical link is also selected from multiple logical links in combination with the network topology.

[0098] Optionally, each logical link usually includes an intermediate network device, a link type, and a link distance. The intermediate network device describes which network devices the data needs to pass through for transmission. The link type can be based on different network protocols and technologies, such as InfiniBand, RoCE, TCP / IP over Ethernet, etc. The link type indicates how many network devices the link needs to pass through, which affects the delay of data transmission.

[0099] Exemplarily, it is assumed that there are three logical links.

[0100] Logical link 1, the intermediate network devices include a switch, the link type is InfiniBand, and the link distance is 1 hop.

[0101] Logical link 2, the intermediate network devices include a switch and a router, the link type is InfiniBand, and the link distance is two hops.

[0102] Logical link 3, the intermediate network devices include 2 switches and 2 routers, the link type is TCP / IP over Ethernet, and the link distance is four hops.

[0103] If the task requirements correspond to low latency and high bandwidth, such as distributed training of deep learning models, then logical link 1 is preferred as the target communication path.

[0104] In this way, the target communication path can be accurately determined according to the target computing node to which the computing device belongs, which can significantly speed up the processing of the target task.

[0105] Since the communication status of the logical link directly affects the efficiency of data transmission and the stability of task execution, it needs to be monitored to ensure efficient use of the link, detect problems in a timely manner, and dynamically optimize the communication path.

[0106] Therefore, in one embodiment, the method further includes: monitoring the communication status of each logical link; if the target logical link does not meet the communication requirement corresponding to the task requirement, switching the target logical link according to the communication status.

[0107] The communication status refers to the current network connection status between computing devices, including information such as the load, delay, and bandwidth utilization of the logical link. These statuses can be obtained by real-time monitoring of the communication link to determine whether a path is congested, has too high a delay, or has increased packet loss.

[0108] Since, in large-scale distributed computing, the network status changes dynamically, if the communication status of the target logical link does not meet the communication requirements corresponding to the task needs, the communication path can be quickly adjusted according to the communication status to ensure that the task execution will not be affected by the communication bottleneck.

[0109] The target logical link does not meet the communication requirements corresponding to the task requirements, including link congestion and failure to meet the communication requirements of the task.

[0110] Optionally, when insufficient bandwidth, increased latency, or packet loss occurs on the target logical link, the link's communication capacity is limited and data cannot be efficiently transmitted as expected. At this point, the link is considered to be congested and cannot meet the communication needs of the task.

[0111] Optionally, some tasks may require faster data synchronization or low-latency real-time data transmission. If the logical link cannot provide sufficient bandwidth or latency to meet the communication requirements of the task, the system will consider that the link does not meet the communication requirements of the task and switch to a logical link with better performance to ensure smooth execution of the task.

[0112] Furthermore, switching the target logical link according to the communication status includes: acquiring available logical links according to the communication status; calculating the load status of each available logical link; and switching the target logical link according to the load status.

[0113] Among them, available logical links refer to logical links that are currently idle or have available bandwidth, because some logical links may have been occupied by other tasks or are transmitting data, making them temporarily unavailable for communication of the current task. Therefore, when selecting links, we first check which logical links are available at the current moment, and exclude those links that are fully loaded or occupied.

[0114] Optionally, the bandwidth utilization, delay, and packet loss rate of each available logical link may be integrated to quantify the load status of each logical link, and then data transmission may be switched to a new logical link according to the load status.

[0115] Optionally, after completing the communication path switch, the communication framework will continue to monitor the status of the new logical link to ensure that it can maintain efficient data transmission. If the new logical link performs well, the task continues to transmit and calculate along the logical link; if the new logical link also has problems, the communication framework can repeat the above steps again and further switch to other logical links.

[0116] For example, in a distributed deep learning training task, multiple computing nodes (including GPUs) are distributed in different physical locations. These computing nodes are connected through InfiniBand and RoCE logical links to synchronize gradients during model training (such as All-Reduce operations). Each computing node is responsible for processing a portion of the data and regularly exchanges computing results with other nodes during training.

[0117] In the initial stage, computing devices in different computing nodes communicate through InfiniBand logical links. As the amount of training data and computing increases, a certain InfiniBand logical link (used to connect two computing nodes A and B) begins to bear a large amount of data transmission demand. At this time, the system monitors the following phenomena:

[0118] 1. Bandwidth utilization remains above 95%, indicating that the link is close to full capacity.

[0119] 2. The communication delay increased from 2ms to 8ms, exceeding the threshold set by the system.

[0120] 3. The packet loss rate begins to rise and exceeds 1%, resulting in multiple packet retransmissions, further increasing the link load.

[0121] 4. The throughput decreases. Although the data flow increases, the actual amount of data transmitted does not increase but decreases instead.

[0122] At this time, the system detects that the logical link is congested and the communication efficiency is significantly reduced, and the logical link is dynamically switched.

[0123] For example, by checking the current network topology, it is found that there is a RoCE logical link between nodes A and B. Although the bandwidth is relatively low, the current load is relatively light. The communication between nodes A and B is switched from InfiniBand to the RoCE link to divert the data load and avoid increased congestion.

[0124] After switching to the RoCE link, continue to monitor RoCE. If RoCE also has problems, the communication framework can repeat the above steps again and further switch to other logical links.

[0125] In the above embodiment, by monitoring and dynamically switching the communication status of the logical link, it is possible to optimize communication efficiency, improve resource utilization, enhance system fault tolerance, and effectively reduce task delays and transmission errors.

[0126] In one embodiment, the target task is forwarded to the target computing node, and the target computing node collaboratively executes the target task according to the target communication path, including: dividing the target task into target subtasks according to each target computing node; serializing each target subtask to obtain each serialized data, and forwarding each serialized data to each target computing node; each target computing node calculates each serialized data to obtain intermediate data, and sends the intermediate data to other target computing nodes based on the target communication path to collaboratively execute the target task.

[0127] Optionally, when dividing the target task, it is necessary to divide the target task into multiple target subtasks according to the computing power of the target computing node. This is because the computing power of each target computing node is different. Therefore, according to the difference in computing power, the target task will be divided into multiple target subtasks suitable for execution by each target computing node.

[0128] Optionally, each target subtask is serialized to obtain serialized data. The serialization process is to convert the data structure of the subtask into a standard format that can be transmitted on the network to ensure that different computing nodes can correctly parse and use it. Subsequently, the serialized data is forwarded to the corresponding target computing node.

[0129] Optionally, after receiving the serialized data, each target computing node deserializes it, restores the data structure of the subtask, and then performs calculations to obtain intermediate data. Each computing node sends the intermediate data to other target computing nodes based on the target communication path corresponding to the target communication path. This step is to share necessary data between computing nodes and collaboratively complete the overall task. Through the exchange of intermediate data, each target computing node can work together to complete the target task.

[0130] Exemplarily, taking the target task All-Reduce as an example, the local gradient data will be serialized first, and the serialized data will be sent to the target computing node using the target communication protocol. Each target computing node will calculate the local gradient when processing the data assigned to it. These gradients reflect the error information of the model training data on the target computing node. In order to achieve global gradient synchronization, the local gradients calculated by each target computing node need to be exchanged with each other. In other words, each target computing node needs to receive the local gradient data of all other target computing nodes. After receiving the gradient data from other target computing nodes, each target computing node will reduce these gradients with its own local gradients, usually summing them. In this way, each target computing node will have a global gradient sum. Finally, the aggregated gradient data is deserialized and fed back to the upper-level software.

[0131] In the above embodiment, after the target task is assigned to each computing node, the task is divided into subtasks and serialized so that it can be transmitted between different nodes. After each node executes the subtask, it generates intermediate data, and collaboratively exchanges data through the target communication path to ensure that all nodes complete the task together. This not only optimizes resource utilization, but also improves data transmission efficiency, realizes efficient collaborative computing in a heterogeneous computing environment, and simplifies communication operations between different hardware devices.

[0132] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0133] Based on the same inventive concept, the embodiment of the present application also provides a heterogeneous communication device for implementing the heterogeneous communication method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more heterogeneous communication device embodiments provided below can refer to the limitations on the heterogeneous communication method above, and will not be repeated here.

[0134] In an exemplary embodiment, as shown in FIG5 , a heterogeneous communication device is provided, including: a request receiving module 510, a parsing module 520, a path determining module 530 and a task executing module 540, wherein:

[0135] The request receiving module 510 is used to receive a task execution request.

[0136] The parsing module 520 is used to parse the task execution request to obtain the target task and task requirements.

[0137] The path determination module 530 is used to determine the target computing node and the target communication path according to the task requirements and the state of the target environment.

[0138] The task execution module 540 is used to forward the target task to the target computing node, wherein the target computing node collaboratively executes the target task according to the target communication path.

[0139] In one embodiment, the path determination module 530 includes:

[0140] The detection unit is used to detect the target environment and obtain the node status and node status; the target environment includes several initial computing nodes.

[0141] The node determination unit is used to select a target computing node from the initial computing nodes according to the node status and task requirements.

[0142] The path selection unit is used to determine the target communication path according to the target computing node and the node topology.

[0143] In one embodiment, the path selection unit includes:

[0144] The relationship determination subunit is used to determine the target computing node to which each computing device belongs according to the node topology.

[0145] The first selection subunit is used to use the physical link between the computing devices as the target communication path if the computing devices belong to the same target computing node.

[0146] The second selection subunit is used to select a target logical link as a communication path from the logical links in combination with the network topology and task requirements if the computing devices do not belong to the same target computing node.

[0147] In one embodiment, the above device further comprises:

[0148] The link monitoring module is used to monitor the communication status of each logical link.

[0149] The switching module is used to switch the target logical link according to the communication status if the target logical link does not meet the communication requirements corresponding to the task requirements.

[0150] In one embodiment, the switching module includes:

[0151] The available logical link acquisition unit is used to acquire the available logical link according to the communication status.

[0152] The load calculation unit is used to calculate the load status of each available logical link.

[0153] The link switching unit is used to switch the target logical link according to the load status.

[0154] In one embodiment, the task execution module 540 includes:

[0155] The task division unit is used to divide the target task into target subtasks according to each target computing node.

[0156] The execution unit is used to serialize each target subtask to obtain each serialized data, and forward each serialized data to each target computing node; each target computing node calculates each serialized data to obtain intermediate data, and sends the intermediate data to other target computing nodes based on the target communication path to collaboratively execute the target task.

[0157] In one embodiment, the node determination unit includes:

[0158] The resource determination subunit is used to determine the resource availability of each computing device in each initial node according to the node status.

[0159] The reference node selection subunit is used to select a reference computing node from each initial node according to resource availability.

[0160] The target computing node selection subunit is used to obtain the target computing node from the reference computing nodes in combination with a preset method.

[0161] Each module in the above heterogeneous communication device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0162] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG6. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store target tasks. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a heterogeneous communication method is implemented.

[0163] Those skilled in the art will understand that the structure shown in FIG. 6 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0164] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the method in any one of the above embodiments when executing the computer program.

[0165] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in any one of the above embodiments are implemented.

[0166] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the method in any one of the above embodiments when executed by a processor.

[0167] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0168] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0169] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A heterogeneous communication method, characterized in that: The method comprises: Receive task execution request; Parsing the task execution request to obtain the target task and task requirements; Determine the target computing node and the target communication path according to the task requirements and the state of the target environment; The target task is forwarded to the target computing node, wherein the target computing node collaboratively executes the target task according to the target communication path.

2. The method according to claim 1, characterized in that The target environment includes a plurality of initial computing nodes, and the state of the target environment includes a node state and a node topology; determining the target computing node and the target communication path according to the task requirements and the state of the target environment includes: Detecting the target environment to obtain the node status and the node state; According to the node status and the task requirements, the target computing node is selected from the initial computing node; wherein the target computing node includes a plurality of computing devices; A target communication path is determined according to the target computing node and the node topology.

3. The method according to claim 2, characterized in that The state of the target environment includes a network topology; and determining a target communication path according to the target computing node and the node topology includes: Determining the target computing node to which each computing device belongs according to the node topology; In the case where the computing devices belong to the same target computing node, using a physical link between the computing devices as the target communication path; In the case that the computing devices do not belong to the same target computing node, a target logical link is selected from the logical links as the target communication path in combination with the network topology and the task requirement.

4. The method according to claim 3, characterized in that After forwarding the target task to the target computing node, and the target computing node collaboratively executing the target task according to the target communication path, the method further includes: Monitoring the communication status of each of the logical links; If the target logical link does not meet the communication requirement corresponding to the task requirement, the target logical link is switched according to the communication status.

5. The method according to claim 4, characterized in that The switching of the target logical link according to the communication state includes: According to the communication status, obtaining an available logical link; Calculating the load status of each of the available logical links; The target logical link is switched according to the load status.

6. The method according to claim 2, characterized in that The selecting the target computing node from the initial computing node according to the node state and the task requirement includes: Determining resource availability of each of the computing devices in each of the initial nodes according to the node status; selecting a reference computing node from each of the initial nodes according to the resource availability; The target computing node is obtained from the reference computing node in combination with a preset method.

7. A heterogeneous communication device, characterized in that: The device comprises: A request receiving module, used for receiving a task execution request; A parsing module, used to parse the task execution request to obtain the target task and task requirements; A path determination module, used to determine a target computing node and a target communication path according to the task requirements and the state of the target environment; each of the target computing nodes includes different types of computing devices; The task execution module is used to execute the target task on each of the target computing nodes based on the target communication protocol corresponding to the target communication path.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A heterogeneous communication system, characterized in that: The system comprises: a communication framework and a target computing node; The communication framework is used to receive a task execution request; parse the task execution request to obtain a target task and task requirements; determine a target computing node and a target communication path according to the task requirements and the state of the target environment; forward the target task to the target computing node; and receive a calculation result; The target computing node is used to collaboratively execute the target task according to the target communication path and send the computing result to the communication framework.

Images