Node communication methods and devices, storage media and electronic devices

Abstract

This application discloses a node communication method and apparatus, storage medium, and electronic device, relating to the field of artificial intelligence technology. The method includes: determining first node information of a receiving end based on communication domain information, wherein the first node information indicates the node attribute to which the receiving end belongs, and the communication domain includes the receiving end and the sending end; comparing the first node information and second node information to determine a target node relationship, wherein the second node information indicates the node attribute to which the sending end belongs, and the target node relationship indicates whether the sending end and the receiving end are the same physical card; determining a target communication mode based on the target node relationship, and sending target data to the receiving end based on the target communication mode. This solves the technical problem of high cost in multi-GPU architecture communication mechanisms in related technologies, achieving a significant improvement in communication performance.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a node communication method and apparatus, storage medium and electronic device. Background Technology

[0002] Multi-GPU parallel computing not only significantly accelerates model training but also handles more complex and diverse environmental states and action spaces. However, multi-GPU parallel computing environments are not inherently friendly. Achieving efficient parallel computing and communication typically requires addressing communication bottlenecks between GPUs, especially when model training spans multiple physical machines and involves dozens or even hundreds of GPUs. An efficient and flexible communication framework can significantly improve the speed and quality of model training, reduce latency, and ensure accurate synchronization of model parameters, which is crucial for enhancing the overall performance of reinforcement learning frameworks.

[0003] Existing communication frameworks suffer from significant interface differences between various communication mechanisms. This necessitates developers using different APIs for different communication scenarios (same card or different cards), substantially increasing code complexity and maintenance costs. Particularly when using a mix of CUDA IPC and NCCL, additional logic is required to adapt to these two communication modes, further raising the technical barrier. In short, multi-GPU architecture communication mechanisms in related technologies suffer from high costs. Summary of the Invention

[0004] This application provides a node communication method and apparatus, storage medium and electronic device to at least solve the problem of high cost in the communication mechanism of multi-GPU architecture in the related art.

[0005] This application provides a node communication method, comprising: determining first node information of a receiving end based on communication domain information, wherein the first node information is used to indicate the node attribute to which the receiving end belongs, and the communication domain includes the receiving end and the sending end; comparing the first node information and second node information to determine a target node relationship, wherein the second node information is used to indicate the node attribute to which the sending end belongs, and the target node relationship is used to indicate whether the sending end and the receiving end are the same physical card; determining a target communication mode according to the target node relationship, and sending target data to the receiving end based on the target communication mode.

[0006] This application also provides a node communication device, comprising: a node information determination module, configured to determine first node information of a receiving end based on communication domain information, wherein the first node information is used to indicate the node attribute to which the receiving end belongs, and the communication domain includes the receiving end and the sending end; a node relationship determination module, configured to compare the first node information and second node information to determine a target node relationship, wherein the second node information is used to indicate the node attribute to which the sending end belongs, and the target node relationship is used to indicate whether the sending end and the receiving end are the same physical card; and a data transmission module, configured to determine a target communication mode according to the target node relationship, and transmit target data to the receiving end based on the target communication mode.

[0007] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described node communication methods.

[0008] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described node communication methods.

[0009] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described node communication methods.

[0010] This application determines the first node information of the receiving end based on communication domain information, where the first node information indicates the node attribute to which the receiving end belongs, and the communication domain includes the receiving end and the sending end. It then compares the first node information and the second node information to determine the target node relationship, where the second node information indicates the node attribute to which the sending end belongs, and the target node relationship indicates whether the sending end and the receiving end are on the same physical card. Based on the target node relationship, it determines the target communication mode and sends the target data to the receiving end based on the target communication mode. By comparing the node information of the receiving end and the sending end, it can intelligently determine whether they are located on the same physical node (i.e., same-card relationship). If their network addresses are the same but their GPU numbers are different, it is determined to be same-card communication; otherwise, it is different-card communication. This automatic identification capability significantly reduces the user's configuration burden, allowing the framework to dynamically select the most suitable communication strategy based on the runtime environment. Based on the node relationship, the most efficient communication mode is selected. This mode selection ensures that regardless of whether the communication occurs between the same card or different cards, it can be executed in the optimal way, significantly improving overall communication efficiency and computing performance. Therefore, it can solve the problem of high cost in the communication mechanism of multi-GPU architectures in related technologies. Attached Figure Description

[0011] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of the hardware environment of an optional node communication method according to an embodiment of this application;

[0013] Figure 2 This is a flowchart of an optional node communication method according to an embodiment of this application;

[0014] Figure 3 This is a schematic diagram of an optional node communication method according to an embodiment of this application;

[0015] Figure 4 This is a schematic diagram of another optional node communication method according to an embodiment of this application;

[0016] Figure 5 This is a schematic diagram of an optional communication domain creation process according to an embodiment of this application;

[0017] Figure 6 This is a schematic diagram of an optional communication framework architecture according to an embodiment of this application;

[0018] Figure 7 This is a structural block diagram of an optional node communication device according to an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0020] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0021] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] According to one aspect of the embodiments of this application, a node communication method is provided. As an optional implementation, the above-described node communication method can be applied to, but is not limited to, [examples of other methods]. Figure 1 The hardware environment shown illustrates a node communication system. This node communication system may include, but is not limited to, a transmitter 102, a network 110, and a receiver 112. Both transmitter 102 and receiver 112 can run target clients (such as...). Figure 1 As shown, taking a large-scale client as an example. The aforementioned sending end 102 includes a display 108, a processor 106, and a memory 104. The display 108 can be used to display the system interface, etc., and also to provide a human-computer interaction interface to receive human-computer interaction operations on the interface and touch operations on different controls. The processor is used to generate interaction instructions in response to the aforementioned human-computer interaction operations. The memory is used to store communication domain-related files, etc.

[0023] Assumption Figure 1 The sending end 102 runs a client for training a large model. The specific process of this embodiment is as follows: Step S102, determine the first node information of the receiving end based on the communication domain information, wherein the first node information is used to indicate the node attribute to which the receiving end belongs, and the communication domain includes the receiving end and the sending end; Step S104, compare the first node information and the second node information to determine the target node relationship, wherein the second node information is used to indicate the node attribute to which the sending end belongs, and the target node relationship is used to indicate whether the sending end and the receiving end are the same physical card; Step S106, determine the target communication mode according to the target node relationship, and execute step S108, send the target data to the receiving end 112 through the network 110 based on the target communication mode.

[0024] Optionally, in this embodiment, the sending end 102 and the receiving end 112 can be terminal devices configured with a target client, which may include, but are not limited to, at least one of the following: mobile phones (such as Android phones, iOS phones, etc.), laptops, tablets, PDAs, MIDs (Mobile Internet Devices), PADs, desktop computers, smart TVs, etc. The target client may be a client that supports generating text responses. The network may include, but is not limited to, wired networks and wireless networks, wherein the wired network includes: local area networks, metropolitan area networks, and wide area networks, and the wireless network includes: Bluetooth, WIFI, and other networks that enable wireless communication.

[0025] The application scenarios of this application are not limited to the above examples. This application method can be used as long as GPU communication is involved.

[0026] Embodiments of this application provide a node communication method. Figure 2 This is a flowchart of an optional node communication method according to an embodiment of this application; as shown... Figure 2 As shown, the node communication method includes:

[0027] Step S202: Determine the first node information of the receiving end based on the communication domain information, wherein the first node information is used to indicate the node attribute to which the receiving end belongs, and the communication domain includes the receiving end and the sending end;

[0028] It's important to note that communication domain information refers to a dataset in a distributed communication system that defines the set of all nodes participating in the communication and their attributes. This typically includes network addresses (such as IP addresses and ports) for inter-node communication, GPU device information, and the logical order of nodes within the communication domain. The receiver's first node information refers to its node attributes within the communication domain, including its physical location (based on IP address and GPU ID) and its logical rank within the communication domain. This information is used by the sender to identify the receiver's precise location and attributes, thereby selecting the correct communication method.

[0029] The sending end first needs to understand the information of all nodes within the communication domain, including the specific location and attributes of the receiving end. In a distributed computing environment, determining node attributes is a prerequisite for selecting the communication method. The communication domain information provides the configuration of all nodes. By analyzing this information, the sending end can accurately identify the physical location (card information), logical location (IP address and port number), and its relative position (rank) within the communication domain of the receiving end. This accurate identification capability is crucial for subsequently selecting between intra-card communication (IPC) and non-intra-card communication (NCCL).

[0030] In an optional implementation, the sender obtains communication domain information from a configuration file or the system. This is typically a JSON file containing key attributes such as the network address, GPU ID, and logical GPU ID of all nodes. The sender parses the `devices` section of the communication domain information to find entries related to the receiver. Using the receiver's `logic_gpu` ID or its relative position (rank) within the communication domain, the sender can pinpoint the specific `addr` (network address) and `gpu` (GPU ID) attributes of the receiver. The sender may need to encapsulate the receiver's attributes into specific data structures or messages, such as `GetCommInfoRequest`, `IPCTensorMeta`, or `NcclTensorMeta`, for subsequent transmission via gRPC or direct communication. The sender can use the parsed first node information for automatic communication path identification, i.e., comparing its own attributes (second node information) with the receiver's attributes in subsequent steps to determine whether the transmission mode is the same or different from the same GPU. Simultaneously, the parsed node information is also used in subsequent data transmission processes, such as sending data via CUDA IPC or NCCL.

[0031] Step S204: Compare the first node information and the second node information to determine the target node relationship. The second node information is used to indicate the node attribute to which the sending end belongs, and the target node relationship is used to indicate whether the sending end and the receiving end are the same physical card.

[0032] It should be noted that the second node information refers to the node attribute information of the sending end, which also includes IP address, port number, GPU ID, etc., and is used to describe the location and resource status of the sending end. The target node relationship is determined by comparing the first node information and the second node information, which determines the relative positional relationship between the sending end and the receiving end, and is used to guide how to choose the appropriate communication path (same card or different card).

[0033] In an optional implementation, before communication begins, the sending end needs to extract its own (second node information) and the receiving end's (first node information) network addresses and GPU IDs from the communication domain configuration file. This can be achieved by parsing the JSON-formatted configuration file and reading the relevant entries. The sending end uses standard string or numerical comparison operations to check whether the sending end's and receiving end's network addresses (IP + port) and GPU IDs match exactly. If the IPs and GPU IDs are the same, but the ports are different, it can be inferred that the receiving ends are located on the same physical card, i.e., they are on the same card; conversely, if the IPs are different, or even if the IPs are the same but the GPU IDs are different, it is assumed that the receiving ends are located on a different physical card, i.e., they are on different cards.

[0034] Step S206: Determine the target communication mode based on the target node relationship, and send the target data to the receiving end based on the target communication mode.

[0035] It should be noted that the target communication mode is determined based on the relationship between the target nodes. For example, same-card communication uses CUDA IPC, while inter-card communication may use NCCL. The target data can be data that needs to be transmitted between the sending and receiving ends, such as the weights, gradients, or intermediate calculation results of a deep learning model.

[0036] In an optional implementation, during step S206, the sending end first checks the node information of the receiving end to determine the target node relationship between them. Once it is determined whether the node relationship is within the same SIM card or between different SIM cards, the sending end further selects the corresponding target communication mode. For communication within the same SIM card, CUDAIPC is used; for communication between different SIM cards, NCCL or multi-node RDMA communication is enabled. Finally, depending on the selected communication mode, the sending end calls the corresponding API or library function to send the target data to the receiving end. For the same SIM card scenario, data access is directly performed through a shared memory handle of CUDA IPC; for the different SIM card scenario, NCCL broadcast, AllReduce, and other operations are used, while metadata information is transmitted through gRPC to ensure that the receiving end can correctly process the received data.

[0037] Example 1:

[0038] Step S202: The client sends a gRPC request to the master node, requesting information to build the communication domain.

[0039] Step S204:

[0040] The master node responds to the gRPC request by providing information such as the communication domain ID (comm id), rank, worldsize, and transmission mode (IPC or NCCL).

[0041] Step S206:

[0042] Same-card communication implementation method:

[0043] The sending end (which could be any client) checks the node information of the receiving end. Finding that the IP address and GPU number are the same but the port is different, it determines that the target node is on the same GPU. The sending end generates a shared memory handle via CUDA IPC and notifies the receiving end of this handle information and the transmitted tensor metadata via gRPC. The receiving end accesses the shared GPU memory using the received handle information and reads the data. After the sending and receiving ends confirm that the data is correct, the data transmission is complete.

[0044] Implementation of heterogeneous card communication:

[0045] Similarly, the sending end checks the node information of the receiving end, determining the target node relationship as a different GPU based on the difference between the IP address and the GPU ID. The sending end uses the NCCL library to prepare for data broadcasting, while simultaneously transmitting metadata such as data size and type via gRPC. The receiving end listens for gRPC messages, pre-allocates GPU memory based on the received metadata, and calls the NCCL library's receive interface to wait for data arrival. NCCL completes the data broadcast, the receiving end verifies data integrity, and after confirming that there are no errors, the data transmission is complete.

[0046] This application determines the first node information of the receiving end based on communication domain information, where the first node information indicates the node attribute to which the receiving end belongs, and the communication domain includes the receiving end and the sending end. It then compares the first node information and the second node information to determine the target node relationship, where the second node information indicates the node attribute to which the sending end belongs, and the target node relationship indicates whether the sending end and the receiving end are on the same physical card. Based on the target node relationship, it determines the target communication mode and sends the target data to the receiving end based on the target communication mode. By comparing the node information of the receiving end and the sending end, it can intelligently determine whether they are located on the same physical node (i.e., same-card relationship). If their network addresses are the same but their GPU numbers are different, it is determined to be same-card communication; otherwise, it is different-card communication. This automatic identification capability significantly reduces the user's configuration burden, allowing the framework to dynamically select the most suitable communication strategy based on the runtime environment. Based on the node relationship, the most efficient communication mode is selected. This mode selection ensures that regardless of whether the communication occurs between the same card or different cards, it can be executed in the optimal way, significantly improving overall communication efficiency and computing performance. Therefore, it can solve the problem of high cost in the communication mechanism of multi-GPU architectures in related technologies.

[0047] In an optional implementation, determining the target node relationship by comparing the first node information and the second node information includes: obtaining the first node number, first physical number, and first mapping number of the receiving end from the first node information, wherein the first node number is used to indicate the node and port to which the receiving end belongs, the first physical number is used to indicate the serial number of the physical card on the node to which the receiving end belongs, and the first mapping number indicates the logical number to which the receiving end belongs; obtaining the second node number, second physical number, and second mapping number of the sending end from the second node information, wherein the second node number is used to indicate the node and port to which the receiving end belongs, the second physical number is used to indicate the serial number of the physical card on the node to which the receiving end belongs, and the second mapping number indicates the logical number to which the receiving end belongs; and determining the target node relationship by comparing the first node number, the first physical number, the first mapping number, and the second node number, the second physical number, and the second mapping number respectively.

[0048] It's important to note that the node number typically includes the node's IP address and port number, used to uniquely identify a node within the network. The physical card number refers to the actual identifier of the GPU device on the physical node; each GPU has a unique physical card number. In multi-GPU environments, for unified management, each GPU may be assigned a logical number for application-level reference. This number may differ from the physical card number, especially in containerized environments or virtualized GPU scenarios.

[0049] In an optional implementation, the sending end reads the receiver's attribute information (first node number, first physical number, first mapping number) from the communication domain configuration file or system, and similarly extracts its own attribute information (second node number, second physical number, second mapping number). First, the IP address portion of the node number is compared. If the receiver and sender have the same IP address, they are located on the same physical node; if the IP addresses are different, they are located on different physical nodes. Next, the physical card number and logical number are compared to further confirm whether it is same-card communication. If the receiver and sender have the same IP address and physical card number, but different port numbers and logical numbers, they are determined to be in a same-card relationship; if the IP addresses or physical card numbers are different, they are determined to be in a different-card relationship. Based on the determination of the target node relationship, the sending end selects a communication mode. Under the same-card relationship, CUDA IPC is used for data sharing; under the different-card relationship, NCCL or other cross-node communication technologies are used for data transmission.

[0050] Through the above-described embodiments of this application, the communication mode is determined by comparing node information before data transmission, and the most suitable method is used for data transmission. This dynamic adaptability not only improves communication efficiency but also provides a simplified and flexible communication solution for distributed computing tasks (such as deep learning model training).

[0051] In an optional implementation, the target node relationship is determined by comparing the first node number, the first physical number, the first mapping number, the second node number, the second physical number, and the second mapping number, respectively. This includes: determining the target node relationship as a different card relationship when the first node number and the second node number are different; determining the target node relationship as a different card relationship when the first node number and the second node number are the same but the first physical number and the second physical number are different; and determining the target node relationship as a same card relationship when the nodes indicated by the first node number and the second node number are the same but the ports are different, and the first physical number and the second physical number are the same.

[0052] It should be noted that when the network identifiers (node ​​numbers) of the sending and receiving ends do not match, or when they are on the same node but use different physical GPUs, they are considered to be in a different GPU relationship. When the sending and receiving ends are on the same node, use the same physical GPU, but are logically assigned different GPU port numbers, they are considered to be in a same GPU relationship.

[0053] In an optional implementation, the sending end obtains node information in the communication domain from a configuration file or system variables, including the receiver's first node number, first physical number, and first mapping number, and the sending end's second node number, second physical number, and second mapping number. Based on the obtained node information, the sending end compares the first node number with the second node number, the first physical number with the second physical number, and the first mapping number with the second mapping number one by one. For comparing network identifiers, it can directly check whether the node numbers are equal. Comparing physical GPU numbers involves checking whether the first physical number and the second physical number are consistent. Comparing container logical GPU numbers involves checking whether the first mapping number and the second mapping number are the same. Based on the comparison results, the sending end can make the following decisions:

[0054] If the first node number and the second node number are different, or if the node numbers are the same but the physical GPU numbers are different, then the target node relationship is determined to be a different GPU.

[0055] If all identifiers are the same but the port numbers are different, it means that although they are in the same physical location, they are regarded as independent logical entities due to the port differences. In this case, the target node relationship is determined to be the same card.

[0056] Once the target node relationship is determined, the sending end can select CUDA IPC or NCCL as the target communication mode based on whether it is the same card or a different card, and prepare for data transmission.

[0057] Example 2:

[0058] Suppose a distributed deep learning training environment with three nodes: master node, node1, and node2.

[0059] In step S202, each node requests communication domain information from the master via gRPC to obtain parameters such as rank, worldsize, and UUID, as well as transmission mode information for the same or different SIM cards.

[0060] Step S204: The sending end (taking node1 as an example) reads the first node information of the receiving end (such as node2) from the communication domain information, including its network identifier, GPU number, and container logical number. Then, node1 determines the target node relationship by comparing its own (second node's) network identifier, GPU number, and container logical number with the corresponding information of the receiving end.

[0061] Step S206: Based on the target node relationship determined in step S204, node1 selects CUDA IPC (for intra-card transmission) or NCCL (for inter-card transmission) as the target communication mode. Subsequently, node1 starts sending data to the receiving end node2 according to the selected communication mode.

[0062] In the case of different cards (NCCL transmission): the sending end initiates the data transmission process through the NCCL broadcast mechanism, and at the same time uses gRPC to notify the receiving end of the metadata about the data transmission (such as data size, shape, etc.) to ensure that the receiving end can accurately receive and process the data.

[0063] In a scenario with the same GPU (CUDA IPC transfer): the sending end generates a shared memory handle and passes it to the receiving end via gRPC. The receiving end uses the handle to directly access the shared GPU memory, thereby completing the data transfer and reception.

[0064] The above-described embodiments of this application enable intelligent determination of communication paths and automatic selection of the most suitable communication mode, thereby greatly simplifying the developer's usage process and improving data transmission efficiency and system flexibility.

[0065] In an optional implementation, sending target data to the receiving end based on the target communication mode includes: adjusting the target data according to a preset format to obtain metadata to be sent, and sending the metadata to the receiving end; determining the sending resource according to the target communication mode, and generating the sending resource, wherein the sending resource is used to send the target data to the receiving end; and sending the target data to the receiving end based on the sending resource.

[0066] It should be noted that metadata can be auxiliary information about the target data to facilitate data transmission, such as the data size, shape, type, shared memory handle, or communication identifier. Sending resources can be resources generated based on the target communication mode for data transmission, and may include CUDA IPC handles, NCCL communication domain IDs, streams, etc.

[0067] In an optional implementation, the sender converts the target data's attributes (such as shape, type, target code, etc.) into a preset metadata format, such as NcclTensorMeta or IPCTensorMeta. This formatted metadata is sent to the receiver via gRPC to inform the receiver of the expected data format and communication strategy. Based on the target communication mode, the sender selects and generates corresponding transmission resources. If it is inter-card communication (NCCL mode), the sender creates an NCCL communication environment and initializes relevant resources (such as communicator and stream). If it is intra-card communication (IPC mode), the sender generates a CUDAIPC shared memory handle and encapsulates it in the metadata. The sender then actually sends the target data based on the generated transmission resources.

[0068] Inter-card communication: The sending end calls functions such as nccl.Broadcast or nccl.AllReduce of NCCL to send data to the receiving end through the NCCL communication domain. The sending end encapsulates the tensor information to be transmitted (such as shape, type, and checksum) into NcclTensorMeta and sends it to the receiving end via gRPC. The sending end determines to use the NCCL communication mode, creates the NCCL communication environment, and sets the communication stream information. The sending end calls nccl.Broadcast to broadcast the data to the receiving end and synchronizes the stream to ensure the completion of data transmission.

[0069] Same-card communication: The sending end locally calls the relevant CUDA IPC APIs, shares GPU memory with the receiving end, and confirms the start and end of data transmission, as well as data integrity, via gRPC. The sending end encapsulates data information into an IPCTensorMeta message, including the shared memory handle (generated via CUDA IPC) and tensor shape information. The sending end generates a CUDA IPC shared memory handle and encapsulates it into metadata. The sending end sends the metadata to the receiving end via gRPC. After receiving the metadata, the receiving end uses the handle to directly access the shared GPU memory and read the data. The sending and receiving ends then confirm successful data reception and perform data integrity checks via gRPC.

[0070] The embodiments described above in this application intelligently select the most suitable communication method, ensuring efficient and secure data transmission between different physical or logical locations. Whether it's inter-GPU communication in a distributed scenario or intra-GPU communication among multiple logical GPUs on the same physical device, it can automatically adapt, simplifying developer operations and improving the flexibility and robustness of the communication framework.

[0071] In an optional implementation, determining and generating transmission resources based on the target communication mode includes: when the target communication mode is a heterogeneous communication mode, determining the data length of the target data and a first communication sequence number, wherein the first communication sequence number is used to indicate the communication sequence number of the sending end in the communication domain; and determining the second communication sequence number corresponding to each of at least one node included in the communication domain.

[0072] It should be noted that data length refers to the size of the data to be transmitted, used to allocate sufficient memory space at the receiving end. The first communication sequence number is the sender's communication rank in the communication domain, which the receiving end uses to determine the data source. The second communication sequence number refers to the communication rank of other nodes in the communication domain. In inter-card communication mode, the sender needs to know this information to correctly sort and initialize the communication domain in NCCL communication.

[0073] In an optional implementation, the sender calculates the length of the target data, which typically involves the data's byte size. Simultaneously, the sender uses communication domain information to determine its own communication rank. This information is used to construct an NcclTensorMeta message, which is sent to the receiver via gRPC. The sender reads all node information for the communication domain from a configuration file, including each node's network address, GPU ID, and communication rank (second communication rank) within the communication domain. This information is necessary for constructing the NCCL communication domain. In heterogeneous card communication mode, the sender needs to generate an NcclTensorMeta message containing information such as the data length and source communication rank (first communication rank). This message is then sent to the receiver via gRPC so that the receiver can prepare memory space to receive the data. Simultaneously, the sender also needs to use NCCL initialization information (such as ncclUniqueId) to create the communication domain with other nodes within the domain.

[0074] Example 3:

[0075] Assume the communication domain contains three nodes: master (172.20.0.22:30001), node1 (172.20.0.22:30002), and node2 (172.20.0.23:30001). Each node has two physical GPUs. The sender is node1, the receiver is node2, and the communication mode is non-GPU communication (NCCL).

[0076] node1 calculates that the length of the tensor data to be sent is 1MB, and obtains its own communication sequence number as 1 (rank=1) from the communication domain information. node1 reads the configuration file to confirm the communication sequence numbers of all nodes in the communication domain, that is, the sequence number of master is 0, the sequence number of node1 is 1, and the sequence number of node2 is 2.

[0077] The process involves constructing and sending an NcclTensorMeta message: node1 creates an NcclTensorMeta message, fills it with 1MB of data, sets the source communication sequence number to 1 (node1's rank), and sends it to node2 via gRPC. Simultaneously, node1 sends NCCL initialization information (ncclUniqueId, etc.) so that node2 can participate in building the NCCL communication environment. NCCL initialization information is exchanged between the sender and receiver, and among all nodes within the communication domain, creating the communication domain. Node1 and node2 share this information via gRPC or direct communication, ultimately completing the setup of the heterogeneous card communication environment. After the communication environment is created, node1 calls the NCCL send interface to send data to node2. Simultaneously, node2 listens for gRPC messages and NCCL communication from node1, preparing to receive data. Based on the data length and communication sequence number information in the received NcclTensorMeta message, node2 pre-allocates the corresponding GPU memory and calls the NCCL receive interface to receive data from node1. After receiving the data, node2 performs data verification and subsequent processing.

[0078] Through the above-described embodiments of this application, in a heterogeneous SIM card communication scenario, a complete process is achieved: the sending end prepares and sends resources, constructs the communication environment, and the receiving end receives and processes data based on the sent resource information. This not only ensures efficient and reliable data transmission between multiple nodes but also demonstrates the framework's intelligent selection and adaptability, simplifying the user's communication configuration and implementation process.

[0079] In an optional implementation, sending target data to the receiving end based on sending resources includes: calling a data broadcast interface and sending the target data to at least one node included in the communication domain; and stopping access to the communication domain after sending to at least one node has been completed.

[0080] It should be noted that the data broadcast interface can be an interface or function in the communication framework used to send data from one source node to all other nodes in the communication domain, such as ncclBroadcast in NCCL or a custom tensorlink.Broadcast in TensorLink.

[0081] In an optional implementation, the sender first prepares the corresponding transmission resources based on the determined target communication mode. For intra-card transmission, this may involve creating a CUDA IPC handle; for inter-card transmission, it is necessary to ensure the correct configuration of the NCCL environment. Once the resources are ready, the sender calls the broadcast interface in the communication framework, such as tensorlink.Broadcast, to begin data transmission. When calling the broadcast interface, the sender needs to specify the source location of the target data and the target communication domain. The broadcast interface copies the data from the sender to the cache or GPU memory of each receiver in the communication domain. If NCCL is used for multi-node data exchange, the data will be broadcast to all nodes in the sender's NCCL communication domain; while in intra-card transmission, the data is accessed directly through shared memory. The sender needs to monitor the progress of data broadcasting and wait for feedback from all receiving nodes confirming that the data has been received correctly. This is typically achieved through mechanisms such as gRPC to provide real-time feedback on the data transmission status. Once the sending end confirms that all nodes within the communication domain have successfully received the data, it will stop accessing the communication domain. This means releasing communication resources (such as NCCL communicator, IPC handle, etc.) and completing the broadcast operation to avoid unnecessary communication overhead later.

[0082] Suppose there are three nodes (master, node1, and node2) training a deep learning model, and the goal is to broadcast the latest weight parameters of the model among these nodes. The sending node (let's call it master) checks the relationship between the target nodes and finds that node1 and node2 are on different GPUs, so it decides to use NCCL for data broadcasting. The master calls the `tensorlink.Broadcast` function of TensorLink, specifying the target data as the model weights and the target communication domain as the set containing all nodes. Through the `tensorlink.Broadcast` interface, the master broadcasts the model weight data to the communication domain, i.e., the GPU memory of node1 and node2. The master listens for gRPC feedback from node1 and node2 to confirm that they have successfully received the data. If the verification fails, a retry process may be required. After node1 and node2 have confirmed that the data has been received correctly, the master stops accessing the communication domain and releases related communication resources, such as closing the NCCL communicator.

[0083] Through the above-described implementation methods of this application, data broadcasting is performed via NCCL in heterogeneous card scenarios, and resources are correctly released after the task is completed, ensuring system stability and efficient resource utilization. This mechanism is particularly important in distributed deep learning training, and can significantly improve the speed and quality of model training.

[0084] In an optional implementation, sending the target data to at least one node included in the communication domain includes: traversing at least one node according to the communication sequence number corresponding to each of the at least one node, and sending the target data to at least one node; and accessing the target data if it is determined that at least one node has received the target data.

[0085] It should be noted that the communication sequence number can be a unique identifier assigned to each node in the communication domain, used to identify the role of the node in the communication process, usually referred to as rank.

[0086] In an optional implementation, during data broadcasting, the sender must ensure that every node within the communication domain correctly receives the data. The sender obtains a list of communication sequence numbers for all nodes within the communication domain and sorts them numerically to form an ordered list of communication nodes. The sender iterates through this list of communication nodes, sending the target data to each node. This involves selecting a suitable data transmission interface based on the target communication mode (same card or different card), for example, using NCCL's ncclBroadcast or CUDA IPC's shared memory mechanism. The sender must wait for a reception acknowledgment signal from each node within the communication domain, typically via gRPC or a similar mechanism, to ensure error-free data transmission. Once it is confirmed that all nodes have successfully received the data, the sender will cease accessing the data. This means releasing the shared memory handle when using CUDA IPC, or ceasing data broadcasting operations in NCCL mode.

[0087] In an optional implementation, the sender first initializes the communication environment and creates appropriate communication resources, such as the NCCL communicator, based on the communication domain configuration information. The sender then iterates through each node sequentially according to its communication sequence number within the communication domain. For example, for nodes with communication sequence numbers 0, 1, and 2, the sender will send data in the order of 0, 1, and 2. For each traversed node, the sender calls the corresponding data broadcast interface to send data. If NCCL is used, the ncclBroadcast function is called to copy the data from the sender's source address to the target address of all other nodes. In addition to the actual data broadcast, the sender also needs to send metadata (such as NcclTensorMeta or IPCTensorMeta) to all nodes via gRPC to ensure that the receivers know how to receive and process the data. After receiving the data and completing the verification, each receiving node sends an acknowledgment signal back to the sender via gRPC. The sender needs to collect all acknowledgment signals before considering the data broadcast successful. After all receiving nodes have confirmed receipt of the data, the sender closes the resource (such as the NCCLcommunicator) according to the communication mode, ceasing access to the data and preventing resource leaks.

[0088] The above-described implementation of this application enables efficient and unified data broadcasting to all nodes within the communication domain, regardless of whether these nodes are based on the same card (local GPU) or different card (remote GPU) communication mode.

[0089] In an optional implementation, the transmission resources are determined and generated according to the target communication mode, including: when the target communication mode is a card-to-card communication mode, determining the video memory address of the target data; and creating a shared memory handle based on the video memory address.

[0090] It's important to note that in same-card communication mode, the determination and generation of sending resources primarily revolve around the creation of a shared memory handle. The sending end loads the target data into the GPU memory and records its memory address. This is typically done using deep learning framework APIs (such as `.to(device)` in PyTorch and `.device` in TensorFlow). Using the CUDAIPC API, the sending end creates a shared memory handle based on the GPU memory address of the target data. In CUDA, this step can be accomplished by calling the `cudaIpcGetMemHandle()` function, which accepts a pointer to GPU memory and returns an IPC memory handle for sharing GPU memory resources. The sending end packages the shared memory handle into an `IPCTensorMeta` message and sends it to the receiving end via gRPC. This ensures that the receiving end can obtain the correct handle for accessing GPU memory. In addition to the shared memory handle, the sending end also needs to send metadata about the target data, such as data shape, type, and checksum, to ensure that the receiving end can process the data correctly. The sending end waits for the receiving end to confirm receipt of the metadata and shared memory handle before proceeding to the next step.

[0091] Example 4:

[0092] Suppose two Python processes (process A and process B) are running on the same physical machine, sharing a GPU card. Process A needs to send a batch of tensor data to process B for data synchronization or model parameter updates.

[0093] Process A determines the video memory address: Process A loads the target data (assuming a tensor with elements such as [10, 20, 30]) into the video memory of GPU card 5 and retains the address of the tensor in the video memory.

[0094] Process A creates a shared memory handle: Process A calls cudaIpcGetMemHandle(&shared_handle), where &shared_handle is a pointer to the video memory address, to create a shared memory handle shared_handle.

[0095] Process A constructs and sends the IPCTensorMeta message: Process A creates the IPCTensorMeta message, fills the shared memory handle `shared_handle`, and includes the data's shape ([10, 1, 1]), type, checksum, etc. The IPCTensorMeta message is then sent to process B via gRPC.

[0096] Process B receives and processes metadata: Process B listens for gRPC, and upon receiving an IPCTensorMeta message, reads the shared memory handle and metadata from it. Based on the shape, type, and checksum of the metadata, process B knows how much video memory space needs to be allocated and prepares to receive shared data.

[0097] Process B opens a shared memory handle: Process B calls cudaIpcOpenMemHandle(tensor_ptr, shared_handle), where tensor_ptr is a pointer to the shared memory used by the receiving end to complete access to the shared memory.

[0098] Data Reading and Confirmation: Process B uses tensor_ptr to read data from the video memory and performs necessary verifications (such as checking if the checksum matches). After confirming that the data is correct, it sends a successful reception confirmation to process A via gRPC.

[0099] Processes A and B close the shared memory handle: After confirming successful data transmission and reception, both processes call cudaIpcCloseMemHandle() to close the shared memory handle, ensuring the proper release of GPU resources and preventing resource leaks.

[0100] Through the above-described embodiments of this application, in a same-card communication scenario, a shared memory handle is created through the CUDA IPC mechanism, thereby efficiently sharing data among multiple processes located on the same GPU card. This approach not only reduces data copying and communication latency but also simplifies the communication process and improves program readability and maintainability.

[0101] In an optional implementation, sending target data to the receiving end based on sending resources includes: sending mode determination information to the receiving end, wherein the mode determination information is used to notify the receiving end that this communication is a same-card communication; and sending a shared memory handle to the receiving end.

[0102] It's important to note that mode determination information is a signal sent to the receiving end during communication, informing it that the current communication uses the same-card communication mode. This information is typically contained in metadata or specific messages, allowing the receiving end to select the correct communication strategy. A shared memory handle is an identifier used to share memory resources on the GPU. The receiving end can directly access data in the sending end's GPU memory using this handle, without needing to copy the data.

[0103] In the same-card communication mode, the operation of sending target data to the receiving end mainly includes the following steps: The sending end encapsulates the shape, type, checksum, and other information of the target data into an IPCTensorMeta message, and attaches the same-card communication mode determination information. The sending end sends the constructed IPCTensorMeta message to the receiving end based on gRPC or a similar RPC framework. After receiving this metadata, the receiving end knows that it will receive data via same-card communication. The sending end also encapsulates the shared memory handle into the message and sends it to the receiving end via gRPC. After receiving the shared memory handle, the receiving end can directly access the sending end's GPU memory. After receiving the IPCTensorMeta message and the shared memory handle, the receiving end creates a locally appropriate memory region based on the metadata information, and uses the shared memory handle to directly access the sending end's memory to read the target data. After completing the data reading, the receiving end sends an acknowledgment message to the sending end via gRPC, indicating that the data reception was successful.

[0104] Through the above-described embodiments of this application, efficient communication that directly shares data between GPUs without data copying is achieved in co-GPU communication scenarios. This approach not only significantly reduces communication latency but also simplifies communication code and lowers the barrier to entry for developers. It is suitable for large-scale data processing and model training scenarios such as deep learning and reinforcement learning, and can effectively improve the overall system performance.

[0105] In an optional implementation, after sending the shared memory handle to the receiving end, the method includes: upon receiving a reference instruction, obtaining a first handle from the reference instruction, wherein the first handle is used to indicate a reference video memory storing reference data; reading reference data from the reference video memory; and closing the first handle.

[0106] It should be noted that the reference instruction can be an instruction received by the receiving end in the same-card communication mode, containing shared memory handle information; it is typically a gRPC message. The reference memory can be a memory area stored on the sending GPU, containing reference data, which the receiving end accesses through the first handle. The reference data can be actual data stored in the sending GPU's memory, such as model parameters and calculation results, which the receiving end needs to retrieve.

[0107] In an optional implementation, the receiving end listens to the gRPC channel and receives an IPCTensorMeta message (reference instruction) from the sending end. The first handle is parsed from the IPCTensorMeta message; this is a shared handle to the sending end's GPU memory, allowing the receiving end to access the sending end's reference memory. The receiving end calls cudaIpcOpenMemHandle() via the CUDA IPC mechanism, using the first handle to open access to the sending end's reference memory. The receiving end accesses the opened shared memory region and reads the reference data. This may involve copying the data to the receiving end's local memory for subsequent computation or processing. After reading the data, the receiving end performs data integrity verification based on a checksum (such as target_code) to ensure the accuracy of data transmission. Once the data has been read and verified, the receiving end must call cudaIpcCloseMemHandle() to close the first handle, releasing access to the shared memory and preventing resource consumption and leakage.

[0108] Through the above-described embodiments of this application, video memory resources are properly managed while ensuring data integrity, avoiding resource leakage, and guaranteeing the efficiency of communication and the stability of the system.

[0109] Figure 3 This is a schematic diagram of an optional node communication method according to an embodiment of this application; as shown... Figure 3 As shown, the sending end executes step S302, calling the unified interface. The communication framework provides a unified Broadcast interface for broadcasting data within the communication domain. This interface call can automatically adapt to communication methods between the same and different GPUs, automatically selecting IPC or NCCL for data transmission based on the target device's location. The sending end first traverses all nodes in the communication domain to determine the destination of the target data. For each receiving node, the sending end checks its network identifier, physical GPU number, and other information to determine the relationship between the target node and the sending end (same GPU or different GPU).

[0110] Next, proceed to step S304, mode preparation:

[0111] Same-card communication mode preparation: If the target node and the sender are located on the same physical GPU card, that is, the same-card condition is met, the sender encapsulates the metadata of the target data (such as data shape, type, checksum, shared memory handle, etc.) into the IPCTensorMeta message, and prepares to share data through CUDA IPC.

[0112] Preparation for inter-GPU communication mode: If the target node and the sender belong to different physical GPUs, the sender encapsulates the metadata of the target data into an NcclTensorMeta message, preparing to broadcast the data via NCCL's nccl.Broadcast.

[0113] Then, step S306 is executed, and metadata is sent:

[0114] Same-card transmission: For nodes on the same card, the sending end sends the shared memory handle (the handle field in IPCTensorMeta) to the receiving end through the CUDA IPC mechanism and notifies the receiving end to start receiving data.

[0115] Transmission via different SIM cards: For nodes with different SIM cards, the sending end uses the NCCL broadcast function nccl.Broadcast to send data, and sends an NcclTensorMeta message via gRPC to notify the receiving end that the data will be transmitted via NCCL.

[0116] Next, step S308 is executed at the receiving end to receive metadata: the receiving end waits for the metadata sent by the sending end via gRPC, including IPCTensorMeta or NcclTensorMeta messages.

[0117] NcclTensorMeta contains metadata about the data transmitted in NCCL communication mode, such as the amount of data, the rank of the sending source, and the target_code and force_update flags used for data integrity and consistency verification.

[0118] IPCTensorMeta: Used for IPC communication mode. In addition to containing the shape of the data and target_code, it also includes a handle for shared memory, which the receiving end uses to access the shared memory.

[0119] Receiver 3 executes step S310-2, reading data based on the handle: The receiver opens the shared memory according to the handle in IPCTensorMeta and directly reads the data from the GPU memory without additional data copying steps. After completing data reading and verification, the receiver closes the shared memory handle to avoid memory leaks, and then returns a successful reception confirmation to the sender via gRPC. For IPC mode, the receiver opens the IPC handle, reads the shared tensor data, and finally needs to explicitly close the handle to prevent memory leaks.

[0120] Receiver 1 and Receiver 2 execute step S310-1, waiting for the NCCL broadcast to complete, and then directly acquire the broadcast data. Regardless of whether communication is within the same SIM card or between different SIM cards, the receiver needs to verify the acquired data. After completing data reception, the receiver directly returns a successful reception confirmation to the sender via gRPC. For NCCL mode, data reception is automatically completed through the nccl.Broadcast operation; the receiver confirms data reception via stream.sync before continuing execution.

[0121] Next, step S312 is executed to determine synchronization. Stream synchronization can be used, specifically `stream.sync`. CUDA streams are a synchronization mechanism in GPU computing that allows parallel operations to execute in an ordered manner within a specified stream. The `stream.sync` operation ensures the completion of preceding asynchronous operations and is a crucial step in ensuring data integrity and program sequence during data transmission. After NCCL broadcasting, a `stream.sync` call is usually followed to ensure all synchronization operations are completed, avoiding data races and inconsistencies. In IPC mode, although data sharing has already occurred, `stream.sync` calls may still be added before and after data processing to ensure the correct timing of data access.

[0122] After sending the data, the sending end executes step S314 to receive feedback and wait for reception confirmation signals from all receiving ends to ensure that the data has been correctly received by all target nodes. Once all receiving ends confirm that the data has been received correctly, steps S316 and S318 can be executed to confirm synchronization through the stream.sync operation and close the unified interface. Then, the sending end can prepare and start the next round of data broadcasting, repeating the above process.

[0123] The following defines the notification message from the sender to the receiver to inform them of received data. In the communication framework described above, the data transmission mechanism between the sender and receiver is meticulously designed to ensure that both intra-card and inter-card communication can proceed in the most efficient and reliable manner. Explanation of the function and information of gRPC messages:

[0124] IPCTensorMeta: This message is primarily used in intra-card communication scenarios, i.e., when data is transferred between different processes or threads located on the same GPU card. It contains key information required by the receiving end to read and process data, specifically including:

[0125] handle: GPU memory management handle, used by the receiving end to enable shared memory so as to directly access the tensor data provided by the sending end.

[0126] shape: Transmits the shape of the tensor, such as the width, height, depth and other dimensional information, which is crucial for allocating the corresponding amount of GPU memory space at the receiving end.

[0127] target_code: A checksum used to verify the integrity and accuracy of transmitted data.

[0128] force_update: A boolean flag indicating whether the receiver should force an update of the tensor data even if data verification fails.

[0129] NcclTensorMeta: Applicable to inter-GPU communication scenarios, i.e., when data is transferred between different GPU cards. The information carried in this message helps the receiving end prepare to receive data and perform necessary verifications:

[0130] tensor_num: The number of tensors transmitted in this transmission, which helps the receiving end to accurately allocate the receive buffer.

[0131] src: The rank number of the source of the data being sent in the communication domain, used to identify the data sender.

[0132] target_code: Similar to IPCTensorMeta, this is the code used for data verification to ensure the accuracy of the received data.

[0133] force_update: Also exists, and its function is the same as force_update in IPCTensorMeta, determining whether to force an update of the tensor when data validation fails.

[0134] SendTensorMetaRequest: This is a request sent by the sender to the receiver before data transmission. It is used to notify the receiver in advance of the tensor metadata that is about to be transmitted, specifically including:

[0135] comm_name: The name of the communication domain, used to uniquely identify the context of this communication.

[0136] args0: Additional information related to data transfer, such as the list of tensors to be transferred or other metadata.

[0137] trans_mode: Indicates whether the data transfer will use IPC (same card) or NCCL (different card) transmission mode.

[0138] oneof meta_info: A union type where, depending on trans_mode, the sender will choose to populate either the IPCTensorMeta or NcclTensorMeta field to transmit specific transport metadata.

[0139] SendTensorMetaResponse: This is the response returned by the receiving end to the sending end after successfully receiving or processing the SendTensorMetaRequest. It is used to confirm the reception status or to report error information. The main fields include:

[0140] code: Status code, used to indicate whether the receiving or processing process was successful. 0 represents success, and non-zero values ​​indicate failure or abnormal conditions.

[0141] errmsg: If the code is not 0, it contains specific error or warning information to help the sender understand the detailed reasons for the reception failure.

[0142] Through IPCTensorMeta and NcclTensorMeta, the TensorLink communication framework can efficiently transmit data metadata according to different communication modes (same card or different cards). SendTensorMetaRequest and SendTensorMetaResponse constitute a complete metadata data transmission and confirmation mechanism, ensuring the accuracy, reliability, and timely feedback of data transmission. These mechanisms work together to enable TensorLink to intelligently select the best communication method and automatically handle errors and retransmissions during transmission, greatly improving data communication efficiency and system stability in multi-GPU, multi-node computing scenarios.

[0143] In an optional implementation, sending metadata to the receiving end includes: sending a first checksum corresponding to the target data to the receiving end; retransmitting the target data upon receiving a retransmission instruction; and after sending the target data to the receiving end based on the transmission resources, including: determining a third checksum based on the reference data upon receiving reference data and a second checksum; and sending a retransmission instruction to the reference node if the second checksum and the third checksum are different.

[0144] It should be noted that the first checksum is a check value calculated by the sending end before sending data, used to ensure data integrity and correctness. The reference data can be data read by the receiving end from the sending end, used to calculate the checksum. In same-card communication, this is accessed directly via shared memory; in different-card communication, it is received via NCCL. The second checksum can be a checksum obtained by the receiving end from the metadata received along with the target data, used to compare with the checksum result of the reference data to verify the accuracy of the data. The third checksum can be a checksum recalculated by the receiving end based on the content of the reference data after reading it, used to compare with the second checksum to determine the accuracy of data transmission.

[0145] In an optional implementation, the sending end generates metadata based on the characteristics of the target data, including a first checksum, and sends it to the receiving end via a communication protocol (such as gRPC). The metadata may also contain information such as data shape and type. The sending end uses gRPC to construct an IPCTensorMeta or NcclTensorMeta message, encapsulating the first checksum and other metadata of the data within it, ensuring that the receiving end can correctly identify and verify the data. Based on the target communication mode (same card or different card), the sending end sends the target data itself to the receiving end, possibly via CUDA IPC or NCCL. After receiving the target data and metadata, the receiving end recalculates the third checksum based on the data content and compares it with the second checksum in the metadata. After reading the data, the receiving end calculates the third checksum using the same verification algorithm and compares it with the second checksum contained in the metadata to confirm the integrity of the data. If the second checksum and the third checksum match, the receiving end confirms that the data reception is error-free and can directly use the data. If the checksum comparison results are inconsistent, the receiving end identifies a possible data transmission error and sends a retransmission command to the sending end via gRPC. The receiving end constructs and sends a SendTensorMetaRequest message via gRPC, setting force_update=False to indicate a data reception error, requiring the sending end to retransmit the data. Upon receiving the retransmission command, the sending end reprepares the data and its first checksum, and re-executes the metadata and data transmission process. The sending end listens on the gRPC channel; upon receiving a retransmission request, it reconstructs the IPCTensorMeta or NcclTensorMeta message and retransmits the target data via CUDA IPC or NCCL.

[0146] Through the above-described embodiments of this application, the verification and retransmission mechanism is used during data transmission to ensure the reliability and accuracy of data transmission. At the same time, in the event of transmission errors, data retransmission can be performed efficiently, reducing training interruptions or result deviations caused by data corruption, and improving the overall robustness of communication and user experience.

[0147] In an optional implementation, retransmitting the target data includes: incrementing the transmission count by one when transmitting the target data; and adjusting the first indicator bit to an update state when the transmission count exceeds a preset number, and sending the target data and the first indicator bit to the receiving end, wherein the update state is used to instruct the receiving end to update the parameters.

[0148] It should be noted that the first indicator bit can be a field in SendTensorMetaRequest used to mark whether to force parameter updates (Force_update). When set to update status, the receiving end will ignore data verification failure and directly update the parameters.

[0149] In optional implementations, data may need to be retransmitted during data transmission, especially in cases of network instability, inter-GPU communication delays, or failures. To ensure that the data is eventually received correctly by the receiver, the sender employs a retry mechanism. Each retransmission records the number of transmissions. When the number of transmissions exceeds a preset retry threshold, the sender adopts a more aggressive strategy—setting the Force_update flag to True, thereby forcing the receiver to update its parameters. Even if the data checksum does not match, this may indicate that some data has been altered or lost during transmission.

[0150] Before data transmission begins, the sender initializes the transmission count to 0. After each failed attempt to send the target data to the receiver, the sender increments the transmission count. The sender checks if the current transmission count exceeds the preset maximum retry count. If it does not, it continues to attempt to resend the target data; if it does, it enters an update state strategy. Once the transmission count exceeds the preset limit, the sender adjusts Force_update (first indicator bit) to the update state (True), instructing the receiver to perform parameter updates even if data integrity issues are encountered. Subsequently, the sender sends the target data and the adjusted Force_update state to the receiver via a SendTensorMetaRequest message. When Force_update is False, the receiver decides whether to retransmit or receive data based on data verification. When receiving a Force_update state of True, the receiver directly updates the parameters, even if target data verification fails, ensuring the training process does not stall due to a single communication failure.

[0151] Through the above-described embodiments of this application, even in the event of poor network conditions or communication failures, the continuity of deep learning training or model synchronization can be ensured, training interruptions caused by communication failures can be avoided, and the robustness and flexibility of the entire communication framework can be improved.

[0152] Figure 4 This is a schematic diagram of another optional node communication method according to an embodiment of this application; as shown Figure 4 As shown, the sending end executes step S402 to send data; then executes step S404 to determine whether the preset number of times has been reached. If the preset number of times has been reached, execute step S406-1 to update the parameters; if the preset number of times has not been reached, execute step S406-2 to resend the target data; finally, execute step S408 to receive feedback.

[0153] The sending end prepares the tensor data to be broadcast. Before the data is officially sent, the sending end generates a checksum (Target_code) based on the data content to verify the accuracy of the data. The sending end sends the metadata (including the Target_code) to the receiving end via gRPC (Remote Procedure Call) communication. The metadata also includes key information such as the data shape and whether a forced update is required. After the metadata is successfully received, the sending end selects the appropriate communication method (same card or different card) based on the communication mode, using CUDA IPC (same card) or NCCL (different card) to send the data entity (tensor) to the receiving end. After reading the data sent by the sending end, the receiving end immediately regenerates the checksum based on the received data content, and then compares this checksum with the Target_code in the metadata to verify the correctness of the data transmission. If the checksum matches, the receiving end considers the data reception successful, and no further action is required. It then exits the receiving process normally and returns a success confirmation to the sending end. If the checksum does not match and the `Force_update` field is False, the receiver will determine that a data transmission error has occurred. It will not use the received erroneous data for model updates but will instead return an error signal to the sender, requesting data retransmission. When the sender receives a transmission error signal, it checks whether the pre-set maximum number of retransmissions has been reached. If not, the sender will keep `Force_update` at False and retry the data transmission. If the maximum number of retransmissions has been reached, the sender will set `Force_update` to True, meaning that even if the received data fails to pass verification, the receiver must still use this data for updates to avoid infinite retry loops that could cause training stagnation or other problems. Based on the latest state of `Force_update`, the sender decides whether data retransmission is necessary. If `Force_update` is still False, the sender will repeat steps 1 to 4 until data verification succeeds or the maximum number of retransmissions is reached. If `Force_update` is True, the sender will perform a one-time data transmission, and the receiver will use the received data for parameter updates regardless of the data verification result.

[0154] In an optional implementation, before determining the first node information of the receiving end based on the communication domain information, the method includes: incrementing the node count value by one when an information acquisition instruction is received; sending first creation information to at least one candidate node when the node count value reaches a preset threshold, wherein the candidate node is the node that sent the information acquisition instruction, and the first creation information is used to indicate the information of the communication domain; and determining that the communication domain creation is completed after receiving a completion instruction from at least one candidate node.

[0155] It should be noted that an information retrieval command can be a request sent by a candidate node participating in communication domain creation to the leading node (such as the Master node) for obtaining the initial information or status required for communication domain creation. The node count value can be a counter used by the Master node to track the number of nodes that have received the information retrieval command, serving as a basis for determining whether a preset threshold has been reached. The preset threshold can be a threshold used during the communication domain creation process to determine when to broadcast the first creation information to candidate nodes, typically equal to the expected total number of nodes in the communication domain or the world size. The first creation information contains key information and parameters required for communication domain creation, such as the communication domain ID (CommID), the role of each node (e.g., rank), and a packet explaining how to establish communication, and is sent by the Master node to the candidate nodes.

[0156] In an optional implementation, the Master node listens for information retrieval commands. Each time it receives a command, it increments the node count, recording how many nodes have requested to join the communication domain creation. Once the node count reaches a preset threshold (typically the total number of expected nodes in the communication domain), the Master node considers all expected nodes ready for communication domain creation. At this point, the Master node constructs and sends the first creation information to all candidate nodes that have sent information retrieval commands; these are the participating nodes in the communication domain. Each candidate node, upon receiving the first creation information, creates a communication domain instance locally based on the provided information, including but not limited to assigning a rank, setting a CommID, and initializing communication parameters. After completing local communication domain creation and performing necessary initialization checks, the candidate node sends a completion command to the Master node, indicating that its local communication domain creation is complete. When the Master node receives completion commands from all candidate nodes, it means that all nodes have successfully created the communication domain, and the overall communication domain creation process is considered complete.

[0157] The Master node needs to be able to receive and process information retrieval instructions from multiple nodes, as well as a mechanism to record node counts. The Master node should be able to construct and send initial creation information based on the expected information of the communication domain. This typically includes necessary information such as the communication domain ID, the rank assignment of each node, and the communication protocol. Each candidate node should have logic for processing the initial creation information, including creating and initializing a local communication domain instance based on the received information, and a mechanism for sending completion instructions. The Master node needs to design a mechanism to count and confirm completion instructions from all candidate nodes to ensure the overall completion of communication domain creation.

[0158] The above-described implementation methods of this application ensure the coordination and efficiency of communication domain creation. Through centralized scheduling of the Master node and collaborative response between nodes, the distributed communication domain is successfully constructed, providing a solid foundation for subsequent large-scale data transmission and model training synchronization.

[0159] In an optional implementation, first creation information is sent to at least one candidate node, including at least one of the following:

[0160] 1) Send the domain number of the communication domain to at least one candidate node;

[0161] 2) Send at least one communication sequence number corresponding to each candidate node to at least one candidate node;

[0162] 3) Send the number of nodes in the communication domain to at least one candidate node;

[0163] 4) Send the second node information to at least one candidate node.

[0164] It should be noted that candidate nodes can be any of the nodes participating in communication when the communication domain is created. They may be located in the same or different physical locations (machines or GPU cards). The domain sequence number (the domain number of the communication domain) is used to uniquely identify the number of the communication domain, allowing nodes and operations in different communication domains to be distinguished and managed. The communication sequence number can be a unique number for each candidate node within the communication domain, used to identify the node's specific position and role in the communication process.

[0165] In an optional implementation, at the initial stage of creating a communication domain, the master node needs to distribute creation information to all participating candidate nodes to ensure that each node can correctly initialize the communication environment based on this information, including but not limited to identifying the communication domain, determining its own position and role in the domain, knowing the total number of nodes in the domain, and obtaining the necessary node configuration information.

[0166] The master sends the domain sequence number of the communication domain to all candidate nodes via gRPC, allowing nodes to identify the communication domain they are participating in. The master sends each candidate node its communication sequence number within the communication domain to help nodes understand their position in the communication process. The master sends the total number of nodes in the communication domain to the candidate nodes, which helps nodes estimate the communication load and allocate memory resources appropriately. The master also needs to provide candidate nodes with necessary secondary node information, such as the node's network address and GPU ID, to ensure that data transmission between nodes can proceed according to the optimal path.

[0167] Example 5:

[0168] Suppose that the master node (NodeM) needs to create a communication domain that includes three candidate nodes: NodeA, NodeB, and NodeC, which are located on GPU cards 5, 6, and 7, respectively.

[0169] Preparation of creation information: NodeM initializes the communication domain creation process and prepares creation information, including the domain number of the communication domain, the communication number of each node (such as NodeA=0, NodeB=1, NodeC=2), the total number of nodes in the communication domain (3), and the second node information of each node (network address, GPU number).

[0170] Domain Sequence Number Distribution: NodeM sends the domain sequence number of the communication domain to all candidate nodes via gRPC, such as Comm_id=12345.

[0171] Communication sequence number allocation: NodeM sends communication sequence number 0 to NodeA, communication sequence number 1 to NodeB, and communication sequence number 2 to NodeC, through the rank field in SendTensorMetaRequest.

[0172] Node count notification: NodeM sends the total number of nodes in the communication domain, i.e., worldsize=3, to all candidate nodes as part of the communication domain creation information.

[0173] Second-node information transmission: NodeM transmits the network address of each node (such as NodeA's addr="172.20.0.22:30001"), GPU number, and other second-node information to assist the nodes in configuring the environment and preparing for communication.

[0174] Confirmation Received: After receiving the creation information, the candidate node sends a SendTensorMetaResponse via gRPC to confirm successful reception of the information, and may also return code indicating its own initialization status.

[0175] Communication domain creation and confirmation: After all candidate nodes successfully receive and confirm the creation information, NodeM and candidate nodes begin to create the actual communication environment according to the communication domain configuration. After the creation is completed, they inform the master node that the creation was successful or failed through confirmation signals (such as SendTensorMetaResponse).

[0176] Through the above-described embodiments of this application, each node can accurately initialize its role and configuration in the communication domain based on the received creation information, thereby laying a solid foundation for subsequent data communication and synchronization operations.

[0177] In an optional implementation, after sending first creation information to at least one candidate node, the process includes: upon receiving second creation information, determining third node information based on the second creation information, wherein the third node information is used to indicate node information of the master node of the reference communication domain; creating the reference communication domain based on the second node information and the third node information, and sending a completion instruction to the master node.

[0178] It should be noted that after receiving the first creation information from the Master node, the candidate node will perform preliminary communication domain creation and configuration based on this information. Next, the candidate node needs to receive second creation information, which is usually sent by the master node of the reference communication domain. This contains more detailed creation parameters to help the candidate node understand and configure the specific details of the reference communication domain. By parsing the second creation information, the candidate node can determine the third node information, namely the master node information of the reference communication domain. This step is necessary because the candidate node needs to know how to communicate with the master node to obtain further creation guidance or confirm the creation status of the communication domain. Once all the necessary information is determined, the candidate node will create or join the reference communication domain based on the second creation information and the third node information, completing the initialization of the local communication domain instance. Finally, the candidate node sends a completion command to the master node, indicating that it has successfully created the communication domain and is ready to await further instructions or begin data exchange.

[0179] In an optional implementation, the Master node sends first creation information containing basic creation parameters, such as the communication domain ID and the node's assigned rank, to all candidate nodes. The Master node listens for second creation information from the reference communication domain's master node, which typically contains more detailed communication domain settings and master node information. After receiving completion instructions from all candidate nodes, the Master node confirms that the entire communication domain creation process has been successfully completed and can begin operations such as data broadcasting and parameter synchronization.

[0180] Upon receiving the first creation information from the Master node, the candidate node performs preliminary creation of its local communication domain, such as assigning a rank and setting the communication domain ID. The candidate node then receives the second creation information from the master node of the reference communication domain, parsing the detailed parameters within the information, particularly determining the third node information, i.e., the detailed information of the master node of the reference communication domain. Based on the parameters in the first and second creation information, the candidate node creates or joins the reference communication domain, completing the detailed configuration and initialization of its local communication domain instance. After completing the local instance creation of the reference communication domain, the candidate node sends a completion command to the Master node, indicating that it is ready to participate in data exchange and synchronization within the communication domain, and awaits confirmation from the Master node that the entire communication domain creation is complete.

[0181] The above-described implementation methods of this application simplify the process of creating new communication domains and ensure the consistency and efficiency of communication domain creation, providing a stable and reliable communication infrastructure for distributed deep learning and reinforcement learning tasks.

[0182] In an optional implementation, creating a reference communication domain based on the second node information includes: determining the reference node relationship based on the third node information and the second node information; and creating the reference communication domain when the reference node relationship is a different card.

[0183] It's important to note that during the initial creation of the communication domain, the Master node sends second-level node information to candidate nodes. This information includes the node's communication sequence number and physical location within the domain. Simultaneously, each candidate node holds third-level node information provided by a configuration file, which includes detailed attributes of all nodes in the communication domain. Based on this information, candidate nodes can resolve their relationships with other nodes.

[0184] In an optional implementation, candidate nodes identify which nodes reside on the same physical GPU card (same-card communication) and which reside on different physical GPU cards (different-card communication) by comparing their own attributes in the second node information with the attributes of other nodes in the third node information. Once a reference node for different-card communication is determined, the candidate node creates a reference communication domain according to the specifications of NCCL or other communication libraries suitable for different-card communication. The candidate node initializes the NCCL communication domain using the ncclCommInitRank function, which requires providing the world size (total number of nodes) within the communication domain, its own communication sequence number, and other potential parameters (such as link priority).

[0185] When determining reference node relationships, candidate nodes should possess the ability to dynamically parse node information to adapt to changes in node configuration within the communication domain, such as the addition or removal of nodes or reconfiguration of GPU cards. During the creation of the reference communication domain, an error handling mechanism is required. For example, if inter-node communication failure or incorrect parameter configuration is detected, a mechanism should be in place to resend configuration information or abort the communication domain creation. To ensure that all candidate nodes can correctly create the reference communication domain, the Master node and candidate nodes need to maintain information synchronization, ensuring that each node holds the latest node information.

[0186] Through the above-described embodiments of this application, the candidate node can accurately determine its communication relationship with other nodes based on the received second node information and the third node information in the configuration file, and create an efficient inter-card communication domain when necessary, ensuring the flexibility and high speed of data transmission and providing a stable foundation for subsequent communication operations.

[0187] Figure 5This is a schematic diagram of an optional communication domain creation process according to an embodiment of this application; as shown... Figure 5 As shown, the master node executes step S502. In the communication domain creation process, the master node completes system initialization, enters listening mode, and prepares to receive client requests. After completing its own initialization, the client sends a request to the master node to obtain key information required for building the communication domain, such as the communication sequence number (rank), world size (worldsize), and communication domain ID (commid). Node1, Node2, and Node3 execute steps S504-1, S504-2, and S504-3 respectively to obtain creation information. They can send a GetCommInfoRequest request to attempt to obtain the communication domain creation information. After receiving all requests, the master node uses an internal counter to ensure that this is the last request, and then executes step S506, sending the creation information GetCommInfoResponse to Node1, Node2, and Node3. The content includes the assigned rank, worldsize (2, only counting inter-card communication nodes), and commid (e.g., 12345). Upon receiving the information, nodes 1, 2, and 3 execute steps S508-1 and S508-2 respectively to create a communication domain. A communication domain can contain all nodes. For nodes communicating with each other on the same SIM card, an actual NCCL communication domain does not need to be established, but they still exist as nodes within the communication domain; each node is assigned a rank number. This allows any node within the communication domain to act as a sender during subsequent communication, determining the relationship with any other node and thus the data transmission method. Node 3, being on the same SIM card as the master, does not need to create a communication domain again. Once all client nodes (whether via IPC or NCCL) have completed the preparations for communication domain creation, they send a completion command indicating successful local communication domain creation. Afterward, the master node executes steps S510 and S512, waiting for creation requests and creating the communication domain.

[0188] In the TensorLink communication framework, the Master node acts as the coordinator for communication domain creation. After completing system initialization, it enters a listening state, waiting to receive information retrieval requests from various clients. Each client, upon initialization, sends a GetCommInfoRequest to the Master node to obtain necessary information for creating the communication domain, such as rank, worldsize, and command. Upon receiving these requests, the Master node tracks the request status of all expected clients using an internal counter. Only when the last request arrives does it trigger the actual creation of the communication domain. This is because ensuring all clients are ready and requesting to join the communication domain is a prerequisite for successful domain creation and effective collaboration. It's worth noting that during communication domain creation, if there is intra-card transmission (i.e., multiple logical GPUs reside on the same physical GPU card), data transmission between these intra-card nodes will be achieved through IPC mechanisms, eliminating the need to create an NCCL communication domain. This is because NCCL is primarily designed for inter-GPU card communication, while intra-card transmission relies more on low-latency memory sharing mechanisms such as IPC. Therefore, in communication domains where same-card transmission exists, the NCCL communication domain creation steps only apply to nodes communicating with different cards, while nodes communicating with the same card communicate directly through IPC.

[0189] Figure 6 This is a schematic diagram of an optional communication framework architecture according to an embodiment of this application; as shown... Figure 6 As shown, this represents the TensorLink communication framework architecture. In the fields of deep learning and high-performance computing, "tensor" typically refers to a multidimensional array, which is the primary representation of data and model parameters. "Link" signifies connection or link, referring here to the communication link established by TensorLink between different GPU devices. Whether communicating within the same GPU (sharing resources on the same GPU card) or across multiple GPUs or servers, TensorLink provides a stable and high-speed data transmission path. During use, steps S602-S608 are executed: determining the configuration file; initialization; creating the communication domain; and data transmission.

[0190] Among them, the General Remote Procedure Call (gRPC) communication library is a high-performance, open-source framework used to implement remote communication between different applications. Through gRPC, clients can invoke methods on remote servers as if they were local methods, providing efficient data transfer and serialization support.

[0191] Server-side wrapper layer: NCCL (NVIDIA Collective Communication Library) server communication wrapper. This is actually a server-side wrapper layer for the NCCL library. Its goal is to translate the C++ interface of NCCL into a Python interface, enabling applications in a Python environment to use NCCL for data communication between GPUs. This wrapper helps improve the compatibility and flexibility of deep learning frameworks.

[0192] Internal Process Communication Encapsulation: TensorLink's internal process communication (IPC) encapsulation, based on PyTorch's C++ extension mechanism, encapsulates the CUDA IPC interface, enabling direct calls to CUDA IPC functions in Python for data sharing and communication between different processes on the same GPU card.

[0193] Collective Communication Library: The NCCL communication library is a set of libraries specifically designed for efficient data communication between GPU devices. It supports a variety of collective communication operations, such as broadcast, reduction, and AllReduce, and is particularly suitable for data synchronization and exchange in multi-GPU parallel computing scenarios.

[0194] Internal process communication: CUDA IPC is a mechanism provided by the CUDA framework that allows different processes running on the same GPU card to share video memory resources, thereby enabling fast data exchange and reducing the overhead of data copying.

[0195] like Figure 6 As shown, the TensorLink communication framework can be divided into three main parts:

[0196] 1. Configuration File and Parsing: This part is responsible for reading and parsing communication configuration information, such as the name of the communication domain, the ID and logical number of the member device, and the communication mode (IPC or NCCL), which are used for the initialization and construction of communication later.

[0197] 2. TensorLink Interface Layer: Provides a unified set of communication interfaces, allowing users to send and receive data without needing to understand the underlying communication details. This layer intelligently determines the communication type (same card or different card) and calls the corresponding encapsulation layer for data transmission.

[0198] 3. TensorLink Encapsulation Layer: Includes NCCL Server communication encapsulation and TensorLink IPC communication encapsulation. These are designed for different card communication scenarios and same card communication scenarios, respectively. They encapsulate the NCCL and CUDA IPC interfaces, enabling TensorLink to manage and use these underlying communication functions in a unified manner.

[0199] like Figure 6 As shown, step S602 requires determining the configuration file. The communication domain configuration file is a core component of the TensorLink framework. It is defined in JSON format and used to describe the attributes and node information of the communication domain, facilitating framework initialization and management. The following are the key elements of the configuration file design:

[0200] Communication domain definition: In the configuration file, each item represents a communication domain, such as comm1 and comm2. Each communication domain has its own master node and a series of slave nodes (devices) information.

[0201] Master node: The master node is responsible for initializing the communication domain, allocating communication parameters, and managing the communication process within the domain. In the configuration file, the master node is identified by the master field, which usually includes its network address, such as "172.20.0.22:30001".

[0202] Device Information (devices): In each communication domain, the devices field contains information about all GPU devices participating in the communication. Each piece of device information includes: addr: The network address of the device, used to identify the device's location within the network. gpu: The GPU ID on the physical node to which the device belongs, used to specify the GPU for communication in a multi-GPU environment. logic_gpu: The logical ID of the device in the container environment, used to map GPU resources in containerized deployment scenarios.

[0203] The configuration file also includes a mechanism for automatically identifying transfers between the same SIM card and transfers between different SIM cards:

[0204] Same-card transfer: When two devices have the same IP address, different port numbers, and the same GPU number, they are identified as having same-card transfer. This means that data exchange will take place within the same GPU card, utilizing low-latency, high-bandwidth communication methods such as CUDA IPC.

[0205] Inter-GPU transfer: When two devices have different IP addresses, or even if they have the same IP address but different GPU IDs, it is considered an inter-GPU transfer. In this case, data exchange will cross different GPU cards or physical machines, typically requiring the use of high-performance communication libraries such as NCCL.

[0206] Example 6:

[0207] The following describes the scenarios in which the above-mentioned solution of this application can be applied:

[0208] In large-scale natural language processing tasks, the BERT model is widely used due to its superior performance. Faced with a massive number of model parameters, traditional single-GPU training struggles to meet performance requirements, making distributed training an inevitable choice. Using the communication library of this invention, cross-GPU synchronization of model parameters, gradients, and activation values ​​is effectively handled in a multi-GPU configuration, ensuring that each GPU node can update global information in a timely manner and achieving efficient parallel computation.

[0209] Generative pre-trained models (GPT) involve training massive parameter matrices and complex computational graphs. This invention enables seamless synchronization of all GPT model parameters in a multi-machine, multi-GPU environment, leveraging the combined advantages of NCCL and IPC to achieve rapid propagation and synchronization of model parameters throughout the training cluster.

[0210] In large-scale neural network training, tensor parallelism and data parallelism strategies are used simultaneously to maximize model parallelism and data throughput. Based on this invention, there is no need to manually distinguish the communication details between the same and different GPUs. The communication library automatically identifies and adopts the optimal communication strategy for parameter synchronization, providing a unified and simplified interface for parallel model computing, reducing configuration complexity and improving R&D efficiency.

[0211] In real-time inference systems, such as the perception module of autonomous vehicles, neural networks with different functions need to be deployed on multiple GPUs to process complex visual and perceptual information. The communication library of this invention plays a crucial role in this scenario, facilitating the efficient transfer of intermediate inference results between different GPUs, reducing latency, and improving the overall system's inference throughput and response speed.

[0212] In tasks such as video encoding and image segmentation, multiple GPUs each undertake different stages of work, such as preprocessing, feature extraction, and image synthesis. Using the communication library of this invention, low-latency data frame transmission can be achieved between GPU nodes, ensuring rapid data flow at each stage of the video processing pipeline and improving the real-time performance and smoothness of video processing.

[0213] In graph neural networks, the model needs to frequently exchange boundary node information between subgraphs distributed across different GPUs to achieve global connectivity. By integrating IPC and NCCL communication mechanisms, this invention can accelerate the synchronization process of node embedding and reduce cross-GPU communication latency. Especially in heterogeneous computing architectures, it significantly improves the training speed and model convergence rate of GNNs.

[0214] In financial risk analysis, Monte Carlo simulations require significant computational resources and are typically run in parallel on multiple GPUs to accelerate the simulation. This invention's communication library provides a high-performance inter-GPU data synchronization channel, enabling GPU nodes to exchange boundary data and simulation results at high speed in Monte Carlo simulation scenarios, effectively improving the accuracy and efficiency of the simulation.

[0215] In computational fluid dynamics (CFD) and molecular dynamics simulations, each GPU processes a local region or a subset of particles, requiring frequent exchange of boundary conditions and computational results with other GPUs. This invention utilizes the efficient data transmission capabilities of its communication library to achieve GPU-to-GPU boundary information synchronization, avoiding data transmission bottlenecks that limit computational performance. This is particularly significant in large-scale parallel computing environments, substantially improving simulation speed and computational efficiency.

[0216] In an optional implementation, a heartbeat-based resource discovery service is developed to periodically monitor the status of all nodes in the network, including GPU utilization, memory usage, and network bandwidth, to achieve real-time resource monitoring. Based on the results of dynamic resource discovery, machine learning algorithms are used to predict future resource demands, thereby intelligently allocating communication resources, optimizing communication paths, and reducing cross-node data transmission latency. A network flow control algorithm is designed to dynamically adjust the communication traffic between nodes, avoiding single-point overload and improving overall communication efficiency.

[0217] Employing an adaptive routing algorithm, the framework automatically finds alternative paths to maintain uninterrupted communication when a communication path failure is detected. When the topology changes, the framework automatically updates the communication domain configuration without requiring a system restart, enhancing its adaptability and flexibility.

[0218] During data transmission, communication errors are detected in real time, and a retransmission mechanism is immediately initiated without manual intervention, thus improving communication reliability. A fault recovery process is designed so that when a node failure is detected, its communication responsibilities can be quickly transferred to a backup node or reassigned, reducing recovery time and improving system availability.

[0219] Building upon the existing identification of same-card and different-card transmission, finer-grained identification of inter-node communication types has been added, such as local communication, cross-node communication, and cross-host communication, providing support for more complex communication scenarios. The intelligent scheduling algorithm is deeply integrated with the TensorLink communication framework to implement dynamic scheduling strategies based on communication type, network conditions, and resource utilization, improving overall communication efficiency. The heartbeat monitoring function of the TensorLink framework has been enhanced, periodically checking node status; once an anomaly is detected, a fault detection process is immediately initiated, automatically isolating faulty nodes. Stronger data verification and integrity checks have been added during data transmission to ensure that transmitted data remains intact and error-free even under retransmission mechanisms.

[0220] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0221] Embodiments of this application also provide a node communication device. Figure 7 This is a structural block diagram of an optional node communication device according to an embodiment of this application, such as... Figure 7 As shown, the device includes:

[0222] The node information determination module 702 is used to determine the first node information of the receiving end based on the communication domain information, wherein the first node information is used to indicate the node attribute to which the receiving end belongs, and the communication domain includes the receiving end and the sending end;

[0223] The node relationship determination module 704 is used to compare the first node information and the second node information to determine the target node relationship. The second node information is used to indicate the node attribute to which the sending end belongs, and the target node relationship is used to indicate whether the sending end and the receiving end are the same physical card.

[0224] The data transmission module 706 is used to determine the target communication mode based on the target node relationship and to send the target data to the receiving end based on the target communication mode.

[0225] Optionally, the node relationship determination module 704 is further configured to: obtain the first node number, first physical number, and first mapping number of the receiving end from the first node information, wherein the first node number is used to indicate the node and port to which the receiving end belongs, the first physical number is used to indicate the serial number of the physical card on the node to which the receiving end belongs, and the first mapping number indicates the logical number to which the receiving end belongs; obtain the second node number, second physical number, and second mapping number of the sending end from the second node information, wherein the second node number is used to indicate the node and port to which the receiving end belongs, the second physical number is used to indicate the serial number of the physical card on the node to which the receiving end belongs, and the second mapping number indicates the logical number to which the receiving end belongs; and compare the first node number, first physical number, first mapping number, and second node number, second physical number, and second mapping number respectively to determine the target node relationship.

[0226] Optionally, the node relationship determination module 704 is further configured to: determine the target node relationship as a different card relationship when the first node number and the second node number are different; determine the target node relationship as a different card relationship when the first node number and the second node number are the same but the first physical number and the second physical number are different; and determine the target node relationship as a same card relationship when the nodes indicated by the first node number and the second node number are the same but the ports are different, and the first physical number and the second physical number are the same.

[0227] Optionally, the data sending module 706 described above is further configured to: adjust the target data according to a preset format to obtain metadata to be sent, and send the metadata to the receiving end; determine the sending resource according to the target communication mode, and generate the sending resource, wherein the sending resource is used to send the target data to the receiving end; and send the target data to the receiving end based on the sending resource.

[0228] Optionally, the data sending module 706 is further configured to: determine the data length of the target data and the first communication sequence number when the target communication mode is a heterogeneous communication mode, wherein the first communication sequence number is used to indicate the communication sequence number of the sending end in the communication domain; and determine the second communication sequence number corresponding to each of the at least one node included in the communication domain.

[0229] Optionally, the data sending module 706 described above is further configured to: call the data broadcast interface and send the target data to at least one node included in the communication domain; and stop accessing the communication domain after sending to at least one node has been completed.

[0230] Optionally, the data sending module 706 is further configured to: traverse at least one node according to the communication sequence number corresponding to each of the at least one node, and send the target data to at least one node; and access the target data if it is determined that at least one node has received the target data.

[0231] Optionally, the data sending module 706 is further configured to: determine the video memory address of the target data when the target communication mode is the same card communication mode; and create a shared memory handle based on the video memory address.

[0232] Optionally, the data sending module 706 described above is further configured to: send mode determination information to the receiving end, wherein the mode determination information is used to notify the receiving end that this communication is a same-card communication; and send a shared memory handle to the receiving end.

[0233] Optionally, the data sending module 706 described above is further configured to: upon receiving a reference instruction, obtain a first handle from the reference instruction, wherein the first handle is used to indicate the reference video memory storing reference data; read reference data from the reference video memory; and close the first handle.

[0234] Optionally, the data transmission module 706 described above is further configured to: send the first check code corresponding to the target data to the receiving end; retransmit the target data upon receiving a retransmission instruction; determine the third check code based on the reference data upon receiving reference data and the second check code; and send a retransmission instruction to the reference node if the second check code and the third check code are different.

[0235] Optionally, the data sending module 706 is further configured to: increment the sending count by one when sending target data; adjust the first indicator bit to the update state when the sending count is greater than the preset count, and send the target data and the first indicator bit to the receiving end, wherein the update state is used to instruct the receiving end to update the parameters.

[0236] Optionally, the node information determination module 702 is further configured to: increment the node count value by one when an information acquisition instruction is received; send first creation information to at least one candidate node when the node count value reaches a preset threshold, wherein the candidate node is the node that sent the information acquisition instruction, and the first creation information is used to indicate the information of the communication domain; and determine that the communication domain creation is completed after receiving a completion instruction from at least one candidate node.

[0237] Optionally, the node information determination module 702 is further configured to: send the domain number of the communication domain to at least one candidate node; send the communication number corresponding to each of the at least one candidate node to at least one candidate node; send the number of nodes in the communication domain to at least one candidate node; and send second node information to at least one candidate node.

[0238] Optionally, the node information determination module 702 is further configured to: upon receiving the second creation information, determine the third node information based on the second creation information, wherein the third node information is used to indicate the node information of the master node of the reference communication domain; create the reference communication domain based on the second node information and the third node information, and send a completion instruction to the master node.

[0239] Optionally, the node information determination module 702 is further configured to: determine the reference node relationship based on the third node information and the second node information; and create a reference communication domain when the reference node relationship is a different card.

[0240] For a description of the features in the embodiments corresponding to the node communication device, please refer to the relevant descriptions in the embodiments corresponding to the node communication method, which will not be repeated here.

[0241] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above-described node communication method embodiments.

[0242] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described node communication method embodiments at runtime.

[0243] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0244] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described node communication method embodiments.

[0245] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described node communication method embodiments.

[0246] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0247] The foregoing has provided a detailed description of a node communication method, apparatus, storage medium, and electronic device provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A node communication method, comprising: determining first node information of a receiving end based on communication domain information, wherein the first node information is used to indicate a node attribute to which the receiving end belongs, and the communication domain includes the receiving end and a sending end; comparing the first node information and second node information to determine a target node relationship, wherein the second node information is used to indicate a node attribute to which the sending end belongs, and the target node relationship is used to indicate whether the sending end and the receiving end are the same physical card; determining a target communication mode according to the target node relationship, and sending target data to the receiving end based on the target communication mode. In the step of comparing the first node information and second node information to determine a target node relationship, the following steps are included: obtaining a first node number, a first physical number and a first mapping number of the receiving end from the first node information, wherein the first node number is used to indicate a node and a port to which the receiving end belongs, the first physical number is used to indicate a sequence number of a physical card on the node to which the receiving end belongs, and the first mapping number is used to indicate a logical number to which the receiving end belongs; obtaining a second node number, a second physical number and a second mapping number of the sending end from the second node information, wherein the second node number is used to indicate a node and a port to which the receiving end belongs, the second physical number is used to indicate a sequence number of a physical card on the node to which the receiving end belongs, and the second mapping number is used to indicate a logical number to which the receiving end belongs; comparing the first node number, the first physical number, the first mapping number, the second node number, the second physical number and the second mapping number respectively to determine the target node relationship.

2. The method of claim 1, wherein in the step of comparing the first node number, the first physical number, the first mapping number, the second node number, the second physical number and the second mapping number respectively to determine the target node relationship, the following steps are included: in a case where the first node number is different from the second node number, determining that the target node relationship is a different card relationship; in a case where the first node number and the second node number are the same but the first physical number and the second physical number are different, determining that the target node relationship is a different card relationship; in a case where the first node number and the second node number indicate the same node but different ports, and the first physical number and the second physical number are the same, determining that the target node relationship is a same card relationship.

3. The method of claim 1, wherein in the step of sending target data to the receiving end based on the target communication mode, the following steps are included: adjusting the target data according to a preset format to obtain metadata to be sent, and sending the metadata to the receiving end; determining a sending resource according to the target communication mode, and generating the sending resource, wherein the sending resource is used to send the target data to the receiving end; sending the target data to the receiving end based on the sending resource.

4. The method of claim 3, wherein in the step of determining a sending resource according to the target communication mode, and generating the sending resource, the following steps are included: ​ In a case that the target communication mode is a different card communication mode, determine a data length of the target data and a first communication sequence number, wherein the first communication sequence number is used to indicate a communication sequence number of the sending end in a communication domain; Determine a second communication sequence number corresponding to each of at least one node included in the communication domain.

5. The method of claim 4, wherein: the sending the target data to the receiving end based on the sending resource comprises: calling a data broadcast interface, and sending the target data to the at least one node included in the communication domain; in a case that the target data is sent to the at least one node, stop accessing the communication domain.

6. The method of claim 5, wherein: the sending the target data to the at least one node included in the communication domain comprises: traversing the at least one node according to a communication sequence number corresponding to each of the at least one node, and sending the target data to the at least one node; in a case that the target data is received by the at least one node, access the target data.

7. The method of claim 3, wherein: the determining a sending resource according to the target communication mode and generating the sending resource comprises: in a case that the target communication mode is a same card communication mode, determining a video memory address of the target data; creating a shared memory handle according to the video memory address.

8. The method of claim 7, wherein: the sending the target data to the receiving end based on the sending resource comprises: sending mode determination information to the receiving end, wherein the mode determination information is used to inform the receiving end that this communication is a same card communication; sending the shared memory handle to the receiving end.

9. The method of claim 8, wherein: after the sending the shared memory handle to the receiving end, comprises: in a case that a reference instruction is received, obtaining a first handle from the reference instruction, wherein the first handle is used to indicate a reference video memory storing reference data; reading the reference data from the reference video memory, and closing the first handle.

10. The method of claim 3, wherein: the sending the metadata to the receiving end comprises: sending a first check code corresponding to the target data to the receiving end; in a case that a retransmission instruction is received, retransmit the target data; after the sending the target data to the receiving end based on the sending resource, comprises: in a case that reference data and a second check code are received, determining a third check code according to the reference data; in a case that the second check code is different from the third check code, sending a retransmission instruction to a reference node.

11. The method of claim 10, wherein: the retransmitting the target data comprises: in a case that the target data is sent, increasing a sending number by one. In a case where the sending times are greater than a preset number of times, the first indication bit is adjusted to an update state, and the target data and the first indication bit are sent to the receiving end, where the update state is used to instruct the receiving end to perform parameter updating.

12. The method of any one of claims 1-11, characterized in that, Before the first node information of the receiving end is determined based on the communication domain information, the method comprises: In a case where an information acquisition instruction is received, the node count value is increased by one; In a case where the node count value reaches a preset threshold, first creation information is sent to at least one candidate node, where the candidate node is a node that sends the information acquisition instruction, and the first creation information is used to instruct information of the communication domain; After a completion instruction of the at least one candidate node is received, it is determined that the communication domain is created.

13. The method of claim 12, characterized in that, The first creation information sent to the at least one candidate node comprises at least one of the following: The domain serial number of the communication domain is sent to the at least one candidate node; The respective communication serial numbers of the at least one candidate node are sent to the at least one candidate node; The number of nodes of the communication domain is sent to the at least one candidate node; The second node information is sent to the at least one candidate node.

14. The method of claim 12, characterized in that, After the first creation information is sent to the at least one candidate node, the method comprises: In a case where second creation information is received, third node information is determined according to the second creation information, where the third node information is used to instruct node information of a master node of a reference communication domain; The reference communication domain is created according to the second node information and the third node information, and a completion instruction is sent to the master node.

15. The method of claim 14, characterized in that, The reference communication domain is created according to the second node information, which comprises: Reference node relationships are determined according to the third node information and the second node information; In a case where the reference node relationships are different cards, the reference communication domain is created.

16. A node communication apparatus, characterized in that, comprises: a node information determination module configured to determine first node information of a receiving end based on communication domain information, where the first node information is used to instruct a node attribute to which the receiving end belongs, and the communication domain comprises the receiving end and a sending end; a node relationship determination module configured to compare the first node information and second node information to determine a target node relationship, where the second node information is used to instruct a node attribute to which the sending end belongs, and the target node relationship is used to instruct whether the sending end and the receiving end are the same physical card; a data sending module configured to determine a target communication mode according to the target node relationship, and send target data to the receiving end based on the target communication mode. The node relationship determination module is further configured to: acquire a first node number, a first physical number and a first mapping number of a receiving end from the first node information, wherein the first node number is used to indicate a node and a port to which the receiving end belongs, the first physical number is used to indicate a serial number of a physical card on the node to which the receiving end belongs, and the first mapping number is used to indicate a logical number to which the receiving end belongs; acquire a second node number, a second physical number and a second mapping number of a sending end from the second node information, wherein the second node number is used to indicate a node and a port to which the receiving end belongs, the second physical number is used to indicate a serial number of a physical card on the node to which the receiving end belongs, and the second mapping number is used to indicate a logical number to which the receiving end belongs; and compare the first node number, the first physical number, the first mapping number, the second node number, the second physical number and the second mapping number respectively to determine the target node relationship. 17.An electronic device, comprising: a memory configured to store a computer program; a processor configured to execute the computer program to implement the steps of the node communication method according to any one of claims 1 to 15. 18.A computer readable storage medium, comprising: a computer program stored in the computer readable storage medium, wherein the computer program is executed by a processor to implement the steps of the node communication method according to any one of claims 1 to 15. 19.A computer program product, comprising computer instructions, wherein the computer instructions are executed by a processor to implement the steps of the node communication method according to any one of claims 1 to 15. ​ ​

Images