Method, apparatus, device, medium and program product for data transmission

Abstract

The embodiment of the invention provides a method, device and equipment for data transmission, a medium and a program product. The method comprises: obtaining link information of a plurality of links associated with a plurality of network cards of a network device, the link information indicating a connection relationship of each link in the plurality of links, the plurality of links comprise at least one internal link among a plurality of network cards of a network device and / or at least one external link between a plurality of network cards of the network device and a network card of another network device; determining one or more target links from the plurality of links based on the link information; and causing data to be transmitted of computing resources associated with the network device to be transmitted via the one or more target links.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to methods, apparatus, devices, computer-readable storage media, and computer program products for data transmission. Background Technology

[0002] With the rapid development of computer network technology, the demand for network fault tolerance is increasing. In particular, the development of machine learning technology has greatly increased the scale of networks, making the reliability of network links even more crucial. It is desirable to ensure the effectiveness and reliability of data transmission when using network links. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for data transmission is provided. The method includes: acquiring link information of a plurality of links associated with a plurality of network interface cards (NICs) of a network device, the link information indicating the connection relationship of each of the plurality of links, the plurality of links including at least one internal link between the plurality of NICs of the network device and / or at least one external link between the plurality of NICs of the network device and a NIC of another network device; determining one or more target links from the plurality of links based on the link information; and transmitting data to be transmitted from computing resources associated with the network device via the one or more target links.

[0004] In a second aspect of this disclosure, an apparatus for data transmission is provided. The apparatus includes: an acquisition module configured to acquire link information of a plurality of links associated with a plurality of network interface cards (NICs) of a network device, the link information indicating the connection relationship of each of the plurality of links, the plurality of links including at least one internal link between the plurality of NICs of the network device and / or at least one external link between the plurality of NICs of the network device and a NIC of another network device; a determination module configured to determine one or more target links from the plurality of links based on the link information; and a transmission module configured to transmit data to be transmitted from computing resources associated with the network device via the one or more target links.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. When executed by the at least one processor, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The medium stores computer-executable instructions that, when executed by a processor, implement the method of the first aspect.

[0007] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product is tangibly stored in a computer storage medium and includes computer-executable instructions that, when executed by a device, cause the device to perform the method of the first aspect.

[0008] It should be understood that the description in this section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0011] Figure 2 A schematic diagram of a process for data transmission according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A schematic diagram of an example structure of multiple internal links associated with multiple network interface cards (NICs) according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A schematic diagram of an example structure of multiple links associated with multiple network interface cards (NICs) according to some embodiments of the present disclosure is shown;

[0014] Figure 5 A schematic diagram of an example process for link detection according to some embodiments of the present disclosure is shown;

[0015] Figure 6 A schematic diagram of an example structure for transmitting data using an intralink of multiple targets according to some embodiments of the present disclosure is shown;

[0016] Figure 7 A flowchart illustrating an example process for data transmission according to some embodiments of this disclosure is shown;

[0017] Figure 8 A schematic diagram of an example architecture for data transmission according to some embodiments of the present disclosure is shown;

[0018] Figure 9 A schematic structural block diagram of an apparatus for data transmission according to some embodiments of the present disclosure is shown; and

[0019] Figure 10An electronic device in which one or more embodiments of the present disclosure may be implemented is shown. Detailed Implementation

[0020] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0021] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0022] In this document, unless explicitly stated otherwise, performing a step in response to A does not mean that the step is performed immediately after A, but may include one or more intermediate steps.

[0023] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0024] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0025] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information, thereby enabling the user to choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers or storage media that perform the operation of the technical solution disclosed herein, based on the prompt message.

[0026] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, such as a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0027] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0028] As used in this paper, the term "model" refers to a model that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs using multiple layers of processing units. A neural network model is an example of a deep learning-based model. In this paper, "model" may also be referred to as a "machine learning model," "learning model," "machine learning network," or "learning network," and these terms are used interchangeably.

[0029] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. Environment 100 includes multiple network devices, such as network device 110 and network device 120. As an example, network device 110 and network device 120 can at least be used to provide services for training tasks of machine learning models. In some embodiments, network device 110 may include a server. It should be understood that network device 110 is not limited to a server and may be a device in the network capable of performing training tasks of machine learning models or providing any other suitable services.

[0030] In some embodiments, network device 110 and network device 120 may each include at least one network interface card (NIC) and computing resources. As an example, the at least one NIC may include a general-purpose NIC, a dedicated NIC, or other types of NICs. As an example, computing resources may include, but are not limited to, processors such as GPUs and CPUs. It should be understood that computing resources can be any suitable resources, such as devices or equipment, in a computing device capable of processing computing tasks.

[0031] In some embodiments, a link is a path for data transmission, and a link may involve several network interface cards (NICs) and computing resources, etc. The NICs and computing resources on a link can be considered associated, transmitting data between them and thus forming the corresponding link. For example... Figure 1 As shown, in network device 110, network interface cards (NICs) 112-1, 112-2, ..., 112-N can be associated with computing resource 115, thereby forming multiple links within network device 110. In network device 120, NICs 122-1, 122-2, ..., 122-N can be associated with computing resource 125, thereby forming multiple links within network device 120.

[0032] Furthermore, network devices 110 and 120 can also transmit data to each other, for example, through their respective network interface cards (NICs), thereby forming external links for network devices 110 and 120. It should be noted that... Figure 1 The computing resources 115 and 125 shown may each include one or more computing resources.

[0033] Network devices 110 and / or 120 may include, for example, computing systems / servers, such as mainframes, edge computing nodes, computing devices in cloud environments, etc. Server-side devices may be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Server-side devices may include, for example, computing systems / servers, such as mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0034] It should be understood that the structure and function of the various elements in environment 100 and the number of elements are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0035] Currently, the trend in distributed machine learning model training is to use more graphics processing units (GPUs) to train larger-scale neural network models to solve more complex problems. In the past, the size of distributed training clusters has grown from a few hundred GPUs to tens of thousands. With the increase in the number of GPUs involved in training, distributed training services also require more network interface cards (NICs), switches, and active optical cables (AOCs) to form network links. This increase in network size also leads to a higher probability of network component failures. While some network tiers (e.g., tier 2 and tier 3) typically have sufficient redundant links to handle network failures, the link between the top-of-rack switch (TOR) and the NIC is usually a single point of failure. These single-point network component failures have a significant impact on distributed training services, often causing the entire distributed training task to be interrupted or even fail. This can result in a large number (e.g., hundreds) of network devices (e.g., servers) used for machine learning model training becoming unavailable during the failure, leading to huge cost waste.

[0036] To minimize the disruption or failure of machine learning model training tasks due to various reasons, several methods have been developed to address single-point network component failures and reduce losses.

[0037] One way to handle failures is by using a checkpointing mechanism. Specifically, the complete state of the model can be saved periodically. This allows training to resume from the saved checkpoint if training fails, avoiding the wasted training time of starting from scratch every time training fails. However, checkpointing puts pressure on storage; saving checkpoints too frequently not only increases storage costs but also impacts training performance. Therefore, checkpoints are usually saved infrequently, with some applications saving them on an hourly basis. In this case, if a training task fails, it could result in a large number of servers each losing a period of training time (e.g., tens of minutes, or even hours).

[0038] Another approach to handling faults is to use pre-training fault detection. Specifically, this method performs fault detection on the machine before training, removing abnormal nodes in advance. However, this method cannot address network component failures during training.

[0039] Another way to handle failures is by using a heartbeat mechanism, the key to which is timely fault detection. Machine learning model training tasks are typically synchronous. In this mode, distributed training tasks have many synchronous operation points, and each node needs to perform the same operation, waiting for each node's operation to complete before moving on to the next synchronous operation point. The default synchronization wait time is usually several minutes or even tens of minutes, which means that detecting a single point of failure in the network at the training task level can take several minutes or even tens of minutes.

[0040] In addition, another approach to handling faults involves dual-plane network components, which uses a dual-port, dual-uplink network solution. However, this network architecture requires dual-port network cards, making it hardware-dependent, costly to implement, and less flexible.

[0041] In view of this, embodiments of the present disclosure provide a scheme for data transmission. Specifically, in embodiments of the present disclosure, link information of multiple links associated with multiple network interface cards (NICs) of a network device is obtained, and one or more target links are determined from the multiple links based on the link information. The link information indicates the connection relationship of each of the multiple links, and the multiple links include at least one internal link between the multiple NICs of the network device and / or at least one external link between the multiple NICs of the network device and the NIC of another network device. Furthermore, the data to be transmitted from the computing resources associated with the network device is transmitted via one or more target links.

[0042] In this manner, embodiments of this disclosure can ensure normal data transmission through rapid link switching when one or more internal and / or external links of a network device malfunction. This solution enhances the fault tolerance of network components, enabling tasks such as training machine learning models to continue even when network hardware failures occur. This improves the network's fault tolerance, thereby enhancing its reliability and availability.

[0043] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0044] Figure 2 A schematic diagram of a data transmission process 200 according to some embodiments of the present disclosure is shown. For ease of discussion, reference will be made to... Figure 1 These embodiments are described in the context of environment 100. These embodiments can be implemented in... Figure 1 The embodiments are implemented at network device 110 or network device 120. The following discussion of embodiments of this disclosure will use network device 110 as an example. It should be understood that this is merely exemplary and is not intended to limit the embodiments of this disclosure in any way.

[0045] In box 210, network device 110 acquires link information for multiple links associated with its multiple network interface cards (NICs). The link information indicates the connectivity of each of the multiple links.

[0046] As an example, network device 110 can at least be used to provide services for training machine learning models. In some embodiments, network device 110 may include a server. Accordingly, the server may be configured to perform training tasks for machine learning models. It should be understood that network device 110 is not limited to a server and may be any device in the network capable of performing training tasks for machine learning models or providing any other suitable services.

[0047] In some embodiments, the network interface cards (NICs) of network device 110 may be, for example, multiple general-purpose NICs, multiple remote-controlled NICs (RNICs), or multiple NICs of other types. As an example, the multiple NICs of network device 110 may be of the same type. Of course, even if the multiple NICs are not entirely the same type, but they require the same function in the network, they can still be used to implement embodiments of this disclosure. Multiple links can be formed between the multiple NICs.

[0048] Here, multiple links include at least one internal link between multiple network interface cards (NICs) of network device 110 and / or at least one external link between multiple NICs of network device 110 and a NIC of another network device. That is, for multiple NICs located within the same network device (e.g., network device 110), at least one internal link can be formed between them within network device 110. Conversely, multiple NICs located within network device 110 can form at least one external link with one or more NICs located in another network device (e.g., network device 120). As an example, a single link among the multiple links can be a link between two NICs. Accordingly, if two NICs are located within the same network device, an internal link can be formed between them. If two NICs are located in different network devices, an external link can be formed between them.

[0049] In some embodiments, different internal links can have different priorities. For example, multiple internal links may include a first internal link and at least one second internal link. The priority of the first internal link may be higher than the priority of any second internal link. In some cases, the first internal link may also be referred to as the primary internal link, and the second internal link may also be referred to as the backup internal link. For ease of understanding, the following will combine... Figure 3 Let's discuss in detail the internal links that are associated with multiple network interface cards (NICs) in a network device.

[0050] Figure 3 A schematic diagram of an example structure 300 of multiple internal links associated with multiple network interface cards (NICs) according to some embodiments of the present disclosure is shown. Structure 300 can be implemented in Figure 1 In environment 100, structure 300 is the structure to which some steps of process 200 are applicable.

[0051] like Figure 3 As shown, suppose a network device includes multiple network interface cards (NICs), such as NIC 0, NIC 1, NIC 2, and NIC 3. These NICs can be associated with multiple computing resources (such as computing resource 0, computing resource 1, and other computing resources), thus forming multiple links. For these computing resources, input data can be input through NICs 0 to NIC 3, and output data can also be output through NICs 0 to NIC 3. Therefore, there can be multiple internal links from input to output. These multiple internal links can include a primary internal link (i.e., the first internal link, denoted as 310) and multiple backup internal links (i.e., the second internal links, not shown). These backup internal links can include, for example, links formed by NICs 1 and 2 with multiple computing resources, links formed by NICs 1 and 3 with multiple computing resources, links formed by NICs 2 and 3 with multiple computing resources, and so on.

[0052] As an example, computing resources may include, but are not limited to, processors such as GPUs and CPUs. It should be understood that computing resources can be any suitable resources such as devices or equipment within a computing device capable of handling computing tasks (such as performing training tasks for machine learning models).

[0053] In addition to internal links having different priorities, external links can also have different priorities. In some embodiments, the priority of a first external link (e.g., also referred to as the primary external link) can be higher than the priority of a second external link (e.g., also referred to as a backup external link). For ease of understanding, the following will combine... Figure 4 Let's discuss in detail the external links associated with multiple network interface cards (NICs) in a network device.

[0054] Figure 4 A schematic diagram of an example structure 400 of multiple links associated with multiple network interface cards (NICs) according to some embodiments of the present disclosure is shown. Structure 400 can be implemented in Figure 1 In environment 100, structure 400 is the structure to which some steps of process 200 are applicable.

[0055] like Figure 4 As shown, assume there are two network devices, namely network device 410 and network device 420. Network device 410 and network device 420 may include a primary external link 401 and multiple backup external links. These backup external links may include, for example, the link between network interface card 5-2 of network device 410 and network interface card 8-1 of network device 420, and the link between network interface card 5-2 and network interface card 9-1 of network device 420, etc.

[0056] It should be understood that Figure 4 The structures in network devices 410 and 420 are merely examples, and the connection methods between network devices 410 and 420 are also merely examples. The connection methods and number of each node and network card in network device 410 and network device 420 are also merely examples, and are not intended to impose any limitations.

[0057] Return to reference Figure 2 In box 220, network device 110 determines one or more target links from a plurality of links based on link information. The link information indicates the connectivity of each of the plurality of links. Each link may include at least one network interface card (NIC) and other nodes. Other nodes may include, for example, other devices or equipment on the link, and are not limited thereto.

[0058] The target link can be an internal link or an external link. Furthermore, the target link can be multiple internal links, and data can be transmitted evenly across these internal links. Similarly, the target link can be multiple external links, and data can be transmitted evenly across these external links. The embodiments of this disclosure are not limited in these respects. In other words, the target link can include one or more target internal links from at least one internal link, and / or one or more target external links from at least one external link. The determination of one or more target internal links and one or more target external links will be discussed below with reference to specific embodiments.

[0059] Combination Figure 3 In some embodiments, to determine one or more target links from multiple links, network device 110 can detect whether the first internal link 310 is available based on link information. In other words, it can detect whether the first internal link 310 is abnormal. For example, it can detect whether the network card or other nodes connected to the first internal link 310 are damaged, or whether the first internal link 310 is congested, etc. It should be understood that these are merely examples, and other methods for detecting whether the first internal link 310 is available may exist in actual applications, which are not limited here.

[0060] In some embodiments, if the first internal link 310 is available, the network device 110 can determine that the first internal link 310 is a target internal link among one or more target links. For internal links, the first internal link 310 has the best transmission performance; therefore, if the first internal link 310 is usable, it is considered a target internal link.

[0061] Conversely, if the first internal link 310 is unavailable, based on the link information, network device 110 can determine one or more target internal links from at least one second internal link as part of one or more target links. In other words, in the event of a failure of the primary internal link, a backup internal link can be selected.

[0062] In some embodiments, each of the at least one second internal link may be associated with at least one network interface card (NIC) among a plurality of network interface cards (NICs). To determine one or more target internal links from the at least one second internal link, network device 110 may, based on link information, determine the relative positional relationship between the at least one NIC associated with each second internal link and the associated computing resources, as well as the correlation between each second internal link and a first internal link. The correlation may indicate whether each second internal link has at least one node identical to that of the first internal link.

[0063] As an example, the relative positional relationship between a network interface card (NIC) and its associated computing resources can affect the performance of the corresponding internal link. As an example, the relative positional relationship between at least one NIC associated with each second internal link and its associated computing resources can indicate the distance between at least one NIC and its associated computing resources.

[0064] by Figure 3 For example, in the input section, network interface card 0 is closest to computing resource 0. In the output section, network interface card 0 is closest to computing resource 1. Therefore, network interface card 0 is closest to its associated computing resource (such as...). Figure 3 As shown, the first internal link 310, composed of computing resources 0, 1, and others, is the best-performing primary internal link. The distances between the backup network interface cards (NICs 1, 2, and 3) and the computing resources gradually increase. Therefore, when the first internal link 310 fails, the link formed by NIC 1 and the computing resources is the best backup internal link and can be selected first. Correspondingly, the link formed by NIC 3 and the computing resources can be the last backup internal link to be selected. Assuming that NICs 0, 1, and 2 are all damaged, the link formed by NIC 3 and the computing resources can be considered to ensure the normal operation of the network.

[0065] As an example, if a second internal link has the same nodes as the first internal link, then the second internal link is correlated with the first internal link, which can serve as a principle for selecting backup internal links. Furthermore, the correlation principle can be combined with the principle of relative positional relationships. Figure 4 For example, in network device 410, assuming that the link formed by network interface cards 4-1 and 4-2 with node 0, computing resources 2 and 3 is the main internal link A, after this link fails, the internal link B associated with network interface cards 5-1 and 5-2 can be given priority. This is because internal link B shares the same node 0 as the main internal link A, and the distance between network interface cards 5-1 and 5-2 and the computing resources is closer than the distance between other network interface cards and the computing resources.

[0066] Furthermore, network device 110 can determine one or more target internal links that meet the link requirements from at least one second internal link. Link requirements may include at least one of relative positional relationship requirements and correlation requirements. Therefore, based on the connection relationships of the nodes within the network device in the link information, the relative positional relationships between the nodes can be determined, and it can be determined which backup internal links have the same nodes as the primary internal link, thereby matching high-performance internal links according to the link requirements in the actual application.

[0067] In this way, embodiments of this disclosure can search and sort the links within the network device, thereby selecting other internal links when an internal link fails, ensuring normal data transmission through internal link switching. This effectively improves the fault tolerance capability within the network device, making the use of internal network links more reliable.

[0068] Combination Figure 4 In some embodiments, to determine one or more target links from multiple links, network device 410 can detect whether the first external link 401 is available based on link information. In other words, it can detect whether the first external link 401 is abnormal. For example, it can detect whether the network card or other nodes connected to the first external link 401 are damaged, or whether the first external link 401 is congested, etc. It should be understood that these are merely examples, and other methods for detecting whether the first external link 401 is available may exist in actual applications, which are not limited here.

[0069] In some embodiments, if the first external link 401 is available, the network device 410 may identify the first external link 401 as a target external link among one or more target links. The first external link 401 has the best transmission performance among external links, therefore, if the first external link 401 is usable, it is designated as the target external link.

[0070] Conversely, if the first external link 401 is unavailable, based on the connection relationships of each of the at least one second external link indicated by the link information, the network device 410 can determine one or more target links from at least one second external link. In other words, in the event of a failure of the primary external link, a backup external link can be selected.

[0071] In some embodiments, in order to determine one or more target external links from at least one second external link, network device 110 may detect the link status of each of the at least one second external links based on the connection relationship of each of the at least one second external links indicated by link information. The link status may include at least one of the availability status and load status of each second external link.

[0072] As an example, detecting the availability status of a second external link can include checking if the connected network card is turned off or damaged, or detecting congestion on the second external link, which could render it unavailable. Detecting the load status of the second external link can include checking its transmission latency, load, etc. The following will combine... Figure 5 Let's discuss the detection of external links in detail.

[0073] Figure 5 A schematic diagram of an example process 500 for link state detection according to some embodiments of the present disclosure is shown. Link state detection can also be referred to as global link state awareness. Process 500 can be implemented in... Figure 1 In environment 100, process 500 can be considered an example of process 200 performing external link detection.

[0074] like Figure 5 As shown, the passively acquired network event sensing 510, network card event sensing 520, and actively acquired link quality probe 530 can all be used as inputs to the global state link sensing 540. Then, based on these inputs, the link state table 550 shown in Table 1 below can be output.

[0075] Table 1 Link Status Table

[0076]

[0077]

[0078] In Table 1, for network interface card (NIC) event sensing 520, it can detect, for example, whether the NIC changes from an on to an off state, or detect other state changes of the NIC; there are no restrictions here. For network event sensing 510, it can detect, for example, whether there is data loss during network transmission, whether there is a transmission timeout, or other abnormal events; there are no restrictions here. In network event sensing 510, for example, a five-tuple can be used to identify the network connection (e.g., it can include source IP address, source port, destination IP address, destination port, and protocol type). For link quality probe 530, it can actively probe the availability and load of the link through probe packets. It should be understood that the above methods for detecting the availability and load status of external links are only examples and are not intended to impose any limitations; in actual applications, any other suitable detection methods can be used.

[0079] Furthermore, based on the link states of at least one second external link, network device 110 can determine a second external link that meets the link state requirements from the at least one second external link as one or more target external links. As an example, link state requirements may include at least one of availability state requirements and load state requirements for the second external link. For example, it may be required that the network interface card on the second external link is in normal working order, or that the transmission latency of the second external link meets certain latency requirements, and so on. In practical applications, any suitable link state requirements can be set according to specific needs, and no restrictions are placed here.

[0080] In this way, the embodiments of this disclosure can detect the status of external links to the network device, and select other external links when an external link is abnormal, thereby ensuring normal data transmission through external link switching. This effectively improves network fault tolerance and optimizes the use of network links.

[0081] Return to reference Figure 2 In box 230, network device 110 enables data to be transmitted from its associated computing resources to be transmitted via one or more target links. As an example, transmitting data to be transmitted via one or more target links may include sending or receiving data to be transmitted via one or more target links.

[0082] In some embodiments, if a target internal link is determined from multiple links, all data to be transmitted can be transmitted via that target internal link. If a target external link is determined from multiple links, all data to be transmitted can be transmitted via that target external link. If a target internal link and a target external link are determined from multiple links, the data to be transmitted can be transmitted using the corresponding target internal link or target external link based on internal link transmission requirements or external link transmission requirements.

[0083] In some embodiments, if multiple target links are identified from multiple links, network device 110 can determine the share of data to be transmitted corresponding to each of the multiple target links based on the load status of each target link. Then, network device 110 can transmit the corresponding share of data in the data to be transmitted via the multiple target links.

[0084] For example, assuming multiple target external links are identified from at least one external link, the load of these multiple target external links can be detected, and then the share of data to be allocated to each target external link can be determined, thereby transmitting the corresponding share of data through the multiple target external links. Similarly, assuming multiple target internal links are identified from at least one internal link, the load of these multiple target internal links can be detected, and then the share of data to be allocated to each target internal link can be determined, thereby transmitting the corresponding share of data through the multiple target internal links. The following will combine... Figure 6 Let's take an example to illustrate how to use internal links between multiple targets to transmit data.

[0085] Figure 6 A schematic diagram of an example structure 600 for transmitting data using multiple target intralinks according to some embodiments of the present disclosure is shown. Structure 600 can be implemented in Figure 1 In environment 100, structure 600 can be shown as the structure used for some steps in process 200.

[0086] Structure 600 illustrates multiple channels, such as eight channels (channels 0 to 7), that exist when operating a single node in a Collective Communication Library (CCL). Each channel may correspond to an internal link for transmitting data streams. At the input of each channel, the data stream from the network is received via the network interface card (NIC), processed by multiple computing resources, and then sent back to the network via the NIC. In these eight channels, assuming a failure of NIC 2 causes channels 2 and 6 (which could be the primary or backup internal link associated with NIC 2) to malfunction, the data to be transmitted can be redistributed. For example, the data to be transmitted can be allocated to these six channels based on the load conditions of channels 0, 1, 3, 4, 5, and 7. This avoids simply assigning data from channels 2 and 6 to a single backup channel, which could lead to excessive load on that single channel.

[0087] By detecting the load status of the links, the allocation ratio (i.e., share) of each channel can be adjusted according to the load size when allocating data. In another embodiment, if all channels established by the aggregated communication library pass through the same network interface card (NIC) and all NICs are faulty, the data will no longer be redistributed on the channels, but other computing resources can be used as data relay nodes.

[0088] This implementation method enables load balancing of network links. This allows for faster data transmission and reduced latency.

[0089] In some embodiments, to transmit data via one or more target links, if one or more target links are determined from a plurality of links, network device 110 can detect whether there is data to be retransmitted. Data to be retransmitted includes data that was not successfully transmitted before the one or more target links were determined. If data to be retransmitted exists, network device 110 can transmit the data to be retransmitted via one or more target links. Then, if the data to be retransmitted is transmitted, network device 110 can transmit the data to be transmitted via one or more target links. The following will combine... Figure 7 The process of data retransmission will be discussed in detail with reference to the embodiments discussed above.

[0090] Figure 7 A flowchart of an example process 700 for data transmission according to some embodiments of the present disclosure is shown. Process 700 can be implemented in Figure 1 In environment 100. Process 700 can be shown as a specific embodiment of process 200. Accordingly, process 700 can be implemented in Figure 1The process 700 is implemented at network device 110 or network device 120. The following discussion uses the implementation of process 700 at network device 110 as an example to illustrate embodiments of this disclosure. It should be understood that this is merely exemplary and is not intended to limit the embodiments of this disclosure in any way. It should be noted that process 700 can be applied to data transmission on internal links as well as data transmission on external links.

[0091] In process 700, after the data transmission process is initiated (710), network device 110 can obtain link information (720) and then determine whether the main link (which can be a main internal link or a main external link) is available (730) based on the link information. If the main link is available, the data to be transmitted is transmitted via the main link (740). However, if the main link is unavailable, data packets may fail to be transmitted due to the main link being abnormal (also known as data packet loss). Therefore, a target link (which can be a target internal link or a target external link) needs to be selected (750). After selecting the target link, it is necessary to determine whether there is data to be retransmitted (760). If there is no data to be retransmitted, the data to be transmitted can be transmitted directly via the target link. If there is data to be retransmitted, the data to be retransmitted can be retransmitted via the target link first (770). Then, it can be determined whether the retransmission is complete (780). If it is not complete, the data to be retransmitted is retransmitted via the target link again. If the retransmission is complete, the data to be transmitted can be transmitted via the target link again (790).

[0092] Therefore, reliable data transmission can be guaranteed even in the event of network failure. This enhances the fault tolerance of single-point network components, ensuring uninterrupted tasks when network hardware fails and improving the utilization rate of machine learning model clusters.

[0093] Figure 8 A schematic diagram of an example architecture 800 for data transmission according to some embodiments of the present disclosure is shown. Architecture 800 can be implemented in... Figure 1 In environment 100, architecture 800 can be the architecture used in implementation process 200.

[0094] In architecture 800, process 200 can realize the transmission between the aggregated communication library 810 and network interface card (NIC) hardware (e.g., NIC 820 and dedicated NIC 830), which can be used for the workload 801 used for machine learning model training and the NIC hardware, and can also be used for collaboration between distributed training nodes. Specifically, the function of the single-machine topology awareness module 850 is to identify all possible inbound and outbound single-machine links for each computing resource based on the single-machine topology of the network device, and sort them according to performance. The single-machine topology awareness module 850 can solve the selection of backup links within the network device and select the optimal internal link. Network devices typically contain multiple computing resources and NICs, which may be located under different network nodes. In addition, the relative position of computing resources and NICs has a significant impact on link performance. The single-machine topology awareness module 850 can identify the single-machine topology of the network device and select the optimal internal topology.

[0095] The adaptive connection management module 860 manages the entire lifecycle of connections between network devices. The integrated communication library 810 needs to establish point-to-point connections, through which all subsequent data transmission and reception occurs. During connection creation, the connection management module 860 establishes connections based on the distance between the network interface card (NIC) and computing resources (resources required for the connection are allocated at this stage). The connection established on the NIC closest to the computing resources is the primary connection, offering the best performance. As the distance increases, multiple backup links with progressively decreasing priority are established. The data transmission path is determined based on the link status. Specifically, the data transmission path can be selected based on link availability and load, and the data transmission volume for each path can be dynamically selected based on link availability and load. When a connection needs to be released, this module ensures that there is no uncompleted data transmission before releasing the resources used by the connection.

[0096] The global link awareness module 870 can initiate a real-time link status awareness process when the data transmission process starts, sensing link availability and load to provide a basis for traffic routing decisions. The reliable data transmission module 880 can resolve the issue of incomplete data transmission when a link failure occurs; this module ensures correct and reliable data transmission. The API (Application Programming Interface) abstraction layer 840 can present the same API interface to different collection communication libraries 810 to ensure interface consistency.

[0097] Therefore, the embodiments of this disclosure effectively improve the network's fault tolerance through the search and sorting of backup links by the single-machine topology awareness module 850, the multi-link information acquisition mechanism in the global link status awareness module 870, and the link switching in the connection management module 860 and the data reliable transmission module 880. Furthermore, to address the current limitation of dual-port dual-uplink dual-active architectures, the single-machine topology awareness module 850 can allocate multiple backup network cards to each primary network card, preventing scenarios where even a single backup link fails. The embodiments of this disclosure can automatically locate unavailable network links when network components fail, and then switch the data to be transmitted from the unavailable link to the backup link. This ensures that the training task of the machine learning model is not interrupted. When the network fault is repaired, the availability of the network link can be automatically detected and the data to be transmitted can be switched back from the backup link to the primary link. This scheme enhances the fault tolerance of single-point network components, ensuring that tasks are not interrupted when network hardware fails, and improving the cluster utilization of the machine learning model.

[0098] Figure 9 A schematic structural block diagram of an apparatus 900 for data transmission according to some embodiments of the present disclosure is shown. The apparatus 900 may be implemented as or included in network device 110 or network device 120. Various modules / components in the apparatus 900 may be implemented by hardware, software, firmware, or any combination thereof.

[0099] As shown in the figure, the device 900 includes an acquisition module 910 configured to acquire link information of multiple links associated with multiple network interface cards (NICs) of a network device, the link information indicating the connection relationship of each of the multiple links, the multiple links including at least one internal link between the multiple NICs of the network device and / or at least one external link between the multiple NICs of the network device and the NIC of another network device; a determination module 920 configured to determine one or more target links from the multiple links based on the link information; and a transmission module 930 configured to transmit data to be transmitted from computing resources associated with the network device via one or more target links.

[0100] In some embodiments, at least one internal link includes a first internal link and at least one second internal link, wherein the first internal link has a higher priority than the at least one second internal link, and the determining module 920 is further configured to detect whether the first internal link is available based on link information; in response to the first internal link being available, determine the first internal link as a target internal link among one or more target links; or in response to the first internal link being unavailable, determine one or more target internal links from at least one second internal link as part of one or more target links based on link information.

[0101] In some embodiments, each of the at least one second internal link is associated with at least one network interface card (NIC) among a plurality of NICs, and the apparatus 900 is further configured to determine, based on link information, the relative positional relationship between the at least one NIC associated with each second internal link and the associated computing resources, and the correlation between each second internal link and a first internal link, the correlation indicating whether each second internal link has at least one node identical to the first internal link; and to determine, from the at least one second internal link, one or more target internal links that meet link requirements, the link requirements including at least one of relative positional relationship requirements and correlation requirements.

[0102] In some embodiments, at least one external link includes a first external link and at least one second external link, wherein the first external link has a higher priority than the at least one second external link, and the determining module 920 is further configured to detect whether the first external link is available based on link information; in response to the first external link being available, determine the first external link as a target external link among one or more target links; or in response to the first external link being unavailable, determine one or more target links from at least one second external link based on the connection relationship of each of the at least one second external link indicated by the link information.

[0103] In some embodiments, the apparatus 900 is further configured to detect the link status of each of the at least one second external links based on the connection relationship of each of the at least one second external links indicated by the link information, the link status including at least one of the availability status and load status of each second external link; and to determine, based on the respective link status of the at least one second external link, a second external link that meets the link status requirements as one of one or more target external links.

[0104] In some embodiments, the transmission module 930 is further configured to, in response to determining a plurality of target links from a plurality of links, determine a share of data to be transmitted corresponding to each of the plurality of target links based on the load status of each of the plurality of target links; and transmit the corresponding share of data in the data to be transmitted via the plurality of target links.

[0105] In some embodiments, the transmission module 930 is further configured to detect whether there is data to be retransmitted in response to determining one or more target links from a plurality of links, wherein the data to be retransmitted includes data that was not successfully transmitted before the one or more target links were determined; to transmit the data to be retransmitted via the one or more target links in response to the existence of data to be retransmitted; and to transmit the data to be retransmitted via the one or more target links in response to the completion of the data transmission to be retransmitted.

[0106] In some embodiments, the network device includes a server configured to perform training tasks for machine learning models.

[0107] The modules and / or units included in device 900 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more modules and / or units can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the modules and / or units in device 900 can be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0108] Figure 10 A block diagram is shown illustrating an electronic device 1000 in which one or more embodiments of the present disclosure may be implemented. It should be understood that... Figure 10 The electronic device 1000 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 1 Network device 110 or network device 120 or Figure 9 The device 900 can be implemented as or included in one or more electronic devices 1000.

[0109] like Figure 10 As shown, electronic device 1000 is in the form of a general-purpose computing device. Components of electronic device 1000 may include, but are not limited to, one or more processors 1010, memory 1020, storage device 1030, one or more communication units 1040, one or more input devices 1050, and one or more output devices 1060. Processor 1010 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 1020. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 1000.

[0110] Electronic device 1000 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 1000, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 1020 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 1030 can be a removable or non-removable medium and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within electronic device 1000.

[0111] Electronic device 1000 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 10 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 1020 may include computer program product 1025 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0112] The communication unit 1040 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 1000 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 1000 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0113] Input device 1050 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 1060 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 1000 can also communicate with one or more external devices (not shown) via communication unit 1040 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 1000, or with any device that enables electronic device 1000 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0114] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0115] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0116] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0117] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0118] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0119] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for data transmission, comprising: Obtain link information of multiple links associated with multiple network interface cards (NICs) of a network device, wherein the link information indicates the connection relationship of each of the multiple links, and the multiple links include at least one internal link between the multiple NICs of the network device and / or at least one external link between the multiple NICs of the network device and a NIC of another network device; Based on the link information, one or more target links are determined from the plurality of links; as well as The data to be transmitted from the computing resources associated with the network device is transmitted via the one or more target links.

2. The method of claim 1, wherein the at least one internal link comprises a first internal link and at least one second internal link, wherein the first internal link has a higher priority than the at least one second internal link, and Determining one or more target links from the plurality of links includes: Based on the link information, detect whether the first internal link is available; In response to the availability of the first internal link, the first internal link is determined to be a target internal link among the one or more target links; or In response to the first internal link being unavailable, based on the link information, one or more target internal links are determined from the at least one second internal link as part of the one or more target links.

3. The method of claim 2, wherein each of the at least one second internal link is associated with at least one of the plurality of network interface cards (NICs), and Determining one or more target internal links from the at least one second internal link includes: Based on the link information, the relative positional relationship between at least one network interface card associated with each second internal link and the associated computing resources is determined, as well as the correlation between each second internal link and the first internal link, wherein the correlation indicates whether each second internal link has at least one node that is the same as the first internal link; as well as One or more target internal links that meet the link requirements are determined from the at least one second internal link, the link requirements including at least one of relative positional relationship requirements and correlation requirements.

4. The method of claim 1, wherein the at least one external link comprises a first external link and at least one second external link, wherein the first external link has a higher priority than the at least one second external link, and Determining one or more target links from the plurality of links includes: Based on the link information, detect whether the first external link is available; In response to the availability of the first external link, the first external link is identified as a target external link among the one or more target links; or In response to the first external link being unavailable, the one or more target links are determined from the at least one second external link based on the connection relationship of each of the at least one second external link indicated by the link information.

5. The method of claim 4, wherein determining the one or more target external links from the at least one second external link comprises: Based on the connection relationship of each of the at least one second external links indicated by the link information, the link status of each of the at least one second external links is detected, and the link status includes at least one of the availability status and load status of each second external link; as well as Based on the link status of each of the at least one second external link, a second external link that meets the link status requirements is determined from the at least one second external link and is selected as one of the one or more target external links.

6. The method of claim 1, wherein transmitting the data to be transmitted via the one or more target links comprises: In response to identifying multiple target links from the multiple links, a share of the data to be transmitted to be allocated for each of the multiple target links is determined based on the load status of each of the multiple target links. as well as The corresponding share of the data to be transmitted is transmitted via the multiple target links.

7. The method of claim 1, wherein transmitting the data to be transmitted via the one or more target links comprises: In response to identifying one or more target links from the plurality of links, it is detected whether there is data to be retransmitted, wherein the data to be retransmitted includes data that was not successfully transmitted before the one or more target links were identified; In response to the existence of the data to be retransmitted, the data to be retransmitted is transmitted via the one or more target links; as well as In response to the completion of the data transmission to be retransmitted, the data to be transmitted is transmitted via the one or more target links.

8. The method of claim 1, wherein the network device includes a server configured to perform training tasks for machine learning models.

9. An apparatus for data transmission, comprising: The acquisition module is configured to acquire link information of multiple links associated with multiple network interface cards (NICs) of a network device. The link information indicates the connection relationship of each of the multiple links, and the multiple links include at least one internal link between the multiple NICs of the network device and / or at least one external link between the multiple NICs of the network device and a NIC of another network device. The determination module is configured to determine one or more target links from the plurality of links based on the link information; as well as The transmission module is configured to transmit data to be transmitted from computing resources associated with the network device via the one or more target links.

10. An electronic device, comprising: At least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 8 when executed by the at least one processor.

11. A computer-readable storage medium having stored thereon computer-executable instructions that can be executed by a processor to implement the method according to any one of claims 1 to 8.

12. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 8.

Images