A method and apparatus for performing a communication task in an accelerator card system

Abstract

The present disclosure provides a method for performing a communication task in an accelerator card system, which can be implemented in a computing device, wherein the computing device can be included in a combined processing device, which can further include a general-purpose interconnect interface and other processing devices. The computing device interacts with the other processing devices to collectively complete a user-specified computing operation. The combined processing device can further include a storage device connected to the computing device and the other processing devices, respectively, for data of the computing device and the other processing devices.

Description

Technical Field

[0001] This disclosure relates to the field of computers, and more specifically, to the field of computer communications. Background Technology

[0002] Currently, mainstream frameworks (such as TensorFlow and PyTorch) use only a dedicated communication queue (comm_queue) to execute communication tasks. When the communication library responsible for communication tasks obtains a task, it usually directly places the task into the framework's comm_queue or the library's internal task queue (internal_queue) for execution, such as the NCCL library responsible for inter-GPU communication. Currently, all communication tasks are executed in a single queue, and if an error occurs, the entire communication task needs to be re-executed from the beginning, thus reducing overall communication efficiency. Furthermore, current technologies cannot enable users to perform fault tolerance or retransmission operations for communication tasks without their awareness. Summary of the Invention

[0003] One objective of this disclosure is to overcome the shortcomings of existing technologies, such as the inability to distribute a task to multiple queues for parallel execution, the inability to fully utilize communication or computing resources, and the low fault tolerance.

[0004] According to a first aspect of this disclosure, a method for performing communication tasks in an accelerator card system is provided, wherein the accelerator card system includes a plurality of accelerator cards capable of communicating with each other, and one of the plurality of accelerator cards is capable of communicating with another accelerator card through a communication path; the method includes: establishing a communication task queue, the communication task queue including communication tasks and a status identifier for monitoring the execution status of the communication tasks; establishing a communication task execution queue for executing communication tasks between the accelerator cards through a communication path; and changing the status identifier in response to the execution of the communication tasks to monitor the execution status of the communication tasks.

[0005] According to a second aspect of this disclosure, an electronic device is provided, comprising: one or more processors; and a memory storing computer-executable instructions that, when executed by the one or more processors, cause the electronic device to perform the method described above.

[0006] According to a third aspect of this disclosure, a computer-readable storage medium is provided, including computer-executable instructions that, when executed by one or more processors, perform the method described above.

[0007] At least one beneficial effect of the technical solution disclosed herein is that it distinguishes between communication task queues and communication task execution queues, enabling users to perform fault tolerance or retransmission operations without their awareness. The technical solution disclosed herein can also allocate a total communication task to different sub-communication task queues, thereby accelerating the execution of the total communication task. Furthermore, even if an error occurs in the execution of a certain sub-communication task queue, it is not necessary to re-execute all sub-communication tasks, thus reducing the burden of task execution. Attached Figure Description

[0008] The above and other objects, features, and advantages of exemplary embodiments of the present disclosure will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the present disclosure are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein:

[0009] Figure 1a To disclose a schematic diagram of the acceleration unit structure in one embodiment.

[0010] Figure 1b , Figure 2 , Figure 3 , Figure 4 as well as Figures 5a-5c These are schematic diagrams of multiple structures of the acceleration unit in embodiments of this disclosure;

[0011] Figures 6-11 These are schematic diagrams of multiple structures of the acceleration components in embodiments of this disclosure;

[0012] Figures 12a-12c A schematic diagram representing the network topology to accelerate components;

[0013] Figure 13 This is a schematic diagram of an acceleration device including multiple acceleration units, according to an embodiment of this disclosure.

[0014] Figure 14 This is a schematic diagram of the network topology corresponding to the acceleration device in one embodiment;

[0015] Figure 15 This is a schematic diagram of the network topology corresponding to the acceleration device in another embodiment;

[0016] Figures 16-20 These are several schematic diagrams of an acceleration device including multiple acceleration components, as described in embodiments of this disclosure;

[0017] Figure 21 This is a schematic diagram of the network topology for yet another type of acceleration device;

[0018] Figure 22 A schematic diagram of a matrix network topology based on wireless extension of an acceleration device;

[0019] Figure 23 This is a schematic diagram of the acceleration device in yet another embodiment of this disclosure;

[0020] Figure 24 This is a schematic diagram of the network topology for yet another type of acceleration device;

[0021] Figure 25 This is a schematic diagram of the network topology for yet another type of acceleration device;

[0022] Figure 26 This is a schematic diagram of the combined device structure in one embodiment of this disclosure;

[0023] Figure 27 This is a schematic diagram of the board structure in one embodiment of this disclosure;

[0024] Figure 28 A flowchart of a method for performing a communication task in an accelerator card system according to one embodiment of the present disclosure is shown;

[0025] Figure 29a A flowchart illustrating a method for performing a communication task according to one embodiment of the present disclosure is shown;

[0026] Figure 29b A schematic diagram of a task dispatch queue and a communication task execution queue according to one embodiment of the present disclosure is shown;

[0027] Figure 30a A flowchart illustrating the division of a total task in a task queue into multiple sub-tasks according to one embodiment of the present disclosure is shown;

[0028] Figure 30b A schematic diagram illustrating the insertion of an identifier into a queue according to one embodiment of the present disclosure is shown;

[0029] Figure 31 A schematic diagram of a queue according to another embodiment of this disclosure is shown; and

[0030] Figure 32 A schematic diagram showing the modification of the second waiting identifier for insertion according to one embodiment of the present disclosure is shown. Detailed Implementation

[0031] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0032] It should be understood that the terms "first," "second," "third," and "fourth," etc., in the claims, specification, and drawings of this disclosure are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" as used in the specification and claims of this disclosure indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0033] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure. As used in this disclosure and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this disclosure and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0034] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0035] The specific embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0036] Figure 28 A method for performing a communication task in an accelerator card system according to one embodiment of the present disclosure is shown, wherein the accelerator card system includes a plurality of accelerator cards capable of communicating with each other, and one of the plurality of accelerator cards is capable of communicating with another accelerator card through a communication path; the method includes: in operation S2810, establishing a communication task queue, the communication task queue including communication tasks and a status identifier for monitoring the execution status of the communication tasks; in operation S2820, establishing a communication task execution queue for executing communication tasks between accelerator cards through a communication path; and in operation S2830, in response to the execution of the communication tasks, changing the status identifier to monitor the execution status of the communication tasks.

[0037] First, the various implementations of the accelerator card system will be described in detail below with reference to the accompanying drawings. The accelerator card system described herein consists of multiple accelerator cards capable of communicating with each other. These accelerator cards can be communicatively connected via different communication paths, allowing travel from one accelerator card to another via different communication paths, thus forming different communication topologies. It should be understood that the connection mentioned below refers to a communicative connection, meaning that each accelerator card can communicate and transmit data with each other.

[0038] Furthermore, the accelerator card system described above can be configured as an acceleration unit, acceleration component, or acceleration device. It is important to understand that although different terms are used depending on the specific scenario, they are all essentially systems comprising multiple accelerator cards.

[0039] Figure 1a This document discloses a schematic diagram of an acceleration unit structure in one embodiment. According to one embodiment of this disclosure, the acceleration card system may include an acceleration unit, which may include M local acceleration cards. Each local acceleration card includes an internal port, and each local acceleration card is connected to other local acceleration cards through the internal port. The M local acceleration cards logically form an L*N scale acceleration card matrix, where L and N are integers not less than 2.

[0040] like Figure 1a As shown, multiple accelerator cards can be used to form an accelerator card matrix. The accelerator cards are interconnected, enabling the transmission and communication of data or instructions. For example, accelerator cards MC00 to MC0N form the 0th row of the accelerator card matrix, accelerator cards MC10 to MC1N form the 1st row, and so on, with accelerator cards MCL0 to MCLN forming the Lth row.

[0041] It should be understood that, for ease of understanding, accelerator cards within the same acceleration unit are referred to as "accelerator cards of this unit," while accelerator cards in other acceleration units are referred to as "accelerator cards of other units." This terminology is merely for ease of description and does not constitute a limitation on the technical solution disclosed herein.

[0042] Each accelerator card can have multiple ports, which can connect to accelerator cards within the same unit or to accelerator cards in other units. In this disclosure, the connection ports between accelerator cards within the same unit can be referred to as internal ports, while the connection ports between an accelerator card within the same unit and an accelerator card in other units can be referred to as external ports. It should be understood that the terms "external port" and "internal port" are used merely for convenience, and the same port can be used for both. This will be described below.

[0043] It is important to understand that M can be any integer. M accelerator cards can be arranged into a 1*M or M*1 matrix, or the M matrices can be arranged into other types of matrices. The accelerator unit disclosed herein does not limit the specific matrix size and format.

[0044] Furthermore, accelerator cards can be connected via one or more communication paths, such as between accelerator cards within the same unit or between an accelerator card in the same unit and an accelerator card in an external unit. This will be described in detail later.

[0045] It is also important to understand that, in the context of this disclosure, although the positions of multiple accelerator cards are described using a rectangular network, the resulting matrix is ​​not necessarily in a matrix form in physical space. It can be located anywhere; for example, multiple accelerator cards can form a straight line or be arranged irregularly. The aforementioned matrix is ​​merely logical; as long as the connections between the accelerator cards form a matrix relationship, it is acceptable.

[0046] According to one embodiment of this disclosure, M can be 4, whereby 4 accelerator cards of this unit can logically form a 2*2 accelerator card matrix; M can be 9, whereby 9 accelerator cards of this unit can logically form a 3*3 accelerator card matrix; M can be 16, whereby 16 accelerator cards of this unit can logically form a 4*4 accelerator card matrix. M can also be 6, whereby 6 accelerator cards of this unit can logically form a 2*3 or 3*2 accelerator card matrix; M can also be 8, whereby 8 accelerator cards of this unit can logically form a 2*4 or 4*2 accelerator card matrix.

[0047] According to one embodiment of this disclosure, each local accelerator card is connected to at least one other local accelerator card via two paths.

[0048] In the topology described in this disclosure, two accelerator cards can be connected via a single communication path or via multiple paths (e.g., two), provided the number of ports is sufficient. Connecting via multiple communication paths helps ensure the reliability of communication between accelerator cards and facilitates the formation of different topologies. This will be explained and described in more detail in the examples below.

[0049] According to one embodiment of this disclosure, the diagonal accelerator cards in the four corners of the accelerator card matrix are connected by two paths. For a matrix, it is preferable to connect two pairs of accelerator cards located at opposite corners. For certain topologies, connecting accelerator cards at diagonal positions helps to form two complete communication loops. This will be explained and described in more detail in the examples below.

[0050] More specifically, according to one embodiment of this disclosure, at least one of the unit accelerator cards may include an external port. For example, each accelerator unit may include four unit accelerator cards, each unit accelerator card may include six ports, and four ports of each unit accelerator card are internal ports for connecting to the other three unit accelerator cards; the remaining two ports of at least one unit accelerator card are external ports for connecting to external unit accelerator cards.

[0051] It's important to understand that of the six ports on each accelerator card in this unit, four ports can be used to connect to the accelerator card within that unit, while the remaining two ports can be used to connect to accelerator cards in other accelerator units. These remaining ports can also remain unused, without connecting to any external devices, or they can be directly or indirectly connected to other devices or ports.

[0052] For illustrative and simplification purposes, the acceleration units, acceleration components, acceleration devices, and electronic devices described below are all based on the example of each acceleration unit comprising four acceleration cards. It should be understood that each acceleration unit may include more or fewer acceleration cards.

[0053] For ease of description, the acceleration unit may include four acceleration cards, namely the first acceleration card, the second acceleration card, the third acceleration card and the fourth acceleration card. Each acceleration card is provided with an internal port and an external port, and each acceleration card is connected to the other three acceleration cards through the internal port.

[0054] Figure 1b This is a schematic diagram of the acceleration unit structure in one embodiment of the present disclosure. The acceleration unit 100 includes four acceleration cards: MC0, MC1, MC2, and MC3. Each acceleration card may include an external port and an internal port. The internal port of acceleration card MC0 is connected to the internal ports of acceleration cards MC1, MC2, and MC3; the internal port of acceleration card MC1 is connected to the internal ports of acceleration cards MC2 and MC3; and the internal port of acceleration card MC2 is connected to the internal port of acceleration card MC3. In other words, the internal port of each acceleration card is connected to the internal ports of the other three acceleration cards. Information exchange between the four acceleration cards can be achieved through the interconnection of their internal ports. This embodiment of the present disclosure utilizes the interconnection between the four acceleration cards in the acceleration unit to improve the computing power of the acceleration unit and achieve high-speed processing of massive amounts of data, while minimizing the path between each acceleration card and the other acceleration cards and reducing communication latency.

[0055] As described above, the number of accelerator cards in this disclosure is not limited to four, but can be any other number. For example, in one embodiment, the number of accelerator cards N equals 3, each accelerator card is provided with an internal port and an external port, and each accelerator card is connected to the other two accelerator cards through its internal port, realizing interconnection between the three accelerator cards. In another embodiment, the number of accelerator cards N equals 5, each accelerator card is provided with an internal port and an external port, and each accelerator card is connected to the other four accelerator cards through its internal port, realizing interconnection between the five accelerator cards, thereby improving the computing power of the acceleration unit and realizing high-speed processing of massive amounts of data. In yet another embodiment, the number of accelerator cards N is greater than 5, each accelerator card is provided with an internal port and an external port, and each accelerator card is connected to all other accelerator cards through its internal port, realizing interconnection between N accelerator cards and realizing high-speed processing of massive amounts of data.

[0056] based on Figure 1b The provided acceleration unit 100 further allows each acceleration card to be connected to at least one other acceleration card via two paths. Specifically, there are three possible connection methods: the first method is that each acceleration card can be connected to one of the other three acceleration cards via two paths; the second method is that each acceleration card can be connected to two of the other three acceleration cards via two paths; and the third method is that each acceleration card can be connected to all three acceleration cards via two paths, in which case it is possible that each acceleration card has more ports. To facilitate understanding of the above-mentioned two-path connection methods, the first connection method will be used as an example below, combined with... Figure 2 An exemplary description is provided.

[0057] Figure 2 This is a schematic diagram of the acceleration unit structure in another embodiment of this disclosure. Figure 2 In the acceleration unit 200 shown, each acceleration card can be connected to at least one other acceleration card via two paths. For example, acceleration card MC0 and acceleration card MC2 can be connected via two paths, as can acceleration card MC1 and acceleration card MC3. With this configuration, there can be two links (or paths) for information exchange between two acceleration cards. Thus, if one link fails, the two acceleration cards can still be connected via the other link, effectively improving the security of the acceleration unit.

[0058] The above is combined with Figure 1 and Figure 2 The acceleration unit according to this disclosure and the connection methods between its plurality of acceleration cards have been described exemplaryly. Those skilled in the art should understand that the above description is exemplary and not restrictive; for example, the arrangement of the acceleration cards in the acceleration unit may not be limited to that shown in Figure 1 and... Figure 2 As shown in the diagram, in one embodiment, the four accelerator cards of the acceleration unit can be logically arranged in a quadrilateral pattern, which will be discussed below. Figure 3 Describe it.

[0059] Figure 3 This is a schematic diagram of the acceleration unit structure in yet another embodiment of this disclosure. Figure 3 In the acceleration unit 300 shown, the four acceleration cards MC0, MC1, MC2, and MC3 can logically be arranged in a quadrilateral pattern, with each card occupying one of the four vertices of the quadrilateral. The wiring between the acceleration cards MC0, MC1, MC2, and MC3 also forms a quadrilateral, making the wiring arrangement clearer and easier to set up. It should be noted that... Figure 3 The four accelerator cards shown are arranged in a rectangle or a 2x2 matrix. However, this is a logic interconnection diagram, and the rectangular form is used for ease of description. The specific quadrilateral shape can be freely set, such as a parallelogram, trapezoid, or square. In actual layout and routing, the four accelerator cards can also be arranged arbitrarily. For example, in an actual system, the four accelerator cards are arranged side by side in a straight line, and the order could be MC0, MC1, MC2, and MC3. It should also be understood that the logical quadrilateral shown in this embodiment is exemplary. In reality, the arrangement shape of multiple accelerator cards can vary greatly, and the quadrilateral is just one of them. For example, when there are five accelerator cards, they can be logically arranged in a pentagon.

[0060] based on Figure 2 For further details regarding the connection relationships of the provided acceleration unit 200, please refer to [link / reference]. Figure 4 , Figure 4 This is a schematic diagram of the acceleration unit structure in yet another embodiment of this disclosure. Figure 4 In the acceleration unit 400 shown, the four acceleration cards MC0, MC1, MC2, and MC3 can logically be arranged in a quadrilateral, with each card occupying one of the four vertices of the quadrilateral. As further illustrated, the internal ports of acceleration cards MC1 and MC3 can be connected via two paths, as can the internal ports of MC0 and MC2. This not only simplifies the wiring configuration of the acceleration unit 400 but also enhances its security.

[0061] Figure 5a This is a schematic diagram of the acceleration unit structure in one embodiment of this disclosure. Figure 5aIn the acceleration unit 500 shown, the numerical markings on each acceleration card represent ports. Each acceleration card may include six ports: port 0, port 1, port 2, port 3, port 4, and port 5. Ports 1, 2, 4, and 5 are internal ports, while ports 0 and 3 are external ports. For the four acceleration cards MC0, MC1, MC2, and MC3, each card's two external ports can be connected to other acceleration units for interconnection between multiple acceleration units. Each acceleration card's four internal ports can be used to interconnect with the other three acceleration cards in the same acceleration unit.

[0062] like Figure 5a As further illustrated, the four accelerator cards can logically be arranged, for example, in a quadrilateral. Accelerator cards MC0 and MC2 can be diagonally connected, with port 2 of MC0 and port 2 of MC2 connected, and port 5 of MC0 and port 5 of MC2 connected. This means there can be two communication links between accelerator cards MC0 and MC2. Similarly, accelerator cards MC1 and MC3 can be diagonally connected, with port 2 of MC1 and port 2 of MC3 connected, and port 5 of MC1 and port 5 of MC3 connected. This also means there can be two communication links between accelerator cards MC1 and MC3.

[0063] With this configuration, each accelerator card has two external ports and four internal ports. Furthermore, in two diagonally arranged pairs of accelerator cards, each pair can be connected via two internal ports to form two links, effectively improving the security and stability of the acceleration unit. The logical quadrilateral arrangement of the four accelerator cards also makes the wiring layout of the entire acceleration unit clear and logical, facilitating wiring operations within each unit. It should be further noted that... Figure 5b In the interconnection lines between the four accelerator cards shown, the connection lines between port 1 of accelerator card MC1 and port 1 of MC0, port 2 of accelerator card MC0 and port 2 of MC2, port 1 of accelerator card MC2 and port 1 of MC3, and port 2 of accelerator card MC3 and port 2 of MC1, these four lines form a vertical figure-eight network, as shown below. Figure 5b As shown. The connection lines between port 4 of accelerator card MC1 and port 4 of MC2, the connection line between port 5 of accelerator card MC2 and port 5 of MC0, the connection line between port 4 of accelerator card MC0 and port 4 of MC3, and the connection line between port 5 of accelerator card MC3 and port 5 of MC1, these four lines form a horizontal figure-eight network, as shown. Figure 5c As shown, these two fully connected square networks can form a double-ring structure, providing redundancy and enhancing system reliability.

[0064] According to one embodiment of this disclosure, the accelerator card described herein may be a Mezzanine Card (MC Card), which may be a separate circuit board. The MC Card may house an ASIC chip and some necessary peripheral control circuitry. The MC Card may be connected to the substrate via a snap-on connector. Power and control signals from the substrate can be transmitted to the MC Card via the snap-on connector. According to another embodiment of this disclosure, the internal and / or external ports described herein may be SerDes ports. For example, in one embodiment, each MC Card may provide six bidirectional SerDes ports, each with eight channels and a data transmission rate of 56Gbps. The total bandwidth of each port can then reach up to 400Gbps, supporting massive data exchange between accelerator cards and facilitating high-speed data processing by the acceleration unit.

[0065] The SerDes mentioned above is a portmanteau of the English words serializer and de-serializer, and is referred to as a serial deserializer. SerDes interfaces can be used to build high-performance processor clusters. The main function of SerDes is to convert multiple low-speed parallel signals into serial signals at the transmitting end, transmit them through the transmission medium, and finally convert the high-speed serial signals back into low-speed parallel signals at the receiving end. Therefore, it is very suitable for end-to-end long-distance high-speed transmission requirements. In another embodiment, the external port in the accelerator card can be connected to the QSFP-DD interface of other accelerator units. The QSFP-DD interface is a commonly used optical module interface in SerDes technology, which, when used with cables, can be used to interconnect with other external devices.

[0066] Furthermore, according to yet another embodiment of this disclosure, an acceleration unit can house four acceleration cards, and the interconnection of the four acceleration cards can be accomplished using printed circuit board (PCB) traces. On a high-speed substrate with a low dielectric constant, signal integrity can be maximized through reasonable layout and wiring, thereby ensuring that the communication bandwidth between the four acceleration cards approaches the theoretical value.

[0067] The acceleration unit disclosed in this disclosure comprises four acceleration cards. Each acceleration card connects to the other three acceleration cards via its internal port. Each acceleration card can directly communicate with the other three acceleration cards. This communication architecture is a fully connected quad network topology. The advantage of this fully connected network architecture is that the path between each acceleration card and the other acceleration cards is the shortest, the total number of hops is the smallest, and the latency is the lowest. This disclosure uses hops to describe the system latency. Hop in communication represents the number of hops, i.e., the number of communication times. Specifically, Hop represents the shortest path from a node, traversing all nodes in the network, and returning to the initial node. The interconnection of the four acceleration cards forms a fully connected quad network topology with the shortest latency. Furthermore, the double-ring structure formed by the interconnection of two diagonally connected acceleration cards can improve the robustness of the system, ensuring that services can still operate normally even if a single acceleration card fails. When performing various arithmetic and logical operations, each ring in the double-ring structure can complete a part of the operation, thereby improving the overall computational efficiency and maximizing the utilization of the topology bandwidth.

[0068] The above combination Figures 1a-5c Several embodiments of the acceleration unit according to this disclosure have been described. Based on the above-described acceleration unit, this disclosure also discloses an acceleration component that may include multiple of the above-described acceleration units. Several embodiments of the acceleration component will be described exemplarily below.

[0069] Figure 6 This is a schematic diagram of the acceleration component structure in one embodiment of this disclosure. Figure 6 As shown, the acceleration component 600 may include n of the aforementioned acceleration units. In other words, the acceleration card system can be embodied as an acceleration component, which includes multiple acceleration units, namely acceleration unit A1, acceleration unit A2, acceleration unit A3, ..., acceleration unit An. Acceleration unit A1 and acceleration unit A2 are connected through an external port, and acceleration unit A2 and acceleration unit A3 are connected through an external port, that is, each acceleration unit is connected to each other through its external port. In one embodiment, the external port of acceleration card MC0 in acceleration unit A1 can be connected to the external port of acceleration card MC0 in acceleration unit A2, and the external port of acceleration card MC0 in acceleration unit A2 can be connected to the external port of acceleration card MC0 in acceleration unit A3, that is, each acceleration unit is connected through the external port of acceleration card MC0.

[0070] Those skilled in the art will understand that the connection between the acceleration units in this disclosure is not limited to the connection of the external port of acceleration card MC0, but may also include one or more of the following: the connection of the external ports of acceleration card MC1, acceleration card MC2, and acceleration card MC3. That is, in this disclosure, the connection method between acceleration unit A1 and acceleration unit A2 may include one or more of the following: connecting the external port of MC0 in A1 to the external port of MC0 in A2, connecting the external port of MC1 in A1 to the external port of MC1 in A2, connecting the external port of MC2 in A1 to the external port of MC2 in A2, and connecting the external port of MC3 in A1 to the external port of MC3 in A2. Similarly, the connection methods between acceleration unit A2 and acceleration unit A3 can include one or more of the following: connecting the external port of MC0 in A2 to the external port of MC0 in A3; connecting the external port of MC1 in A2 to the external port of MC1 in A3; connecting the external port of MC2 in A2 to the external port of MC2 in A3; and connecting the external port of MC3 in A2 to the external port of MC3 in A3. This can be extrapolated to the connection between acceleration unit An-1 and acceleration unit An. It should be noted that the above description is exemplary; for example, the connection between different acceleration units is not limited to the connection of acceleration cards corresponding to the labels, but can be set to the connection of acceleration cards with the same label as needed.

[0071] It should be noted that, Figure 6 The diagram shows n acceleration units, where n is greater than 3. However, the number of acceleration units is not limited to the number greater than 3 shown in the diagram. It can also be set to, for example, 2 or 3. The connection relationship between two acceleration units is the same as or similar to the connection relationship between acceleration units A1 and A2 mentioned above. The connection relationship between three acceleration units is the same as or similar to the connection relationship between acceleration units A1, A2, and A3 mentioned above. These details will not be repeated here.

[0072] Furthermore, the structures of multiple acceleration units within an acceleration component can be identical or different. Figure 6 For ease of demonstration, the structures of the multiple acceleration units shown are identical. However, in reality, the structures of multiple acceleration units can be different. For example, in some acceleration units, the multiple accelerator cards are arranged in a polygonal layout; in others, they are arranged in a line; in still others, they are connected by a single line; and in some acceleration units, they are connected by two links. Some acceleration units include four accelerator cards, while others include three or five. In other words, the structure of each acceleration unit can be set independently, and different acceleration units can have the same or different structures.

[0073] The acceleration component disclosed herein allows not only interconnection of accelerator cards within the acceleration unit, but also interconnection of accelerator cards from different acceleration units, thereby enabling the construction of a hybrid three-dimensional network. With this configuration, each accelerator card can process data while simultaneously sharing data through interconnection between acceleration units. Since data sharing allows for direct data acquisition, reducing data propagation paths and time, it significantly improves data processing efficiency.

[0074] Figure 7 This is a schematic diagram of the acceleration component structure in another embodiment of this disclosure. Figure 7 As shown, the acceleration component 700 may include n aforementioned acceleration units, namely acceleration unit A1, acceleration unit A2, acceleration unit A3, ..., acceleration unit An. The multiple acceleration units in the acceleration component 700 can logically form a multi-layered structure (shown as dashed lines in the diagram). Each layer may include one acceleration unit, and the acceleration card of each acceleration unit is connected to the acceleration card in another acceleration unit via an external port. This progressively layered configuration allows each acceleration card to share data via a high-speed serial link while processing data at high speed, enabling unlimited interconnection of acceleration cards to meet customizable computing power requirements and achieve flexible configuration of the processor cluster's hardware computing power. As further shown in the diagram, each layer of acceleration units may include four acceleration cards, which can logically be arranged in a quadrilateral configuration, with the four acceleration cards positioned at the four vertices of the quadrilateral.

[0075] Those skilled in the art should understand that the above combination Figure 7 The described acceleration components are exemplary and not limiting. For example, the structures of multiple acceleration units can be the same or different. The number of layers in the acceleration component can be 2, 3, 4, or more, and the number of layers can be freely set as needed. For each pair of connected acceleration units, the number of connection paths between them can be 1, 2, 3, or 4. For ease of understanding, the following will combine... Figure 8 - Figure 12 provides an exemplary description.

[0076] Figure 8 This is a schematic diagram of the acceleration component structure in yet another embodiment of this disclosure. Figure 8 As shown, the acceleration component 701 can have two acceleration units, which are connected by a path. Specifically, they can be connected through, for example, the external port of the acceleration card MC0 in acceleration unit A1 and the external port of the acceleration card MC0 in acceleration unit A2, so that information exchange between acceleration unit A1 and acceleration unit A2 can be realized.

[0077] like Figure 9As shown, the acceleration component 702 can contain two acceleration units, connected via two paths. The external port of acceleration card MC0 in acceleration unit A1 is connected to the external port of acceleration card MC0 in acceleration unit A2, and the external port of acceleration card MC1 in acceleration unit A1 is connected to the external port of acceleration card MC1 in acceleration unit A2. This ensures that if one path fails, the other path still supports communication between the acceleration units, further enhancing the security of the acceleration component.

[0078] Please refer to the following. Figure 10 , Figure 10 This is a schematic diagram of the acceleration component structure in yet another embodiment of this disclosure. Figure 10 In the acceleration component 703 shown, there can be two acceleration units. The two acceleration units are connected via three paths: the external port of acceleration card MC0 in acceleration unit A1 is connected to the external port of acceleration card MC0 in acceleration unit A2; the external port of acceleration card MC1 in acceleration unit A1 is connected to the external port of acceleration card MC1 in acceleration unit A2; and the external port of acceleration card MC2 in acceleration unit A1 is connected to the external port of acceleration card MC2 in acceleration unit A2. Thus, even if two of the paths fail, there is still a third path to support communication between the acceleration units, further improving the security of the acceleration component.

[0079] Please refer to the following. Figure 11 , Figure 11 This is a schematic diagram of the acceleration component structure in yet another embodiment of this disclosure. Figure 11 In the acceleration component 704 shown, there can be two acceleration units. The two acceleration units can be connected via four paths. For example, the external port of acceleration card MC0 in acceleration unit A1 can be connected to the external port of acceleration card MC0 in acceleration unit A2; the external port of acceleration card MC1 in acceleration unit A1 can be connected to the external port of acceleration card MC1 in acceleration unit A2; the external port of acceleration card MC2 in acceleration unit A1 can be connected to the external port of acceleration card MC2 in acceleration unit A2; and the external port of acceleration card MC3 in acceleration unit A1 can be connected to the external port of acceleration card MC3 in acceleration unit A2. Thus, even if three of these paths fail, there is still one path to support communication between the acceleration units, further improving the security of the acceleration component.

[0080] Figure 12a A schematic diagram illustrating the network topology to accelerate component representation. For example... Figure 12aAs shown, the acceleration component 705 may include two acceleration units, each acceleration unit may include four acceleration cards, and there may be two links between acceleration cards MC1 and MC3 in each acceleration unit, and two links between acceleration cards MC0 and MC2. Figure 12a The acceleration device 705 in the left figure can form the three-dimensional representation shown in the right figure. Figure 12a In the right-hand diagram, circles represent accelerator cards, and lines represent link connections. Within each circle, number 0 represents accelerator card MC0, number 1 represents accelerator card MC1, number 2 represents accelerator card MC2, and number 3 represents accelerator card MC3. The right-hand diagram still represents the accelerator component 705, but as a different representation, showing a network topology. The numbers embedded in the vertical lines in the right-hand diagram indicate the port numbers for connections. For example, MC0 units are connected via port 0, MC1 units are connected via port 0, MC2 units are connected via port 3, and MC3 units are connected via port 3.

[0081] for Figure 12a The right figure shows an acceleration unit as a node. Two nodes have eight acceleration cards, forming an eight-card interconnect. The one-machine-four-card interconnection relationship within each node is fixed. When two nodes are interconnected, MC0 and MC1 in the upper node (acceleration unit A1) are connected to MC0 and MC1 in the lower node (acceleration unit A2) through port 0, respectively; MC2 and MC3 in the upper node are connected to MC2 and MC3 in the lower node through port 3, respectively. This node topology is called a hybrid cube mesh network topology, meaning that the acceleration component 705 is a hybrid cube mesh network topology.

[0082] exist Figure 12a In the 8-card topology shown, two independent rings can also be formed. For example... Figure 12b and Figure 12c As shown, this allows for maximum utilization of the topology bandwidth for reduction operations.

[0083] exist Figure 12b In the acceleration unit A1, acceleration cards MC1 and MC3 are connected via their respective internal ports 5, acceleration cards MC0 and MC2 are connected via their respective internal ports 5, and acceleration cards MC2 and MC3 are connected via their respective internal ports 1. Acceleration cards MC1 in acceleration unit A1 and MC1 in acceleration unit A2 are connected via their respective external ports 0, and acceleration cards MC0 in acceleration unit A1 and MC0 in acceleration unit A2 are connected via their respective external ports 0. Thus, an independent loop is formed among the eight cards in Figure 12.

[0084] exist Figure 12cIn the acceleration unit A1, acceleration cards MC1 and MC3 are connected via their respective internal ports 2, acceleration cards MC0 and MC2 are connected via their respective internal ports 2, and acceleration cards MC0 and MC1 are connected via their respective internal ports 1. Acceleration cards MC2 in acceleration unit A1 and MC2 in acceleration unit A2 are connected via their respective external ports 3, and acceleration cards MC3 in acceleration unit A1 and MC3 in acceleration unit A2 are connected via their respective external ports 3. Thus, another independent loop is formed among the eight cards in Figure 12.

[0085] The above only shows two exemplary connection methods. In reality, the four connection paths between the two acceleration units are actually equivalent. Therefore, any one to three of these four paths can be used to connect the two acceleration units and form a ring connection with the acceleration card within each acceleration unit. This will not be elaborated further here.

[0086] Figure 13 This is a schematic diagram of an acceleration device in yet another embodiment of this disclosure. Figure 13 As shown, the acceleration device 800 may include n acceleration units, namely acceleration unit A1, acceleration unit A2, acceleration unit A3, ..., acceleration unit An. The multiple acceleration units in the acceleration device 800 are logically arranged in a multi-layered structure (shown as dashed lines in the figure). This multi-layered structure may include an odd number of layers or an even number of layers. Each layer may include one acceleration unit. The acceleration card of each acceleration unit is connected to the acceleration card of another acceleration unit through an external port. Specifically, acceleration unit A1 and acceleration unit A2 are connected through an external port, acceleration unit A2 and acceleration unit A3 are connected through an external port, and so on, with acceleration unit An-1 and acceleration unit An connected through an external port. Furthermore, the last acceleration unit can be connected to the first acceleration unit, thus forming a ring structure with the multiple acceleration units connected end-to-end. For example, in the figure, the external port of acceleration card MC0 of acceleration unit An is connected to the external port of acceleration card MC0 of acceleration unit A1. This layered configuration allows each accelerator card to share data via a high-speed serial link while processing data at high speed, enabling unlimited interconnection of accelerator cards to meet customizable computing power requirements and achieve flexible configuration of the hardware computing power of the processor cluster.

[0087] It should be noted that the connection relationships of the acceleration units in the acceleration device of this disclosure can vary, as described in detail above. For specific examples, please refer to the above. Figure 6The details of the connection relationships between the acceleration units are not repeated here. Furthermore, there are several ways to connect the last acceleration unit to the first acceleration unit, specifically including one or more of the following: connecting the external port of MC0 in acceleration unit A1 to the external port of MC0 in An; connecting the external port of MC1 in acceleration unit A1 to the external port of MC1 in An; connecting the external port of MC2 in acceleration unit A1 to the external port of MC2 in An; and connecting the external port of MC3 in acceleration unit A1 to the external port of MC3 in An. For ease of understanding, the following will combine... Figure 14 and Figure 15 An exemplary description will be provided. In the following description, those skilled in the art will understand that... Figure 14 and Figure 15 The acceleration device shown is Figure 13 The acceleration device 800 shown has various specific manifestations, therefore regarding Figure 13 The description of the accelerator 800 can also be applied to Figure 14 and Figure 15 The acceleration device in the middle.

[0088] refer to Figure 14 , Figure 14 This is a schematic diagram of the network topology corresponding to the acceleration device in one embodiment. For example... Figure 14 The acceleration device 801 shown can consist of four acceleration units. Circles represent acceleration cards, and lines represent link connections. Within each circle, number 0 represents acceleration card MC0, number 1 represents MC1, number 2 represents MC2, and number 3 represents MC3. The numbers embedded in the vertical lines indicate the port numbers of the connections. The last acceleration unit is connected to the first, with a total of 5 hops. Each acceleration unit is a node, and interconnection between nodes allows for the interconnection of 4 nodes (16 cards). The four acceleration units form a small cluster, internally interconnected, called a super pod. This topology is the preferred form for ultra-large-scale clusters, using high-speed SerDes ports, with a total of 5 hops and minimal latency. The cluster exhibits good manageability and robustness.

[0089] refer to Figure 15 , Figure 15 This is a schematic diagram of the network topology corresponding to the acceleration device in another embodiment. Figure 15 and Figure 14 The difference is that, Figure 15 The acceleration device 802 shown has a greater number of acceleration units. As can be seen from the diagram, the last acceleration unit of acceleration device 802 is connected to the first acceleration unit. With this configuration, the total number of hops is the number of nodes plus one, meaning the total number of hops is the number of acceleration units plus one.

[0090] The above combination Figures 13-15 An exemplary acceleration device including multiple acceleration units has been described. According to the technical solution of this disclosure, an acceleration device including multiple of the aforementioned acceleration components is also provided. The following will describe it in detail with reference to multiple embodiments.

[0091] Figure 16 This is a schematic diagram of an acceleration device in another embodiment of the present disclosure. The acceleration system of the present disclosure can be implemented as an acceleration device. The acceleration device 900 may include m of the aforementioned acceleration components. Each acceleration component, in addition to the external ports that need to be used for connection between acceleration units within the acceleration component, also has idle external ports. The acceleration components are interconnected through the idle external ports. Specifically, the external port of the acceleration card MC1 of acceleration unit A1 in acceleration component B1 can be connected to the external port of the acceleration card MC1 of acceleration unit A1 in acceleration component B2, the external port of the acceleration card MC1 of acceleration unit A1 in acceleration component B2 can be connected to the external port of the acceleration card MC1 of acceleration unit A1 in acceleration component B3, and so on, with multiple acceleration components interconnected. It is understood that... Figure 16 The acceleration device shown is exemplary and not limiting; for example, the structures of multiple acceleration components may be the same or different. Furthermore, the method by which different acceleration components are connected via unused external ports is not limited to... Figure 16 The methods shown may also include other methods. For ease of understanding, the following will combine... Figures 17-25 An exemplary description is provided.

[0092] based on Figure 16 The provided acceleration device, further, refers to Figure 17 , Figure 17 This is a schematic diagram of the network topology corresponding to the acceleration device in another embodiment. Acceleration device 901 may include two acceleration components. Acceleration component B1 may include four acceleration units, and acceleration component B2 may include four acceleration units. The first acceleration unit in acceleration component B1 is connected to the first acceleration unit in acceleration component B2, and the last acceleration unit in acceleration component B1 is connected to the last acceleration unit in acceleration component B2. The total number of hops in this network topology is 9. Those skilled in the art will understand that... Figure 17 The network structure consisting of multiple acceleration units in each acceleration component is logical; in practical applications, the arrangement of these acceleration units can be adjusted as needed. The number of acceleration units in each acceleration component is not limited to the four shown in the diagram; it can be set to more or fewer as required, such as six or eight.

[0093] based on Figure 16 The provided acceleration device, further, refers to Figure 18 , Figure 18 This is a schematic diagram of an acceleration device in another embodiment of the present disclosure. The acceleration device 902 may include four acceleration components, namely acceleration components B1, B2, B3, and B4. Each of the four acceleration components may include two acceleration units A1 and A2. Each acceleration component can be interconnected with one of the acceleration units A1 and A2 of the other acceleration components. For example, acceleration unit A1 in acceleration component B1 is connected to acceleration unit A1 in acceleration component B2, acceleration unit A1 in acceleration component B2 is connected to acceleration unit A1 in acceleration component B3, and acceleration unit A1 in acceleration component B3 is connected to acceleration unit A1 in acceleration component B4. These connections are all made through external ports of the acceleration units.

[0094] It should be noted that, in addition to the connection methods between acceleration components, Figure 18 Besides the connection method shown, there can be many other types. For example, the connection methods between acceleration components can specifically include: acceleration unit A1 or A2 in acceleration component B1 is connected to acceleration unit A1 or A2 in acceleration component B2; acceleration unit A1 or A2 in acceleration component B2 is connected to acceleration unit A1 or A2 in acceleration component B3; and acceleration unit A1 or A2 in acceleration component B3 is connected to acceleration unit A1 or A2 in acceleration component B4.

[0095] based on Figure 18 For further information on the provided acceleration device, please refer to [link / reference]. Figure 19 , Figure 19 This is a schematic diagram of an acceleration device in yet another embodiment of this disclosure. Figure 19 In the acceleration device 903 shown, each acceleration component can be interconnected with one of the first acceleration unit and the second acceleration unit of other acceleration components via two paths. For example, the first acceleration unit (e.g., acceleration unit A1) in acceleration component B1 and the first acceleration unit (e.g., acceleration unit A1) in acceleration component B2 can be connected via two paths, acceleration unit A1 in acceleration component B2 and acceleration unit A1 in acceleration component B3 can be connected via two paths, and acceleration unit A1 in acceleration component B3 and acceleration unit A1 in acceleration component B4 can be connected via two paths.

[0096] It should be noted that, Figure 19 The symbol indicates a connection between two paths, but it can also include connections between more than two paths. Besides the methods for accelerating connections between components... Figure 19The connection method shown may also include other methods, such as acceleration unit A1 or A2 in acceleration component B1 being connected to acceleration unit A1 or A2 in acceleration component B2 via two paths, acceleration unit A1 or A2 in acceleration component B2 being connected to acceleration unit A1 or A2 in acceleration component B3 via two paths, and acceleration unit A1 or A2 in acceleration component B3 being connected to acceleration unit A1 or A2 in acceleration component B4 via two paths.

[0097] based on Figure 16 For further information on the provided acceleration device, please refer to [link / reference]. Figure 20 , Figure 20 This is a schematic diagram of an acceleration device in another embodiment of the present disclosure. The acceleration device 904 includes four acceleration components: acceleration component B1, acceleration component B2, acceleration component B3, and acceleration component B4. Each acceleration component includes two acceleration units, and each acceleration unit includes two pairs of acceleration cards. In each acceleration unit, MC0 and MC1 are the first pair of acceleration cards, and MC2 and MC3 are the second pair of acceleration cards. Specifically, the second pair of acceleration cards of acceleration unit A1 in acceleration component B1 is connected to the second pair of acceleration cards of acceleration unit A2 in acceleration component B2; the first pair of acceleration cards of acceleration unit A2 in acceleration component B2 is connected to the first pair of acceleration cards of acceleration unit A1 in acceleration component B3; the second pair of acceleration cards of acceleration unit A2 in acceleration component B3 is connected to the second pair of acceleration cards of acceleration unit A1 in acceleration component B4; and the first pair of acceleration cards of acceleration unit A1 in acceleration component B4 is connected to the first pair of acceleration cards of acceleration unit A2 in acceleration component B1.

[0098] refer to Figure 21 , Figure 21 This is a network topology diagram of yet another type of acceleration device. Figure 21 The acceleration device 905 shown is Figure 20 This is a specific embodiment of the acceleration device 904 shown, therefore the above description of the acceleration device 904 can also be applied to... Figure 21 The acceleration device 905 in the middle. For example... Figure 21 As shown, each acceleration component of the acceleration device 905 can form a hybrid three-dimensional network unit. The interconnection relationship within each hybrid three-dimensional network unit can be as shown in the figure, realizing the interconnection of 8 nodes and 32 cards in the acceleration device 905. The four acceleration components can be interconnected through, for example, QSFP-DD interfaces and cables to form a matrix network topology.

[0099] Specifically, in this embodiment, ports 0 of the accelerator cards MC2 and MC3 of the upper-level nodes of acceleration component B1 can be connected to the accelerator cards MC2 and MC3 of the lower-level nodes of acceleration component B2, respectively. Ports 3 of MC0 and MC1 of the lower-level nodes of acceleration component B2 can be connected to MC0 and MC1 of the upper-level nodes of acceleration component B3, respectively. Ports 0 of MC2 and MC3 of the lower-level nodes of acceleration component B3 can be connected to MC2 and MC3 of the upper-level nodes of acceleration component B4, respectively. Ports 3 of MC0 and MC1 of the upper-level nodes of acceleration component B4 can be connected to MC0 and MC1 of the lower-level nodes of acceleration component B1, respectively. This interconnection of the hybrid three-dimensional networks can form two bidirectional ring structures (as described above). Figure 5b , Figure 5c , Figure 12b and Figure 12c As described, it has advantages such as good reliability and security, and is suitable for deep learning training with high computational efficiency. In the accelerator 905, the matrix network topology consisting of 8 nodes has a total of 11 hops.

[0100] Furthermore, such as Figure 21 As shown, the first pair of accelerator cards and the second pair of accelerator cards in different accelerator units within the same accelerator assembly can be indirectly connected. For example, accelerator cards MC0 and MC1 of the upper-level accelerator unit in accelerator assembly B1 are indirectly connected to accelerator cards MC2 and MC3 of the lower-level accelerator unit.

[0101] exist Figure 21 Based on the existing network topology, matrix network topology can be further extended into larger network topologies. Figure 22 This is a schematic diagram of a matrix network topology based on wireless extension of an acceleration device. (Example:) Figure 22 As shown, the acceleration device 906 may include multiple acceleration components, and each acceleration component (shown in a box in the figure) may include multiple acceleration units (not shown in perspective, but can be referenced). Figure 21 The acceleration component structure can include, for example, four interconnected acceleration cards as shown in the figure, so the matrix network topology can theoretically be expanded indefinitely.

[0102] based on Figure 16 For further information on the provided acceleration device, please refer to [link / reference]. Figure 23 , Figure 23This is a schematic diagram of an acceleration device in another embodiment of the present disclosure. The acceleration device 908 may include m (m 2) acceleration components, each acceleration component may include n (n 2) acceleration units, and the m acceleration components may be connected in a ring. Specifically, the acceleration unit An of acceleration component B1 may be connected to the acceleration unit A1 of acceleration component B2, the acceleration unit An of acceleration component B2 may be connected to the acceleration unit A1 of acceleration component B3, and so on down to acceleration component Bm, where the acceleration unit An of acceleration component Bm may be connected to the acceleration unit A1 of acceleration component B1. Thus, these m acceleration components are connected end-to-end in a ring.

[0103] based on Figure 23 Please refer to Figure 24 , Figure 24 This is a network topology diagram of another acceleration device. The acceleration device 909 may include 6 acceleration components, each acceleration component may include two acceleration units, and the second acceleration unit of each acceleration component may be connected to the first acceleration unit of the next acceleration component, forming an interconnection of 12 nodes and 48 cards, forming a larger matrix network topology. The total Hop under this network topology is 13 times.

[0104] based on Figure 24 Please refer to Figure 25 , Figure 25 This is a network topology diagram of another acceleration device. The acceleration device 910 includes 8 acceleration components, each of which includes two acceleration units. The second acceleration unit of each acceleration component can be connected to the first acceleration unit of the next acceleration component, forming an interconnection of 16 nodes and 64 cards, forming a larger matrix network topology. The total Hop under this network topology is 17.

[0105] exist Figure 25 Based on this, it can be continuously scaled vertically to form ultra-large-scale matrix networks, such as 20 nodes with 80 cards, 24 nodes with 96 cards, etc. Theoretically, it can be expanded indefinitely, with the total number of hops being the number of nodes plus one. By optimizing the interconnection method between nodes, the latency of the entire system can be minimized, maximizing the ability to meet the real-time requirements of the system while processing massive amounts of data.

[0106] The above combination Figures 16-25 An exemplary acceleration device comprising multiple acceleration components has been described. Those skilled in the art will understand that the above description is exemplary and not restrictive; for example, the number, structure, and connection relationships between the acceleration components can be adjusted as needed. Those skilled in the art can also combine the above embodiments to form an acceleration device as needed, which is also within the scope of this disclosure.

[0107] Additionally, it should be noted that the accelerator card matrix, fully connected square network (topology), hybrid three-dimensional network (topology), matrix network (topology) and other similar terms mentioned in this disclosure are all logical, and the specific deployment can be adjusted as needed.

[0108] The topology disclosed in this disclosure can also perform data reduction operations. These reduction operations can be performed on each accelerator card, each accelerator unit, and within the accelerator apparatus. Specific operational steps are as follows.

[0109] Taking reduction summation as an example, the reduction operation process in an acceleration unit may include: transferring the data stored in the first acceleration card to the second acceleration card, and performing addition on the data originally stored in the second acceleration card and the data received from the first acceleration card; then, transferring the addition result in the second acceleration card to the third acceleration card, and performing addition again, and so on, until all the data stored in the acceleration cards has been added and each acceleration card has received the final result.

[0110] by Figure 4 Taking the acceleration unit shown as an example, acceleration card MC0 stores data (0,0), acceleration card MC1 stores data (1,2), acceleration card MC2 stores data (3,1), and acceleration card MC3 stores data (2,4). The data (0,0) in acceleration card MC0 can be passed to acceleration card MC1, and after addition, the result (1,2) is obtained. Next, the result (1,2) is passed to acceleration card MC2 to obtain the next result (4,3). Then, the next result (4,3) is passed to acceleration card MC3 to obtain the final result (6,7).

[0111] Subsequently, in the reduction operation of this disclosure, the final result (6,7) is passed to each accelerator card MC0, MC1, MC2 and MC3, so that the data (6,7) is stored in all accelerator cards, thus completing the reduction operation in one accelerator unit.

[0112] Figure 4 The acceleration unit can form two independent rings, each of which can complete half of the data reduction operations, thereby speeding up the operation and improving the efficiency.

[0113] In addition, the aforementioned acceleration unit can also achieve concurrent computation of multiple acceleration cards during reduction operations, thereby accelerating the computation speed. For example, acceleration card MC0 stores data (0,0), acceleration card MC1 stores data (1,2), acceleration card MC2 stores data (3,1), and acceleration card MC3 stores data (2,4). Part of the data (0) in acceleration card MC0 can be transferred to acceleration card MC1, and after addition, the result (1) is obtained. Simultaneously, part of the data (2) in acceleration card MC1 can be transferred to acceleration card MC2, and after addition, the result (3) is obtained. This achieves concurrent computation between acceleration cards MC1 and MC2. And so on, to complete the entire reduction operation.

[0114] The aforementioned concurrent computation can also include grouping acceleration units to perform addition operations first, and then performing a reduction operation between the results of the current group of acceleration units and the results of the operations of another group of acceleration units. For example, if acceleration card MC0 stores data (0,0), acceleration card MC1 stores data (1,2), acceleration card MC2 stores data (3,1), and acceleration card MC3 stores data (2,4), the data in acceleration card MC0 can be transferred to acceleration card MC1 for computation to obtain the first set of results (1,2); synchronously or asynchronously, the data in acceleration card MC2 can be transferred to acceleration card MC3 for computation to obtain the second set of results (5,5). Next, the first set of results and the second set of results are then processed to obtain the final reduction result (6,7).

[0115] Similarly, in addition to performing reduction operations within an acceleration unit, reduction operations can also be performed within an acceleration component or acceleration device. It should be understood that an acceleration device can also be considered as an acceleration component connected end-to-end.

[0116] When performing reduction operations in an acceleration component or acceleration device, it may include: performing a first reduction operation on the data in the acceleration card of the same acceleration unit to obtain a first reduction result in each acceleration unit; and performing a second reduction operation on the first reduction results in multiple acceleration units to obtain a second reduction result.

[0117] Taking reduction summation as an example, the first step mentioned above has already been described. For an acceleration device that includes multiple acceleration units, local reduction operations can be performed in each acceleration unit first. After the reduction operation in each acceleration unit is completed, the acceleration card in the same acceleration unit will obtain the result of the local reduction operation, which is referred to as the first reduction result.

[0118] Next, the first reduction results from all acceleration units can be passed between adjacent acceleration units and added together. Thus, similar to performing a reduction operation within a single acceleration unit, the first acceleration unit passes the first reduction result to the second acceleration unit, where addition operations are performed on the acceleration cards. Then, the result is passed and added again. After the final addition operation, the final result is passed to each acceleration unit.

[0119] It should be noted that, since the acceleration components mentioned above are not necessarily connected end-to-end, the final result can be transmitted in reverse order to each acceleration unit, rather than in a loop as when the acceleration units are connected end-to-end. This disclosure does not specifically limit how the final result is transmitted.

[0120] Furthermore, according to one embodiment of this disclosure, the acceleration device may also be configured to perform reduction operations, including: performing a first reduction operation on data in the acceleration card of the same acceleration unit to obtain a first reduction result; performing an intermediate reduction operation on the first reduction result in multiple acceleration units of the same acceleration component to obtain an intermediate reduction result; and performing a second reduction operation on the intermediate reduction result in multiple acceleration components to obtain a second reduction result.

[0121] In this implementation, reduction operations can be performed in the same acceleration unit first, as described above, and will not be repeated here.

[0122] Next, reduction operations can be performed in each acceleration component, so that each accelerator card in each acceleration component obtains the local reduction result of its own acceleration component. Then, reduction operations are performed in multiple acceleration components, so that each accelerator card obtains the global reduction result of the acceleration device.

[0123] The above describes various implementation methods of the accelerator card system. The following will introduce more specific methods for communication based on this accelerator card system.

[0124] In this disclosure, the communication task queue and the communication task execution queue are separate. Distinguishing between the communication task queue and the communication task execution queue allows users to perform fault tolerance or retransmission operations without being aware of the task.

[0125] In this disclosure, communication tasks can be distributed asynchronously to any accelerator card in the accelerator card system, forming a communication task queue. The communication task queue and the communication task execution queue can reside on different accelerator cards. Communication tasks within the same communication queue will be executed sequentially. The execution of these communication tasks can be performed by another accelerator card, thus keeping the communication task queue and the communication task execution queue on different accelerator cards.

[0126] Preferably, for communication tasks with large amounts of data, they can be divided into multiple communication tasks for execution. Communication tasks on the same queue will be executed serially according to the order in which they are issued, while tasks on different queues can be executed concurrently. Thus, dividing a total communication task into multiple sub-communication tasks and executing them in parallel will greatly improve the execution efficiency of the communication task. For communication tasks such as Allreduce, data can be transferred from one accelerator card to another through different communication paths. Therefore, when a total communication task is divided into multiple parallel sub-communication tasks, these sub-communication tasks can be executed through different communication paths.

[0127] by Figures 5a-5c Taking the accelerator card connection method shown as an example, when data needs to be transmitted from accelerator card MC1 to accelerator card MC3, the data can be transmitted through the following multiple communication paths:

[0128] 1. For example Figure 5b As shown, data can be transmitted from port 1 of accelerator card MC1 to port 1 of accelerator card MC0, then from port 2 of accelerator card MC0 to port 2 of accelerator card MC2, and finally from port 1 of accelerator card MC2 to port 1 of accelerator card MC3.

[0129] 2. For example Figure 5b As shown, data can also be transmitted directly from port 2 of accelerator card MC1 to port 2 of accelerator card MC3.

[0130] 3. For example Figure 5c As shown, data can be transmitted from port 4 of accelerator card MC1 to port 4 of accelerator card MC2, then from port 5 of accelerator card MC2 to port 5 of accelerator card MC0, and finally from port 4 of accelerator card MC0 to port 4 of accelerator card MC3.

[0131] 4. For example Figure 5c As shown, data can also be transmitted directly from port 5 of accelerator card MC1 to port 5 of accelerator card MC3.

[0132] Therefore, in the publicly available technical solutions, communication between two accelerator cards can be achieved through multiple communication paths; in other words, communication between two accelerator cards can be achieved through different topologies. Thus, when a total communication task is divided into multiple sub-communication tasks, each sub-communication task can be executed through different communication paths.

[0133] It is important to understand that the above combination Figures 5a-5c The description is just a simple example. When the accelerator card system contains more accelerator cards, the communication paths will be more complex and diverse, which can also divide the total communication task into more sub-communication tasks.

[0134] In the communication task queue, multiple status flags can be set. These flags can monitor the execution status of communication tasks and control the execution status of other communication tasks. The execution of a communication task changes the status flags, and changes in the status flags will correspondingly affect the execution of other communication tasks. These status flags will be described in more detail below.

[0135] According to one embodiment of this disclosure, although the communication task queue can be loaded onto any accelerator card, it is preferable to load the communication task queue onto a low-load accelerator card. It should be understood that when multiple accelerator cards participate in computation and communication, the load on each accelerator card may differ; therefore, a low-load accelerator card can be preferably selected to carry the communication task queue. The accelerator card can receive communication tasks from the host or other accelerator cards, form a queue, and control the execution of each communication task in the queue. This approach helps to fully utilize accelerator card resources and improve the overall operating efficiency of the system.

[0136] Figure 29a A flowchart illustrating a method for performing a communication task according to one embodiment of the present disclosure is shown; Figure 29b A schematic diagram of a task dispatch queue and a communication task execution queue according to one embodiment of the present disclosure is shown.

[0137] like Figure 29a As shown, according to one embodiment of the present disclosure, the method of the present disclosure further includes: in operation S2910, dividing a total communication task in the communication task queue into multiple sub-communication tasks, each sub-communication task being in a different communication task execution queue; in operation S2920, executing the multiple sub-communication tasks in parallel through different communication paths; and in operation S2930, in response to the completion of the execution of the sub-communication tasks, causing the total communication task to be completed.

[0138] The following is combined Figure 29b Let me describe the above method in detail.

[0139] exist Figure 29b The system includes two types of queues: a communication task queue (LQ) and a communication task execution queue (PQ). The communication task queue can receive multiple general communication tasks, such as general communication tasks A, B, and C. These general communication tasks A, B, and C are sequentially grouped together when they enter the communication task queue (LQ), with the execution order being A, B, and C. That is, while communication task A is executing, communication tasks B and C must wait; communication task B can only execute after communication task A has finished; and communication task C must wait for communication task B to finish. This task execution method cannot fully utilize the system's parallel processing resources, especially when the execution time of a particular communication task is particularly long or the amount of communication data is particularly large. This can significantly block the execution of other communication tasks, thus impacting system performance.

[0140] The communication tasks in the communication task queue LQ can be considered as the total communication task. This total communication task can then be divided into multiple parallel sub-communication tasks, which are then placed in the communication task execution queue PQ for execution. When a total communication task is divided into multiple sub-communication tasks for parallel execution, the execution efficiency of the task can be significantly improved.

[0141] In this disclosure, taking the overall communication task B as an example, the overall communication task B can be divided into multiple sub-communication tasks b1, b2, etc. Here, we will use two sub-communication tasks b1 and b2 as an example for illustration. It should be noted that the number of sub-communication tasks can be other than the number required, which can be determined based on the topology of the accelerator card system. For example, if there are more communication paths from one accelerator card to another, then the overall communication task can be divided into more sub-communication tasks; conversely, if there are fewer paths, then the overall communication task can be divided into fewer sub-communication tasks. Or, if the amount of data involved in the overall communication task is larger, then it can be divided into more sub-communication tasks.

[0142] After dividing the total communication task B into sub-communication tasks b1 and b2, and placing these sub-communication tasks into different communication task execution queues PQ1 and PQ2 respectively, the two sub-communication tasks b1 and b2 can be executed in parallel in the communication task execution queues PQ1 and PQ2.

[0143] The execution of the main communication task B and the sub-communication tasks must meet the following rules: 1. When the main communication task B has not yet started, the sub-communication tasks b1 and b2 should also be in an unstarted state; 2. When the main communication task B starts, the sub-communication tasks b1 and b2 should also start; 3. Other tasks in the communication task queue LQ that are after task B (such as C) need to wait for task B to finish before they can be executed; 4. When all sub-communication tasks b1 and b2 have finished, the main communication task B should also be completed.

[0144] Figure 30a A flowchart illustrating the division of a total communication task in a communication task queue into multiple sub-communication tasks according to one embodiment of the present disclosure is shown.

[0145] Therefore, according to one embodiment of this disclosure, dividing a total communication task in a communication task queue into multiple sub-communication tasks S2910 includes: in operation S29110, setting a first write flag that allows the total communication task to start execution; in operation S29120, setting a first wait flag that prevents the sub-communication tasks from starting execution; and in operation S29130, when the first write flag is not executed, executing the first wait flag to prevent the sub-communication tasks from starting execution.

[0146] Figure 30b A schematic diagram illustrating the insertion of an identifier into a queue according to one embodiment of the present disclosure is shown below. Figure 30b To describe in detail Figure 30a Specific implementation methods.

[0147] First, to control the execution of communication task B, a write flag needs to be set. In other words, a write flag, denoted as F0, needs to be inserted before the communication task to be executed. Communication task B will only begin execution when the write flag F0 is reached, or when F0 is changed to allow the execution of subsequent tasks. If the write flag F0 is not reached, the corresponding communication task will not begin execution. This write flag can be inserted using an atomic operation. An atomic operation is an operation that will not be interrupted by the thread scheduling mechanism; once started, it runs continuously until completion without any context switching.

[0148] Accordingly, a wait flag f0 can be inserted before each sub-communication task. This wait flag indicates that the execution of any sub-communication task following this flag is prohibited. It is important to understand that... Figure 30b Although the first write flag F0 and the wait flag f0 are named differently, they both point to the same flag to detect whether that same flag has changed. It is also important to understand that... Figure 30b The insertion positions of the first write flag F0 and the wait flag f0 shown are for ease of understanding only and are not necessarily as they appear. Figure 30b That way, it can be inserted into the sub-communication task.

[0149] According to one embodiment of this disclosure, executing the plurality of sub-communication tasks in parallel includes: turning off the first waiting flag in response to the first write flag being executed, thereby executing the plurality of sub-communication tasks in parallel.

[0150] The write flag F0 and the wait flag f0 before the main communication task are related. Only when the write flag F0 allows the execution of the subsequent main communication task will the wait flag f0 end and the corresponding sub-communication task start running. If the write flag F0 does not allow the execution of the subsequent main communication task, the wait flag f0 will also cause the execution of the sub-communication task to be in a waiting state.

[0151] Figure 31 A schematic diagram of a queue according to another embodiment of this disclosure is shown.

[0152] According to one embodiment of this disclosure, a second waiting flag can be set to prohibit other communication tasks after the execution of the total communication task.

[0153] like Figure 31 As shown, a second wait flag can be inserted after the first write flag. When the execution reaches the second wait flag, it indicates that other total communication tasks after the current total communication task need to be in a waiting state. Other total communication tasks cannot start execution before the current total communication task is completed.

[0154] As can be seen from the above description, when the first write flag F0 is executed, the main communication task B corresponding to the first write flag F0 begins to execute. That is, the sub-communication tasks b1 and b2 of the main communication task B end the waiting state and begin to execute. After that, when the second waiting flag F1 is executed, the other tasks after the main communication task B enter the waiting state and do not execute when the main communication task B is executed.

[0155] Figure 32 A schematic diagram showing the modification of the second waiting identifier according to one embodiment of the present disclosure is illustrated.

[0156] According to one embodiment of this disclosure, whenever a sub-communication task is completed, the second waiting flag F1 is modified until all sub-communication tasks are completed; and in response to the completion of all sub-communication tasks, the second waiting flag F1 is modified to a waiting end flag, thereby causing the total communication task to be completed.

[0157] Next, as Figure 32As shown, each sub-communication task b1 and b2 is executed in the execution queue PQ. Each time a sub-communication task b1 or b2 completes, the second waiting flag F1 can be modified accordingly, for example, by incrementing it by one. The number of times the second waiting flag F1 is modified is the same as the number of times the sub-communication task is completed. Therefore, the second waiting flag F1 can initially be set with a target value. As sub-communication tasks b1 or b2 complete, the second waiting flag F1 gradually approaches this target value. When the second waiting flag F1 reaches the preset target value, it means that all sub-communication tasks b1 and b2 have been completed. It should be understood that there are many ways to modify the second waiting flag F1, and it is not limited to simply "incrementing" as described above. For example, it can be decremented by one each time it is modified, until the second waiting flag F1 is less than a predetermined threshold. This disclosure does not impose any limitations on how the second waiting flag is modified.

[0158] The "second waiting flag F1 has reached the target value" mentioned above can also be understood as a waiting end flag, which means that the current total communication task B has been completed and other tasks can be started.

[0159] It is important to understand that, in Figure 30b , Figure 31 and Figure 32 Although the identifier f0 is displayed in the sub-communication task, this is only for ease of understanding. The identifier f0 is actually in the communication task queue to facilitate the monitoring of the execution of each sub-communication task.

[0160] When dividing the total communication task into multiple sub-communication tasks, there are several ways to divide it. The total communication task can be randomly divided into multiple sub-communication tasks; the total communication task can be divided into a fixed number of sub-communication tasks; the total communication task can be divided into a number of sub-communication tasks corresponding to the number of processors based on the number of communication paths, and so on.

[0161] According to a preferred embodiment of this disclosure, a total communication task in a task queue can be divided into multiple sub-communication tasks with equivalent execution time.

[0162] The aforementioned equivalent execution time does not mean that the size of each sub-communication task is the same. For example, if the communication speed of each port is 40Gbps, then for 160G of data, if a single communication path is used to transmit this data, it would theoretically take 4 seconds to complete the data transmission. Therefore, the 160G of data can be split into multiple sub-communication tasks, such as 2, 3, or 4. When split into 4 sub-communication tasks, 4 communication paths can be used to transmit data in parallel, theoretically requiring only 1 second to complete the 160G data transmission, reducing the communication time to only 25% of the original communication time. Obviously, this will help shorten the execution time of the total communication task.

[0163] Furthermore, the transmission speeds of multiple communication paths are not necessarily the same. Therefore, the division of communication tasks can be adjusted by considering the speed of each communication path to regulate the data size of each sub-task. For example, if the transmission speeds of four communication paths are 16Gbps, 18Gbps, 22Gbps, and 24Gbps, then 160G of data can be divided into four sub-tasks: 32G, 36G, 44G, and 48G. Each communication path takes 2 seconds to complete the data transmission, thus ensuring that each communication path can complete its sub-tasks simultaneously or nearly simultaneously.

[0164] Different communication paths can correspond to different topologies, such as Figures 5a-5c As described.

[0165] According to one embodiment of this disclosure, the total communication task can be divided into multiple sub-communication tasks in response to the total data volume exceeding a specific threshold. It should be understood that dividing the total communication task into multiple sub-communication tasks also requires consideration of the total data volume involved in each task. If the total data volume involved in a certain total communication task is small, then it is unnecessary to further divide the total communication task.

[0166] According to one embodiment of this disclosure, the method further includes: rerunning the faulty sub-communication task in response to an error occurring in one or more sub-communication tasks.

[0167] When multiple sub-communication tasks are executed in the communication task execution queue (PQ), errors may occur, such as transmission failures due to data lines, data throughput errors, and packet loss during data transmission. In traditional solutions, if the overall communication task is not divided into multiple sub-communication tasks, if an error occurs during task execution, the entire overall communication task needs to be re-executed, which severely wastes processing power and degrades the overall system performance.

[0168] In the scheme disclosed herein, since multiple sub-communication tasks reside in different execution queues, these queues operate independently without interfering with each other. Therefore, even if an error occurs during the execution of one sub-communication task, it will not affect the execution of other sub-communication tasks. Thus, if an error occurs in the execution of a sub-communication task, only the erroneous sub-communication task needs to be re-executed, without having to rerun all sub-communication tasks or the entire communication task. While the erroneous sub-communication task is running, other queues can be idle or can execute other sub-communication tasks simultaneously. Therefore, dividing a total communication task into multiple parallel sub-communication tasks in this disclosure improves the utilization of system processing resources and increases processing efficiency.

[0169] According to one embodiment of the present disclosure, in response to an error occurring in one or more sub-communication tasks, the erroneous sub-communication task is further divided into multiple sub-tasks for parallel execution.

[0170] When a sub-communication task encounters an error and needs to be re-executed, the erroneous sub-communication task can be added as a new main communication task to the communication task allocation queue (LQ). This sub-communication task is then further divided into multiple subtasks, and the erroneous sub-communication task is re-executed once in multiple parallel execution queues (PQ). Further dividing the erroneous sub-communication task into multiple subtasks for re-execution significantly improves system efficiency, greatly reducing the time and processing resources required to correct errors even if they occur.

[0171] Therefore, based on the above-mentioned scheme of this disclosure, when the amount of communication data is large, a communication task can be split into any number of sub-tasks and distributed to multiple different communication task execution queues for concurrent execution, thereby improving bandwidth utilization.

[0172] Furthermore, different algorithms can be selected for communication based on the maximum bandwidth of the specific topological connection between chips, further optimizing the communication efficiency for large data volumes. Thus, based on the physical topological connection between accelerator cards, a flexible communication logical topology can be constructed for data communication, further improving communication efficiency.

[0173] It can also independently monitor the execution of sub-communication tasks in each communication task execution queue. If an error occurs in a sub-communication task, only that sub-communication task needs to be resent or re-executed, without having to re-execute the entire communication task. This allows for partial retransmission of communication tasks without the user's awareness, reducing the cost of fault tolerance and retransmission in communication tasks and improving overall communication efficiency.

[0174] The accelerator card that issues tasks can be any accelerator card in the accelerator card system. According to one embodiment of this disclosure, since the communication task queue only involves waiting and writing operations, and the actual communication tasks are executed in the communication task execution queue, the communication task queue can correspond to any one of the accelerator card systems. This helps reduce the probability of programming errors by developers, and allows the selection of accelerator cards with smaller task loads to perform the waiting and writing control of the communication task queue.

[0175] This disclosure also provides an electronic device, including: one or more processors; and a memory storing computer-executable instructions that, when executed by the one or more processors, cause the electronic device to perform the method described above.

[0176] This disclosure also provides a computer-readable storage medium including computer-executable instructions that, when executed by one or more processors, perform the method described above.

[0177] The technical solution disclosed herein can be applied to the field of artificial intelligence, and can be implemented as or in an artificial intelligence chip. This chip can exist independently or be included in a computing device.

[0178] Figure 26 The figure shows a schematic diagram of the combined processing device structure in one embodiment of this disclosure.

[0179] Figure 26 A combined processing device 2600 is shown, which includes the aforementioned computing device 2602, interconnection interface 2604, and other processing devices 2606. The computing device according to this disclosure interacts with the other processing devices to jointly perform user-specified operations. Figure 26 This is a schematic diagram of the combined processing device.

[0180] Other processing devices include one or more processor types such as central processing unit (CPU), graphics processing unit (GPU), and neural network processor. There is no limit to the number of processors included in other processing devices. These other processing devices serve as interfaces between the machine learning computing device and external data and control, including data transfer and basic control such as starting and stopping the machine learning computing device. Other processing devices can also collaborate with the machine learning computing device to complete computational tasks.

[0181] Interconnect interfaces are used to transfer data and control commands between computing devices (including, for example, machine learning computing devices) and other processing devices. The computing device can obtain necessary input data from other processing devices and write it to on-chip storage; it can obtain control commands from other processing devices and write them to on-chip control caches; and it can read data from the computing device's storage modules and transmit it to other processing devices.

[0182] Optionally, the structure may further include a storage device 2608, which is connected to both the computing device and the other processing device. The storage device is used to store data in the computing device and the other processing device, and is particularly suitable for data that cannot be fully stored in the internal storage of the computing device or other processing device, requiring computation.

[0183] This combined processing device can serve as a System-on-a-Chip (SoC) for devices such as mobile phones, robots, drones, and video surveillance equipment, effectively reducing the core area of ​​the control unit, increasing processing speed, and reducing overall power consumption. In this case, the interconnect interface of the combined processing device is connected to certain components of the device, such as cameras, monitors, mice, keyboards, network cards, and Wi-Fi interfaces.

[0184] In some embodiments, this disclosure also discloses a chip packaging structure that includes the aforementioned chip.

[0185] In some embodiments, this disclosure also discloses a circuit board that includes the above-described chip packaging structure. See also... Figure 27 The present invention provides an exemplary board, which, in addition to the chip 2702, may also include other supporting components, including but not limited to: a storage device 2704, an interface device 2706, and a controller 2708.

[0186] The storage device is connected to the chip within the chip package structure via a bus and is used to store data. The storage device may include multiple sets of storage cells 2710. Each set of storage cells is connected to the chip via a bus. It can be understood that each set of storage cells may be DDR SDRAM (Double Data Rate SDRAM).

[0187] DDR can double the speed of SDRAM without increasing the clock frequency. DDR allows data to be read on both the rising and falling edges of the clock pulse. DDR is twice as fast as standard SDRAM. In one embodiment, the storage device may include four groups of storage cells. Each group of storage cells may include multiple DDR4 chips. In one embodiment, the chip may internally include four 72-bit DDR4 controllers, of which 64 bits are used for data transmission and 8 bits are used for ECC verification. In one embodiment, each group of storage cells includes multiple Double Data Rate Synchronous Dynamic Random Access Memory (DRAM) units connected in parallel. DDR can transmit data twice per clock cycle. A controller for controlling DDR is provided in the chip for controlling data transmission and data storage in each storage cell.

[0188] The interface device is electrically connected to the chip within the chip package structure. The interface device is used to realize data transmission between the chip and an external device 2712 (e.g., a server or computer). For example, in one embodiment, the interface device can be a standard PCIe interface. For instance, data to be processed is transferred from the server to the chip via a standard PCIe interface, realizing data transfer. In another embodiment, the interface device can also be other interfaces; this disclosure does not limit the specific form of the other interfaces mentioned above, as long as the interface unit can realize the switching function. Furthermore, the calculation results of the chip are still transmitted back to the external device (e.g., the server) by the interface device.

[0189] The controller is electrically connected to the chip. The controller monitors the state of the chip. Specifically, the chip and the controller can be electrically connected via an SPI interface. The controller may include a microcontroller (MCU). The chip may include multiple processing chips, multiple processing cores, or multiple processing circuits, capable of driving multiple loads. Therefore, the chip can operate in different states, such as high load and low load. The controller can regulate the operating states of multiple processing chips, multiple processing cores, and / or multiple processing circuits within the chip.

[0190] In some embodiments, this disclosure also discloses an electronic device or apparatus that includes the aforementioned board.

[0191] Electronic devices or apparatuses include data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, mobile phones, dashcams, navigators, sensors, cameras, servers, cloud servers, cameras, camcorders, projectors, watches, headphones, mobile storage, wearable devices, vehicles, home appliances, and / or medical devices.

[0192] The means of transportation include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical equipment includes MRI scanners, ultrasound scanners and / or electrocardiographs.

[0193] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0194] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0195] In the several embodiments provided in this disclosure, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, optical, acoustic, magnetic, or other forms.

[0196] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0197] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0198] If the integrated unit is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, when the technical solution of this disclosure can be embodied in the form of a software product, the computer software product is stored in a storage device and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage device includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0199] The embodiments of this disclosure have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this disclosure. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of this disclosure. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this disclosure. Therefore, the content of this specification should not be construed as a limitation of this disclosure.

Claims

1. A method for performing communication tasks in an accelerator card system, wherein, The accelerator card system includes multiple accelerator cards capable of communicating with each other, wherein one accelerator card can communicate with another accelerator card through a communication path; the method includes: A communication task queue is established, which includes communication tasks and status identifiers for monitoring the execution status of the communication tasks. Establish a communication task execution queue for executing communication tasks between accelerator cards via communication paths; In response to the execution of the communication task, the status identifier is changed to monitor the execution status of the communication task.

2. The method according to claim 1, wherein, The accelerator card system includes an acceleration unit, which comprises M accelerator cards. Each accelerator card includes an internal port, and each accelerator card can communicatively connect to other accelerator cards in the same unit via its internal port. The M accelerator cards in this unit are logically formed into an L*N accelerator card matrix, where L and N are integers not less than 2.

3. The method according to claim 1, wherein, The accelerator card system includes multiple acceleration units, each acceleration unit includes M accelerator cards, each accelerator card includes an internal port, and each accelerator card can communicatively connect to other accelerator cards in the same unit through the internal port. The M accelerator cards of this unit are logically formed into an accelerator card matrix of size L*N, where L and N are integers not less than 2. At least one of the M accelerator cards of this unit includes an external port, and the accelerator units are communicatively connected to each other through the external port.

4. The method according to claim 1, wherein, The accelerator card system includes multiple acceleration units, each acceleration unit includes M accelerator cards, each accelerator card includes an internal port, and each accelerator card can communicatively connect to other accelerator cards in the same unit through the internal port. The M accelerator cards are logically arranged into an L*N matrix, where L and N are integers not less than 2. At least one of the M accelerator cards includes an external port. The accelerator units are communicatively connected to each other through the external port. The multiple accelerator units are logically arranged in a multi-layer structure, with each layer including one accelerator unit. The accelerator card of each accelerator unit is communicatively connected to the accelerator card of the other unit through the external port. The last accelerator unit is communicatively connected to the first accelerator unit, thus forming a ring structure with the multiple accelerator units connected end to end.

5. The method according to claim 1, wherein, The accelerator card system includes a first acceleration component, a second acceleration component, a third acceleration component, and a fourth acceleration component. Each acceleration component includes a first acceleration unit and a second acceleration unit. Each acceleration component is communicatively connected to one of the first and second acceleration units of other acceleration components through one of the first and second acceleration units. Each acceleration unit includes M accelerator cards for this unit. Each accelerator card for this unit includes an internal port. Each accelerator card for this unit is communicatively connected to other accelerator cards for this unit through its internal port. The M accelerator cards in this unit are logically formed into an accelerator card matrix of size L*N, where L and N are integers not less than 2. At least one of the M accelerator cards in this unit includes an external port, and the accelerator units can be communicatively connected through an idle external port. The second pair of acceleration cards of the first acceleration unit of the first acceleration component is connected to the second pair of acceleration cards of the second acceleration unit of the second acceleration component; The first pair of acceleration cards of the second acceleration unit of the second acceleration component is connected to the first pair of acceleration cards of the first acceleration unit of the third acceleration component; The second pair of acceleration cards of the second acceleration unit of the third acceleration component is connected to the second pair of acceleration cards of the first acceleration unit of the fourth acceleration component; The first pair of acceleration cards of the first acceleration unit of the fourth acceleration component is connected to the first pair of acceleration cards of the second acceleration unit of the first acceleration component.

6. The method according to any one of claims 2-5, wherein, The M accelerator cards in this unit are logically arranged into an accelerator card matrix of 2*2, 3*3, or 4*4.

7. The method according to any one of claims 2-6, wherein, Each local accelerator card can be communicatively connected to at least one other local accelerator card via two paths.

8. The method according to any one of claims 2-7, wherein, The accelerator cards in the four corner units of the accelerator card matrix are communicatively connected via two communication paths.

9. The method according to any one of claims 2-8, wherein, When the acceleration unit includes four local acceleration cards, each local acceleration card includes six ports, and four of the ports of each local acceleration card are internal ports for communicatively connecting with the other three local acceleration cards; the remaining two ports of at least one local acceleration card are external ports for communicatively connecting with external acceleration cards.

10. The method according to any one of claims 2-9, wherein, The internal and external ports are SerDes ports.

11. The method according to any one of claims 3-10, wherein, The multiple acceleration units have the same structure.

12. The method according to any one of claims 3-11, wherein, The acceleration unit is configured to perform reduction operations on the data in the acceleration card of the acceleration unit to obtain the reduction result.

13. The method according to claim 12, wherein, The plurality of acceleration units are configured to perform reduction operations, including: Perform a first reduction operation on the data in the accelerator card of the same accelerator unit to obtain the first reduction result in each accelerator unit; The first reduction result from multiple acceleration units is subjected to a second reduction operation to obtain the second reduction result.

14. The method according to any one of claims 1-13, wherein, The communication task queue and the communication task execution queue are located on different accelerator cards.

15. The method according to claims 1-14, wherein, The communication task queue is located on any of the plurality of accelerator cards, preferably on a low-load accelerator card.

16. The method according to any one of claims 1-15, further comprising: A total communication task in the communication task queue is divided into multiple sub-communication tasks, and each sub-communication task is in a different communication task execution queue; The multiple sub-communication tasks are executed in parallel through different communication paths; In response to the completion of the sub-communication task, the overall communication task is completed.

17. The method of claim 16, further comprising: Set a first write flag to allow the start of the total communication task; Set a first waiting flag to prevent the sub-communication task from starting execution; When the first write flag is not executed, the first wait flag is executed to prevent the sub-communication task from starting.

18. The method according to claim 17, wherein, Executing the multiple sub-communication tasks in parallel includes: In response to the first write flag being executed, the first wait flag is turned off, thereby executing the multiple sub-communication tasks in parallel.

19. The method according to any one of claims 16-18, further comprising: A second waiting flag is set to prevent other communication tasks from being executed after the main communication task.

20. The method of claim 19, further comprising: Each time a sub-communication task is completed, the second waiting flag is modified until all sub-communication tasks are completed. In response to the completion of all sub-communication tasks, the second waiting flag is modified to the waiting end flag, thereby completing the execution of the total communication task.

21. The method according to any one of claims 16-20, wherein a total communication task in the communication task queue is divided into multiple sub-communication tasks with equivalent execution time in different communication paths.

22. The method according to any one of claims 16-21, wherein, In response to the total amount of data in the total communication task exceeding a certain threshold, the total communication task is divided into multiple sub-communication tasks.

23. The method according to any one of claims 16-22, further comprising: In response to an error in one or more sub-communication tasks, re-execute the sub-communication task that caused the error.

24. The method according to any one of claims 16-23, wherein in response to an error in one or more sub-communication tasks, the erroneous sub-communication task is further divided into multiple sub-communication tasks for parallel execution.

25. An electronic device, comprising: One or more processors; as well as A memory storing computer-executable instructions that, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1-24.

26. A computer-readable storage medium comprising computer-executable instructions that, when executed by one or more processors, perform the method as described in any one of claims 1-24.

Images