Consistent interconnection control devices, methods, electronic devices, products and computing systems

Abstract

This invention discloses a consistency interconnect control device, method, electronic device, product, and computing system, relating to the field of computer technology. By providing a consistency interconnect control device, including a consistency interconnect network, a first controller, and a first connector, the consistency interconnect network includes multiple interconnected consistency interconnect nodes. Each consistency interconnect node is connected to a first card slot where an accelerator card is installed via the first connector. The first controller manages the state of the cache lines of the accelerator cards and controls the consistency interconnect network to execute consistency interconnect read / write tasks between different accelerator cards. This invention realizes a consistency interconnect system for accelerated computing clusters. By achieving cache consistency between accelerator cards, when the accelerated computing cluster performs distributed computing, cache hits can be performed first, reducing access to local memory and other accelerator card memory. This solves the problem of high communication time between different accelerators in the accelerated computing cluster and improves the efficiency of distributed computing.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more particularly to consistent interconnection control devices, methods, electronic devices, products and computing systems. Background Technology

[0002] With the development of artificial intelligence technology, the demand for computing power has increased dramatically. By connecting accelerator cards to server hosts and interconnecting these accelerator cards, large-scale accelerated computing clusters can be built to achieve distributed computing, solving the computational problems of large-scale artificial intelligence models. However, when accelerated computing clusters perform distributed computing, the communication costs between different accelerators continuously increase, and the communication time significantly impacts the efficiency of distributed computing. Summary of the Invention

[0003] This invention provides a consistent interconnection control device, method, electronic device, product, and computing system to at least solve the problem of long communication time between different accelerators in accelerated computing clusters in related technologies.

[0004] This invention provides a conformance interconnection control device, comprising: a conformance interconnection network, a first controller, and a first connector;

[0005] The consistent interconnect network includes multiple interconnected consistent interconnect nodes, each of which is connected to the first connector in a one-to-one correspondence.

[0006] The other end of the first connector is connected to the first slot, which is used to install the accelerator card;

[0007] The first controller is used to manage the state of the cache lines of the accelerator cards and to control the coherent interconnect network to perform coherent interconnect read and write tasks between different accelerator cards.

[0008] The present invention also provides a consistent interconnection control method, applied to a first controller, comprising:

[0009] The status of the accelerator card's cache lines is acquired and managed through the first connector;

[0010] The coherent interconnect network is controlled to perform coherent interconnect read / write tasks between different accelerator cards;

[0011] The consensus interconnect network includes multiple interconnected consensus interconnect nodes, each of which is connected to a first connector. The other end of the first connector is connected to a first slot, which is used to install the accelerator card.

[0012] The present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described coherent interconnection control methods when executing the computer program.

[0013] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described coherent interconnection control methods.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described coherent interconnection control methods.

[0015] This invention provides a consistent interconnect control device, comprising a consistent interconnect network, a first controller, and a first connector. The consistent interconnect network includes multiple interconnected consistent interconnect nodes, each connected to a corresponding first connector. The other end of the first connector is connected to a first slot for installing accelerator cards. The first controller manages the state of the cache lines of the accelerator cards and controls the consistent interconnect network to perform consistent interconnect read / write tasks between different accelerator cards. This invention provides a consistent interconnect system for accelerated computing clusters composed of multiple accelerators. By achieving cache consistency between accelerator cards, when the accelerated computing cluster performs distributed computing, cache hits can be performed first, reducing access to local memory and other accelerator card memory, thereby reducing the number of communication operations in distributed computing. This solves the problem of high communication time between different accelerators in the accelerated computing cluster and improves the efficiency of distributed computing. Attached Figure Description

[0016] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a connection diagram of a consistency interconnection control device provided in an embodiment of the present invention;

[0018] Figure 2 This is a schematic diagram of the structure of a consistency interconnection control device provided in an embodiment of the present invention;

[0019] Figure 3 A schematic diagram of the structure of a control module provided in an embodiment of the present invention;

[0020] Figure 4An arbitration schematic diagram of an arbitration module provided in an embodiment of the present invention;

[0021] In this configuration, 100 is a conformance interconnection control device, 101 is a conformance interconnection network, 102 is a first connector, 103 is a second connector, and 104 is a first controller. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0023] It should be noted that, in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0024] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] Here, we will first explain some key terms used in the embodiments of this invention.

[0026] Cache coherence, also known as cache consistency or inter-cache consistency, refers to the mechanism in a computer system with a hierarchical storage structure that ensures the data in the cache is identical to the data in the main memory. When one processor modifies certain data, the corresponding data in the caches of other processors must be updated or invalidated to ensure that all processors see consistent data.

[0027] Memory coherence refers to the requirement that multiple processors must follow a certain order when accessing shared memory to ensure the correct execution of a program. The memory coherence model defines the visibility and order of read and write operations in a program, ensuring that different processors see the same order of memory operations.

[0028] Current server systems face high hardware costs and consistency maintenance burdens for large-scale expansion, and only achieve consistent interconnection between hosts and between hosts and accelerator cards. When performing distributed computing tasks using an accelerated computing system composed of multiple interconnected accelerator cards, if one accelerator card needs to obtain data from another, the data must be moved from remote memory to local memory via the interconnect network before computation can begin. This results in high communication overhead and consumes significant time in data transfer, impacting the computational efficiency of artificial intelligence models.

[0029] To address this, this invention first provides a consistent interconnect control device for implementing consistent interconnection between accelerator cards. The device includes a consistent interconnection network, a first controller, and a first connector. The consistent interconnection network comprises multiple interconnected consistent interconnection nodes, each connected to a corresponding first connector. The other end of the first connector is connected to a first slot for installing an accelerator card. The first controller manages the state of the cache lines of the accelerator cards and controls the consistent interconnection network to perform consistent interconnection read / write tasks between different accelerator cards. Based on the consistent interconnection control device provided by this invention, a consistent interconnection system for an accelerated computing cluster composed of multiple accelerators can be implemented. By achieving cache consistency between accelerator cards, when the accelerated computing cluster performs distributed computing, cache hits can be performed first, reducing access to local memory and other accelerator card memory. This reduces the number of communication operations in distributed computing, solves the problem of high communication time between different accelerators in the accelerated computing cluster, and improves the efficiency of distributed computing.

[0030] Figure 1 This is a connection diagram of a consistency interconnection control device provided in an embodiment of the present invention.

[0031] The coherent interconnect control device provided in this embodiment of the invention may include: a coherent interconnect network, a first controller, and a first connector; the coherent interconnect network includes multiple interconnected coherent interconnect nodes, and each coherent interconnect node is connected to the first connector in a one-to-one correspondence; the other end of the first connector is connected to a first slot, which is used to install an accelerator card; the first controller is used to manage the state of the cache lines of the accelerator card, and to control the coherent interconnect network to perform coherent interconnect read and write tasks between different accelerator cards.

[0032] In this embodiment of the invention, the Cache Coherence Network (CCN) and the first controller can be implemented based on the logic circuitry of a programmable controller. The programmable controller can be a Field Programmable Gate Array (FPGA).

[0033] like Figure 1 As shown, a coherent interconnect network 101 can be connected to multiple first slots via multiple first connectors 102, and the first connectors 102 can specifically be connected to second connectors 103 on the accelerator card side. The first connectors 102 and 103 can be multi-channel I / O (MCIO) connectors. The first controller ( Figure 1 (Not shown) is used to control the forwarding of data packets received from the first connector 102 by the consistency interconnection network 101.

[0034] The accelerator card can be a homogeneous or heterogeneous accelerator card, and its type can be a graphics processing unit (GPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or other types of accelerator cards. The first slot is used to install the accelerator card, and the first slot can be a high-speed serial computer expansion bus (Peripheral Component Interconnect Express, PCIe) slot.

[0035] like Figure 1 As shown, the rack can contain a computing system consisting of a host, a first switch, and an accelerated computing system. The first switch can be a PCIe switch or an Ethernet switch. The uplink port of the first switch is connected to the host, and the downlink port of the first switch is connected to the first slot via a third connector. Thus, based on the first switch, interconnection between the host and the accelerator card, as well as between different accelerator cards, can be achieved.

[0036] Figure 1 This paper illustrates a 64-card computing system. The host can be a dual-processor (CPU) host node, compatible with mainstream CPU models. The first switch can consist of multiple switch controllers; four switch controllers can be configured in a 64-card cluster. Each switch controller's downlink port connects to two computing boards via a PCIe bus, and its uplink port is connected to the CPU, enabling communication between the CPU and the accelerator cards.

[0037] Each computing board integrates 16 accelerator cards. The serial (Serdes) interface of a single accelerator card is divided into two parts: one part is connected to the first switch through the first slot, and the other part is connected to the coherence interconnect network 101 through the dedicated coherence bus provided in this embodiment of the invention. This supports multiple data transmission protocols, such as PCIe and Compute Express Link (CXL) protocol. This ensures that all 16 accelerator cards on each computing board are connected to the coherence interconnect network 101. The coherence interconnect networks 101 are connected to each other via a high-speed coherence bus, achieving coherence interconnection among 64 accelerator cards. Furthermore, four computing boards are connected to the first switch (a PCIe switch) through a third connector (which can be a PCIe interface) to achieve communication with the CPU.

[0038] Figure 1 The four conformance interconnect networks 101 shown can be located in four conformance interconnect control devices 100. The conformance interconnect control devices 100 can be connected to each other via chip-to-chip connectors to form a network as shown in the figure. Figure 1 The ring topology shown. The topology composed of the consistency interconnect control device 100 and the multiple consistency interconnect control devices 100 provided in this embodiment of the invention can be extended to the consistency interconnect of 128-card and 256-card systems.

[0039] The consistency interconnect control device 100 provided in this embodiment of the invention includes a consistency interconnect network 101, a first controller 104, and a first connector 102. The consistency interconnect network 101 includes multiple interconnected consistency interconnect nodes, each corresponding to a first connector 102. The other end of the first connector 102 is connected to a first slot for installing accelerator cards. The first controller 104 manages the state of the cache lines of the accelerator cards and controls the consistency interconnect network 101 to perform consistency interconnect read / write tasks between different accelerator cards. This implements a consistency interconnect system for an accelerated computing cluster composed of multiple accelerators. By achieving cache consistency between accelerator cards, when the accelerated computing cluster performs distributed computing, it can first hit the cache, reducing access to local memory and other accelerator card memory, thereby reducing the number of communication times in distributed computing. This solves the problem of high communication time between different accelerators in the accelerated computing cluster and improves the efficiency of distributed computing.

[0040] Figure 2 This is a schematic diagram of a consistency interconnection control device provided in an embodiment of the present invention.

[0041] Based on the above embodiments, the internal structure of the consistency interconnection control device 100 will be further described in this embodiment.

[0042] As described in the above embodiments, the consistency interconnect network 101 and the first controller can be implemented based on the logic circuit of a programmable controller. Therefore, the logic circuit of the programmable controller can be divided into multiple logic regions as consistency interconnect nodes (e.g., ...). Figure 2 As shown in 00, 01, ..., 15), each consensus interconnect node corresponds to a first connector 102, that is, to an accelerator card. The consensus interconnect nodes are interconnected to form a consensus interconnect network 101.

[0043] The topology of the Consistency Interconnect Network 101 determines the physical layout and connections between the consistency interconnect nodes and channels in the network, and determines the message hop count, the distance between nodes, network latency, etc. Considering that too many ports will result in low bandwidth for a single port, while reducing the number of ports will increase the message hop count in the Consistency Interconnect Network 101, in this embodiment of the invention, each consistency interconnect node can be assigned four ports for connecting to other consistency interconnect nodes, forming a network as follows: Figure 2 The two-dimensional network topology (2D mesh) shown allows each consistent interconnect node on the bus to be observed by all devices on the bus, achieving full interconnection between nodes (accelerator cards).

[0044] In embodiments of the present invention, each consensus interconnection network 101 may include, for example: Figure 2 The 16 coherent interconnect nodes shown correspond to 16 first connectors 102, enabling coherent interconnection of 16 accelerator cards. The coherent interconnect nodes can be designed to be compatible with different types of accelerator cards and interconnected through a unified interface.

[0045] In this embodiment of the invention, the consistency interconnection control device 100 may include two modules for control: a consistency management module and a node management module. The consistency management module operates within the first controller 104.

[0046] like Figure 2 As shown, the consistency management module can include a control module, an input module, and an output module. The control module is used to maintain the primary directory. The consistency management module can also include a caching component, the size of which can be set as needed, for example, 5MB.

[0047] In this embodiment of the invention, the first controller 104 manages the status of the cache lines of the accelerator card, which may include: the first controller 104 updating the first directory according to the status of the cache lines; the first directory includes a list of sharers corresponding to the cache lines, a memory address corresponding to the cache lines, and the cache line status of the cache lines.

[0048] In this embodiment of the invention, the first directory may include the status of cache lines in all accelerator cards managed by the first controller 104. The number of entries in the first directory can be the ratio of the number of accelerator cards managed by the first controller 104 multiplied by the memory size of a single accelerator card, to the size of a single cache line. For example, for 16 accelerator cards, the corresponding entry number N = 16 * single accelerator memory / size of (cache line size).

[0049] Table 1 shows the storage information structure of a single entry in a first directory provided by an embodiment of the present invention.

[0050] Table 1

[0051]

[0052] As shown in Table 1, the sharer list indicates which accelerator cards store the information of the cache line. There are a total of 16 accelerator cards, so the sharer list can include 16 bits of information, with each bit corresponding to one accelerator card. If 0 represents a non-sharer and 1 represents a sharer, assuming that the entry is the entry corresponding to a cache line in accelerator card 0, it can be said that the sharers of the cache line are accelerator cards 2 and 4, and the caches of accelerator cards 2 and 4 contain data consistent with the cache line.

[0053] The memory address indicates which accelerator card's memory this cache line belongs to, totaling... Bit information.

[0054] In this embodiment of the invention, the state of the cache line can be one of the following: modified state, exclusive state, shared state, or invalid state.

[0055] The modified status indicates that the data in the cache line has been modified by the current accelerator card, and these modifications have not yet been written back to main memory. Therefore, the data in this cache line is inconsistent with the data in main memory and also inconsistent with the data in the sharer. That is, there is the latest copy in this cache line, but no valid copy of this cache line exists in other caches.

[0056] Exclusive state means that the data in this cache line is exclusively owned by the current accelerator card. Other accelerator cards do not have a copy of this data in their caches. The data in this state is consistent with the data in main memory. The current accelerator card can perform read and write operations on this cache line without notifying other accelerator cards.

[0057] The shared state indicates that the data in the cache line may be shared by multiple accelerator cards. That is, the data may also exist in the cache of other accelerator cards. In this state, the data in the cache line is consistent with the data in the main memory.

[0058] An invalid state indicates that the data in the cache line is invalid and cannot be used by the current accelerator card. This usually occurs when the data has been modified by other accelerator cards and the cache line of the current accelerator card needs to be updated.

[0059] State transitions are used to ensure cache consistency between different accelerator cards. Below are some examples of state transitions.

[0060] From invalid state to shared or exclusive state: When an accelerator card reads a data block from main memory, if the data block is clean (unmodified) in main memory and there is no copy of the data in the caches of other accelerator cards, the cache line state of the data block is set to exclusive. If a copy of the data already exists in the caches of other accelerator cards, the cache line state of the data block is set to shared.

[0061] From shared state to exclusive or modified state: When an accelerator card performs a write operation on a cache line that is in a shared state, the cache line's state changes from shared to modified. Simultaneously, the data block in the caches of other accelerator cards is marked as invalid.

[0062] From Exclusive State to Modified State: When an accelerator card performs a write operation on a cache line that is in an exclusive state, the state of that cache line will change from exclusive state to modified state.

[0063] From Modified State to Shared or Invalid State: When an accelerator card reads a cache line that is in the modified state, the state of that cache line remains unchanged. When an accelerator card writes to a cache line that is in the modified state, the state of that cache line remains unchanged. When an accelerator card writes back a cache line that is in the modified state, the state of that cache line changes from modified to shared or invalid, depending on the cache states of other accelerator cards.

[0064] The above four cache line states can be represented by 2 bits of information, such as 00, 01, 10, and 11.

[0065] Figure 3 This is a schematic diagram of the structure of a control module provided in an embodiment of the present invention.

[0066] In this embodiment of the invention, the first controller 104 controls the consistency interconnect network 101 to perform consistency interconnect read / write tasks between different accelerator cards, which may include: the first controller 104 determining the sending accelerator card and the receiving accelerator card of the first data packet received by the consistency interconnect control device 100, determining the target forwarding path between the input node and the output node in the consistency interconnect network 101, and controlling the consistency interconnect network 101 to forward the first data packet according to the target forwarding path, so as to send it to the receiving accelerator card through the output node and its corresponding first connector 102; wherein, the input node is the consistency interconnect node corresponding to the sending accelerator card, and the output node is the consistency interconnect node corresponding to the receiving accelerator card.

[0067] like Figure 3 As shown, in this embodiment of the invention, the control module may include a port (such as...) Figure 3 The components shown are ports 0-15), consistency engine, forwarding module, route allocation module, arbitration module, and control and status register.

[0068] The consistency engine can include a memory request control module, a cache request control module, and an accelerator card cache management module. The accelerator card cache management module is used to update the first directory according to the status of the cache lines of the accelerator card.

[0069] The memory control module is connected to the memory of the accelerator card. The memory of the accelerator card may include at least one of Double Data Rate Synchronous Dynamic Random-Access Memory (DDR) and High Bandwidth Memory (HBM). The memory control module is responsible for reading and writing back these two parts of memory.

[0070] Port (e.g.) Figure 3 Ports 0-15 shown are responsible for connecting to the computing unit of the accelerator card, mainly to realize the physical link. The accelerator card's memory request and control information is transmitted through this physical channel.

[0071] The forwarding module is responsible for parsing the information transmitted through the physical channel and forwarding the memory control information or cache control information to the memory request control module or cache request control module.

[0072] The forwarding module's workflow can include: ① Header Flit identification: The first Flit of each data packet is marked, containing key information such as protocol type (cache request / memory request), source node, destination node, and transaction ID. ② Parsing the Flit into a cache request or a memory request. If it is a cache request, the lower 32 bits of the physical address are extracted as the destination identifier; if it is a memory request, the complete 64-bit address is extracted, and it is checked whether it belongs to the address space managed by the memory controller. ③ Packet reassembly: Multiple Flits of the same data packet are reassembled sequentially into a complete message, while performing Cyclic Redundancy Check (CRC). If an error is detected, a retransmission mechanism is triggered or an error counter is recorded.

[0073] The routing allocation module is used to determine the target forwarding path between the input node and the output node in the self-consistent interconnection network 101 after receiving the transmission information from the forwarding module or the arbitration module. The workflow of the routing allocation module may include: (1) routing strategy selection, determining the output port according to the configured routing algorithm.

[0074] In this embodiment of the invention, the first controller 104 determines the target forwarding path between the input node and the output node in the self-consistent interconnection network 101, which may include: the first controller 104 searching for the shortest forwarding path between the input node and the output node according to the interconnection topology of the self-consistent interconnection network 101; if the shortest forwarding path is occupied, then continuing to search for the shortest forwarding path among the remaining forwarding paths between the input node and the output node; if an unoccupied forwarding path is found, then the currently searched forwarding path is used as the target forwarding path; if all forwarding paths between the input node and the output node are occupied, then the target forwarding path is determined according to the data priority of the first data packet.

[0075] In some optional embodiments of the present invention, the routing algorithm may be: using a breadth-first search (BFS) algorithm at the input node to calculate the shortest path from the source node to the output node. Specific operations may include: 1) initializing a queue and adding the input node to the queue; simultaneously, initializing a two-dimensional array to record the parent node of each node for subsequent path backtracking; retrieving a node from the queue and checking its four adjacent nodes (up, down, left, right); if the adjacent nodes have not been visited and the connection is not occupied, adding the node to the queue and recording its parent node; repeating the above steps until the output node is found or the queue is empty; if the output node is found, constructing the shortest path by backtracking the parent node array.

[0076] 2) Path Selection. Shortest Path Available: If all connections on the shortest path are unoccupied, the input node selects this shortest path and transmits data along it to the output node. Shortest Path Occupied: The input node begins calculating the second shortest path. Second Shortest Path Available: If the second shortest path is unoccupied, the input node selects this second shortest path for data transmission. Second Shortest Path Also Occupied: When all paths are occupied, a decision is made based on the importance of the routing data.

[0077] In this embodiment of the invention, the first controller 104 determines the target forwarding path based on the data priority of the first data packet, which may include: if the data priority of the first data packet is higher than a first threshold, then the shortest forwarding path found for the first time is determined as the target forwarding path, so as to share the shortest forwarding path with the data being transmitted by the shortest forwarding path; if the data priority of the first data packet is lower than a second threshold, then the first data packet is controlled to enter a waiting state, and the occupancy status of the forwarding path is monitored to determine an idle forwarding path as the target forwarding path.

[0078] For high-importance data, meaning the data priority of the first data packet is higher than the first threshold, a transmission channel is chosen to be shared with the shortest path. For low-importance data, meaning the data priority of the first data packet is lower than the second threshold, the input node can be controlled to enter a waiting state and continuously monitor the occupancy of the forwarding path (which may only be the shortest forwarding path).

[0079] In this embodiment of the invention, the data priority of the first data packet can be calculated based on the data importance parameter and the device importance parameter of the first data packet. The data importance parameter of the first data packet is used to characterize the importance of the data transmitted by the first data packet. For example, the data importance parameter of the first data packet requesting data for calculation is higher than the data importance parameter of the first data packet requesting control information. The device importance parameter of the first data packet is used to characterize the importance of the device that sent the first data packet. Accelerator cards can be prioritized, and different device importance parameters can be set for different accelerator cards. The data priority value of the first data packet can be obtained by summing the data importance parameter and the device importance parameter.

[0080] 3) Waiting for processing. During the waiting process, the input node periodically checks the occupancy status of the shortest forwarding path. Once the shortest forwarding path becomes available, the input node immediately selects that path for data transmission.

[0081] (2) When multiple input nodes simultaneously request the same output port of an output node, the arbitration module selects the winning input node through arbitration.

[0082] The arbitration module may include an arbitrator and a crossbar. The arbitrator arbitrates and orders requests from multiple input nodes, and the winning input data is connected to the target output port via the crossbar. The switching structure supports single-cycle multi-port parallel transmission, ensuring high throughput.

[0083] In this embodiment of the invention, the first controller 104 can also be used to determine the data transmission order of different input nodes based on the data priority of the consistent interconnection read / write tasks of different input nodes when multiple input nodes simultaneously request the same output port of the same output node. That is, the arbitration module in the first controller 104 can determine the data transmission order of different input nodes by calculating the priority of the first data packet of different input nodes, and the method for determining the priority can refer to the above-described method for calculating the data priority of the first data packet.

[0084] In this embodiment of the invention, the first controller 104 can also be used to perform phased pipelining of the consistent interconnect read / write tasks corresponding to each input node when multiple input nodes simultaneously request the same output port of the same output node, and to execute the consistent interconnect read / write tasks of different input nodes in a pipelining parallel manner. That is to say, the arbitration module can also mark the request sequence number, perform conflict detection, and pipelining in stages to improve throughput. For example, if three input nodes request the same output port of the same output node, since the entire data forwarding process includes multiple steps and these steps occupy different resources, the data transmission requests of the three input nodes can be processed in pipelining parallel.

[0085] In addition, the arbitration module can also be used to sort requests by global sequence number, ensuring order. The request with the lowest current sequence number is selected via a priority encoder. Figure 4 This is a schematic diagram of an arbitration module provided in an embodiment of the present invention. Figure 4 As shown, if requests from accelerator card 1, accelerator card 4, and accelerator card 6 are received simultaneously, the arbitrator in the arbitration module will arbitrate the requests from each accelerator through request reception and request scheduling, arrange them in the order of accelerator card 1 request, accelerator card 4 request, and accelerator card 6 request, and perform queue state management to ensure order, and select the request with the lowest current sequence number through the priority encoder.

[0086] The consistency engine configures the arbitration module through control and status registers to implement the arbitration strategy.

[0087] The control module also includes an output scheduling and sending module, which is used to maintain a scheduling queue at the output port and select the next data packet to be sent according to the Quality of Service (QoS) policy (such as priority-weighted round-robin).

[0088] The physical link request control unit establishes all physical links for link control. Signal transmission for memory and cache control is established on physical channels. Before any information transmission can proceed, a physical path must first be established, and a transmission channel between the two accelerators must be established via routing.

[0089] like Figure 4 As shown, in the consistency management module of the first controller 104, the requests from input nodes are stored in the cache component. The input module retrieves the input requests from the cache component, parses them, and forwards them to the control module for request reception. The output module is responsible for responding to the requests from the accelerator card, such as generating read / write completion signals in the first directory.

[0090] The conformance interconnect control device 100 provided in this embodiment of the invention may further include a protocol conversion interface; the protocol conversion interface is located between the first connector 102 and the conformance interconnect node, and is used to perform a protocol conversion task between the conformance interconnect protocol of the accelerator card corresponding to the first connector 102 and the conformance interconnect protocol of the first controller 104.

[0091] The coherence interconnect control device 100 provided in this embodiment of the invention can be used for coherence interconnection between heterogeneous accelerator cards. That is, the coherence interconnection node can connect to different types of accelerator cards. These accelerator cards may adopt different coherence interconnection protocols, such as CXL, Coherent Hub Interface (CHI) protocol, Cache Coherent Interconnect for Accelerators (CCIX) protocol, and other accelerator card coherence interconnection protocols.

[0092] The conformance interconnect control device 100 provided in this embodiment of the invention can use the conformance interconnect protocol (CCN-interconnect, which can be referred to as CCN interconnect protocol) provided in this embodiment of the invention. The conformance interconnect protocol is based on the CCN interconnect protocol for forwarding. The protocol conversion interface of each conformance interconnect node performs the protocol conversion task between the conformance interconnect protocol of the accelerator card and the conformance interconnect protocol of the first controller 104.

[0093] In this embodiment of the invention, the protocol conversion interface performs a protocol conversion task, which may include: extracting the message field of the input message according to the consistency conversion protocol of the input party; and performing message type conversion, address encoding conversion, and data format conversion on the message field according to the consistency conversion protocol of the output party.

[0094] like Figure 2As shown, the node management module of the consistency interconnect control device 100 is used to manage the consistency interconnect nodes in the consistency interconnect control device 100. It includes a protocol conversion interface between each consistency interconnect node and the physical layer interface. This protocol conversion interface is a bridge connecting the CCN interconnect protocol and the consistency interconnect protocol of the accelerator card. It mainly consists of a message parsing module, a protocol mapping module, a transaction management module, and an error handling module.

[0095] The message parsing module receives messages from the CCN interconnection protocol and parses them into individual fields, such as message type, address, data, and transaction ID. This is the foundation for subsequent conversion work; only by accurately parsing the information of each field can effective protocol mapping be performed. The workflow of the message parsing module may include: (1) receiving the message stream of the CCN interconnection protocol; (2) cutting and analyzing the message stream according to the message format specification of the CCN interconnection protocol and extracting each field; and (3) performing preliminary verification on the extracted fields to ensure that their format and value range meet the requirements of the new protocol.

[0096] The protocol mapping module is used to map the message type, address, data, and other fields of the CCN interconnect protocol to the corresponding format of the target accelerator card's consistent interconnect protocol, based on the characteristics of the CCN interconnect protocol and the consistent interconnect protocol of the target accelerator card. This is the core part of the entire protocol conversion interface, implemented based on the details and differences of the two protocols. The workflow of the protocol mapping module can include: (1) Message type mapping. Establish a correspondence table between the CCN interconnect protocol message type and the target protocol message type. For example, the "read request" message of the CCN interconnect protocol may correspond to the "atomic read request" message of the CXL protocol, or the "unbuffered read request" message of the CHI protocol. According to the correspondence table, the parsed CCN interconnect protocol message type is converted into the message type of the target protocol. (2) Address mapping. Since different protocols may use different address encoding methods and address space divisions, the address needs to be adjusted. For example, the address of the CCN interconnect protocol may be based on a specific combination of device number and offset, while the CXL protocol has its own address format, so the address of the CCN interconnect protocol needs to be converted into an address that conforms to the requirements of the CXL protocol. (3) Data mapping. Check whether the length, byte order, etc. of the data conform to the requirements of the target protocol. If it does not meet the requirements, make the corresponding adjustments, such as data padding or byte order conversion.

[0097] The transaction management module is used to maintain the status and mapping relationship of transactions to ensure that requests and responses can be correctly matched. Since the transaction processing methods of different protocols may be different, a unified transaction management mechanism needs to be established. The workflow of the transaction management module may include: (1) Transaction ID mapping. Assign a corresponding target protocol transaction ID to each CCN interconnection protocol transaction ID and record it in the transaction mapping table. When a request message from the CCN interconnection protocol is received, a new target protocol transaction ID is generated and the correspondence between the two is recorded. (2) Transaction status tracking. Track the status of each transaction, such as request sent, response received, etc. When a response message from the target protocol is received, find the corresponding CCN interconnection protocol transaction ID according to the transaction mapping table, convert the response message back to the CCN interconnection protocol format, and send it to the device that initiated the request.

[0098] The error handling module is used to detect and handle various error situations during protocol conversion, such as message format errors and protocol incompatibility, to ensure the stability and reliability of the system. The workflow of the error handling module can include: (1) Error detection: checking for errors at each stage of message parsing, protocol mapping, and transaction processing. For example, checking whether the error check bits of the message are correct, and whether there are fields that cannot be mapped during the mapping process. (2) Error reporting: generating an error report message when an error is detected, containing information such as the error type and error location. Sending the error report message to the relevant device or system management module for subsequent processing. (3) Error recovery: taking corresponding recovery measures according to the type and severity of the error. For example, for retryable errors, such as transmission errors, the message can be resent; for unrecoverable errors, such as protocol incompatibility, it may be necessary to terminate the transaction and notify the relevant device.

[0099] The consistency interconnect control device 100 provided in this embodiment of the invention can replace the first switch to perform read / write interactions between accelerator cards. For example, when accelerator card 0 needs to obtain first data but cannot find the first target data locally, accelerator card 0 queries the sharer list corresponding to the corresponding cache line through the first directory in the consistency interconnect control device 100, and determines that accelerator card 2 and accelerator card 4 are the sharers of the cache line. At this time, accelerator card 0 sends a read request for the first data in accelerator card 2 and accelerator card 4 to the consistency interconnect control device 100. The consistency interconnect control device 100 receives the read request from accelerator card 0 through the first connector 102 corresponding to accelerator card 0, parses and determines the output port, and sends the read request to accelerator card 2 and accelerator card 4 respectively after arbitration and queuing. If accelerator card 2 or accelerator card 4 finds the first data in its local cache, it returns the first data to accelerator card 0 through the consistency interconnect control device 100. If the local cache does not find the data, it queries the local memory to obtain the first data and returns it to accelerator card 0 through the consistency interconnect control device 100. If the data is not in the local memory either, a read failure message is returned.

[0100] If accelerator card 0 needs to write second data to its cache line, accelerator card 0 queries the sharer list corresponding to the cache line through the first directory in the consistency interconnect control device 100, and determines that accelerator card 2 and accelerator card 4 are the sharers of the cache line. At this time, accelerator card 0 sends write requests to accelerator card 2 and accelerator card 4 to the consistency interconnect control device 100. The consistency interconnect control device 100 receives the write request from accelerator card 0 through the first connector 102 corresponding to accelerator card 0, parses and determines the output port, and sends the write request to accelerator card 2 and accelerator card 4 respectively after arbitration and queuing. The consistency interconnect control device 100 can write the second data to the memory of accelerator card 2 and accelerator card 4 through Remote Direct Memory Access (RDMA).

[0101] This invention also provides a computing system, which may include a host, multiple accelerator cards, a first switch, and a coherent interconnect control device. The accelerator cards are connected to the host through the first switch. The coherent interconnect control device includes a coherent interconnect network, a first controller, and a first connector. The coherent interconnect network includes multiple interconnected coherent interconnect nodes, and each coherent interconnect node is connected to a corresponding first connector. The other end of the first connector is connected to an accelerator card. The first controller is used to manage the state of the cache lines of the accelerator cards and to control the coherent interconnect network to perform coherent interconnect read and write tasks between different accelerator cards.

[0102] The computing system provided in this embodiment of the invention can be referred to the description of the above-described consistency interconnection control device embodiment.

[0103] This invention also provides a coherent interconnect control method applied to a first controller. The coherent interconnect control method may include: acquiring and managing the state of cache lines of accelerator cards through a first connector; controlling the coherent interconnect network to perform coherent interconnect read and write tasks between different accelerator cards; wherein the coherent interconnect network includes multiple interconnected coherent interconnect nodes, and the coherent interconnect nodes are connected to the first connector one by one; the other end of the first connector is connected to a first slot, and the first slot is used to install accelerator cards.

[0104] The consistency interconnection control method provided in this embodiment of the invention can be referred to the description of the above-described consistency interconnection control device embodiment.

[0105] Embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above embodiments of the coherent interconnection control method.

[0106] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program configured to execute the steps in any of the above-described embodiments of the consistency interconnection control method at runtime.

[0107] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0108] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described embodiments of the consistency interconnection control method.

[0109] Embodiments of the present invention also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described embodiments of the coherence interconnection control method.

[0110] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0111] The foregoing has provided a detailed description of the consistency interconnection control device, method, electronic device, product, and computing system provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of this invention. It should be noted that those skilled in the art can make various improvements and modifications to this invention without departing from its principles, and these improvements and modifications also fall within the protection scope of this invention.

Claims

1. A consistency interconnection control device, characterized in that, include: A consistent interconnect network, a first controller, and a first connector; The consistent interconnect network includes multiple interconnected consistent interconnect nodes, each of which is connected to the first connector in a one-to-one correspondence. The other end of the first connector is connected to the first slot, which is used to install the accelerator card; The first controller is used to manage the state of the cache lines of the accelerator cards and to control the coherent interconnect network to perform coherent interconnect read and write tasks between different accelerator cards; The first controller manages the state of the cache lines of the accelerator cards and controls the coherent interconnect network to perform coherent interconnect read / write tasks between different accelerator cards, including: The first controller updates the first directory based on the status of the cache line; the first directory includes a list of sharers corresponding to the cache line, the memory address corresponding to the cache line, and the cache line status of the cache line; the first directory includes the status of the cache lines of all the accelerator cards managed by the first controller; The first controller determines the sending accelerator card and the receiving accelerator card of the first data packet received by the consistency interconnect control device, determines the target forwarding path between the input node and the output node in the consistency interconnect network, and controls the consistency interconnect network to forward the first data packet according to the target forwarding path, so as to send it to the receiving accelerator card through the output node and its corresponding first connector. The input node is the consistency interconnect node corresponding to the sending accelerator card, and the output node is the consistency interconnect node corresponding to the receiving accelerator card.

2. The consistency interconnection control device according to claim 1, characterized in that, The cache line is in one of the following states: modified, exclusive, shared, or invalid.

3. The consistency interconnection control device according to claim 1, characterized in that, The first controller determines the target forwarding path between the input node and the output node from the consensus interconnection network, including: The first controller searches for the shortest forwarding path between the input node and the output node based on the interconnection topology of the consistent interconnection network; If the shortest forwarding path is occupied, continue searching for the shortest forwarding path among the remaining forwarding paths from the input node to the output node; If an unoccupied forwarding path is found, the currently found forwarding path is used as the target forwarding path. If all forwarding paths between the input node and the output node are occupied, the target forwarding path is determined according to the data priority of the first data packet.

4. The consistency interconnection control device according to claim 3, characterized in that, The first controller determines the target forwarding path based on the data priority of the first data packet, including: If the data priority of the first data packet is higher than the first threshold, then the shortest forwarding path found for the first time is determined as the target forwarding path, so that the shortest forwarding path is shared with the data being transmitted by the shortest forwarding path; If the data priority of the first data packet is lower than the second threshold, the first data packet is controlled to enter a waiting state, and the occupancy status of the forwarding path is monitored to determine the idle forwarding path as the target forwarding path.

5. The consistency interconnection control device according to claim 1, characterized in that, The first controller is further configured to determine the data transmission order of different input nodes based on the data priority of the consistent interconnect read / write tasks of different input nodes when multiple input nodes simultaneously request the same output port of the same output node.

6. The consistency interconnection control device according to claim 1, characterized in that, The first controller is further configured to, when it detects that multiple input nodes simultaneously request the same output port of the same output node, perform phased pipelined execution of the consistent interconnect read / write tasks corresponding to each input node, and execute the consistent interconnect read / write tasks of different input nodes in a pipelined parallel manner.

7. The consistency interconnection control device according to claim 1, characterized in that, It also includes a protocol conversion interface; The protocol conversion interface is located between the first connector and the coherent interconnect node, and is used to perform protocol conversion tasks between the coherent interconnect protocol of the accelerator card corresponding to the first connector and the coherent interconnect protocol of the first controller.

8. The consistency interconnection control device according to claim 7, characterized in that, The protocol conversion interface performs the protocol conversion task, including: Extract the message fields of the input message according to the consistency conversion protocol of the input party; According to the consistency conversion protocol of the output party, the message field is converted into message type, address encoding and data format.

9. A computing system, characterized in that, include: Host, multiple accelerator cards, first switch, and consistency interconnect control device; The accelerator card is connected to the host via the first switch; The coherent interconnect control device includes a coherent interconnect network, a first controller, and a first connector; the coherent interconnect network includes multiple interconnected coherent interconnect nodes, each corresponding to a first connector; the other end of the first connector is connected to the accelerator card. The first controller is used to manage the state of the cache lines of the accelerator cards and to control the coherent interconnect network to perform coherent interconnect read and write tasks between different accelerator cards; The first controller manages the state of the cache lines of the accelerator cards and controls the coherent interconnect network to perform coherent interconnect read / write tasks between different accelerator cards, including: The first controller updates the first directory based on the status of the cache line; the first directory includes a list of sharers corresponding to the cache line, the memory address corresponding to the cache line, and the cache line status of the cache line; the first directory includes the status of the cache lines of all the accelerator cards managed by the first controller; The first controller determines the sending accelerator card and the receiving accelerator card of the first data packet received by the consistency interconnect control device, determines the target forwarding path between the input node and the output node in the consistency interconnect network, and controls the consistency interconnect network to forward the first data packet according to the target forwarding path, so as to send it to the receiving accelerator card through the output node and its corresponding first connector. The input node is the consistency interconnect node corresponding to the sending accelerator card, and the output node is the consistency interconnect node corresponding to the receiving accelerator card.

10. A consistency interconnection control method, characterized in that, Applied to the first controller, including: The status of the accelerator card's cache lines is acquired and managed through the first connector; Control the consistency interconnect network to perform consistency interconnect read and write tasks between different accelerator cards; The consensus interconnect network includes multiple interconnected consensus interconnect nodes, each of which is connected to a first connector; the other end of the first connector is connected to a first slot, which is used to install the accelerator card. The status of the accelerator card's cache lines is acquired and managed through the first connector, including: The first directory is updated according to the status of the cache line; the first directory includes a list of sharers corresponding to the cache line, the memory address corresponding to the cache line, and the cache line status of the cache line; the first directory includes the status of the cache lines of all the accelerator cards managed by the first controller; Controlling the coherent interconnect network to perform coherent interconnect read / write tasks between different accelerator cards includes: The sender accelerator card and receiver accelerator card of the first data packet received by the consistency interconnection control device are determined. The target forwarding path between the input node and the output node is determined from the consistency interconnection network. The consistency interconnection network is controlled to forward the first data packet according to the target forwarding path, so as to send it to the receiver accelerator card through the output node and its corresponding first connector. The input node is the consistency interconnect node corresponding to the sending accelerator card, and the output node is the consistency interconnect node corresponding to the receiving accelerator card.

11. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the coherent interconnect control method as described in claim 10 when executing the computer program.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the consistency interconnect control method as described in claim 10.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the consistency interconnection control method as described in claim 10.

Images