Consistency interconnection control device and method, electronic device, product and computing system

Abstract

The invention discloses a consistency interconnection control device and method, an electronic device, a product and a computing system, and relates to the technical field of computers, the consistency interconnection control device is provided and comprises a consistency interconnection network, a first controller and a first connector, the consistency interconnection network comprises a plurality of interconnected consistency interconnection nodes, the consistency interconnection node is connected with the first card slot for installing the acceleration card through the first connector, and the first controller manages the state of the cache line of the acceleration card and controls the consistency interconnection network to execute the consistency interconnection read-write task between different acceleration cards, so that the consistency interconnection system for the acceleration computing cluster is realized. By implementing the cache consistency between the acceleration cards, when the acceleration computing cluster executes distributed computing, the cache can be hit firstly, so that access to a local memory and memories of other acceleration cards is reduced, the problem of high time consumption of communication between different accelerators of the acceleration computing cluster is solved, and the efficiency of distributed computing is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to consistent interconnection control devices, methods, electronic devices, products and computing systems. Background Art

[0002] With the development of artificial intelligence (AI) technology, the demand for computing power has increased significantly. By connecting accelerator cards to server hosts and interconnecting them, large-scale accelerated computing clusters can be built to achieve distributed computing, solving the computational challenges of large-scale AI models. However, when distributed computing is performed in an accelerated computing cluster, the communication cost between different accelerators continues to increase, and the communication time consumed significantly affects the efficiency of distributed computing. Summary of the Invention

[0003] The present invention provides a consistent interconnection control device, method, electronic device, product and computing system to at least solve the problem of time-consuming communication between different accelerators in an accelerated computing cluster in the related art.

[0004] The present invention provides a consistent interconnection control device, comprising: a consistent interconnection network, a first controller and a first connector; The coherent interconnect network includes a plurality of interconnected coherent interconnect nodes, and the coherent interconnect nodes are connected to the first connectors in a one-to-one correspondence; The other end of the first connector is connected to a first slot, and the first slot is used to install an accelerator card; The first controller is used to manage the status of the cache line of the acceleration card and control the coherent interconnection network to perform coherent interconnection read and write tasks between different acceleration cards.

[0005] The present invention also provides a consistent interconnection control method, which is applied to a first controller and includes: Acquire and manage the state of the cache line of the accelerator card through the first connector; Controlling the coherent interconnect network to execute coherent interconnect read and write tasks between different accelerator cards; The coherent interconnect network includes a plurality of interconnected coherent interconnect nodes, and the coherent interconnect nodes are connected to the first connector in a one-to-one correspondence; the other end of the first connector is connected to the first slot, and the first slot is used to install the acceleration card.

[0006] The present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any one of the above-mentioned consistent interconnection control methods when executing the computer program.

[0007] The present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned coherent interconnection control methods are implemented.

[0008] The present invention also provides a computer program product, comprising a computer program, which implements the steps of any of the above-mentioned coherent interconnection control methods when executed by a processor.

[0009] Through the present invention, a consistency interconnection control device is provided, which includes a consistency interconnection network, a first controller and a first connector. The consistency interconnection network includes a plurality of interconnected consistency interconnection nodes, which are connected to the first connector in a one-to-one correspondence. The other end of the first connector is connected to the first slot, which is used to install an accelerator card. The first controller manages the state of the cache line of the accelerator card and controls the consistency interconnection network to perform consistency interconnection read and write tasks between different accelerator cards. A consistency interconnection system for an accelerated computing cluster composed of multiple accelerators is realized. By realizing cache consistency between accelerator cards, when the accelerated computing cluster performs distributed computing, it can first hit from the cache, reducing access to local memory and other acceleration card memories, thereby reducing the number of communications in distributed computing, solving the problem of time-consuming communication between different accelerators in the accelerated computing cluster, and improving the efficiency of distributed computing. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0011] Figure 1 A connection diagram of a consistent interconnection control device provided by an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a consistent interconnection control device provided by an embodiment of the present invention; Figure 3 A schematic structural diagram of a control module provided in an embodiment of the present invention; Figure 4 An arbitration schematic diagram of an arbitration module provided in an embodiment of the present invention; Wherein, 100 is a coherent interconnection control device, 101 is a coherent interconnection network, 102 is a first connector, 103 is a second connector, and 104 is a first controller. DETAILED DESCRIPTION

[0012] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0013] It should be noted that, in the description of the present invention, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. The terms "first," "second," etc., in the present invention are used to distinguish similar objects, and are not used to describe a particular order or precedence.

[0014] In order to enable those skilled in the art to better understand the solutions of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0015] Here, some key terms used in the embodiments of the present invention are explained.

[0016] Cache coherence, also known as cache consistency or inter-cache consistency, refers to a mechanism that ensures that the data in the cache memory is the same as the data in the main memory in a computer system that uses a hierarchical storage system. When a processor modifies a certain data, the corresponding data in the cache of other processors must be updated or invalidated to ensure that the data seen by all processors is consistent.

[0017] Memory coherence means that multiple processors must follow certain ordering rules when accessing shared memory to ensure correct program execution. The memory consistency model defines the visibility and order of read and write operations in a program, ensuring that different processors see the same order of memory operations.

[0018] The hardware costs and consistency maintenance required for large-scale server system expansion are currently high, and consistent interconnection is only achieved between hosts and between hosts and accelerator cards. When using an accelerated computing system consisting of multiple interconnected accelerator cards to perform distributed computing tasks, if one accelerator card needs to obtain data from another, the data must be moved from remote memory to local memory via the interconnected network before the calculation can be performed. This results in high communication overhead and consumes a significant amount of time in data transfer, which affects the computational efficiency of artificial intelligence models.

[0019] To this end, an embodiment of the present invention first provides a consistency interconnection control device, which is used to achieve consistency interconnection between acceleration cards. The consistency interconnection control device includes a consistency interconnection network, a first controller and a first connector. The consistency interconnection network includes a plurality of interconnected consistency interconnection nodes, and the consistency interconnection nodes are connected to the first connector in a one-to-one correspondence. The other end of the first connector is connected to the first slot, and the first slot is used to install the acceleration card. The first controller manages the state of the cache line of the acceleration card and controls the consistency interconnection network to perform consistency interconnection read and write tasks between different acceleration cards. Based on the consistency interconnection control device provided by the embodiment of the present invention, a consistency interconnection system for an acceleration computing cluster composed of multiple accelerators can be implemented. By achieving cache consistency between acceleration cards, when the acceleration computing cluster performs distributed computing, it can first hit from the cache, reducing access to local memory and other acceleration card memories, thereby reducing the number of communications in distributed computing, solving the problem of time-consuming communication between different accelerators in the acceleration computing cluster, and improving the efficiency of distributed computing.

[0020] Figure 1 A connection diagram of a consistent interconnection control device provided by an embodiment of the present invention.

[0021] The coherent interconnect control device provided by an embodiment of the present invention may include: a coherent interconnect network, a first controller and a first connector; the coherent interconnect network includes a plurality of interconnected coherent interconnect nodes, and the coherent interconnect nodes are connected to the first connector in a one-to-one correspondence; the other end of the first connector is connected to the first slot, and the first slot is used to install an accelerator card; the first controller is used to manage the status of the cache line of the accelerator card, and to control the coherent interconnect network to perform coherent interconnect read and write tasks between different accelerator cards.

[0022] In an embodiment of the present invention, the cache coherence network (CCN) and the first controller can be implemented based on logic circuit programming of a programmable controller, which can be a field programmable gate array (FPGA).

[0023] like Figure 1 As shown, a coherent interconnect network 101 can be connected to multiple first slots based on multiple first connectors 102. The first connector 102 can be connected to the second connector 103 on the accelerator card side. The first connector 102 and the second connector 103 can use multi-channel input and output (MCIO) connectors. The first controller ( Figure 1102 ) is used to control the coherent interconnection network 101 to forward the data packet received from the first connector 102 .

[0024] The accelerator card can be a homogeneous or heterogeneous accelerator card, and can be a graphics processing unit (GPU), field programmable gate array (FPGA), application-specific integrated circuit (ASIC), or other type of accelerator card. The first slot is used to install the accelerator card, and the first slot can be a high-speed serial computer expansion bus (Peripheral Component Interconnect Express (PCIe)) slot.

[0025] like Figure 1 As shown, in the entire cabinet, a computing system can be composed of a host, a first switch, and an accelerated computing system. The first switch can be a PCIe switch or an Ethernet switch. The uplink port of the first switch is connected to the host, and the downlink port of the first switch is connected to the first slot via a third connector. This allows interconnection between the host and the accelerator card, as well as between different accelerator cards, based on the first switch.

[0026] Figure 1 This paper shows a 64-card computing system. The host can be a dual-core central processing unit (CPU) host node, compatible with mainstream CPU models. The first switch can be composed of multiple switch controllers. In a 64-card cluster, four switch controllers can be configured. Each switch controller's downlink port supports two computing boards via the PCIe bus. The uplink port of the switch controller is connected to the CPU, enabling communication between the CPU and the accelerator card.

[0027] Each computing board integrates 16 accelerator cards. The serial (SerDes) interface of a single accelerator card is divided into two parts: one part is connected to the first switch through the first slot, and the other part is connected to the coherent interconnection network 101 through the dedicated coherent bus provided by the embodiment of the present invention. It supports multiple data transmission protocols, such as PCIe and Compute Express Link (CXL) protocol. In this way, all 16 accelerator cards on each computing board are connected to the coherent interconnection network 101. The coherent interconnection networks 101 are connected via a high-speed coherent bus, realizing coherent interconnection between 64 accelerator cards. In addition, the four computing boards are connected to the first switch (PCIe switch) through a third connector (which can be a PCIe interface) and realize communication with the CPU.

[0028] Figure 1The four coherent interconnect networks 101 shown can be located in four coherent interconnect control devices 100. The coherent interconnect control devices 100 can be connected via chip-to-chip connectors to form a coherent interconnect network. Figure 1 The topology formed by the coherent interconnection control device 100 and multiple coherent interconnection control devices 100 provided by the embodiment of the present invention can be extended to the coherent interconnection of 128-card and 256-card systems.

[0029] The consistent interconnection control device 100 provided by an embodiment of the present invention includes a consistent interconnection network 101, a first controller 104 and a first connector 102. The consistent interconnection network 101 includes multiple interconnected consistent interconnection nodes, which are connected to the first connector 102 in a one-to-one correspondence. The other end of the first connector 102 is connected to the first slot, which is used to install an accelerator card. The first controller 104 manages the state of the cache line of the accelerator card and controls the consistent interconnection network 101 to perform consistent interconnection read and write tasks between different accelerator cards, thereby realizing a consistent interconnection system for an accelerated computing cluster composed of multiple accelerators. By realizing cache consistency between accelerator cards, when the accelerated computing cluster performs distributed computing, it can first hit from the cache, reducing access to local memory and other accelerator card memories, thereby reducing the number of communications in distributed computing, solving the problem of time-consuming communication between different accelerators in the accelerated computing cluster, and improving the efficiency of distributed computing.

[0030] Figure 2 A schematic diagram of the structure of a consistent interconnection control device provided by an embodiment of the present invention.

[0031] Based on the above embodiment, the embodiment of the present invention continues to describe the internal structure of the coherent interconnection control device 100 .

[0032] In the above embodiment, the coherent interconnection network 101 and the first controller can be implemented based on the logic circuit programming of the programmable controller. Then the logic circuit of the programmable controller can be divided into multiple logic areas as coherent interconnection nodes (such as Figure 2 00, 01, ..., 15), each coherent interconnect node corresponds to a first connector 102, that is, corresponds to an accelerator card. The coherent interconnect nodes are interconnected to form a coherent interconnect network 101.

[0033] The topology of the coherent interconnect network 101 determines the physical layout and connections between the coherent interconnect nodes and channels in the coherent interconnect network 101, and determines the number of message hops, the distance between nodes, network latency, etc. Considering that too many ports will result in a smaller bandwidth of a single port, and reducing the number of ports will increase the number of message hops in the coherent interconnect network 101, in an embodiment of the present invention, each coherent interconnect node can be divided into four ports for connecting to other coherent interconnect nodes, forming a network structure as follows: Figure 2 In the two-dimensional network topology (2D mesh) shown, each consistent interconnected node on the bus can be observed by all devices on the bus, achieving full interconnection between nodes (accelerator cards).

[0034] In the embodiment of the present invention, each coherent interconnection network 101 may include: Figure 2 The 16 coherent interconnect nodes shown correspond to 16 first connectors 102, thus achieving coherent interconnection of 16 accelerator cards. The coherent interconnect nodes can be designed to be compatible with different types of accelerator cards and interconnected through a unified interface.

[0035] In the embodiment of the present invention, the coherent interconnection control device 100 may include two control modules, namely a coherence management module and a node management module, wherein the coherence management module runs on the first controller 104 .

[0036] like Figure 2 As shown, the consistency management module may include a control module, an input module, and an output module. The control module is used to maintain the first directory. The consistency management module may also include a cache component. The size of the cache component can be set as needed, for example, 5MB.

[0037] In an embodiment of the present invention, the first controller 104 manages the status of the cache line of the accelerator card, which may include: the first controller 104 updates the first directory according to the status of the cache line; the first directory includes a sharer list corresponding to the cache line, a memory address corresponding to the cache line, and the cache line status of the cache line.

[0038] In this embodiment of the present invention, the first directory may include the status of cache lines in all accelerator cards managed by the first controller 104. The number of entries in the first directory may be the ratio of the number of accelerator cards managed by the first controller 104 multiplied by the memory size of a single accelerator card and the size of a single cache line. For example, if there are 16 accelerator cards, the corresponding number of entries N is 16 * single accelerator memory / size of (cache line size).

[0039] Table 1 is a storage information structure of a single entry of a first directory provided by an embodiment of the present invention.

[0040] Table 1

[0041] As shown in Table 1, the sharer list indicates which accelerator cards store the information of the cache line. There are 16 accelerator cards in total, so the sharer list can include 16 bits of information, and each bit corresponds to an accelerator card. If 0 is used to represent a non-sharer and 1 is used to represent a sharer, assuming that the entry corresponds to a cache line in accelerator card 0, it can be said that the sharers of the cache line are accelerator card 2 and accelerator card 4, and there is data consistent with the cache line in the cache of accelerator card 2 and accelerator card 4.

[0042] The memory address indicates which accelerator card the cache line belongs to. bit information.

[0043] In the embodiment of the present invention, the state of the cache line may be one of a modified state, an exclusive state, a shared state, and an invalid state.

[0044] Among them, the modified state indicates that the data in the cache line has been modified by the current accelerator card, and these modifications have not yet been written back to the main memory. Therefore, the data in the cache line is inconsistent with the data in the main memory and the data in the sharer. That is, there is the latest copy in the cache line, and there is no valid copy of the cache line in other caches.

[0045] The exclusive state indicates that the data in the cache line is exclusively owned by the current accelerator card. No other accelerator card has a copy of the data in its cache. The data in this state is consistent with the data in the main memory. The current accelerator card can read and write the cache line without notifying other accelerator cards.

[0046] The shared state indicates that the data in the cache line may be shared by multiple accelerator cards. That is, copies of the data may also exist in the caches of other accelerator cards. In this state, the data in the cache line is consistent with the data in the main memory.

[0047] The invalid state indicates that the data in the cache line is invalid and cannot be used by the current accelerator card. This usually occurs when the data is modified by other accelerator cards and the cache line of the current accelerator card needs to be updated.

[0048] State transitions are used to ensure cache consistency between different accelerator cards. The following are several examples of state transitions.

[0049] From invalid state to shared state or exclusive state: When an accelerator card reads a data block from main memory, if the data block is clean (unmodified) in main memory and no other accelerator card has a copy of the data in its cache, the cache line state of the data block is set to exclusive state. If a copy of the data already exists in the cache of another accelerator card, the cache line state of the data block is set to shared state.

[0050] From Shared to Exclusive or Modified: When an accelerator card writes to a cache line in the Shared state, the cache line's state changes from Shared to Modified. Simultaneously, the data block in the caches of other accelerator cards is marked as invalid.

[0051] From Exclusive state to Modified state: When an accelerator card writes to a cache line in Exclusive state, the state of the cache line changes from Exclusive state to Modified state.

[0052] From Modified to Shared or Invalid: When an accelerator card reads a cache line in the Modified state, the state of the cache line remains unchanged. When an accelerator card writes a cache line in the Modified state, the state of the cache line remains unchanged. When an accelerator card writes back a cache line in the Modified state, the state of the cache line changes from Modified to Shared or Invalid, depending on the cache state of the other accelerator card.

[0053] The above four cache line states can be represented by 2 bits of information, such as 00, 01, 10, and 11.

[0054] Figure 3 A schematic structural diagram of a control module provided in an embodiment of the present invention.

[0055] In an embodiment of the present invention, the first controller 104 controls the coherent interconnection network 101 to perform coherent interconnection read and write tasks between different accelerator cards, which may include: the first controller 104 determines the sender accelerator card and the receiver accelerator card of the first data packet received by the coherent interconnection control device 100, determines the target forwarding path between the input node and the output node from the coherent interconnection network 101, and controls the coherent interconnection network 101 to forward the first data packet according to the target forwarding path, so as to send it to the receiver accelerator card through the output node and its corresponding first connector 102; wherein, the input node is the coherent interconnection node corresponding to the sender accelerator card, and the output node is the coherent interconnection node corresponding to the receiver accelerator card.

[0056] like Figure 3 As shown, in an embodiment of the present invention, the control module may include a port (such as Figure 3Ports 0 to 15 shown), consistency engine, forwarding module, routing distribution module, arbitration module, control and status registers.

[0057] The consistency engine may include a memory request control module, a cache request control module and an accelerator card cache management module, and the accelerator card cache management module is used to update the first directory according to the status of the cache line of the accelerator card.

[0058] The memory control module is connected to the accelerator card's memory, which can include at least one of double data rate synchronous dynamic random access memory (DDR) and high bandwidth memory (HBM). The memory control module is responsible for reading and writing back these two types of memory.

[0059] Port (such as Figure 3 Ports 0 to 15 shown are responsible for connecting to the computing unit of the accelerator card, mainly realizing the physical link. The accelerator card's memory requests and control information are transmitted through this physical channel.

[0060] The forwarding module is responsible for parsing the information transmitted by the physical channel and forwarding the memory control information or the cache control information to the memory request control module or the cache request control module.

[0061] The forwarding module's workflow may include the following: ① Header Flow Control Unit (Flit) identification: The first Flit (Header Flit) of each data packet is marked with key information such as the protocol type (cache request / memory request), source node, destination node, and transaction ID. ② Parsing the Flit as a cache request or a memory request. If it is a cache request, the lower 32 bits of the physical address are extracted as the target identifier; if it is a memory request, the full 64-bit address is extracted and checked to see if it belongs to the address space managed by the memory controller. ③ Packet reassembly: Multiple Flits of the same data packet are reassembled into a complete message in sequence, and a cyclic redundancy check (CRC) is performed at the same time. If an error is detected, a retransmission mechanism is triggered or an error counter is recorded.

[0062] The routing assignment module is used to determine the target forwarding path between the input node and the output node from the coherent interconnection network 101 after receiving the transmission information from the forwarding module or the arbitration module. The workflow of the routing assignment module may include: (1) routing strategy selection, determining the output port according to the configured routing algorithm.

[0063] In an embodiment of the present invention, the first controller 104 determines the target forwarding path between the input node and the output node from the consistent interconnection network 101, which may include: the first controller 104 searches for the shortest forwarding path between the input node and the output node according to the interconnection topology of the consistent interconnection network 101; if the shortest forwarding path is occupied, continues to search for the shortest forwarding path among the remaining forwarding paths from the input node to the output node; if an unoccupied forwarding path is searched, the currently searched forwarding path is used as the target forwarding path; if all forwarding paths between the input node and the output node are occupied, determines the target forwarding path according to the data priority of the first data packet.

[0064] In some optional implementations of the present invention, the routing algorithm may be: using a breadth-first search (BFS) algorithm at the input node to calculate the shortest path from the source node to the output node. Specific operations may include: 1) initializing a queue and adding the input node to the queue; simultaneously, initializing a two-dimensional array to record the parent node of each node for subsequent path backtracking; removing a node from the queue and checking its four adjacent nodes (upper, lower, left, and right); if the adjacent nodes have not been visited and the connection is not occupied, adding the node to the queue and recording its parent node; repeating the above steps until the output node is found or the queue is empty; if the output node is found, constructing the shortest path by backtracking the parent node array.

[0065] 2) Path Selection. Shortest Path Available: If all links in the shortest path are unoccupied, the input node selects that shortest path and transmits the data along it to the output node. Shortest Path Occupied: The input node begins calculating the next shortest path. Next Shortest Path Available: If the next shortest path is unoccupied, the input node selects that path for data transmission. Next Shortest Path Occupied: If all paths are occupied, a decision is made based on the importance of the routing data.

[0066] In an embodiment of the present invention, the first controller 104 determines the target forwarding path based on the data priority of the first data packet, which may include: if the data priority of the first data packet is higher than a first threshold, determining the shortest forwarding path searched for the first time as the target forwarding path, so as to share the shortest forwarding path with the data being transmitted by the shortest forwarding path; if the data priority of the first data packet is lower than a second threshold, controlling the first data packet to enter a waiting state, and monitoring the occupancy status of the forwarding path to determine an idle forwarding path as the target forwarding path.

[0067] For high-importance data, i.e., the data priority of the first data packet is higher than the first threshold, the shortest path is selected to share a transmission channel. For low-importance data, i.e., the data priority of the first data packet is lower than the second threshold, the input node can be controlled to enter a waiting state and continuously monitor the occupancy of the forwarding path (which can only detect the shortest forwarding path).

[0068] In an embodiment of the present invention, the data priority of the first data packet can be calculated based on the data importance parameter of the first data packet and the device importance parameter of the first data packet. The data importance parameter of the first data packet is used to characterize the importance of the data transmitted by the first data packet. For example, the data importance parameter of the first data packet requesting data required for calculation is higher than the data importance parameter of the first data packet requesting control information. The device importance parameter of the first data packet is used to characterize the importance of the device that sends the first data packet. The accelerator cards can be prioritized to set different device importance parameters for different accelerator cards. The data priority value of the first data packet can be obtained by summing the data importance parameter and the device importance parameter.

[0069] 3) Waiting process. During the waiting process, the input node periodically checks the occupancy of the shortest forwarding path. Once the shortest forwarding path is available, the input node immediately selects the path for data transmission.

[0070] (2) When multiple input nodes simultaneously request the same output port of an output node, the arbitration module selects the winning input node through arbitration.

[0071] The arbitration module can include an arbiter and a crossbar switch. The arbiter is used to arbitrate and sort requests from multiple input nodes. The winning input data is connected to the target output port through the crossbar switch. The switching fabric supports single-cycle multi-port parallel transmission, ensuring high throughput.

[0072] In this embodiment of the present invention, the first controller 104 may also be configured to determine the data transmission order of different input nodes based on the data priorities of the coherent interconnect read / write tasks of the different input nodes when detecting that multiple input nodes simultaneously request the same output port of the same output node. In other words, the arbitration module in the first controller 104 may determine the data transmission order of different input nodes by calculating the priorities of the first data packets of the different input nodes. The method for determining the priorities may refer to the method for calculating the data priority of the first data packet described above.

[0073] In an embodiment of the present invention, the first controller 104 can also be used to pipeline the consistent interconnect read and write tasks corresponding to each input node in stages when it detects that multiple input nodes are simultaneously requesting the same output port of the same output node, and to execute the consistent interconnect read and write tasks of different input nodes in a pipelined parallel manner. In other words, the arbitration module can also mark the request sequence, perform conflict detection, and pipeline in stages to improve throughput. For example, if three input nodes request the same output port of the same output node, since the entire data forwarding process includes multiple steps and these steps occupy different resources, the data transmission requests of the three input nodes can be pipelined in parallel.

[0074] In addition, the arbitration module can also be used to sort requests by global sequence number to ensure order. The priority encoder selects the request with the lowest current sequence number. Figure 4 FIG1 is an arbitration diagram of an arbitration module provided by an embodiment of the present invention. Figure 4 As shown, if requests from accelerator card 1, accelerator card 4, and accelerator card 6 are received simultaneously, the arbitrator in the arbitration module arbitrates the requests of each accelerator through request reception and request scheduling, arranges them in the order of accelerator card 1 request, accelerator card 4 request, and accelerator card 6 request, and performs queue status management to ensure sequence, and selects the request with the current lowest sequence number through the priority encoder.

[0075] The consistency engine configures the arbitration module through the control and status registers to implement the arbitration strategy.

[0076] The control module also includes an output scheduling and sending module, which is used to maintain a scheduling queue at the output port and select the next data packet to be sent based on the Quality of Service (QoS) strategy (such as priority weighted polling).

[0077] The physical link request control unit establishes the physical links for all link controls. Signal transmission for memory and cache control is based on physical channels. Before all information is transmitted, a physical path must be established and forwarded via routing to establish a transmission channel between the two accelerators.

[0078] like Figure 4 As shown, in the consistency management module of the first controller 104, the input node's request is stored in the cache component. The input module extracts the input request from the cache component, parses it, and forwards it to the control module for request reception. The output module is responsible for responding to the accelerator card's request, such as generating a signal indicating read / write completion in the first directory.

[0079] The consistency interconnection control device 100 provided in an embodiment of the present invention may also include a protocol conversion interface; the protocol conversion interface is arranged between the first connector 102 and the consistency interconnection node, and is used to perform the protocol conversion task between the consistency interconnection protocol of the accelerator card corresponding to the first connector 102 and the consistency interconnection protocol of the first controller 104.

[0080] The coherent interconnect control device 100 provided in an embodiment of the present invention can be used for coherent interconnection between heterogeneous accelerator cards, that is, the coherent interconnect node can connect different types of accelerator cards, and these accelerator cards may adopt different coherent interconnect protocols, such as CXL, Coherent Hub Interface (CHI) protocol, Cache Coherent Interconnect for Accelerators (CCIX) protocol and other accelerator card coherent interconnect protocols.

[0081] The coherent interconnect control device 100 provided in this embodiment of the present invention can use the coherent interconnect protocol (CCN-interconnect, hereinafter referred to as the CCN interconnect protocol) provided in this embodiment of the present invention. The coherent interconnect protocol is forwarded based on the CCN interconnect protocol. The protocol conversion interface of each coherent interconnect node performs protocol conversion between the accelerator card's coherent interconnect protocol and the first controller 104's coherent interconnect protocol.

[0082] In an embodiment of the present invention, the protocol conversion interface performs a protocol conversion task, which may include: extracting the message field of the input message according to the consistency conversion protocol of the input party; and performing message type conversion, address encoding conversion, and data format conversion on the message field according to the consistency conversion protocol of the output party.

[0083] like Figure 2 As shown, the node management module of the consistency interconnection control device 100 is used to manage the consistency interconnection nodes in the consistency interconnection control device 100, including a protocol conversion interface provided between each consistency interconnection node and the physical layer interface. The protocol conversion interface is a bridge connecting the CCN interconnection protocol and the consistency interconnection protocol of the accelerator card, which is mainly composed of a message parsing module, a protocol mapping module, a transaction management module and an error handling module.

[0084] The message parsing module is used to receive messages from the CCN interconnection protocol and parse them into individual fields, such as message type, address, data, transaction ID, etc. This is the basis for subsequent conversion work. Only by accurately parsing the information of each field can effective protocol mapping be performed. The workflow of the message parsing module can include: (1) receiving the message stream of the CCN interconnection protocol. (2) cutting and analyzing the message stream according to the message format specification of the CCN interconnection protocol to extract each field. (3) performing preliminary verification on the extracted fields to ensure that their format and value range meet the requirements of the new protocol.

[0085] The protocol mapping module is used to map the message type, address, data and other fields of the CCN interconnection protocol to the format corresponding to the consistency interconnection protocol of the target accelerator card based on the characteristics of the CCN interconnection protocol and the consistency interconnection protocol of the target accelerator card. This is the core part of the entire protocol conversion interface, which is implemented based on the details and differences of the two protocols. The workflow of the protocol mapping module may include: (1) Message type mapping. Establish a correspondence table between the CCN interconnection protocol message type and the target protocol message type. For example, the "read request" message of the CCN interconnection protocol may correspond to the "atomic read request" message of the CXL protocol, or the "non-cached read request" message of the CHI protocol. According to the correspondence table, the parsed CCN interconnection protocol message type is converted to the target protocol message type. (2) Address mapping. Since different protocols may use different address encoding methods and address space divisions, the address needs to be adjusted. For example, the address of the CCN interconnection protocol may be based on a specific device number and offset combination, while the CXL protocol has its own address format. The address of the CCN interconnection protocol needs to be converted to an address that meets the requirements of the CXL protocol. (3) Data mapping. Check whether the length, byte order, etc. of the data meet the requirements of the target protocol. If it does not match, make corresponding adjustments, such as data padding, byte order conversion, etc.

[0086] The transaction management module is used to maintain the status and mapping relationship of transactions to ensure that requests and responses can be correctly matched. Since different protocols may have different transaction processing methods, it is necessary to establish a unified transaction management mechanism. The workflow of the transaction management module may include: (1) Transaction ID mapping. Assign a corresponding target protocol transaction ID to each CCN interconnection protocol transaction ID and record it in the transaction mapping table. When a request message from the CCN interconnection protocol is received, a new target protocol transaction ID is generated and the correspondence between the two is recorded. (2) Transaction status tracking. Track the status of each transaction, such as request sent, response received, etc. When a response message from the target protocol is received, find the corresponding CCN interconnection protocol transaction ID according to the transaction mapping table, convert the response message back to the format of the CCN interconnection protocol, and send it to the device that initiated the request.

[0087] The error handling module is used to detect and handle various error situations during the protocol conversion process, such as message format errors, protocol incompatibility, etc., to ensure the stability and reliability of the system. The workflow of the error handling module can include: (1) Error detection, checking whether there are errors in each link of message parsing, protocol mapping and transaction processing. For example, checking whether the error check bit of the message is correct, whether there are any fields that cannot be mapped during the mapping process, etc. (2) Error reporting, when an error is detected, an error report message is generated, containing information such as the error type and error location. The error report message is sent to the relevant device or system management module for subsequent processing. (3) Error recovery, taking appropriate recovery measures based on the type and severity of the error. For example, for retryable errors, such as transmission errors, the message can be resent; for unrecoverable errors, such as protocol incompatibility, it may be necessary to terminate the transaction and notify the relevant devices.

[0088] The coherent interconnect control device 100 provided in an embodiment of the present invention can replace the first switch to perform read and write interactions between accelerator cards. For example, when accelerator card 0 needs to obtain first data and cannot find the first target data locally, accelerator card 0 queries the sharer list corresponding to the corresponding cache line through the first directory in the coherent interconnect control device 100, and determines that accelerator card 2 and accelerator card 4 are sharers of the cache line. At this time, accelerator card 0 sends a read request for the first data in accelerator card 2 and accelerator card 4 to the coherent interconnect control device 100. The coherent interconnect control device 100 receives the read request from accelerator card 0 through the first connector 102 corresponding to accelerator card 0, parses and determines the output port, and then sends the read request to accelerator card 2 and accelerator card 4 respectively after arbitration and queuing. If accelerator card 2 or accelerator card 4 finds a hit in the local cache for the first data, the first data is returned to accelerator card 0 through the coherent interconnect control device 100. If the first data is not found in the local cache, the first data is retrieved from the local memory and returned to accelerator card 0 through the coherent interconnect control device 100. If the data is not found in the local memory, a read failure message is returned.

[0089] If accelerator card 0 needs to write the second data to its cache line, accelerator card 0 queries the sharer list corresponding to the corresponding cache line through the first directory in the coherent interconnect control device 100, and determines that accelerator card 2 and accelerator card 4 are sharers of the cache line. At this time, accelerator card 0 sends a write request to accelerator card 2 and accelerator card 4 to the coherent interconnect control device 100. The coherent interconnect control device 100 receives the write request from accelerator card 0 through the first connector 102 corresponding to accelerator card 0, parses and determines the output port, and sends the write request to accelerator card 2 and accelerator card 4 respectively after arbitration and queuing. The coherent interconnect control device 100 can write the second data to the memory of accelerator card 2 and accelerator card 4 through Remote Direct Memory Access (RDMA).

[0090] An embodiment of the present invention also provides a computing system, which may include a host, multiple accelerator cards, a first switch and a coherent interconnection control device; the accelerator card is connected to the host through the first switch; the coherent interconnection control device includes a coherent interconnection network, a first controller and a first connector; the coherent interconnection network includes multiple interconnected coherent interconnection nodes, and the coherent interconnection nodes are connected to the first connector one-to-one; the other end of the first connector is connected to the accelerator card; the first controller is used to manage the status of the cache line of the accelerator card, and control the coherent interconnection network to perform coherent interconnection read and write tasks between different accelerator cards.

[0091] The computing system provided by the embodiment of the present invention may refer to the description of the above-mentioned embodiment of the coherent interconnection control device.

[0092] An embodiment of the present invention also provides a consistency interconnection control method, which is applied to a first controller. The consistency interconnection control method may include: obtaining and managing the status of the cache line of the accelerator card through the first connector; controlling the consistency interconnection network to perform consistency interconnection read and write tasks between different accelerator cards; wherein the consistency interconnection network includes a plurality of interconnected consistency interconnection nodes, and the consistency interconnection nodes are connected to the first connector in a one-to-one correspondence; the other end of the first connector is connected to the first slot, and the first slot is used to install the accelerator card.

[0093] The coherent interconnection control method provided by the embodiment of the present invention may refer to the description of the above-mentioned coherent interconnection control device embodiment.

[0094] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned coherent interconnection control method embodiments.

[0095] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps of any of the above-mentioned coherent interconnection control method embodiments when running.

[0096] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0097] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned coherent interconnection control method embodiments are implemented.

[0098] An embodiment of the present invention further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned coherent interconnection control method embodiments are implemented.

[0099] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0100] The above describes in detail the consistent interconnection control device, method, electronic device, product, and computing system provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only intended to help understand the method and core concept of the present invention. It should be noted that for those skilled in the art, without departing from the principles of the present invention, various improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A consistent interconnection control device, characterized in that: include: a coherent interconnect network, a first controller, and a first connector; The coherent interconnect network includes a plurality of interconnected coherent interconnect nodes, and the coherent interconnect nodes are connected to the first connectors in a one-to-one correspondence; The other end of the first connector is connected to a first slot, and the first slot is used to install an accelerator card; The first controller is used to manage the status of the cache line of the acceleration card and control the coherent interconnection network to perform coherent interconnection read and write tasks between different acceleration cards.

2. The coherent interconnection control device according to claim 1, characterized in that: The first controller manages the state of the cache line of the accelerator card, including: The first controller updates the first directory according to the state of the cache line; The first directory includes a sharer list corresponding to the cache line, a memory address corresponding to the cache line, and a cache line status of the cache line.

3. The coherent interconnection control device according to claim 2, characterized in that: The state of the cache line is one of a modified state, an exclusive state, a shared state, and an invalid state.

4. The coherent interconnection control device according to claim 1, characterized in that: The first controller controls the coherent interconnect network to perform coherent interconnect read and write tasks between different accelerator cards, including: The first controller determines a sender accelerator card and a receiver accelerator card of a first data packet received by the coherent interconnect control device, determines a target forwarding path between an input node and an output node in the coherent interconnect network, and controls the coherent interconnect network to forward the first data packet according to the target forwarding path so as to be sent to the receiver accelerator card through the output node and its corresponding first connector. The input node is the coherent interconnect node corresponding to the sender accelerator card, and the output node is the coherent interconnect node corresponding to the receiver accelerator card.

5. The coherent interconnection control device according to claim 4, characterized in that: The first controller determines a target forwarding path between an input node and an output node from the coherent interconnect network, including: The first controller searches for the shortest forwarding path between the input node and the output node according to the interconnection topology of the coherent interconnection network; If the shortest forwarding path is occupied, continue searching for the shortest forwarding path among the remaining forwarding paths from the input node to the output node; If an unoccupied forwarding path is found, the currently searched forwarding path is used as the target forwarding path; If all the forwarding paths between the input node and the output node are occupied, the target forwarding path is determined according to the data priority of the first data packet.

6. The coherent interconnection control device according to claim 5, characterized in that: The first controller determines the target forwarding path according to the data priority of the first data packet, including: If the data priority of the first data packet is higher than a first threshold, determining the shortest forwarding path found for the first time to be the target forwarding path, so as to share the shortest forwarding path with the data being transmitted by the shortest forwarding path; If the data priority of the first data packet is lower than a second threshold, the first data packet is controlled to enter a waiting state, and the occupancy state of the forwarding path is monitored to determine the idle forwarding path as the target forwarding path.

7. The coherent interconnection control device according to claim 4, characterized in that: The first controller is further configured to determine the data transmission order of different input nodes according to the data priorities of the coherent interconnection read and write tasks of different input nodes when it is detected that multiple input nodes simultaneously request the same output port of the same output node.

8. The coherent interconnection control device according to claim 4, characterized in that: The first controller is also used to pipeline the consistent interconnection read and write tasks corresponding to each input node in stages when it is detected that multiple input nodes simultaneously request the same output port of the same output node, and to execute the consistent interconnection read and write tasks of different input nodes in a pipeline parallel manner.

9. The coherent interconnection control device according to claim 1, characterized in that: Also includes a protocol conversion interface; The protocol conversion interface is provided between the first connector and the coherent interconnection node, and is used to perform a protocol conversion task between the coherent interconnection protocol of the accelerator card corresponding to the first connector and the coherent interconnection protocol of the first controller.

10. The coherent interconnection control device according to claim 9, characterized in that: The protocol conversion interface performs the protocol conversion task, including: Extract the message fields of the input message according to the input party's consistency conversion protocol; According to the consistency conversion protocol of the output party, the message field is converted into a message type, an address code, and a data format.

11. A computing system, characterized in that: include: A host, a plurality of accelerator cards, a first switch, and a coherent interconnection control device; The accelerator card is connected to the host through the first switch; The coherent interconnect control device includes a coherent interconnect network, a first controller, and a first connector; the coherent interconnect network includes a plurality of interconnected coherent interconnect nodes, and the coherent interconnect nodes are connected to the first connectors in a one-to-one correspondence; the other end of the first connector is connected to the accelerator card; The first controller is used to manage the status of the cache line of the acceleration card and control the coherent interconnection network to perform coherent interconnection read and write tasks between different acceleration cards.

12. A consistent interconnection control method, characterized in that: Applied to the first controller, comprising: Acquire and manage the state of the cache line of the accelerator card through the first connector; Controlling the coherent interconnect network to execute coherent interconnect read and write tasks between different accelerator cards; The coherent interconnect network includes a plurality of interconnected coherent interconnect nodes, and the coherent interconnect nodes are connected to the first connector in a one-to-one correspondence; the other end of the first connector is connected to the first slot, and the first slot is used to install the acceleration card.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the coherent interconnect control method according to claim 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the coherent interconnection control method according to claim 12 when executed by a processor.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the coherent interconnection control method according to claim 12 are implemented.

Images