Method and on-chip bus interconnect system for accessing memory

Abstract

The embodiment of the application provides a method for accessing memory and an on-chip bus interconnection system, the system comprises at least one chip, each chip comprises at least one die, each die comprises at least one request node, at least one consistency maintenance node and at least one slave node, and the method comprises the following steps: a first request node can determine first attribute information of a target NUMA node to be accessed by a first access request initiated by the first request node, determine a first consistency maintenance node located in the same chip and / or the same die as the first request node according to the first attribute information, and send the first access request to the first consistency maintenance node; the first consistency maintenance node processes the first access request, obtains second attribute information of the target NUMA node, and forwards the first access request to a target slave node directly or through a second consistency maintenance node according to the second attribute information.

Description

TECHNICAL FIELD

[0001] The present document relates to the technical field of computer, and particularly relates to a method for accessing memory and an on-chip bus interconnection system. BACKGROUND

[0002] The on-chip bus interconnection system includes a plurality of chips, each chip includes a plurality of dies, wherein each die can form a distributed storage space by mounting a storage device (such as a double data rate synchronous dynamic random access memory (DDR)), and the distributed storage space is used to store instructions and working data of a plurality of CPUs in the system.

[0003] Under the distributed storage system, software performs memory management by dividing non-uniform memory access nodes (NUMA nodes). Specifically, different dies and / or different chips can be divided into the same NUMA node, and a single die can also be divided into a plurality of NUMA nodes. This division method significantly increases the routing complexity of the CPU accessing the memory in the on-chip bus interconnection system, and if the routing mechanism cannot be flexibly adapted to the NUMA division method, the application scenarios of the on-chip bus interconnection system will be limited.

[0004] Therefore, it is necessary to propose a routing mechanism that can flexibly adapt to different NUMA node division methods. SUMMARY

[0005] Embodiments of the present application provide a method for accessing memory and an on-chip bus interconnection system to flexibly adapt to different NUMA node division methods.

[0006] To solve the above technical problems, embodiments of the present application are implemented as follows:

[0007] In a first aspect, a method for accessing memory in an on-chip bus interconnection system is provided, the on-chip bus interconnection system including at least one chip, each chip including at least one die, each die including a plurality of nodes, the plurality of nodes including at least one request node, at least one coherence maintenance node, and at least one slave node, the slave node being connected to a storage device, the storage device having memory accessed by the request node, and the method including:

[0008] The first request node initiates a first access request, determines first attribute information of a target non-uniform memory access (NUMA) node to be accessed by the first access request, determines a first consistency maintenance node according to the first attribute information, and sends the first access request to the first consistency maintenance node, wherein the first request node is any request node in any die in any chip, the first attribute information is used to indicate a positional relationship between the target NUMA node and the first request node, and the first consistency maintenance node is located in the same chip and / or the same die as the first request node.

[0009] The first consistency maintenance node processes the first access request, obtains second attribute information of the target NUMA node, and forwards the first access request according to the second attribute information, wherein the second attribute information is used to indicate a positional relationship between the target NUMA node and the first consistency maintenance node.

[0010] The first consistency maintenance node forwards the first access request according to the second attribute information, including:

[0011] The first consistency maintenance node forwards the first access request to a target slave node according to the second attribute information, wherein the target slave node and the first consistency maintenance node are located in the target NUMA node, and the target slave node and the first consistency maintenance node are located in the same chip and / or the same die.

[0012] Alternatively,

[0013] The first consistency maintenance node forwards the first access request to a second consistency maintenance node according to the second attribute information, wherein the second consistency maintenance node and the first consistency maintenance node are located in different NUMA nodes, and / or the second consistency maintenance node and the first consistency maintenance node are located in different chips and / or different dies; and the method further includes: the second consistency maintenance node forwards the first access request to a target slave node.

[0014] In a second aspect, an on-chip bus interconnection system is provided, including at least one chip, each chip including at least one die, each die including a plurality of nodes, the plurality of nodes including at least one request node, at least one consistency maintenance node, and at least one slave node, the slave node being connected to a storage device, the storage device having a memory accessed by the request node; wherein:

[0015] A first request node configured to initiate a first access request, determine first attribute information of a destination non-uniform memory access (NUMA) node to be accessed by the first access request, determine a first consistency maintenance node according to the first attribute information, and send the first access request to the first consistency maintenance node, wherein the first request node is any request node in any die in any chip, the first attribute information is used to indicate a positional relationship between the destination NUMA node and the first request node, and the first consistency maintenance node is located in the same chip and / or the same die as the first request node.

[0016] The first consistency maintenance node is configured to process the first access request, obtain second attribute information of the destination NUMA node, and forward the first access request according to the second attribute information, wherein the second attribute information is used to indicate a positional relationship between the destination NUMA node and the first consistency maintenance node.

[0017] The first consistency maintenance node is configured to forward the first access request to a destination slave node according to the second attribute information, wherein the destination slave node and the first consistency maintenance node are located in the destination NUMA node, and the destination slave node and the first consistency maintenance node are located in the same chip and / or the same die.

[0018] Alternatively,

[0019] The first consistency maintenance node is configured to forward the first access request to a second consistency maintenance node according to the second attribute information, wherein the second consistency maintenance node and the first consistency maintenance node are located in different NUMA nodes, and / or the second consistency maintenance node and the first consistency maintenance node are located in different chips and / or different dies; and the second consistency maintenance node is configured to forward the first access request to a destination slave node.

[0020] In a third aspect, a processor is provided, which includes an on-chip bus interconnection system as described in the second aspect.

[0021] In the embodiments of the present application, for an on-chip bus interconnection system, the system includes at least one chip, each chip includes a plurality of dies, each die includes a request node (RN), a consistency maintenance node (CHN) and a slave node (SN). When any one of the request nodes in any one of the dies initiates a first access request, first attribute information of a target NUMA node is determined, a first consistency maintenance node located in the same die and / or the same chip as the request node is determined according to the first attribute information, and the first access request is forwarded to the first consistency maintenance node; then, the first consistency maintenance node processes the first access request, obtains second attribute information of the target NUMA node, and forwards the first access request to the target slave node directly or through a second consistency maintenance node according to the second attribute information. The routing mechanism of the access request initiated by the first request node through one or two levels of consistency maintenance nodes can flexibly adapt to various NUMA node division modes, including different scenarios such as cross-chip, cross-die or division into multiple NUMA nodes in the same die. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0023] Figure 1A is a structural schematic diagram of an on-chip bus interconnection system provided by an embodiment of the present application.

[0024] Figure 1A is a structural schematic diagram of a chip with a Mesh structure provided by an embodiment of the present application.

[0025] Figure 2 is a flowchart of a method for accessing a memory in an on-chip bus interconnection system provided by an embodiment of the present application.

[0026] Figure 3 is a structural schematic diagram of a first routing component provided by an embodiment of the present application.

[0027] Figure 4 is a structural schematic diagram of a second routing component provided by an embodiment of the present application.

[0028] Figure 5 is a routing path schematic diagram of a memory access request in a cross-die NUMA node division scenario provided by an embodiment of the present application.

[0029] Figure 6 is a routing path diagram of a memory access request in an intra-die NUMA node division scenario provided by an embodiment of the present application.

[0030] Figure 7 is a routing path diagram of a memory access request in a cross-die NUMA node division scenario provided by an embodiment of the present application.

[0031] Figure 8 is a comparison diagram of a routing path of a memory access request in a cross-die NUMA node division scenario provided by an embodiment of the present application, in which a local CHN caches data under a memory address of a remote NUMA and does not cache data under a memory address of a remote NUMA.

[0032] Figure 9 is a routing path diagram of a memory access request in an intra-die NUMA node division scenario provided by an embodiment of the present application, in which a SN and a CHN in the same die are not bound to each other.

[0033] Figure 10 is a routing path diagram of a memory access request in an intra-die NUMA node division scenario provided by an embodiment of the present application, in which a SN and a CHN in the same die are bound to each other. DETAILED DESCRIPTION

[0034] In order to make the person skilled in the art better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in one or more embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor should belong to the protection scope of the present document.

[0035] The terms "first", "second", and the like in the present application and claims are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein. In addition, "and / or" in the present application and claims means at least one of the connected objects, and the character " / " generally means that the front and rear associated objects are in an "or" relationship.

[0036] In order to flexibly adapt to different NUMA node division modes, the present application provides a method for accessing memory in a bus interconnection system on a chip, a bus interconnection system on a chip, and a processor, wherein the method for accessing memory in the bus interconnection system on a chip can be applied to the bus interconnection system on a chip, which will be described one by one.

[0037] In the embodiments of the present application, the on-chip bus interconnection system can be a multi-core coherent bus interconnection system.

[0038] First, the structure of the on-chip bus interconnection system to which the method for accessing the memory in the on-chip bus interconnection system proposed in the embodiments of the present application can be applied is exemplified. The on-chip bus interconnection system can include at least one chip, each chip can include at least one die, and each die can include a plurality of nodes, which can include at least one request node (RN), at least one coherent home node (CHN), and at least one subordinate node (SN), wherein the SN can be connected to a storage device having a memory for access by the request node, and the storage device can be a double data rate synchronous dynamic random access memory (DDR), a register, or other volatile memory storage device. The storage device connected to each SN in the on-chip bus interconnection system can serve as the storage space of the entire system and be used to store the instructions and working data of a plurality of CPUs.

[0039] In some embodiments, the on-chip bus interconnection system is a mesh network, and the on-chip bus interconnection system can use a coherent bus, such as a coherent hub interface (CHI) bus, to connect the components in the mesh network together. The mesh network is a physical connection topology in which the computing nodes (such as CPU cores, memory controllers) on the chip are arranged in a two-dimensional grid, each node is directly connected to the adjacent nodes, and the network can solve the bandwidth bottleneck of the traditional bus and support large-scale parallel communication.

[0040] In some embodiments, the on-chip bus interconnection system can adopt a symmetric multiprocessing (SMP) structure, i.e., the on-chip bus interconnection system includes a plurality of symmetric chips (i.e., a plurality of chips with the same structure), and each two chips can communicate through a chip-to-chip (C2C) node, each chip includes at least one die, and each die includes at least one RN, at least one CHN, and at least one SN.

[0041] In some embodiments, as Figure 1AAs shown, the on-chip bus interconnect system may include chip 0 and chip 1. Chip 0 may include die 0, die 1, die 2, and die 3. Die 0 may include RN0, RN1, CHN0, CHN1, and at least one SN. The structures of die 1, die 2, and die 3 are the same as those of die 0. Furthermore, the structure of chip 1 is also the same as that of chip 0. Figure 1A The structure of chip 1 is not shown in the diagram. Please refer to chip 0 for details on the structure of chip 1.

[0042] In some embodiments, the grain may also be referred to as a CCD (cpu die), and the CCD may be a physical grain integrating at least one RN, at least one CHN and at least one SN.

[0043] In some embodiments, such as Figure 1A As shown, different chips (chip 0 and chip 1) can be connected via C2C, and different dies within the same chip (such as die 0, die 1, die 2 and die 3) can be connected via Die to Die (DTD).

[0044] The functions of RN, CHN, C2C, SN, and DTD are briefly introduced below.

[0045] In some embodiments, the RN can be used to connect to the CPU or other master devices and can initiate different types of access requests, such as read / write, to the consistent memory space.

[0046] In some embodiments, CHN can be used to cache the memory space address cache line it manages, maintaining cache consistency between the CPU and the storage device.

[0047] In some embodiments, C2C can be used for cross-chip routing transmission between chips. Cross-chip communication generally uses a different transmission protocol than on-chip communication, such as Channel-to-Channel Protocol or Peripheral Component Interconnect Express (PCIE).

[0048] In some embodiments, the SN can be used to mount or connect a Double Data Rate Synchronous Dynamic Random Access Memory (DDR) or other types of volatile memory storage devices in response to an access request for memory space initiated by the RN.

[0049] In some embodiments, DTD can be used for data transmission between two dies to complete data packet verification and timing processing for data transmission between dies, ensuring the reliability of data transmission.

[0050] In some embodiments, the first routing component is configured in the request node (RN) and the cross-chip node (C2C), and the second routing component is configured in the consistency maintenance node (CHN).

[0051] In some embodiments, the first routing component is configured with a request node address routing table (RN Address Route Table, RART) containing a mapping relationship between a memory address and a node identifier (such as a node ID), which is used to process a memory address to be accessed in a node initiating an access request and find a target node of a next level of routing. For example, for the first routing component in the request node (RN), the request node address routing table configured therein contains a mapping relationship between a memory address and a CHN identifier, which is used to process a memory address to be accessed by an access request initiated by the request node (RN) in the request node (RN) and find a first consistency maintenance node (such as a first CHN, which can be referred to as a primary CHN). For another example, for the first routing component in the cross-chip node (C2C), the request node address routing table configured therein contains a mapping relationship between a memory address and an identifier of a consistency maintenance node, which is used to process a memory address to be accessed by an access request received by the cross-chip node (C2C) in the cross-chip node (C2C) and find a second consistency maintenance node (such as a second CHN, which can be referred to as a secondary CHN).

[0052] In some embodiments, the second routing component is configured with a local node address routing table (CHN Address Route Table, HART) containing a mapping relationship between a memory address and a node identifier (such as a node ID), which is used to process a memory address to be accessed in a consistency maintenance node (CHN) according to a routing rule and find a target node of a next level of routing. For example, for the second routing component in the first consistency maintenance node (first CHN), the local node address routing table configured therein contains a mapping relationship between a memory address and an SN identifier / CHN identifier / C2C identifier, which is used to process a memory address to be accessed by a first access request received in the first consistency maintenance node (first CHN) and find a destination slave node / second consistency maintenance node / first cross-chip node as described below. For another example, for the second routing component in the second consistency maintenance node (second CHN), the local node address routing table configured therein contains a mapping relationship between a memory address and an identifier of a slave node, which is used to process a memory address to be accessed by an access request received in the second consistency maintenance node (second CHN) and find a destination slave node.

[0053] Figure 1B A structure diagram of one chip with a Mesh structure is shown. As shown in FIG. 1, the chip includes a plurality of request nodes (RN), a plurality of consistency maintenance nodes (CHN), and a plurality of cross-chip nodes (C2C). The request nodes (RN) are connected to the consistency maintenance nodes (CHN) through the cross-chip nodes (C2C). The request nodes (RN) are used to initiate an access request, and the consistency maintenance nodes (CHN) are used to process the access request. The cross-chip nodes (C2C) are used to transmit the access request from the request node (RN) to the consistency maintenance node (CHN). Figure 1BAs shown, chip 0 contains 4 dies, each of which contains multiple RNs, multiple CHNs, and multiple SNs, wherein the multiple SNs are used to mount a DDR storage device. For this chip structure, the routing path of a memory access request under different NUMA partitioning manners can be determined by the first routing component containing RART in the RN and C2C node and the second routing component containing HART in the CHN node.

[0054] A method for accessing memory in a bus interconnection system on a chip is provided in the embodiments of the present application, which will be described below in combination with the accompanying drawings.

[0055] An embodiment of the present application provides a method for accessing memory in a bus interconnection system on a chip, which can be applied to the above-mentioned bus interconnection system on a chip, which can include at least one chip, each of which includes at least one die, each of which includes multiple nodes, the multiple nodes including at least one request node (RN), at least one consistency maintenance node (CHN), and at least one slave node (SN), the slave node (SN) being connected to a storage device having memory accessed by the request node (RN). As shown, Figure 2 The method can include the following steps:

[0056] In step 201, a first request node initiates a first access request, determines first attribute information of a destination non-uniform memory access (NUMA) node to be accessed by the first access request, determines a first consistency maintenance node according to the first attribute information, and sends the first access request to the first consistency maintenance node.

[0057] The first request node can be any request node in any die on any chip included in the bus interconnection system on a chip, the first access request can be any access request initiated by the first request node, the first attribute information can be used to indicate the positional relationship between the destination NUMA node and the first request node, and the first consistency maintenance node can be located in the same chip and / or the same die as the first request node.

[0058] In some embodiments, the first access request can be a read request for a memory address to be accessed, or a write request for a memory address to be accessed by the first access request.

[0059] In some embodiments, the memory address space of the on-chip bus interconnection system can be divided into a plurality of memory address intervals in advance, such as N Hash Map Regions (HMRs), and different memory address intervals can be managed by different NUMA nodes. These memory address intervals are part of the memory address space of the entire system, and a memory address interval can be defined by a base address (start address) and an end address, or a memory address interval can be defined by a base address (its start address) and a region size. On this basis, the destination NUMA node can be the NUMA node that manages the memory address to be accessed by the first access request.

[0060] In some embodiments, in step 201, the first request node determines the first attribute information of the destination NUMA node to be accessed by the first access request, including:

[0061] The first request node determines the target memory address interval hit by the memory address to be accessed by the first access request, wherein the NUMA node to which the target memory address interval belongs is the destination NUMA node;

[0062] The first request node determines the first attribute information of the destination NUMA node according to the configuration information of the target memory address interval. The configuration information of the target memory address interval can include, but is not limited to, at least one of the following information: NUMA node information to which the target memory address interval belongs, and consistent maintenance node information of the die and NUMA node in which the target memory address interval is managed.

[0063] In some embodiments, the first attribute information of the destination NUMA node is used to indicate at least one of the following:

[0064] Whether the destination NUMA node is the NUMA node in which the first request node is located;

[0065] Whether the destination NUMA node is a cross-chip NUMA node;

[0066] Whether the destination NUMA node and the first request node are located in the same chip;

[0067] Whether the destination NUMA node is a cross-die NUMA node, and the like.

[0068] In some embodiments, the first request node determines the first consistent maintenance node according to the first attribute information, including one of the following operations:

[0069] 1) In a case where the first attribute information indicates that the destination NUMA node is the NUMA node where the first request node is located, the destination NUMA node does not belong to the cross-chip NUMA node, and the destination NUMA node belongs to or does not belong to the cross-die NUMA node, the first request node determines the first consistency maintenance node from the NUMA node where the first request node is located and the die, that is, the first request node determines the first consistency maintenance node from the NUMA node where the first request node is located and the die where the first request node is located.

[0070] For example, the destination NUMA node includes die 0 in Chip 1, the first RN initiating the first access request is located in die 0 in Chip 1, that is, the destination NUMA node is the NUMA node where the first request node is located, the destination NUMA node does not belong to the cross-chip NUMA node, and the destination NUMA node does not belong to the cross-die NUMA node, and then the first RN can determine the first CHN from die 0 in Chip 1.

[0071] For example, the destination NUMA node includes die 0 and die 1 in Chip 1, the first RN initiating the first access request is located in die 0 in Chip 1, that is, the destination NUMA node is the NUMA node where the first request node is located, the destination NUMA node does not belong to the cross-chip NUMA node, and the destination NUMA node belongs to the cross-die NUMA node, and then the first RN can determine the first CHN from die 0 in Chip 1.

[0072] 2) In a case where the first attribute information indicates that the destination NUMA node is the NUMA node where the first request node is located, and the destination NUMA node belongs to the cross-chip NUMA node, the first request node determines the first consistency maintenance node from the NUMA node where the first request node is located and the chip, that is, the first request node determines the first consistency maintenance node from the NUMA node where the first request node is located and the chip where the first request node is located.

[0073] For example, the destination NUMA node includes Chip 1 and chip 2, the first RN initiating the first access request is located in Chip 1, that is, the destination NUMA node is the NUMA node where the first request node is located, and the destination NUMA node belongs to the cross-chip NUMA node, and then the first RN can determine the first CHN from Chip 1.

[0074] 3) In a case where the first attribute information indicates that the destination NUMA node is not the NUMA node where the first request node is located, the first request node determines the first consistency maintenance node from the chip and / or die where the first request node is located.

[0075] For example, the destination NUMA node includes Chip1 and Chip2, the first RN initiating the first access request is located in die 0 in Chip0, i.e., the destination NUMA node is not the NUMA node where the first request node is located, and the first RN can determine the first CHN from die 0 in Chip0 or Chip0.

[0076] It can be seen that the first request node always determines the first consistency maintenance node in the die and / or chip where the first request node is located.

[0077] In some embodiments, the consistency maintenance node identifier list is provided in each request node, and the consistency maintenance node identifier list stores an identifier (which can be regarded as a global identifier of the consistency maintenance node) of at least one consistency maintenance node managing the target memory address interval in the die and / or chip where the first request node is located, which can be determined according to the location information defined by the on-chip bus interconnection system. In the Mesh interconnection network, the ID composed of the x and y axes of the network can be used as the identifier, which is used to find the next routing node for processing the access request for accessing the memory address in the request node. On this basis, the first request node determines the first consistency maintenance node from the die and / or chip where the first request node is located, which can include:

[0078] The first request node determines the target memory address interval hit by the memory address to be accessed by the first access request according to the memory address to be accessed by the first access request and the memory address interval managed by each NUMA node of the on-chip bus interconnection system.

[0079] The first request node determines the first hash parameter according to the configuration information of the target memory address interval, wherein the first hash parameter includes the number of consistency maintenance nodes managing the target memory address interval in the die and / or chip where the first request node is located and the hash mode of the consistency maintenance nodes.

[0080] The first request node performs hashing in the die and / or chip where the first request node is located according to the first hash parameter to obtain the first consistency maintenance node.

[0081] In some embodiments, the first routing component is provided in each request node, and the RART is configured in the first routing component. The RART in the first routing component includes the consistency maintenance node identifier list, and the consistency maintenance node identifier list stores an identifier of at least one consistency maintenance node managing the target memory address interval in the die and / or chip where the first request node is located, which is used to find the next routing node for processing the access request for accessing the memory address in the request node.

[0082] As an example, asFigure 3 As shown, the first routing component 300 in the first request node can include: an address matching unit 301, a hash attribute processing unit 302, a hash calculation unit 303, and a CHN identification list 304 (i.e., RART). After the first request node initiates the first access request, the first routing component 300 can input the memory address to be accessed by the first access request, process the memory address, and output the identification of the first consistency maintenance node of the first access request, i.e., the identification of the first CHN.

[0083] Specifically, as shown in FIG. 3, the first routing component 300 can include the following components: Figure 3 As shown, the first routing component 300 processes the memory address to be accessed by the first access request to obtain the identification of the first consistency maintenance node of the first access request, which can include:

[0084] A. The address matching unit 301 can pre-divide the memory address space of the on-chip bus interconnection system into a plurality of memory address intervals, such as N Hash Map Regions (HMRs). Different memory address intervals can be used to support different NUMA nodes. These memory address intervals are part of the memory address space of the entire system. One memory address interval can be defined by a base address (start address) and an end address, or one memory address interval can be defined by a base address (start address) and a region size. On this basis, when the first request node (such as a CPU processor or other external master device) initiates the first access request (a piece of memory space access), the first routing component 300 can input the memory address to be accessed by the first access request. First, the address matching unit 301 can identify the target memory address interval hit by the memory address. Different memory address intervals belong to different NUMA nodes, and the memory address ranges of different NUMA nodes do not overlap. In some embodiments, the address matching unit 301 can check whether the memory address hits the corresponding memory address interval in order from high to low according to the priority of the NUMA node, thereby determining the target memory address interval hit by the memory address.

[0085] B. The hash attribute processing unit 302 obtains a first hash parameter according to the configuration information of the target memory address interval, wherein the first hash parameter includes the number of CHNs in the die and / or chip where the first request node is located and the CHN hash mode for managing the memory address to be accessed by the first access request. The CHN hash mode can include at least one of the following hash calculation modes: power-of-two hash (P2), non-power-of-two hash (NP2), and hierarchical hash (Hierarchy Hash).

[0086] C, the hash calculation unit 303 hashes according to the first hash parameter to obtain the index (index) of the first CHN. For example, the hash calculation unit 303 performs hash calculation on the addresses of the plurality of CHNs managing the memory address in the die or chip where the first request node is located according to the first hash parameter, so as to uniformly hash the memory address on a certain CHN node in the plurality of CHN nodes. Wherein, the plurality of CHNs are located locally to the first request node, that is, the plurality of CHNs and the first request node are located in the same die or the same chip, and optionally, the plurality of CHNs and the first request node can also be located in the same NUMA node; the hash calculation algorithm can include but is not limited to at least one of power-of-two hash (P2), non-power-of-two hash (NP2) and hierarchical hash calculation, and the hash calculation unit 303 can support the combination of different hash calculation algorithms generated under multiple NUMA division modes. For example, as shown in Figure 5 、 Figure 6 or Figure 7 The die and the NUMA node where the RN initiating the first access request is located have 4 CHN nodes, and the power-of-two hash calculation of 4 can be used for the memory address to be accessed, so that the memory address to be accessed by the first access request is uniformly hashed on a certain CHN in the 4 CHNs.

[0087] D, after the hash calculation unit 303 calculates the index (index) of the first CHN, the first routing component 300 queries the CHN identifier list 304 to determine the identifier (such as coordinates) of the first CHN in the network, so as to forward the first access request to the first CHN. Wherein, the index (index) of the first CHN calculated by the hash calculation unit 303 can be regarded as the local identifier of the first CHN, and the CHN identifier list 304 stores the identifier (which can be regarded as the global identifier of the CHN) of the CHN managing the memory address in the NUMA node where the first request node is located in the network, which can be determined according to the position information defined by the on-chip bus interconnection system. In the Mesh interconnection network, the ID composed of the x and y axes of the network can be used as the identifier.

[0088] Step 202, the first consistency maintenance node processes the first access request to obtain the second attribute information of the target NUMA node, and forwards the first access request according to the second attribute information.

[0089] Wherein, the second attribute information is used to indicate the positional relationship between the target NUMA node and the first consistency maintenance node.

[0090] In some embodiments, the first consistency maintenance node forwards the first access request according to the second attribute information, which can include one of the following cases:

[0091] Case 1: The first consistency maintenance node forwards the first access request to a target slave node according to the second attribute information, wherein the target slave node and the first consistency maintenance node are located in a target NUMA node, and the target slave node and the first consistency maintenance node are located in the same chip and / or the same die.

[0092] Case 2: The first consistency maintenance node forwards the first access request to a second consistency maintenance node according to the second attribute information, wherein the second consistency maintenance node and the first consistency maintenance node are located in different NUMA nodes, and / or the second consistency maintenance node and the first consistency maintenance node are located in different chips and / or different dies. At this time, Figure 2 The method shown can further include: the second consistency maintenance node forwards the first access request to the target slave node.

[0093] In some embodiments, in step 202, the first consistency maintenance node processes the first access request to obtain the second attribute information of the target NUMA node, which can include at least one of the following:

[0094] The first consistency maintenance node obtains the first attribute information from the first access request, and determines the second attribute information of the target NUMA node according to the first attribute information. It can be understood that, since the first consistency maintenance node and the first request node are located in the same die and / or the same chip, the location relationship between the target NUMA node and the first consistency maintenance node can be derived according to the location relationship between the target NUMA node and the first request node indicated by the first attribute information, so as to obtain the second attribute information of the target NUMA node.

[0095] The first consistency maintenance node determines a target memory address interval hit by a memory address accessed by the first access request, and determines the second attribute information of the target NUMA node according to configuration information of the target memory address interval, wherein the NUMA node to which the target memory address interval belongs is the target NUMA node.

[0096] In some embodiments, the configuration information of the target NUMA node can include at least one of the following: a memory address interval managed by the target NUMA node, identification information of RNs, CHNs and SNs allocated to the target NUMA node, identification information of dies and chips in which the RNs, CHNs and SNs allocated to the target NUMA node are located.

[0097] In some embodiments, the second attribute information of the target NUMA node can be used to indicate at least one of the following:

[0098] 1) whether the destination NUMA node is the NUMA node where the first coherency maintenance node is located, i.e. the local / remote numa attribute of the destination NUMA node, if the local / remote numa attribute of the destination NUMA node is local numa, it indicates that the memory address to be accessed by the first coherency maintenance node is located in the local NUMA node, i.e. the destination NUMA node is the NUMA node where the first coherency maintenance node is located; otherwise, if the local / remote numa attribute of the destination NUMA node is remote numa, it indicates that the memory address to be accessed by the first coherency maintenance node is not located in the local NUMA node, i.e. the destination NUMA node is not the NUMA node where the first coherency maintenance node is located.

[0099] 2) whether the destination NUMA node is a cross-chip NUMA node. If the destination NUMA node is a non-cross-chip NUMA node, it can be further determined whether the destination NUMA node is located in a local chip according to the local / remote chip attribute of the destination NUMA node, if the local / remote chip attribute of the destination NUMA node is local, it indicates that the destination NUMA node is located in the local chip; otherwise, if the local / remote chip attribute of the destination NUMA node is remote chip, it indicates that the destination NUMA node is located in another chip; if the destination NUMA node is a cross-chip NUMA node, it is necessary to perform hash calculation according to the configured chip quantity to determine the local / remote chip attribute; after obtaining the local / remote chip attribute, it is determined whether it is a local chip access (the access of the local chip does not need to be cross-chip, and the access of the remote chip needs to be cross-chip). It can be understood that when the local / remote numa attribute of the destination NUMA node is local numa, and the local / remote chip attribute of the destination NUMA node is remote chip, it indicates that the accessed destination NUMA node is cross-chip.

[0100] 3) whether the destination NUMA node and the first coherency maintenance node are located in the same chip.

[0101] 4) Whether the destination NUMA node is a cross-die NUMA node. In the case that the destination NUMA node is a non-cross-chip NUMA node, it can be further determined whether the target node is a cross-die NUMA node, and if the NUMA node is a cross-die NUMA node, the local / remote die attribute of the destination NUMA node is determined according to the number of dies contained in the chip where the destination NUMA node is located. For example, the chip where the destination NUMA node is located contains two dies: die0 and die1, and the addresses thereof are calculated by a 2-power-of-2 hash, index = 0 represents die0, and index = 1 represents die1. When the first access request initiated by the first coherency maintenance node is in die0 and the calculated index = 0, it represents that the first access request is an access within the local die, which can be directly routed to the destination SN; index = 1 represents an access to the remote die, and the index = 1 is used to look up the corresponding CHN identification table to obtain the CHN identification of the memory address managed by the other die as the second coherency maintenance node. If the destination NUMA node is not a cross-die NUMA node, it represents that the destination NUMA node is another NUMA node within the chip, and the hash calculation can be directly performed according to the configuration information of the numa and the CHN identification list is queried to find the second coherency maintenance node. It can be understood that when the local / remote numa attribute of the destination NUMA node is local numa, the local / remote chip attribute of the destination NUMA node is local chip, and the local / remote die attribute of the destination NUMA node is remote die, it represents that the accessed destination NUMA node is cross-die.

[0102] It should be noted that in step 202, the implementation of the first coherency maintenance node forwarding the first access request according to the second attribute information is different in the case that the contents indicated by the second attribute information are different. That is, for different NUMA node division modes, the first coherency maintenance node forwards the first access request according to the second attribute information in different ways, which will be described in different cases.

[0103] For the above case 1, it can be further divided into the following subcases:

[0104] Subcase 1-1, the first coherency maintenance node directly forwards the first access request to the destination slave node according to the second attribute information, which can include:

[0105] In a case where the second attribute information indicates that the destination NUMA node is the NUMA node where the first consistency maintenance node is located, the destination NUMA node does not belong to the cross-chip NUMA node, and the destination NUMA node belongs to the cross-die NUMA node, the first consistency maintenance node determines a destination die from dies under jurisdiction of the destination NUMA node, and determines a destination slave node that manages the memory address to be accessed by the first access request from a die where the first consistency maintenance node is located if the destination die is the die where the first consistency maintenance node is located, and directly forwards the first access request to the destination slave node.

[0106] In a case where the second attribute information indicates that the destination NUMA node is the NUMA node where the first consistency maintenance node is located, the destination NUMA node does not belong to the cross-chip NUMA node, and the destination NUMA node belongs to the cross-die NUMA node, the first consistency maintenance node determines a destination die from dies under jurisdiction of the destination NUMA node, and determines a destination slave node that manages the memory address to be accessed by the first access request from a die where the first consistency maintenance node is located if the destination die is the die where the first consistency maintenance node is located, and directly forwards the first access request to the destination slave node.

[0107] In a case where the second attribute information indicates that the destination NUMA node is the NUMA node where the first consistency maintenance node is located, the destination NUMA node does not belong to the cross-chip NUMA node, and the destination NUMA node belongs to the cross-die NUMA node, the first consistency maintenance node determines a destination die from dies under jurisdiction of the destination NUMA node, and determines a destination slave node that manages the memory address to be accessed by the first access request from a die where the first consistency maintenance node is located if the destination die is the die where the first consistency maintenance node is located, and directly forwards the first access request to the destination slave node.

[0108] In a case where the second attribute information indicates that the destination NUMA node is the NUMA node where the first consistency maintenance node is located, the destination NUMA node does not belong to the cross-chip NUMA node, and the destination NUMA node belongs to the cross-die NUMA node, the first consistency maintenance node determines a destination die from dies under jurisdiction of the destination NUMA node, and determines a destination slave node that manages the memory address to be accessed by the first access request from a die where the first consistency maintenance node is located if the destination die is the die where the first consistency maintenance node is located, and directly forwards the first access request to the destination slave node.

[0109] In a case where the second attribute information indicates that the destination NUMA node is the NUMA node where the first consistency maintenance node is located, the destination NUMA node does not belong to the cross-chip NUMA node, and the destination NUMA node belongs to the cross-die NUMA node, the first consistency maintenance node determines a destination die from dies under jurisdiction of the destination NUMA node, and determines a destination slave node that manages the memory address to be accessed by the first access request from a die where the first consistency maintenance node is located if the destination die is the die where the first consistency maintenance node is located, and directly forwards the first access request to the destination slave node.

[0110] For the above case 2, it can be subdivided into the following subcases:

[0111] In a case where the second attribute information indicates that the destination NUMA node is the NUMA node where the first consistency maintenance node is located, the destination NUMA node does not belong to the cross-chip NUMA node, and the destination NUMA node belongs to the cross-die NUMA node, the first consistency maintenance node determines a destination die from dies under jurisdiction of the destination NUMA node, and determines a destination slave node that manages the memory address to be accessed by the first access request from a die where the first consistency maintenance node is located if the destination die is the die where the first consistency maintenance node is located, and directly forwards the first access request to the destination slave node.

[0112] The first consistency maintenance node determines a destination die from the dies under the jurisdiction of the destination NUMA node in a case where the second attribute information indicates that the destination NUMA node is the NUMA node where the first consistency maintenance node is located, the destination NUMA node does not belong to the cross-chip NUMA node, and the destination NUMA node belongs to the cross-die NUMA node, and if the destination die is different from the die where the first consistency maintenance node is located, the first consistency maintenance node determines a second consistency maintenance node from the destination die and directly forwards the first access request to the second consistency maintenance node.

[0113] Subcase 2-2, the first consistency maintenance node directly forwards the first access request to the destination slave node according to the second attribute information, can include:

[0114] The first consistency maintenance node determines a second consistency maintenance node from the destination NUMA node and directly forwards the first access request to the second consistency maintenance node in a case where the second attribute information indicates that the destination NUMA node is not the NUMA node where the first consistency maintenance node is located, the destination NUMA node does not belong to the cross-chip NUMA node, and the destination NUMA node is located in the same chip as the first consistency maintenance node.

[0115] Subcase 2-3, the one or more dies included in each chip further include at least one cross-chip node for cross-chip communication, and the first consistency maintenance node forwards the first access request to the second consistency maintenance node according to the second attribute information, including:

[0116] The first consistency maintenance node determines a destination die from the dies under the jurisdiction of the destination NUMA node in a case where the second attribute information indicates that the destination NUMA node is the NUMA node where the first consistency maintenance node is located, and the destination NUMA node belongs to the cross-chip NUMA node, and if the destination die is not the chip where the first consistency maintenance node is located, the first consistency maintenance node determines a second consistency maintenance node from the destination die and forwards the first access request to the second consistency maintenance node through the first cross-chip node and the second cross-chip node.

[0117] The first cross-chip node is a cross-chip node on the first chip configured for cross-chip communication between the first chip and the second chip, the first chip is the chip where the first consistency maintenance node is located, the second chip is the chip where the second consistency maintenance node is located, and the second cross-chip node is a cross-chip node on the second chip bound to the first cross-chip node.

[0118] Subcase 2-4, the one or more dies included in each chip further include at least one cross-chip node for cross-chip communication, and the first consistency maintenance node forwards the first access request to the second consistency maintenance node according to the second attribute information, including:

[0119] The first consistency maintenance node determines the second consistency maintenance node from the destination NUMA node in a case that the second attribute information indicates that the destination NUMA node is not the NUMA node where the first consistency maintenance node is located, the destination NUMA node does not belong to the cross-chip NUMA node, and the destination NUMA node is not on the same chip as the first request node, and forwards the first access request to the second consistency maintenance node through the first cross-chip node and the second cross-chip node.

[0120] The first cross-chip node is a cross-chip node on the first chip that is configured for cross-chip communication between the first chip and the second chip, the first chip is a chip where the first consistency maintenance node is located, the second chip is a chip where the second consistency maintenance node is located, and the second cross-chip node is a cross-chip node on the second chip that is bound to the first cross-chip node.

[0121] In the above sub-case 2-3 and sub-case 2-4, the first consistency maintenance node forwards the first access request to the second consistency maintenance node through the first cross-chip node and the second cross-chip node, which can include:

[0122] The first consistency maintenance node determines the first cross-chip node on the first chip, and forwards the first access request to the first cross-chip node.

[0123] The first cross-chip node forwards the first access request to the second cross-chip node on the second chip that is bound to the first cross-chip node.

[0124] The second cross-chip node determines the second consistency maintenance node according to a memory address to be accessed by the first access request.

[0125] The second cross-chip node sends the first access request to the second consistency maintenance node.

[0126] In some embodiments, when there are multiple first cross-chip nodes connected between the first chip and the second chip, the first access request can be uniformly hashed on the multiple first cross-chip nodes in a way of hash calculation, so as to balance the load of the first cross-chip nodes.

[0127] In some embodiments, each cross-chip node is provided with a consistency maintenance node identifier list, and the consistency maintenance node identifier list stores an identifier (which can be regarded as a global identifier of the consistency maintenance node) of at least one consistency maintenance node that manages the target memory address interval in the die and / or chip where the cross-chip node is located, which can be determined according to the location information defined by the on-chip bus interconnection system. In the Mesh interconnection network, the ID composed of the x and y axes of the network can be used as the identifier, which is used to find the next routing node in processing the access request for accessing the memory address in the request node. On this basis, the second cross-chip node determines the second consistency maintenance node according to the memory address to be accessed by the first access request, which can include:

[0128] The second cross-chip node determines the target memory address interval hit by the memory address to be accessed by the first access request according to the memory address to be accessed by the first access request and the memory address interval managed by each NUMA node corresponding to the on-chip bus interconnection system.

[0129] The second cross-chip node determines the first hash parameter according to the configuration information of the target memory address interval, wherein the first hash parameter includes the number of consistency maintenance nodes that manage the target memory address interval in the die and / or chip where the second cross-chip node is located and the hash mode of the consistency maintenance nodes.

[0130] The second cross-chip node performs hashing in the die and / or chip where the second cross-chip node is located to obtain the second consistency maintenance node according to the first hash parameter.

[0131] In some embodiments, each cross-chip node is provided with a first routing component, and the first routing component is configured with a request node address routing table (RN Address Route Table, RART). The RART in the first routing component includes a consistency maintenance node identifier list, and the consistency maintenance node identifier list stores an identifier of at least one consistency maintenance node that manages the target memory address interval in the die and / or chip where the cross-chip node is located, which is used to find the next routing node in processing the access request for accessing the memory address in the cross-chip node.

[0132] It should be noted that the first routing component in the second cross-chip node determines the second consistency maintenance node according to the memory address to be accessed by the first access request, which is similar to the process of processing the first access request by the first request node to obtain the first consistency maintenance node of the first access request, which will not be described here.

[0133] It should be noted that in addition to the above several cases, for different NUMA node partitioning manners, the implementation of the second routing component in the first consistency maintenance node determining the second consistency maintenance node of the first access request according to the attribute information can also have other cases, which will not be enumerated.

[0134] The implementation of the second routing component in the first consistency maintenance node (such as the first CHN) forwarding the first access request according to the second attribute information will be described below by an example. As shown in Figure 4 , it is assumed that the second routing component 400 in the first CHN includes an address matching unit 401, a hash attribute arbitration unit 402, a CHN hash calculation unit 403, an SN hash calculation unit 404, a C2C hash calculation unit 405, a CHN identification list 406, an SN identification list 407, and a C2C identification list 408. After receiving the first access request, the first CHN can input the memory address accessed by the first access request into the second routing component 400, and output the identification of the next routing target node of the first access request after processing by the second routing component 400.

[0135] Specifically, as shown in Figure 4 , the second routing component 400 forwarding the first access request according to the second attribute information can include:

[0136] A, the address matching unit 401 can pre-divide the memory address space of the on-chip bus interconnection system into a plurality of memory address intervals, such as N Hash Map Regions (HMRs), and different memory address intervals can be used to support different NUMA nodes. These memory address intervals are part of the memory address space of the entire system, and a memory address interval can be defined by a base address (start address) and an end address, or a memory address interval can be defined by a base address (start address) and a region size. On this basis, when the first CHN receives the first access request, the memory address accessed by the first access request is input into the second routing component 400, and the target memory address interval hit by the memory address accessed by the first access request is first identified by the address matching unit 401. Among them, different memory address intervals belong to different NUMA nodes, and the memory address intervals of different NUMA nodes do not overlap. In some embodiments, the address matching unit 401 can check whether the memory address accessed by the first access request hits the corresponding memory address interval in order from high to low according to the priority of the NUMA node, thereby determining the target memory address interval hit by the memory address accessed.

[0137] B, the hash attribute arbitration unit 402, can determine the target NUMA node according to the target memory address interval hit by the memory address to be accessed, and then determine the second attribute information of the target NUMA node according to the configuration information of the target NUMA node, and then arbitrate according to the second attribute information of the target NUMA node to select different hash calculation units for processing to obtain the identification (index) of the next routing target node. It can be understood that when the second attribute information indicates that the attributes of the target NUMA node are different, the next routing target node can be different. For example, according to the different attributes of the target NUMA node, the hash attribute arbitration unit 402 can perform the following arbitration processing:

[0138] 1) If the target NUMA node does not belong to the cross-chip NUMA node, the local / remote chip attribute of the target NUMA node is local chip (local chip), the local / remote numa attribute of the target NUMA node is local numa (local NUMA node), and the local / remote die attribute of the target NUMA node is local die (local die) (corresponding to the above sub-case 1-1), it indicates that the memory address accessed by the first CHN is located in the local NUMA node, the local chip and the local die, and the next routing target node is the target SN. The hash attribute arbitration unit 402 can select the SN hash calculation unit 404 for processing to determine the target SN.

[0139] Specifically, when there is only one SN in the die where the first CHN is located to manage the memory address to be accessed, the SN hash calculation unit 404 can determine the index (index) of the SN by a direct mapping (direct map) method. When the number of SNs in the die where the first CHN is located to manage the memory address to be accessed by the first access request is greater than one, the SN hash calculation unit 404 can perform hash calculation according to the number of SNs in the die where the first CHN is located to manage the memory address to be accessed (i.e. the number of SNs in the die where the first CHN is located to manage the memory address to be accessed by the first access request). The index (index) of the target SN is obtained. For example, the SN hash calculation unit 404 performs hash calculation on the addresses of the multiple SNs in the die where the first CHN is located to manage the memory address to be accessed, so as to uniformly hash the memory address on a certain SN node in these SN nodes. The hash algorithm supported by the SN hash calculation unit 404 can include but is not limited to at least one of the power of two hash (P2), the non-power of two hash (NP2) and the direct mapping (direct map). The direct mapping indicates that the second CHN only sees one SN and can be directly routed to this SN.

[0140] 2) If the destination NUMA node does not belong to the cross-chip NUMA node, the local / remote chip attribute of the destination NUMA node is a local chip, the local / remote numa attribute of the destination NUMA node is a local numa, and the local / remote die attribute of the destination NUMA node is a remote die (corresponding to the above sub-case 2-1), it indicates that the memory address accessed by the first CHN is located in the local NUMA node, the local chip, and the remote die, that is, the destination SN is not in the local die, and it is necessary to continue to access the next level CHN. The hash attribute arbitration unit 402 can select the CHN hash calculation unit 403 to process, and find the second CHN in the destination die that manages the memory address to be accessed by the first access request.

[0141] Specifically, when there is only one CHN in the destination die that manages the memory address to be accessed, the CHN hash calculation unit 403 can determine the index of the CHN as the index of the second CHN by a direct map manner; when the number of CHNs in the destination die that manage the memory address to be accessed is greater than one, the CHN hash calculation unit 403 can perform hash calculation according to the number of CHNs in the destination die that manage the memory address to be accessed (that is, the number of CHNs in the destination die that are divided to the destination NUMA node), to obtain the index of the second CHN. For example, the CHN hash calculation unit 403 performs hash calculation on the addresses of the plurality of CHNs in the destination die that manage the memory address to be accessed, to uniformly hash the memory address to be accessed on a certain CHN node in these CHN nodes. The hash algorithm supported by the CHN hash calculation unit 403 can include but is not limited to at least one of non-power-of-two hash (NP2), non-power-of-two hash (NP2), and hierarchy.

[0142] 3) If the destination NUMA node does not belong to the cross-chip NUMA node, the local / remote chip attribute of the destination NUMA node is a local chip, the local / remote numa attribute of the destination NUMA node is a remote numa, and the destination NUMA node is located in the same chip as the first CHN (corresponding to the above sub-case 2-2), it indicates that the memory address to be accessed by the first CHN is not in the local NUMA node but in the same chip, and it is necessary to continue to access the next level CHN. The hash attribute arbitration unit 402 can select the CHN hash calculation unit 403 to process, and find the second CHN in the destination NUMA node.

[0143] Specifically, when there is only one CHN in the destination NUMA node that manages the memory address to be accessed, the CHN hash calculation unit 403 can determine the index of the CHN as the index of the second CHN by direct mapping; when the number of CHNs in the destination NUMA node that manage the memory address to be accessed is greater than one, the CHN hash calculation unit 403 can perform hash calculation according to the number of CHNs in the destination NUMA node that manage the memory address to be accessed to obtain the index of the second CHN in the destination NUMA node. For example, the CHN hash calculation unit 403 performs hash calculation on the addresses of the plurality of CHNs in the destination NUMA node that manage the memory address to be accessed to uniformly hash the memory address to be accessed on a certain CHN node among the CHN nodes. The hash algorithm supported by the CHN hash calculation unit 403 can include but is not limited to at least one of a power-of-two hash (P2), a non-power-of-two hash (NP2), and a hierarchy hash.

[0144] 4) If the destination NUMA node belongs to a cross-chip NUMA node, and the local / remote chip attribute of the destination NUMA node is remote chip and the local / remote numa attribute of the destination NUMA node is local numa (corresponding to the above subcases 1-3 or 2-3), the hash attribute arbitration unit 402 needs to first determine the destination chip from the at least two chips included in the destination NUMA node, and then perform one of the following arbitration operations:

[0145] a. If the destination chip is the chip where the first CHN is located (corresponding to the above subcase 1-3), the hash attribute arbitration unit 402 can select the SN hash calculation unit 404 for processing to determine the destination SN from the die where the first CHN is located.

[0146] Specifically, when there is only one SN in the die where the first CHN is located that manages the memory address to be accessed, the SN hash calculation unit 404 can determine the index of the SN as the index of the target SN by direct mapping; when the number of SNs in the die where the first CHN is located that manage the memory address to be accessed is greater than one, the SN hash calculation unit 404 can perform hash calculation according to the number of CHNs in the die where the first CHN is located that manage the memory address to be accessed (i.e., the number of CHNs in the die where the first CHN is located that are divided to the target NUMA node), to obtain the index of the target SN. For example, the SN hash calculation unit 404 performs hash calculation on the addresses of the plurality of SNs in the die where the first CHN is located that manage the memory address to be accessed, to uniformly hash the memory address to be accessed on a certain SN node among the SN nodes. The hash algorithm supported by the SN hash calculation unit 404 can include, but is not limited to, at least one of a power-of-two hash (P2), a non-power-of-two hash (NP2), and direct mapping.

[0147] b. If the target chip is not the chip where the first CHN is located (corresponding to the above-mentioned case sub-case 2-3), the hash attribute arbitration unit 402 can select the C2C hash calculation unit 405 for processing, to determine the first C2C (i.e., the first cross-chip node) in the chip where the first CHN is located; then, the first CHN forwards the first access request to the second C2C (i.e., the second cross-chip node) in the target chip through the first C2C, and the first routing component in the second C2C processes the first access request to obtain the second CHN in the target chip that manages the memory address to be accessed.

[0148] Specifically, when there is only one C2C in the chip where the first CHN is located, the C2C hash calculation unit 405 can determine the index of the C2C as the index of the first C2C by direct mapping; when the number of C2Cs in the chip where the first CHN is located is greater than one, the C2C hash calculation unit 405 can perform hash calculation according to the number of C2Cs in the chip where the first CHN is located, to obtain the index of the first C2C, because the chip where the first CHN is located can be interconnected with multiple chips, and the number of C2Cs connected between any two chips is not necessarily the same. The hash algorithm supported by the C2C hash calculation unit 405 can include, but is not limited to, at least one of a power-of-two hash (P2), a non-power-of-two hash (NP2), and a hierarchy hash.

[0149] 5) In the case that the destination NUMA node does not belong to the cross-chip NUMA node, and the local / remote chip attribute of the destination NUMA node is remote chip, and the local / remote numa attribute of the destination NUMA node is remote numa (corresponding to the above sub-case 2-4), the hash attribute arbitration unit 402 can select the C2C hash calculation unit 405 for processing to determine the first C2C in the chip where the first CHN is located; then, the first CHN forwards the first access request to the second C2C in the destination chip through the first C2C, and the first routing component in the second C2C processes the first access request to obtain the second CHN in the destination chip that manages the memory address accessed by the first access request, wherein the destination chip is the chip where the destination NUMA node is located.

[0150] Specifically, when there is only one C2C in the chip where the first CHN is located, the C2C hash calculation unit 405 can determine the index of the C2C as the index of the first C2C through direct mapping; when the number of C2Cs in the chip where the first CHN is located is greater than one, the C2C hash calculation unit 405 can perform hash calculation according to the number of C2Cs in the chip where the first CHN is located to obtain the identifier (C2C index) of the first C2C in the chip. For example, the C2C hash calculation unit 405 performs hash calculation on the addresses of the plurality of C2Cs in the chip where the first CHN is located to uniformly hash the memory address accessed by the first access request on a certain C2C node in these C2C nodes. The hash algorithm supported by the C2C hash calculation unit 405 can include but is not limited to at least one of a power-of-two hash (P2), a non-power-of-two hash (NP2), and a hierarchical hash (hierarchy).

[0151] It can be understood that for the above subcases 1-3 and subcases 2-3, if the destination NUMA node is a cross-chip NUMA node, the second hash parameter is needed to be used for hashing to obtain the destination chip, wherein the second hash parameter includes the number of chips included in the destination NUMA node and the chip hashing mode, for example, the number of chips can be calculated by the power of two, for example, in the case of dividing 2 chips to the destination NUMA node, the power of two of 2 is calculated, and according to whether the calculation index result is 0 or 1, the memory address accessed by the first access request is allocated to the SN management on the local chip or the remote chip; for the above subcase 2-4, if the destination NUMA node is a non-cross-chip NUMA node, the local / remote attribute of the memory address interval of the destination NUMA node relative to the chip where the first CHN is located needs to be configured, to indicate whether the destination NUMA node belongs to the local chip or the remote chip relative to the first CHN node.

[0152] C, after the index of the second CHN / destination SN / first C2C is calculated by the CHN hash calculation unit 403 / SN hash calculation unit 404 / C2C hash calculation unit 405, the second routing component 400 can correspondingly query the CHN identifier list 406 / SN identifier list 407 / C2C identifier list 408 to determine the identifier (such as the coordinates) of the second CHN / destination SN / first C2C in the entire network, so as to forward the first access request to the second CHN / destination SN / first C2C. Wherein, the identifier of the second CHN / destination SN / first C2C in the network can be determined according to the position information of the second CHN / destination SN / first C2C defined by the on-chip bus interconnection system, and in the Mesh interconnection network, the ID composed of the x and y axes of the network can be used as the identifier of the second CHN / destination SN / first C2C.

[0153] It can be seen that in the case where the content indicated by the second attribute information of the destination NUMA node belongs to the above case 2, Figure 2 The method shown can further include: the second CHN forwards the first access request to the destination SN.

[0154] In some embodiments, the second CHN forwards the first access request to the destination SN can include: a second routing component in the second CHN processes the first access request, determines the destination SN from the SN managing the memory address accessed by the first access request in the target NUMA, and forwards the first access request to the destination SN.

[0155] It should be noted that the process of the second routing component in the second CHN processing the first access request can refer to the process of the second routing component in the first CHN processing the first access request in the above case 1 to find the destination SN, that is, the logic of the first routing component in the first CHN and the second routing component in the second CHN processing the first access request is consistent, and therefore the description is not repeated here.

[0156] The routing path of the first access request initiated by the first request node in different cases will be described below through specific embodiments.

[0157] I. Routing path across the NUMA node of the die

[0158] As shown in Figure 5 , the memory space of the storage device mounted by the die 0 and the die 1 is divided into a NUMA node (NUMA node 0). In the NUMA node division scenario, the storage devices of the two dies form a continuous address space. In the NUMA node division scenario, when the RN0 in the die 0 initiates the first access request, the routing path thereof can include one of the following paths:

[0159] 1) First, the first routing component in the RN0 determines that the first consistency maintenance node of the first access request is one of CHN1-1, CHN1-2, CHN1-3 and CHN1-4 in the die 0, and then the RN0 can route the first access request to CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4, as shown in path 1 in Figure 5 .

[0160] 2) The second routing component in the CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 processes the first access request to obtain the second attribute information of the destination NUMA node. If the second attribute information indicates that the memory address to be accessed by the first access request is in the destination NUMA node NUMA node 0, the NUMA node 0 does not belong to the cross-chip NUMA node, the NUMA node 0 is the NUMA node where the RN0 is located, and the destination die and the die where the RN0 is located are the same die (corresponding to the above sub-case 1-1), the second routing component in the CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 can determine the destination SN in the die 0; the CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 can continue to route the first access request to the destination SN in the die 0, and complete the access, as shown in path 3 in the die 0 in Figure 5 .

[0161] 3) The second routing component in CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 processes the first access request and obtains the second attribute information of the destination NUMA node. If the second attribute information indicates that the destination NUMA node of the memory address to be accessed by the first access request is NUMA node 0, NUMA node 0 is not a cross-chip NUMA node, NUMA node 0 is the NUMA node where RN0 is located, NUMA node 0 is a cross-die NUMA node, and the destination die and the die where RN0 is located are different dies (corresponding to sub-case 2-1 above), then the second routing component in CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 determines the second CHN in die 1 and routes the first access request to the second CHN. Figure 5 As shown in path 2. That is, the memory address accessed by the first access request is assigned to the SN management of the remote die. Then, the second routing component in CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 will route the first access request to the CHN managing that address within another die, such as... Figure 5 Path 2 is shown in the diagram.

[0162] In addition, such as Figure 5 As shown, during routing in a mesh network, cross-crystal access involves DTD forwarding. Furthermore, after the second CHN in crystal 1 receives the first access request forwarded by the DTD, its second routing component can directly route the first access request to the SN in crystal 1, as shown below. Figure 5 Path 3 is shown within grain 1.

[0163] II. Routing paths for dividing multiple NUMA nodes within the die

[0164] like Figure 6 As shown, two NUMA nodes (NUMA node 0 and NUMA node 1) are partitioned within die 0, and the storage space within die 0 is divided into two independent address spaces. In this NUMA node partitioning scenario, when RN0 in die 0 initiates its first access request, its routing path may include one of the following paths:

[0165] 1) If the first routing component in RN0 determines that the first consistency maintenance node for the first access request is one of CHN1-1, CHN1-2, CHN1-3, and CHN1-4 in this NUMA node (NUMA node 0), then RN0 can route the first access request to CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4, such as... Figure 6 The path 1 within NUMA node 0 is shown.

[0166] 2) The second routing component in CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 processes the first access request to obtain the second attribute information of the destination NUMA node. If the second attribute information indicates that the destination NUMA node of the memory address accessed by the first access request is NUMA node 0, NUMA node 0 does not belong to the cross-chip NUMA node, NUMA node 0 is the NUMA node where RN0 is located, and the destination die is the same die as the die where RN0 is located (corresponding to the above sub-case 1-1), the second routing component in CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 can determine the destination SN in NUMA node 0. CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 can continue to route the first access request to the destination SN in NUMA node 0 to complete the access, as shown in path 3 in FIG. 4. Figure 6

[0167] 3) The second routing component in CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 processes the first access request to obtain the second attribute information of the destination NUMA node. If the second attribute information indicates that the destination NUMA node of the memory address accessed by the first access request is NUMA node 1, NUMA node 1 does not belong to the cross-chip NUMA node, NUMA node 1 is not the NUMA node where RN0 is located, and NUMA node 1 is located in the same die as RN0 (corresponding to the above sub-case 2-2), the second routing component in CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 determines the second CHN in NUMA node 1 and routes the first access request to the second CHN. That is, if the memory address accessed by the first access request is allocated to the SN management of the remote NUMA node, the second routing component in CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 will route the first access request to the second CHN (such as CHN2-1 / CHN2-2 / CHN2-3 / CHN2-4) in another die that manages the address after processing, as shown in path 2 in FIG. 5. Figure 6

[0168] In addition, as shown in FIG. 6, after the second CHN in NUMA node 1 receives the first access request, the second routing component therein can route the first access request directly to the SN in NUMA node 1 after processing, as shown in path 3 in FIG. 6. Figure 6 Figure 6

[0169] II. Routing path of cross-chip NUMA node​​​​

[0170] As shown in Figure 7 , the memory space of the storage devices mounted by chip 0 and chip 1 is divided into one NUMA node (NUMA node 0). In the NUMA node division scenario, the storage devices of the two chips form a continuous address space. In the NUMA node division scenario, when RN0 in chip 0 initiates a first access request, the routing path of the first access request can include one of the following paths:

[0171] 1) The first routing component in RN0 determines that the first consistency maintenance node of the first access request is one of CHN1-1, CHN1-2, CHN1-3 and CHN1-4 in the chip (chip 0), and then RN0 can route the first access request to CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4, as shown in path 1 in chip 0 in Figure 7 .

[0172] 2) The second routing component in CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 processes the first access request to obtain second attribute information of the destination NUMA node. If the second attribute information indicates that the memory address accessed by the first access request is in the destination NUMA node NUMA node 0, NUMA node 0 belongs to the cross-chip NUMA node, and NUMA node 0 is the NUMA node where RN0 is located, then the second routing component in CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 determines the destination chip from the two chips included in NUMA node 0, and then processes the following two subcases:

[0173] a. If the destination chip is chip 0 where RN0 is located (corresponding to the above subcases 1-3), the second routing component in CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 determines the destination SN from chip 0 where RN0 is located. CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 can continue to route the first access request to the destination SN in chip 0, and complete the access, as shown in path 3 in chip 0 in Figure 7 .

[0174] b. If the destination chip is not chip 0 where RN0 is located (corresponding to the above subcases 2-3), the second routing component in CHN1-1 / CHN1-2 / CHN1-3 / CHN1-4 determines the first C2C in chip 0 for cross-chip access, as shown in Figure 7The path 2-1 is shown in the diagram. After crossing to the second C2C in chip 1, the first routing component in the second C2C processes the first access request, obtains the second CHN (such as CHN2-1 / CHN2-2 / CHN2-3 / CHN2-4 in chip 1) that manages the memory address to be accessed by the first access request, and routes the first access request to CHN2-1 / CHN2-2 / CHN2-3 / CHN2-4 in chip 1, as shown in the diagram. Figure 7 The path is shown in 2-2.

[0175] In addition, such as Figure 7 As shown, after CHN2-1 / CHN2-2 / CHN2-3 / CHN2-4 in chip 1 receive the first access request, the second routing component within it processes the request and can directly route it to SN in chip 1, as shown. Figure 7 Path 3 is shown within chip 1.

[0176] As described above, the method for accessing memory in an on-chip bus interconnect system proposed in this application involves a first requesting node in any die of the on-chip bus interconnect system initiating a first access request. The first requesting node determines the first attribute information of the destination NUMA node, identifies a first consistency maintenance node located in the same die and / or chip as the requesting node based on the first attribute information, and forwards the first access request to the first consistency maintenance node. Then, the first consistency maintenance node processes the first access request, obtains the second attribute information of the destination NUMA node, and forwards the first access request directly to the destination slave node or through the second consistency maintenance node based on the second attribute information. This routing mechanism, where access requests initiated by the first requesting node are forwarded through one or two levels of consistency maintenance nodes, can flexibly adapt to various NUMA node partitioning methods, including cross-chip, cross-die, or multiple NUMA nodes within the same die.

[0177] Optionally, the method for accessing memory in an on-chip bus interconnect system proposed in this application embodiment may further include, after step 202: after the first consistency maintenance node obtains the data stored at the memory address to be accessed by the first access request, caching the data stored at the memory address.

[0178] It is understandable that by caching the data stored at the memory address to be accessed by the first access request on the first consistency maintenance node, the number of times the first consistency maintenance node initiates access to the destination SN can be reduced, thereby improving access performance.

[0179] Optionally, the first consistency maintenance node caching the data stored at the memory address to be accessed by the first access request may include: when the first attribute information of the destination NUMA node satisfies the first condition, the first consistency maintenance node caching the data stored at the memory address to be accessed by the first access request.

[0180] The first condition may include, but is not limited to, at least one of the following:

[0181] The target NUMA node is a cross-chip NUMA node;

[0182] The destination NUMA node is not the NUMA node where the first requesting node is located;

[0183] The target NUMA node and the first requesting node are located on different chips;

[0184] The die corresponding to the target NUMA node does not contain the die where the first requested node is located.

[0185] It is understood that when the first attribute information of the target NUMA node meets the first condition, the first CHN caches the data stored at the memory address to be accessed by the first access request, which can effectively reduce cross-die and cross-chip accesses of the memory address, reduce access latency, and thus improve access performance.

[0186] For example, such as Figure 8 As shown, assuming: die 0 in chip 0 and chip 1 is assigned to NUMA node 0; die 1 in chip 0 and chip 1 is assigned to NUMA node 1; die 2 in chip 0 and chip 1 is assigned to NUMA node 2; and die 3 in chip 0 and chip 1 is assigned to NUMA node 3; then, by caching data at the memory addresses of remote NUMA nodes (NUMA node 0 / NUMA node 2 / NUMA node 3) through the CHN node within this NUMA node (NUMA node 1), the access can be completed simply by accessing the local CHN node. Figure 8 Path 1 within NUMA node 1 is shown. When the local CHN node lacks data at a specific memory address, or when consistency needs to be maintained or data needs to be written back to the storage device mounted on the SN, routing will first go to the local CHN node and then through the DTD cross-crystal access to the remote NUMA node (e.g., ...). Figure 8 (as shown by paths 2 and 3 within die 3 of chip 1), or cross-chip access to remote NUMA nodes via C2C (e.g.) Figure 8 (See paths 2-1, 2-2, and path 3 within die 1 of chip 1). From Figure 8It can be seen that, compared with the access within the die, the access across the die and across the chip has a long access path, and needs to consume a large amount of time resources and computing resources. Therefore, by caching the data under the memory address of the remote NUMA node in the local CHN, the access frequency across the chip and across the die can be significantly reduced, the routing delay can be reduced, the cpu processing efficiency can be improved, and the access performance of the whole system can be improved.

[0187] Optionally, the method for accessing the memory in the on-chip bus interconnection system also can comprise the following steps.

[0188] The first consistency maintenance node determines whether the data under the memory address accessed by the second access request is cached locally after receiving the second access request;

[0189] The first consistency maintenance node responds to the second access request based on the cached data in the case that the data under the memory address accessed by the second access request is cached locally;

[0190] The first consistency maintenance node performs forwarding processing on the second access request in the case that the data under the memory address accessed by the second access request is not cached locally.

[0191] It can be understood that, in the case that the data under the memory address accessed by the second access request is cached in the first consistency maintenance node, the first consistency maintenance node responds to the second access request, which can effectively reduce the access across the die and across the chip about the memory address accessed by the second access request, reduce the access delay, and improve the access performance.

[0192] In some embodiments, when there are NUMA nodes across the die and across the chip, the two-level consistency maintenance node routing mode can cause the routing path shown in Figure 9 For example, in Figure 9 , the die 0, the die 1, the die 2 and the die 3 in the chip 0 are divided into the NUMA node 0. When the RN in the die 1 at the top left corner initiates an access request, the RN is first routed to the local CHN (the first CHN) node cache, and then the local CHN (the first CHN) is routed to the second CHN of the die 2, as shown in path 2 in Figure 9 , the die 1 at the top left corner crosses the die twice to the die 2 at the bottom right corner. At this time, if the SN is not bound with the CHN of the die, it will cause the second CHN to be routed to the next SN node, and then be routed back to the SN node of the die 1, as shown in path 3 in Figure 9 , the die 2 at the bottom right corner crosses the die twice to the die 1 at the top left corner.

[0193] It is not difficult to understand that in the case of a large Mesh network, two diagonal grain mutual access will cause the path delay to increase greatly. To solve this problem, in the on-chip bus interconnection system provided in some embodiments of the present application, the subordinate nodes in the same grain are bound with the consistency maintenance nodes. For example, Figure 10 As shown, in the case of dividing grain 0, grain 1, grain 2 and grain 3 in chip 0 to NUMA node 0, all CHN nodes on all grains manage the memory address space mounted by SN together, and the first layer can be selected as 4 in the hierarchical hash calculation according to the number of CHN nodes. After the CHN node is found, the CHN in each grain is divided into 2 groups, and each group is bound with an SN. In this way, the first CHN in grain 1 is routed to the second CHN node in grain 0 / grain 2 / grain 3 through path 2, and then directly mapped to the SN node in grain 0 / grain 2 / grain 3, which can avoid the long round trip routing path of "the first access request initiated by the RN in a certain grain is routed to the second CHN node on the farthest grain through path 2 by the first CHN node, and then returned to the SN node in the grain through path 3", thereby reducing the routing delay in the cross-grain and cross-chip NUMA node division scenario, and improving the system access performance.

[0194] The embodiment of the present application also provides an on-chip bus interconnection system, which can include at least one chip, each chip can include at least one grain, and each grain can include a plurality of nodes, the plurality of nodes can include at least one request node (RN), at least one coherent home node (CHN) and at least one subordinate node (SN), the subordinate node is connected to a storage device, and the storage device has a memory accessed by the request node, wherein:

[0195] The first request node is configured to initiate a first access request, determine first attribute information of a target non-coherent memory access (NUMA) node to be accessed by the first access request, determine a first coherent home node according to the first attribute information, and send the first access request to the first coherent home node. The first request node is any request node in any grain on any chip in the at least one chip, the first attribute information is used to indicate the positional relationship between the target NUMA node and the first request node, and the first coherent home node is located in the same chip and / or the same grain as the first request node.

[0196] A first consistency maintenance node configured to process the first access request, obtain second attribute information of the destination NUMA node, and forward the first access request according to the second attribute information, the second attribute information being used to indicate a location relationship between the destination NUMA node and the first consistency maintenance node.

[0197] In some embodiments, the first consistency maintenance node is configured to forward the first access request directly to a destination slave node according to the second attribute information, wherein the destination slave node and the first consistency maintenance node are located in the destination NUMA node, and the destination slave node and the first consistency maintenance node are located in a same chip and / or a same die.

[0198] In some embodiments, the first consistency maintenance node is configured to forward the first access request to a second consistency maintenance node according to the second attribute information, wherein the second consistency maintenance node and the first consistency maintenance node are located in different NUMA nodes, and / or the second consistency maintenance node and the first consistency maintenance node are located in different chips and / or different dies; and the second consistency maintenance node is configured to forward the first access request to a destination slave node.

[0199] In some embodiments, the first request node is configured to determine a target memory address range hit by a memory address accessed by the first access request, and determine first attribute information of the destination NUMA node according to configuration information of the target memory address range, wherein a NUMA node to which the target memory address range belongs is the destination NUMA node.

[0200] In some embodiments, the first attribute information of the destination NUMA node is used to indicate at least one of the following:

[0201] Whether the destination NUMA node is a NUMA node in which the first request node is located;

[0202] Whether the destination NUMA node is a cross-chip NUMA node;

[0203] Whether the destination NUMA node and the first request node are located in a same chip;

[0204] Whether the destination NUMA node is a cross-die NUMA node, and the like.

[0205] In some embodiments, the first request node is configured to determine the first consistency maintenance node from the NUMA node and the die where the first request node is located, in a case that the first attribute information indicates that the destination NUMA node is the NUMA node where the first request node is located, the destination NUMA node does not belong to the cross-chip NUMA node, and the destination NUMA node belongs to or does not belong to the cross-die NUMA node.

[0206] In some embodiments, the first request node is configured to determine the first consistency maintenance node from the NUMA node and the die where the first request node is located, in a case that the first attribute information indicates that the destination NUMA node is the NUMA node where the first request node is located, and the destination NUMA node belongs to the cross-chip NUMA node.

[0207] In some embodiments, the first request node is configured to determine the first consistency maintenance node from the chip and / or the die where the first request node is located, in a case that the first attribute information indicates that the destination NUMA node is not the NUMA node where the first request node is located.

[0208] In some embodiments, the first consistency maintenance node is configured to obtain the first attribute information from the first access request, determine second attribute information of the destination NUMA node according to the first attribute information, determine a target memory address range hit by a memory address accessed by the first access request, and determine the first attribute information of the destination NUMA node according to configuration information of the target memory address range, wherein the NUMA node to which the target memory address range belongs is the destination NUMA node.

[0209] In some embodiments, in some embodiments, the second attribute information of the destination NUMA node can be used to indicate at least one of the following;

[0210] 1) whether the destination NUMA node is the NUMA node where the first coherency maintenance node is located, i.e. the local / remote numa attribute of the destination NUMA node, if the local / remote numa attribute of the destination NUMA node is local numa, it indicates that the memory address to be accessed by the first coherency maintenance node is located in the local NUMA node, i.e. the destination NUMA node is the NUMA node where the first coherency maintenance node is located; otherwise, if the local / remote numa attribute of the destination NUMA node is remote numa, it indicates that the memory address to be accessed by the first coherency maintenance node is not located in the local NUMA node, i.e. the destination NUMA node is not the NUMA node where the first coherency maintenance node is located.

[0211] 2) whether the destination NUMA node is a cross-chip NUMA node. If the destination NUMA node is a non-cross-chip NUMA node, it can be further determined whether the destination NUMA node is located in the local chip according to the local / remote chip attribute of the destination NUMA node, if the local / remote chip attribute of the destination NUMA node is local, it indicates that the destination NUMA node is located in the local chip; otherwise, if the local / remote chip attribute of the destination NUMA node is remote chip, it indicates that the destination NUMA node is located in another chip; if the destination NUMA node is a cross-chip NUMA node, it is necessary to calculate the local / remote chip attribute according to the configured chip number; after obtaining the local / remote chip attribute, it is determined whether it is a local chip access (the access of the local chip does not need to be cross-chip, and the access of the remote chip needs to be cross-chip). It can be understood that when the local / remote numa attribute of the destination NUMA node is local numa, and the local / remote chip attribute of the destination NUMA node is remote chip, it indicates that the accessed destination NUMA node is cross-chip.

[0212] 3) whether the destination NUMA node and the first coherency maintenance node are located in the same chip.

[0213] 4) whether the destination NUMA node is a cross-die NUMA node. In the case that the destination NUMA node is a non-cross-chip NUMA node, it can be further determined whether the destination node is a cross-die NUMA node. If the NUMA node is a cross-die NUMA node, the local / remote die attribute of the destination NUMA node is determined according to the number of dies contained in the chip where the destination NUMA node is located. For example, the chip where the destination NUMA node is located contains two dies: die0 and die1. The addresses thereof are calculated by a 2-power-of-2 hash, index = 0 represents die0, and index = 1 represents die1. When the first access request initiated by the first coherency maintenance node is in die0 and the calculated index = 0, it represents that the first access request is an access in the local die, which can be directly routed to the destination SN. Index = 1 represents an access in the remote die, and the index = 1 is used to look up the corresponding CHN identification table to obtain the CHN identification of the memory address to be accessed in the other die as the second coherency maintenance node. If the destination NUMA node is not a cross-die NUMA node, it represents that the destination NUMA node is another NUMA node in the chip, and the hash calculation can be directly performed according to the configuration information of the numa, and the CHN identification list is queried to find the second coherency maintenance node. It can be understood that when the local / remote numa attribute of the destination NUMA node is local numa, the local / remote chip attribute of the destination NUMA node is local chip, and the local / remote die attribute of the destination NUMA node is remote die, it represents that the accessed destination NUMA node is cross-die.

[0214] In some embodiments, the first coherency maintenance node is configured to, in the case that the second attribute information indicates that the destination NUMA node is the NUMA node where the first coherency maintenance node is located, the destination NUMA node does not belong to a cross-chip NUMA node, and the destination NUMA node belongs to a cross-die NUMA node, determine a destination die from the dies under the jurisdiction of the destination NUMA node. If the destination die is the die where the first coherency maintenance node is located, a destination slave node that manages the memory address to be accessed by the first access request is determined from the die where the first coherency maintenance node is located, and the first access request is directly forwarded to the destination slave node.

[0215] In some embodiments, the first consistency maintenance node is configured to, in a case that the second attribute information indicates that the destination NUMA node is the NUMA node where the first consistency maintenance node is located, the destination NUMA node does not belong to a cross-chip NUMA node, and the destination NUMA node does not belong to a cross-die NUMA node, determine a destination slave node managing a memory address to be accessed by the first access request from a die where the first consistency maintenance node is located, and directly forward the first access request to the destination slave node.

[0216] In some embodiments, the first consistency maintenance node is configured to, in a case that the second attribute information indicates that the destination NUMA node is the NUMA node where the first consistency maintenance node is located, and the destination NUMA node belongs to a cross-chip NUMA node, determine a destination chip from at least two chips included in the destination NUMA node, and in a case that the destination chip is a chip where the first consistency maintenance node is located, determine a destination slave node managing a memory address to be accessed by the first access request from a chip where the first consistency maintenance node is located, and directly forward the first access request to the destination slave node.

[0217] In some embodiments, the first consistency maintenance node is configured to, in a case that the second attribute information indicates that the destination NUMA node is the NUMA node where the first consistency maintenance node is located, the destination NUMA node does not belong to a cross-chip NUMA node, and the destination NUMA node belongs to a cross-die NUMA node, determine a destination die from a die where the destination NUMA node is located, and in a case that the destination die is different from a die where the first consistency maintenance node is located, determine a second consistency maintenance node from the destination die, and directly forward the first access request to the second consistency maintenance node.

[0218] In some embodiments, the first consistency maintenance node is configured to, in a case that the second attribute information indicates that the destination NUMA node is not the NUMA node where the first consistency maintenance node is located, the destination NUMA node does not belong to a cross-chip NUMA node, and the destination NUMA node is located in a same chip as the first consistency maintenance node, determine a second consistency maintenance node from the destination NUMA node, and directly forward the first access request to the second consistency maintenance node.

[0219] In some embodiments, the one or more dies included in each chip further include at least one cross-chip node for cross-chip communication, wherein the first coherency maintenance node is configured to determine a destination chip from at least two chips included in the destination NUMA node if the second attribute information indicates that the destination NUMA node is the NUMA node where the first coherency maintenance node is located and the destination NUMA node belongs to cross-chip NUMA nodes, and determine a second coherency maintenance node from the destination chip if the destination chip is not the chip where the first coherency maintenance node is located, and forward the first access request to the second coherency maintenance node through the first cross-chip node and the second cross-chip node.

[0220] In some embodiments, the one or more dies included in each chip further include at least one cross-chip node for cross-chip communication, wherein the first coherency maintenance node is configured to determine a second coherency maintenance node from the destination NUMA node if the second attribute information indicates that the destination NUMA node is not the NUMA node where the first coherency maintenance node is located, the destination NUMA node does not belong to cross-chip NUMA nodes, and the destination NUMA node is not on the same chip as the first request node, and forward the first access request to the second coherency maintenance node through the first cross-chip node and the second cross-chip node.

[0221] wherein the first cross-chip node is a cross-chip node on the first chip configured for cross-chip communication between the first chip and the second chip, the first chip is the chip where the first coherency maintenance node is located, the second chip is the chip where the second coherency maintenance node is located, and the second cross-chip node is a cross-chip node on the second chip bound to the first cross-chip node.

[0222] In some embodiments, the first coherency maintenance node is configured to determine the first cross-chip node on the first chip and forward the first access request to the first cross-chip node, the first cross-chip node is configured to forward the first access request to the second cross-chip node on the second chip bound to the first cross-chip node, the second cross-chip node is configured to determine a second coherency maintenance node according to a memory address to be accessed by the first access request, and the second cross-chip node is configured to send the first access request to the second coherency maintenance node.

[0223] According to the above description, the on-chip bus interconnection system provided by the embodiments of the present application can be known, wherein when the first request node in any die initiates a first access request, the first request node can determine the first attribute information of the destination NUMA node, determine the first consistency maintenance node located in the same die and / or the same chip as the request node according to the first attribute information, and forward the first access request to the first consistency maintenance node; then, the first consistency maintenance node processes the first access request, obtains the second attribute information of the destination NUMA node, and forwards the first access request to the destination slave node directly or through the second consistency maintenance node according to the second attribute information. The routing mechanism of the access request initiated by the first request node through one or two levels of consistency maintenance nodes can flexibly adapt to various NUMA node division modes, including different scenarios such as cross-chip, cross-die or division into multiple NUMA nodes in the same die.

[0224] In some embodiments, the first consistency maintenance node is further configured to cache the data stored in the memory address to be accessed by the first access request after obtaining the data stored in the memory address.

[0225] In some embodiments, the first consistency maintenance node is further configured to cache the data stored in the memory address to be accessed by the first access request when the first attribute information of the destination NUMA node satisfies a first condition.

[0226] The first condition can include but is not limited to at least one of the following:

[0227] The destination NUMA node is a cross-chip NUMA node;

[0228] The destination NUMA node is not the NUMA node where the first request node is located;

[0229] The destination NUMA node is located in different chips from the first request node;

[0230] The destination NUMA node corresponds to a die that does not contain the die where the first request node is located.

[0231] It can be understood that when the first attribute information of the destination NUMA node satisfies the first condition, the first CHN caches the data stored in the memory address to be accessed by the first access request, which can effectively reduce the cross-die and cross-chip access about the memory address, reduce the access latency, and thus improve the access performance.

[0232] In some embodiments, the first consistency maintenance node is further configured to, after receiving the second access request, determine whether the data at the memory address accessed by the second access request is cached locally; in a case where the data at the memory address accessed by the second access request is cached locally, respond to the second access request based on the cached data; and in a case where the data at the memory address accessed by the second access request is not cached locally, perform forwarding processing on the second access request.

[0233] It can be understood that, in a case where the data at the memory address accessed by the second access request is cached in the first consistency maintenance node, the first consistency maintenance node responds to the second access request, which can effectively reduce cross-grain and cross-chip access with respect to the memory address accessed by the second access request, reduce access latency, and thus can improve access performance.

[0234] In some embodiments, in the on-chip bus interconnection system, the slave nodes in the same grain are bound to the consistency maintenance nodes. Through such binding, the following situation can be avoided: Figure 9 The long round trip routing path shown in the figure, that is, the first access request initiated by the RN in a certain grain is routed to the second CHN node on the farthest grain by the local first CHN node through path 2, and then returned to the SN node of the grain by the second CHN node through path 3, thereby reducing routing latency in the cross-grain and cross-chip NUMA node division scenario and improving system access performance.

[0235] Embodiments of the present application also provide a processor comprising the on-chip bus interconnection system provided by the embodiments of the present application and achieving the same technical effects, which will not be described herein.

[0236] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than the order in which they are recited and still achieve the desired results. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous.

[0237] In summary, the above only describes preferred embodiments of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0238] It is also to be noted that the terms "comprising", "including", and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0239] The various embodiments in this application are described in a progressive manner, and the same or similar parts among the various embodiments can be mutually referred to. Each embodiment focuses on the difference from other embodiments. In particular, for system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

Claims

1. A method for accessing memory in an on-chip bus interconnect system, characterized in that, The on-chip bus interconnect system includes at least one chip, each chip includes at least one die, each die includes multiple nodes, the multiple nodes include at least one requesting node, at least one consistency maintenance node, and at least one slave node, the slave node is connected to a storage device, the storage device has memory accessible by the requesting node, and the method includes: The first requesting node initiates a first access request, determines the first attribute information of the destination non-consistent memory access NUMA node to be accessed by the first access request, determines the first consistency maintenance node based on the first attribute information, and sends the first access request to the first consistency maintenance node. The first requesting node is any requesting node in any die on any chip in the at least one chip. The first attribute information is used to indicate the positional relationship between the destination non-consistent memory access NUMA node and the first requesting node. The first consistency maintenance node and the first requesting node are located on the same chip and / or the same die. The first consistency maintenance node processes the first access request, obtains the second attribute information of the destination non-consistent memory access NUMA node, and forwards the first access request according to the second attribute information. The second attribute information is used to indicate the positional relationship between the destination non-consistent memory access NUMA node and the first consistency maintenance node. Wherein, the first consistency maintenance node forwards the first access request based on the second attribute information, including: The first consistency maintenance node forwards the first access request directly to the destination slave node according to the second attribute information. The destination slave node and the first consistency maintenance node are both located in the destination non-consistent memory access NUMA node, and the destination slave node and the first consistency maintenance node are located in the same chip and / or the same die. or, The first consistency maintenance node forwards the first access request to the second consistency maintenance node according to the second attribute information, wherein the second consistency maintenance node and the first consistency maintenance node are located on different NUMA nodes, and / or the second consistency maintenance node and the first consistency maintenance node are located on different chips and / or different dies; and the method further includes: the second consistency maintenance node forwards the first access request to the destination slave node.

2. The method according to claim 1, characterized in that, The first requesting node determines the first attribute information of the destination non-consistent memory access NUMA node to be accessed by the first access request, including: The first requesting node determines the target memory address range hit by the memory address to be accessed by the first access request, wherein the NUMA node to which the target memory address range belongs is the destination non-consistent memory access NUMA node. The first requesting node determines the first attribute information of the target non-consistent memory access NUMA node based on the configuration information of the target memory address range.

3. The method according to claim 1, characterized in that, The first attribute information is used to indicate at least one of the following; Whether the target non-consistent memory access NUMA node is the NUMA node where the first requesting node is located; Whether the target non-uniform memory access NUMA node is a cross-chip NUMA node; Whether the target non-uniform memory access NUMA node and the first requesting node are located on the same chip; Whether the target non-uniform memory access NUMA node is a cross-grain NUMA node.

4. The method according to claim 3, characterized in that, The first requesting node determines the first consistency maintenance node based on the first attribute information, including: If the first requesting node determines the first consistency maintenance node from the NUMA node where the first requesting node is located and the die when the first attribute information indicates that the destination non-consistent memory access NUMA node is the NUMA node where the first requesting node is located, the destination non-consistent memory access NUMA node does not belong to the cross-chip NUMA node, and the destination non-consistent memory access NUMA node belongs to or does not belong to the cross-die NUMA node; When the first attribute information indicates that the destination non-consistent memory access NUMA node is the NUMA node where the first request node is located, and the destination non-consistent memory access NUMA node is a cross-chip NUMA node, the first request node determines the first consistency maintenance node from the NUMA node where the first request node is located and the chip. If the first requesting node determines that the destination non-consistent memory access NUMA node is not the NUMA node where the first requesting node is located from the chip and / or die where the first requesting node is located, when the first attribute information indicates that the destination non-consistent memory access NUMA node is not the NUMA node where the first requesting node is located.

5. The method according to claim 1, characterized in that, The first consistency maintenance node processes the first access request and obtains the second attribute information of the destination non-consistent memory access NUMA node, including at least one of the following: The first consistency maintenance node obtains the first attribute information from the first access request and determines the second attribute information of the destination non-consistent memory access NUMA node based on the first attribute information; The first consistency maintenance node determines the target memory address range hit by the memory address to be accessed by the first access request, and determines the second attribute information of the destination non-consistent memory access NUMA node according to the configuration information of the target memory address range, wherein the NUMA node to which the target memory address range belongs is the destination non-consistent memory access NUMA node.

6. The method according to claim 1, characterized in that, The second attribute information of the NUMA node for the purpose of non-uniform memory access is used to indicate at least one of the following: Whether the target non-consistent memory access NUMA node is the same NUMA node where the first consistency maintenance node is located; Whether the target non-uniform memory access NUMA node is a cross-chip NUMA node; Whether the target non-consistent memory access NUMA node and the first consistency maintenance node are located on the same chip; Whether the target non-uniform memory access NUMA node is a cross-grain NUMA node.

7. The method according to claim 6, characterized in that, The first consistency maintenance node forwards the first access request directly to the destination slave node based on the second attribute information, including: If the second attribute information indicates that the destination non-consistent memory access NUMA node is the NUMA node where the first consistency maintenance node is located, and the destination non-consistent memory access NUMA node does not belong to a cross-chip NUMA node, or the destination non-consistent memory access NUMA node belongs to a cross-die NUMA node, then the first consistency maintenance node determines the destination die from the dies under the jurisdiction of the destination non-consistent memory access NUMA node. If the destination die is the die where the first consistency maintenance node is located, then the destination slave node that manages the memory address to be accessed by the first access request is determined from the die where the first consistency maintenance node is located, and the first access request is directly forwarded to the destination slave node. When the second attribute information indicates that the destination non-consistent memory access NUMA node is the NUMA node where the first consistency maintenance node is located, and the destination non-consistent memory access NUMA node does not belong to the cross-chip NUMA node and the destination non-consistent memory access NUMA node does not belong to the cross-die NUMA node, the first consistency maintenance node determines the destination slave node that manages the memory address to be accessed by the first access request from the die where the first consistency maintenance node is located, and forwards the first access request directly to the destination slave node. If the second attribute information indicates that the destination non-consistent memory access NUMA node is the NUMA node where the first consistency maintenance node is located, and the destination non-consistent memory access NUMA node is a cross-chip NUMA node, the first consistency maintenance node determines the destination chip from at least two chips included in the destination non-consistent memory access NUMA node. If the destination chip is the chip where the first consistency maintenance node is located, the first consistency maintenance node determines the destination slave node that manages the memory address to be accessed by the first access request from the chip where the first consistency maintenance node is located, and forwards the first access request directly to the destination slave node.

8. The method according to claim 6, characterized in that, The first consistency maintenance node forwards the first access request to the second consistency maintenance node based on the second attribute information, including: If the second attribute information indicates that the destination non-consistent memory access NUMA node is the NUMA node where the first consistency maintenance node is located, and the destination non-consistent memory access NUMA node does not belong to the cross-chip NUMA node, or the destination non-consistent memory access NUMA node belongs to the cross-die NUMA node, the first consistency maintenance node determines the destination die from the die under the jurisdiction of the destination non-consistent memory access NUMA node. If the destination die is a different die from the die where the first consistency maintenance node is located, the second consistency maintenance node is determined from the destination die, and the first access request is directly forwarded to the second consistency maintenance node. If the second attribute information indicates that the destination non-consistent memory access NUMA node is not the NUMA node where the first consistency maintenance node is located, the destination non-consistent memory access NUMA node does not belong to the cross-chip NUMA node, and the destination non-consistent memory access NUMA node is located on the same chip as the first consistency maintenance node, the first consistency maintenance node determines the second consistency maintenance node from the destination non-consistent memory access NUMA node and forwards the first access request directly to the second consistency maintenance node.

9. The method according to claim 6, characterized in that, Each chip includes one or more dies, and at least one cross-chip node for cross-chip communication, wherein the first consistency maintenance node forwards the first access request to the second consistency maintenance node based on the second attribute information, including: If the second attribute information indicates that the destination non-consistent memory access NUMA node is the NUMA node where the first consistency maintenance node is located, and the destination non-consistent memory access NUMA node is a cross-chip NUMA node, the first consistency maintenance node determines the destination chip from at least two chips included in the destination non-consistent memory access NUMA node. If the destination chip is not the chip where the first consistency maintenance node is located, the first consistency maintenance node determines the second consistency maintenance node from the destination chip, and forwards the first access request to the second consistency maintenance node through the first cross-chip node and the second cross-chip node. If the second attribute information indicates that the destination non-consistent memory access NUMA node is not the NUMA node where the first consistency maintenance node is located, the destination non-consistent memory access NUMA node does not belong to the cross-chip NUMA node, and the destination non-consistent memory access NUMA node is not on the same chip as the first request node, the first consistency maintenance node determines the second consistency maintenance node from the destination non-consistent memory access NUMA node, and forwards the first access request to the second consistency maintenance node through the first cross-chip node and the second cross-chip node. Wherein, the first cross-chip node is a cross-chip node on the first chip configured for cross-chip communication between the first chip and the second chip, the first chip is the chip where the first consistency maintenance node is located, the second chip is the chip where the second consistency maintenance node is located, and the second cross-chip node is a cross-chip node on the second chip that is bound to the first cross-chip node.

10. The method according to claim 9, characterized in that, The first consistency maintenance node forwards the first access request to the second consistency maintenance node through the first cross-shard node and the second cross-shard node, including: The first consistency maintenance node determines the first cross-chip node on the first chip and forwards the first access request to the first cross-chip node; The first cross-chip node forwards the first access request to the second cross-chip node on the second chip that is bound to the first cross-chip node; The second cross-shard node determines the second consistency maintenance node based on the memory address to be accessed by the first access request; The second cross-shard node sends the first access request to the second consistency maintenance node.

11. The method according to any one of claims 1 to 10, characterized in that, The method further includes: After obtaining the data stored at the memory address to be accessed by the first access request, the first consistency maintenance node caches the data stored at the memory address.

12. The method according to claim 11, characterized in that, The method further includes: After receiving the second access request, the first consistency maintenance node determines whether to cache the data at the memory address to be accessed by the second access request locally; If the first consistency maintenance node locally caches the data at the memory address to be accessed by the second access request, it responds to the second access request based on the cached data. If the first consistency maintenance node does not cache the data at the memory address to be accessed by the second access request locally, it will perform forwarding processing on the second access request.

13. The method according to claim 11, characterized in that, In the on-chip bus interconnect system, slave nodes and consistency maintenance nodes within the same die are bound to each other.

14. An on-chip bus interconnect system, characterized in that, The on-chip bus interconnect system includes at least one chip, each chip includes at least one die, each die includes multiple nodes, the multiple nodes include at least one requesting node, at least one consistency maintenance node, and at least one slave node, the slave node is connected to a storage device, and the storage device has memory accessible by the requesting node; wherein: A first requesting node is configured to initiate a first access request, determine the first attribute information of the destination non-consistent memory access NUMA node to be accessed by the first access request, determine a first consistency maintenance node based on the first attribute information, and send the first access request to the first consistency maintenance node. The first requesting node is any requesting node in any die on any chip in the at least one chip. The first attribute information is used to indicate the positional relationship between the destination non-consistent memory access NUMA node and the first requesting node. The first consistency maintenance node and the first requesting node are located on the same chip and / or the same die. The first consistency maintenance node is configured to process the first access request, obtain the second attribute information of the destination non-consistent memory access NUMA node, and forward the first access request according to the second attribute information. The second attribute information is used to indicate the positional relationship between the destination non-consistent memory access NUMA node and the first consistency maintenance node; wherein: The first consistency maintenance node is configured to directly forward the first access request to the destination slave node according to the second attribute information, wherein the destination slave node and the first consistency maintenance node are both located in the destination non-consistent memory access NUMA node, and the destination slave node and the first consistency maintenance node are located in the same chip and / or the same die. or, The first consistency maintenance node is configured to forward the first access request to the second consistency maintenance node according to the second attribute information, wherein the second consistency maintenance node and the first consistency maintenance node are located on different NUMA nodes, and / or the second consistency maintenance node and the first consistency maintenance node are located on different chips and / or different dies; and the second consistency maintenance node is configured to forward the first access request to the destination slave node.

15. The system according to claim 14, characterized in that, The first request node is configured to determine the target memory address range hit by the memory address to be accessed by the first access request, and to determine the first attribute information of the destination non-consistent memory access NUMA node according to the configuration information of the target memory address range, wherein the NUMA node to which the target memory address range belongs is the destination non-consistent memory access NUMA node.

16. The system according to claim 14, characterized in that, The first attribute information is used to indicate at least one of the following; Whether the target non-consistent memory access NUMA node is the NUMA node where the first requesting node is located; Whether the target non-uniform memory access NUMA node is a cross-chip NUMA node; Whether the target non-uniform memory access NUMA node and the first requesting node are located on the same chip; Whether the target non-uniform memory access NUMA node is a cross-grain NUMA node.

17. The system according to claim 16, characterized in that, The first requesting node is configured to determine a first consistency maintenance node from the NUMA node where the first requesting node is located and the die when the first attribute information indicates that the destination non-consistent memory access NUMA node is the NUMA node where the first requesting node is located, the destination non-consistent memory access NUMA node does not belong to the cross-chip NUMA node, and the destination non-consistent memory access NUMA node belongs to or does not belong to the cross-die NUMA node. The first requesting node is configured to determine a first consistency maintenance node from the NUMA node where the first requesting node is located and the chip when the first attribute information indicates that the destination non-consistent memory access NUMA node is the NUMA node where the first requesting node is located, and the destination non-consistent memory access NUMA node is a cross-chip NUMA node. The first requesting node is configured to determine a first consistency maintenance node from the chip and / or die where the first requesting node is located if the first attribute information indicates that the destination non-consistent memory access NUMA node is not the NUMA node where the first requesting node is located.

18. The system according to claim 14, characterized in that, The first consistency maintenance node is configured to obtain the first attribute information from the first access request and determine the second attribute information of the destination non-consistent memory access NUMA node based on the first attribute information. The target memory address range to be accessed by the first access request is determined. Based on the configuration information of the target memory address range, the first attribute information of the destination non-consistent memory access NUMA node is determined, wherein the NUMA node to which the target memory address range belongs is the destination non-consistent memory access NUMA node.

19. The system according to claim 14, characterized in that, The second attribute information of the NUMA node for the purpose of non-uniform memory access is used to indicate at least one of the following: Whether the target non-consistent memory access NUMA node is the same NUMA node where the first consistency maintenance node is located; Whether the target non-uniform memory access NUMA node is a cross-chip NUMA node; Whether the target non-consistent memory access NUMA node and the first consistency maintenance node are located on the same chip; Whether the target non-uniform memory access NUMA node is a cross-grain NUMA node.

20. The system according to claim 19, characterized in that, The first consistency maintenance node is configured to, when the second attribute information indicates that the destination non-consistent memory access NUMA node is the NUMA node where the first consistency maintenance node is located, and the destination non-consistent memory access NUMA node does not belong to a cross-chip NUMA node, or the destination non-consistent memory access NUMA node belongs to a cross-die NUMA node, determine the destination die from the die under the jurisdiction of the destination non-consistent memory access NUMA node; if the destination die is the die where the first consistency maintenance node is located, determine the destination slave node that manages the memory address to be accessed by the first access request from the die where the first consistency maintenance node is located, and directly forward the first access request to the destination slave node; The first consistency maintenance node is configured to, when the second attribute information indicates that the destination non-consistent memory access NUMA node is the NUMA node where the first consistency maintenance node is located, and the destination non-consistent memory access NUMA node does not belong to the cross-chip NUMA node and the destination non-consistent memory access NUMA node does not belong to the cross-die NUMA node, determine the destination slave node that manages the memory address to be accessed by the first access request from the die where the first consistency maintenance node is located, and directly forward the first access request to the destination slave node; The first consistency maintenance node is configured to, when the second attribute information indicates that the destination non-consistent memory access NUMA node is the NUMA node where the first consistency maintenance node is located, and the destination non-consistent memory access NUMA node is a cross-chip NUMA node, determine the destination chip from at least two chips included in the destination non-consistent memory access NUMA node; if the destination chip is the chip where the first consistency maintenance node is located, determine the destination slave node that manages the memory address to be accessed by the first access request from the chip where the first consistency maintenance node is located, and directly forward the first access request to the destination slave node.

21. The system according to claim 19, characterized in that, The first consistency maintenance node is configured to, when the second attribute information indicates that the destination non-consistent memory access NUMA node is the NUMA node where the first consistency maintenance node is located, and the destination non-consistent memory access NUMA node does not belong to the cross-chip NUMA node, or the destination non-consistent memory access NUMA node belongs to the cross-die NUMA node, determine the destination die from the dies under the jurisdiction of the destination non-consistent memory access NUMA node; if the destination die is a different die from the die where the first consistency maintenance node is located, determine the second consistency maintenance node from the destination die, and directly forward the first access request to the second consistency maintenance node. The first consistency maintenance node is configured to determine a second consistency maintenance node from the destination non-consistent memory access NUMA nodes when the second attribute information indicates that the destination non-consistent memory access NUMA node is not the NUMA node where the first consistency maintenance node is located, the destination non-consistent memory access NUMA node does not belong to the cross-chip NUMA node, and the destination non-consistent memory access NUMA node is located on the same chip as the first consistency maintenance node, and directly forward the first access request to the second consistency maintenance node.

22. The system according to claim 19, characterized in that, Each chip includes one or more dies, and also includes at least one cross-chip node for cross-chip communication, wherein: The first consistency maintenance node is configured to, when the second attribute information indicates that the destination non-consistent memory access NUMA node is the NUMA node where the first consistency maintenance node is located, and the destination non-consistent memory access NUMA node is a cross-chip NUMA node, determine the destination chip from at least two chips included in the destination non-consistent memory access NUMA node; if the destination chip is not the chip where the first consistency maintenance node is located, determine the second consistency maintenance node from the destination chip, and forward the first access request to the second consistency maintenance node through the first cross-chip node and the second cross-chip node. The first consistency maintenance node is configured to determine a second consistency maintenance node from the destination non-consistent memory access NUMA nodes when the second attribute information indicates that the destination non-consistent memory access NUMA node is not the NUMA node where the first consistency maintenance node is located, the destination non-consistent memory access NUMA node does not belong to the cross-chip NUMA node, and the destination non-consistent memory access NUMA node is not on the same chip as the first request node. The first consistency maintenance node then forwards the first access request to the second consistency maintenance node through the first cross-chip node and the second cross-chip node. Wherein, the first cross-chip node is a cross-chip node on the first chip configured for cross-chip communication between the first chip and the second chip, the first chip is the chip where the first consistency maintenance node is located, the second chip is the chip where the second consistency maintenance node is located, and the second cross-chip node is a cross-chip node on the second chip that is bound to the first cross-chip node.

23. The system according to claim 22, characterized in that, The first consistency maintenance node is configured to identify the first cross-chip node on the first chip and forward the first access request to the first cross-chip node; The first cross-chip node is configured to forward the first access request to the second cross-chip node on the second chip that is bound to the first cross-chip node; The second cross-shard node is configured to determine the second consistency maintenance node based on the memory address to be accessed by the first access request; The second cross-shard node is configured to send the first access request to the second consistency maintenance node.

24. The system according to any one of claims 14 to 23, characterized in that, The first consistency maintenance node is also configured to cache the data stored at the memory address to be accessed by the first access request after obtaining the data stored at that memory address.

25. The system according to claim 24, characterized in that, The first consistency maintenance node is also configured to, upon receiving the second access request, determine whether to cache the data at the memory address to be accessed by the second access request locally; if the data at the memory address to be accessed by the second access request is cached locally, respond to the second access request based on the cached data. If the data at the memory address to be accessed by the second access request is not cached locally, the second access request is forwarded.

26. A processor, characterized in that, include: The on-chip bus interconnect system as described in any one of claims 14-25.

Images