Distributed virtual memory management for data processing networks

Abstract

A data processing network includes request nodes having local memory accessible as distributed virtual memory (DVM) and coupled by an interconnect structure. Multiple DVM domains are assigned, each containing a DVM node for processing DVM operation requests from request nodes in the domain. Upon receiving a request, the DVM node sends a snoop message to other request nodes in its domain and to one or more peer DVM nodes in other DVM domains. The DVM node receives snoop responses from the request nodes and from the one or more peer DVM nodes and sends a completion message to the first request node. Each peer DVM node sends a snoop message to request nodes in its domain, collects snoop responses, and sends a single response to the originating DVM node. In this manner, DVM operations are performed in parallel.

Description

Background Technology

[0001] The data processing network may include multiple requesting nodes, such as processing kernels, that initiate data transactions. The requesting nodes are coupled by an interconnect structure. Requesting nodes may have access to shared memory, such as system memory, and may include local memory, such as one or more caches. The data processing system may also include multiple slave nodes that provide endpoints for transactions initiated by the requesting nodes. Slave units may be, for example, system memory management units (SMMUs) that access system memory or nodes coupled to input / output (I / O) devices.

[0002] For efficiency, a mechanism is provided that allows data stored in the local memory of a requesting node to be accessed by other requesting nodes. By using a logical abstraction layer, local memory, along with other memories such as system memory, can be accessed as a distributed virtual memory (DVM). This reduces the need for processors to access slower memories or storage devices and improves system performance.

[0003] Because data can be shared and copies of data can be stored at different locations within the data processing network, a consistency protocol is implemented to ensure that data is processed in a consistent order and that outdated copies are not used. For this purpose, one or more home nodes are used. All accesses to a specific set of memory addresses are performed through the designated home node, allowing the home node to act as a point of consistency and serialization for the distributed virtual memory.

[0004] Messages are sent via the interconnect structure. Messages for operations related to DVM management (other than read / write operations) can be processed by the DVM node. The DVM node receives the DVM message from the requesting node, performs the requested action, and returns a response to the originating requesting node.

[0005] For example, Arm Limited 5CHI TM The on-chip interconnect specification defines miscellaneous nodes in the interconnect structure that can operate as DVM nodes to process DVM messages.

[0006] As the number of requesting nodes in the network increases, DVM nodes can become congestion points for messages in the network. Furthermore, due to the serial processing of requests, the time required to complete a DVM transaction increases with the number of nodes, resulting in unwanted latency in the network. Attached Figure Description

[0007] The accompanying drawings provide visual representations that will be used to more fully describe various representative embodiments and can be used by those skilled in the art to better understand the disclosed representative embodiments and their inherent advantages. In these drawings, similar reference numerals denote corresponding or similar elements.

[0008] Figure 1 This is a block diagram of a data processing network.

[0009] Figure 2 It is a block diagram of the logical arrangement of nodes in a data processing network with a single DVM node.

[0010] Figure 3 is a transaction diagram of DVM message flow in a data processing network with a single DVM node.

[0011] Figure 4 It is a block diagram of a data processing network based on various representative implementation schemes.

[0012] Figure 5 It is a transaction graph of DVM message flow in a data processing network with multiple DVM nodes according to various representative implementation schemes.

[0013] Figure 6 It is a block diagram of a data processing network based on various representative implementation schemes.

[0014] Figure 7 It is a block diagram of a data processing network consisting of two chips, based on various representative implementation schemes.

[0015] Figure 8 It is a flowchart of methods for DVM management based on various representative implementation schemes.

[0016] Figure 9 It is a block diagram of a data processing network based on various representative implementation schemes. Detailed Implementation

[0017] The various devices and apparatuses described herein provide mechanisms for processing DVM messages within the interconnected structure of data processing networks.

[0018] While this disclosure can have many different embodiments, specific embodiments are shown in the accompanying drawings and will be described in detail herein. It should be understood that the embodiments shown and described herein should be considered as examples providing the principles of this disclosure, and are not intended to limit this disclosure to the specific embodiments shown and described. In the following description, similar reference numerals are used to describe the same, similar, or corresponding parts in several views of the drawings. For simplicity and clarity, reference numerals may be repeated in the drawings to indicate corresponding or similar elements.

[0019] Figure 1This is a block diagram of a data processing network 100. In the data processing network, multiple nodes are coupled by an interconnect structure 104 (also simply referred to as an "interconnect"). Network nodes include requesting nodes 102 (RNs) that initiate data transactions. The example shown includes three requesting nodes, denoted as RN-0, RN-1, and RN-2. Requesting nodes 102 can be processor cores, processor clusters, accelerators, or other devices. For example, a requesting node can be a completely consistent master device.

[0020] Requesting node 102 may have access to shared memory such as system memory 106. Requesting node may include local memory 108, such as one or more caches or random access memory (RAM). The data processing network may also include other nodes, such as requesting node RN-I node 110 coupled to input / output (I / O) device 112, and system memory management unit (SMMU) 114 that accesses system memory 106.

[0021] For efficiency, a mechanism is provided that allows data stored in the local memory of a requesting node to be accessed by other requesting nodes. By using a logical abstraction layer, local memory, along with other memories such as system memory, can be accessed as a distributed virtual memory (DVM). This reduces the need for processors to access slower memory or storage devices and improves system performance. The distributed virtual memory (DVM) is represented as 116.

[0022] Because data can be shared and copies of data can be stored at different locations within the data processing network, a consistency protocol is implemented to ensure that data is processed in a consistent order and that outdated copies are not used. For this purpose, one or more home nodes 118 are used. All accesses to a specific set of memory addresses are performed through the designated home node, enabling the home node to act as a point of consistency and serialization for the distributed virtual memory.

[0023] Messages are sent via the interconnect structure. Messages for operations related to DVM management (other than read / write operations) can be processed by DVM node 120. Such operations include, for example, invalidating Translate Lookaside Buffer (TLB) entries, invalidating branch predictions, invalidating entries in the cache for physical instructions, invalidating entries in the cache for virtual instructions, and synchronizing operations. DVM node 120 receives DVM messages from the requesting node, performs the requested action, and returns a response to the requesting node.

[0024] For example, Arm Limited 5CHI TM The on-chip interconnect specification defines miscellaneous nodes in the interconnect structure that can operate as DVM nodes to process DVM messages.

[0025] Figure 2 This is a block diagram showing the logical arrangement of nodes in the data processing network 200. A single DVM node 202 is responsible for receiving requests from all requesting nodes 204 (RN-0, RN-1, ..., RN-7), sorting the requests, and distributing the requests as listening messages to all other requesting nodes and memory management units in the network. Although in Figure 2 The diagram shows eight request nodes, but the system can contain any number of request nodes. Therefore, DVM node 202 creates a message sending / receiving bottleneck, which becomes more severe as the number of request nodes increases.

[0026] Figure 3 is a transaction diagram 300 of the method for DVM message flow in a data processing network with a single DVM node. Figure 3 shows a timeline 302 for requesting nodes RN-0, RN-1, ..., RN-7 and a timeline 304 for DVM node DN0, with time flowing downwards. In the DVM message indicated by arrow (1), RN3 sends a message to DVM node DN0. This message indicates that RN3 wishes to request a DVM operation. As shown by arrow (2), the DVM node sends a message to RN3 to indicate that resources are available to receive DVM messages. In the message indicated by arrow (3), RN3 completes its request for a DVM operation. Then, DVM node DN0 sends a listener message, specifying the request for a DVM operation to other requesting nodes in the network, as shown by arrow 306. The listener message indicated by arrow 306 can be sent as one or more segments on a listener channel. For example, when the address field in the listener segment is not wide enough, the listener message can be sent in two parts using two segments on the listener channel.

[0027] Each requesting node responds to the listen message by sending a listen response to DVM node DN0, as shown by dashed arrow 308. Finally, DVM node DN0 sends message (4) to the originating requesting node (RN3) to indicate the completion of the requested DVM message.

[0028] Having a single node to handle requests for DVM operations leads to the following performance issues:

[0029] I. All requesting nodes in the chip send DVM messages to a single DVM node, creating a bottleneck on the request channel and straining the DVM node structure resources.

[0030] II. A single DVM node is responsible for sending listening messages to all other requesting nodes. This causes congestion on the listening interface.

[0031] III. All listening messages are serialized, increasing the duration of DVM operations, as shown in bold section 310 of the RN3 timeline.

[0032] IV. DVM nodes are used for a long duration, as indicated by the thickened portion 312 of the timeline for DVM node DN0.

[0033] For example, in a system with 64 request nodes and only one DVM node, 126 loops are needed to send the listening message, assuming each message requires two fragments (two loops).

[0034] According to embodiments of this disclosure, a data processing network is provided, comprising requesting nodes having local memory accessible as distributed virtual memory (DVM) and coupled by an interconnect structure. Multiple DVM domains are assigned, each containing DVM nodes for processing DVM operation requests from requesting nodes assigned to that domain. Upon receiving a request, a DVM node sends listener messages to other requesting nodes in its domain and to one or more peer DVM nodes in other DVM domains. The DVM node receives listener responses from the requesting node and from the one or more peer DVM nodes, and sends a completion message to a first requesting node. Each peer DVM node sends listener messages to requesting nodes in its own domain, collects listener responses, and sends a single response to the originating DVM node. In this manner, DVM operations are performed in parallel.

[0035] Figure 4 This is a block diagram of a data processing network according to various embodiments of this disclosure. The data processing network is implemented in chip 400 and includes multiple DVM nodes. Each DVM node is associated with a set of requesting nodes in the chip. This set of requesting nodes is a subset of the requesting nodes in the chip. The logical group containing the requesting nodes and associated DVM nodes is referred to herein as a DVM domain. All requesting nodes in a DVM domain send DVM messages to the DVM nodes in that domain. A DVM domain is a logical OR operation grouping of elements. Requesting nodes in a DVM domain do not need to be physically grouped. The association between requesting nodes and DVM nodes can be fully programmable, enabling the system to be optimized for different workloads (if needed).

[0036] exist Figure 4In the example shown, the network includes DVM node 402 (DN0) and DVM node 404 (DN1). DVM node 402 is associated with requesting nodes RN-0, RN-1, RN-2, and RN-3, which together form a first DVM domain 406. DVM node 404 is associated with requesting nodes RN-4, RN-5, RN-6, and RN-7, which together form a second DVM domain 408. DN0 is responsible for processing DVM messages from requesting nodes in the first DVM domain, while DN1 is responsible for processing DVM messages from requesting nodes in the second DVM domain. Each DVM node is responsible for receiving DVM messages from requesting nodes in its domain, sorting them, and distributing them to other requesting nodes in the domain. In this way, DVM operations are performed in parallel. This provides increased throughput and reduced latency compared to a network using a single DVM node. This method is scalable for larger systems.

[0037] Figure 5 This is a transaction diagram 500 of a method for DVM message flow in a data processing network with multiple DVM nodes according to various embodiments of this disclosure. In this simplified example, the data processing network is configured as follows: Figure 4 As shown, there are two DVM domains, each with DVM nodes and four request nodes. Figure 5 Timeline 502 is shown for requesting nodes RN-0, RN-1, ..., RN-7; timeline 504 is shown for the first DVM node DN0; and timeline 506 is shown for the second DVM node DN1, with time flowing downwards. In the example transaction shown, RN3 sends a message to DVM node DN0, indicated by arrow (1), because RN-3 is in the domain of DN0. This message indicates that RN3 wishes to request a DVM operation. As shown by arrow (2), the DVM node sends a message to RN3 (e.g., indicated as DataBufferID) to indicate that resources are available to receive DVM messages. In the message indicated by arrow (3), RN3 completes its request for a DVM operation.

[0038] Message (1) may include a first part of a DVM message sent on the request channel. For example, the first part of the DVM message may include a DVM request with associated attributes. Message (3) may include a second part of a DVM message sent as a data fragment on the data channel. The second part of the DVM message may include the target address of the DVM operation. This second part may be, for example, a “NonCopyBackWriteData” instruction indicating that the data should not be written back to memory. DVM node DN0 then sends a listen message to other requesting nodes (RN0, RN1, and RN2) in its DVM domain, specifying the request for DVM operation, as indicated by arrow 508. The listen message (referred to as the “SnpDVM” message) may be sent as one or more fragments, depending on the size of the message and the size of the listen request channel in the interconnect. DN0 collects listen responses 512 from requesting nodes in its domain.

[0039] DVM node DN0 also forwards DVM messages to other DVM nodes in the chip. These nodes are referred to as "peer DVM nodes" in this document. Figure 5 In the example shown, the DVM message is forwarded to the peer DVM node DN1, as indicated by arrow (4). The peer DVM node DN1 sends listener messages to the requesting nodes (RN4, RN5, RN6, and RN7) in its DVM domain, as indicated by arrow 510, and collects the resulting listener responses 514.

[0040] The peer DVM node DN1 then sends a single listener response, indicated by arrow (5), to the initiating DVM node DN0. Finally, the initiating DVM node DN0 sends message (6) to the originating requesting node (RN3) to indicate the completion of the requested DVM operation message.

[0041] The duration of the transaction seen by the originating node RN3 is indicated by box 516. This duration is shorter than the corresponding duration, as indicated by box 518. While this saving is relatively small in the simple example shown, it increases as the network size increases. Additionally, the duration of the transaction seen by the DVM node is also shortened (indicated by the bold portion of the timeline). This results in higher throughput due to better tracker utilization in the DVM node (i.e., the DVM node needs to track transactions for a shorter time).

[0042] Table 1 shows the average lifetime of DVM messages in a DVM node in a data processing network with 32 request nodes. This table demonstrates the significant resource savings achieved when using multiple DVM nodes compared to using a single DVM node.

[0043]

[0044]

[0045] Table 1

[0046] Table 2 shows the performance improvement for an example where the DVM message is an Instruction Cache Invalidation (ICI) request. This example assumes each ICI is for 64 bytes, the interconnect operates at 2 GHz, and the network consists of 32 requesting nodes.

[0047]

[0048] Table 2

[0049] Figure 6 This is a block diagram of a data processing network according to various embodiments of the present disclosure. The data processing network is implemented in chip 600 and includes multiple DVM nodes. The network includes four DVM nodes: 602 (DN0), 604 (DN1), 606 (DN2), and 608 (DN3). DVM node 602 is associated with requesting nodes RN-0, RN-1, and RN-2 that together form a first DVM domain 610. DVM node 604 is associated with requesting nodes RN-3, RN-4, and RN-5 that together form a second DVM domain 612. DVM node 606 is associated with requesting nodes RN-6, RN-7, and RN-8 that together form a third DVM domain 614. DVM node 608 is associated with requesting nodes RN-9, RN-10, and RN-11 that together form a fourth DVM domain 616. Processing DVM messages from requesting nodes in each DVM domain is the responsibility of the DVM node in that domain. Each DVM node is responsible for receiving DVM messages from requesting nodes in its domain, sorting them, and distributing them to other requesting nodes in the domain. In this way, DVM operations are performed in parallel.

[0050] DVM nodes operate in a manner coupled to each other. When the first DVM node receives a DVM message from a requesting node in its domain, it forwards the message to three other DVM nodes (its peer DVM nodes). The peer DVM nodes listen to the requesting node in their respective domains, collect the listener responses, and each peer DVM node sends a single response to the originating DVM node.

[0051] A DVM node can receive DVM messages forwarded from any or all of its peer DVM nodes. In one implementation, to avoid congestion, the number of pending DVM messages a DVM node can have at any given time is limited to a specified maximum. Furthermore, the DVM node is provided with sufficient resources to receive or accommodate DVM messages received from all its peer DVM nodes. Each DVM node is configured to have sufficient resources to guarantee the forwarding progress of at least one synchronous DVM message and one asynchronous DVM message from a peer DVM node.

[0052] To avoid deadlock caused by multiple DVM nodes, DVM nodes can block synchronization requests from other DVM nodes. In one implementation, when multiple requesting nodes send DVM synchronization requests to a DVM node in their respective domain, the DVM node is configured to schedule no more than one synchronization request to the requesting node in its domain and to other DVM nodes. Each of the other DVM nodes is configured to schedule no more than one synchronization request to its domain when it receives a synchronization request (DVMSYNC) from other DVM nodes and from requesting nodes in its own domain.

[0053] In another embodiment of this disclosure, the above mechanism is extended for use in multi-chip systems. Multi-chip systems use nodes called chip-to-chip gateways (CCGs) to send transaction requests to and receive transaction requests from other chips. To enable chip-to-chip DVM message transmission and reception, the CCG is assigned to one of the DVM domains.

[0054] Figure 7 This is a block diagram of a data processing system 700 comprising two chips according to various embodiments of the present disclosure. Chip 0 (702) is configured to have two DVM domains: a DVM domain 704 having DVM node DN00 and request nodes RN-00, RN-01, RN-02 and RN-03, and a DVM domain 706 having DVM node DN01 and request nodes RN-04, RN-05, RN-06 and RN-07. Chip 1 (708) is configured to have two DVM domains: a DVM domain 710 having DVM node DN10 and request nodes RN-10, RN-11, RN-12 and RN-13, and a DVM domain 712 having DVM node DN11 and request nodes RN-14, RN-15, RN-16 and RN-17.

[0055] Chip 0 (702) also includes a chip-to-chip gateway CCG0 (714) assigned to DVM domain 706. Chip 1 (708) also includes a chip-to-chip gateway CCG1 (716) assigned to DVM domain 712.

[0056] In the example DVM operation shown, requesting node RN-00 sends a DVM request to DVM node DN00 in its domain. DN00 sends a listener request to all requesting nodes and any CCGs (RN-01, RN-02, and RN-03 in this example) in its domain and to peer DVM node DN01. DN01 sends a listener request to all requesting nodes and any CCGs (RN-05, RN-06, RN-07, RN-08, and CCG0 (714) in its domain). Chip-to-chip gateway CCG0 (714) forwards the request via link 718 to chip-to-chip gateway node CCG1 (716) on chip 1 (708), and CCG1 (716) passes the request to DVM node (DN11) in its domain. The DVM node treats it in the same way as a request from a requesting node, and the listener request is propagated to all requesting nodes on chip 1 (708) as described above. The responses to the listening requests are collected and returned to the originating requesting node via CCG 716 and 714 and the DVM node.

[0057] As illustrated in the example described above, this disclosure provides a method for managing a distributed virtual memory (DVM) in a data processing network having multiple request nodes and multiple DVM nodes. The distributed virtual memory may include system memory and local memory associated with the multiple request nodes. The method includes assigning multiple DVM domains, each DVM domain including a DVM node and one or more request nodes. In operation, a first request node assigned to a first DVM domain including a first DVM node sends a DVM message requesting DVM operation to the first DVM node. The first DVM node sends listener messages to request nodes in the first DVM domain other than the first request node, and also sends listener messages to peer DVM nodes in DVM domains other than the first DVM domain. The first DVM node collects listener responses from request nodes in the first DVM domain (other than the first request node) and from each peer DVM node, and sends a completion message to the first request node.

[0058] Each peer DVM node sends a listener message to nodes in its DVM domain, collects listener responses from requesting nodes in the peer DVM domain, and sends a listener response to the first DVM node. A requesting node receiving a listener message may, for example, invalidate data in its local memory in response. More generally, a requesting node in the first DVM domain receives a listener message specifying a DVM operation from the first DVM node and then executes the requested DVM operation. If the request is for an asynchronous DVM operation, the listener response may be sent to the first DVM node before the requested DVM operation is completed. For synchronous DVM operations, the listener response is sent to the first DVM node after the requested DVM operation is completed.

[0059] DVM operations include, for example, invalidating entries in the address translation table, invalidating entries in the instruction cache, invalidating the predictor, and DVM synchronization.

[0060] In one implementation, the first DVM node sends a Buffer Identifier message (DBID message) to the first requesting node in response to receiving a DVM message from the first requesting node. The Buffer Identifier message identifies an available buffer for the first DVM node to store the DVM message. The first requesting node then sends the destination address in the distributed virtual memory to the DVM node. This destination address is included in the requesting nodes in the DVM domain sent to the first DVM node and in the listening messages sent to each peer DVM node.

[0061] The message sent by the first requesting node may include a first part of a DVM message sent in an address segment on the request channel. The first part of the DVM message includes a DVM opcode and associated attributes. In response to receiving the first part of the message, the first DVM node replies using a message from its available message buffer, identifying the first DVM node. The first requesting node then sends a second part of the DVM message in a data segment on the data channel. The second part of the DVM message includes the target address for the DVM operation.

[0062] The listen message can be sent in two parts. The first part of the listen message is sent on the listen channel and includes the transaction identifier (TXID), the DVM opcode for the DVM operation, and a first part of the destination address for the DVM operation. The second part of the listen message, also sent on the listen channel, includes the TXID and a second part of the destination address for the DVM operation. Parts of the listen message may arrive out of order.

[0063] Therefore, as described above, the DVM node in the first DVM domain of the first integrated circuit is configured to send a listening message for DVM operation to the DVM node of the second integrated circuit, wherein the listening message is sent via a first chip-to-chip gateway of the first integrated circuit and a second chip-to-chip gateway of the second integrated circuit. The DVM node of the first integrated circuit is also configured to receive a listening response from the DVM node of the second integrated circuit via the second chip-to-chip gateway and the first chip-to-chip gateway, indicating that the DVM operation was completed by the requesting node in the second integrated circuit.

[0064] The DVM node in the second domain is configured to receive listening messages from the DVM node of the first integrated circuit, send listening messages for DVM operation to the second request node of the second integrated circuit, collect listening responses from the second request node of the second integrated circuit, and send a single listening response to the DVM node of the first integrated circuit via the second chip-to-chip gateway and the first chip-to-chip gateway to indicate that the DVM operation is completed by the second request node in the second integrated circuit.

[0065] Figure 8 This is a flowchart 800 illustrating methods for DVM management based on various representative implementation schemes. Figure 8 Following the initial block 802, at block 804, the requesting node (RN) and the DVM node are assigned to DVM domains. This assignment may be user-programmable. In an exemplary DVM operation, the first requesting node sends a DVM message at block 806 to the DVM node assigned to its domain (referred to as the first DVM node). This message may be sent, for example, over a request channel on the interconnect structure. At block 808, the first DVM node responds, for example, with a buffer ID, indicating its readiness to receive data from the first requesting node. At block 810, the first requesting node sends data for the DVM operation to the first DVM node. This data may include, for example, the memory address of the DVM operation and may be sent over a data channel on the interconnect structure. At block 812, the first DVM node sends a DVM listener message to requesting nodes in its domain (other than the first requesting node). At block 814, the first DVM node forwards the DVM message to DVM nodes in other domains. Subsequently, at block 816, other DVM nodes send DVM listener messages to requesting nodes in their domains. Other DVM nodes collect responses to the DVM listener messages at box 818 and send a single completion message back to the first DVM node. At box 820, the first DVM node collects responses from other DVM nodes and from requesting nodes in its domain, and sends a completion message to the first requesting node. This completes the DVM operation, as indicated by termination box 822.

[0066] The use of multiple DVM domains provides increased DVM message throughput by processing multiple DVM messages in the same loop and sending DVM listen messages in parallel. Additionally, it reduces DVM operation latency. The data processing system can have any number of DVM nodes in the interconnect architecture. Any number of request nodes, memory management units, and chip-to-chip gateways can be assigned to DVM domains. Since fewer request nodes send messages to each of the DVM nodes, the pressure on the request channel at each DVM node is reduced. Furthermore, this reduces congestion in the listen message interface, as DVM nodes only send listen messages to request nodes within their domain and to other DVM nodes.

[0067] Figure 9 This is a block diagram of a data processing network 900 according to an embodiment of the present disclosure. Network 900 includes multiple request nodes, including a first request node 902 and other request nodes RN3, RN4, RN11, RN8, and RN6. Each request node includes local memory, such as, for example, a cache. The local memory of the request node, plus any shared memory, can be accessed as a distributed virtual memory (DVM). Network 900 also includes multiple DVM nodes, such as a first DVM node 904 and a second DVM node 906. The local memory of the request nodes can be accessed by other request nodes via one or more DVM nodes. Network 900 includes dispatch logic 908 configured to dispatch one or more request nodes to each DVM node to form a DVM domain for each DVM node. In the example shown, the first request node and request nodes RN3 and RN4 are dispatched to the first DVM node 904. Therefore, the grouping of request node 910 and the first DVM node 904 forms the first DVM domain. Requesting nodes RN6, RN8, and RN11 are assigned to the second DVM node 906. Therefore, the grouping of requesting node 912 and the second DVM node 906 forms the second DVM domain. Domain assignments can be recorded in the DVM domain configuration table 909. This table can be accessed by both the requesting nodes and the DVM nodes to identify other members of their domain.

[0068] Network 900 also includes an interconnect structure 914 that couples the one or more requesting nodes in each DVM domain to DVM nodes in the DVM domain. The interconnect structure also couples to the DVM nodes. In one embodiment, interconnect structure 914 includes a request channel 916, a data channel 918, and a listening channel 920.

[0069] When the first requesting node 902 sends a DVM message to the first DVM node 904 requesting DVM operation, the first DVM node 904 sends a listen message to the requesting node 910 and also sends a listen message on the listen channel 920 to the second DVM node 906 (and any other peer DVM nodes in the network). The first DVM node 904 receives a listen response from the requesting node 910 and also receives a listen response from the second DVM node 906 (and any other DVM nodes in the network). Then, the first DVM node 904 sends a completion message to the first requesting node 902.

[0070] Listening messages are processed by the listening controller 922 in the first DVM node 904.

[0071] The first DVM node 904 includes a first message buffer 924 for storing one or more DVM messages received from requesting nodes in its DVM domain, and a second message buffer 926 for storing listener messages received from peer DVM nodes in other DVM domains. The resources of the DVM node (such as the first and second message buffers 924, 926) are managed by a resource manager 928. The first DVM node 904 is configured to send a buffer identifier message to the first requesting node 902 in response to receiving a DVM message from the first requesting node 902 on the request channel 916. The buffer identifier message identifies the available first buffer of the first DVM node 904.

[0072] The first requesting node 902 is configured to send a target address in the distributed virtual memory to DVM node 904 in response to receiving a buffer identifier message from DVM node 904. The target address can be sent on data channel 918. The target address is included in the requesting messages sent to the first DVM domain and the listening messages sent to the second DVM node 906.

[0073] The second DVM node 906, acting as a peer DVM node, is configured to send listener messages to the requesting nodes in its domain. In the example shown, requesting nodes RN6, RN8, and RN11 are in the domain of the second DVM node 906. The second DVM node 906 collects listener responses from the requesting nodes in its DVM domain and sends these responses to the first DVM node 904.

[0074] In one implementation, a request node, a DVM node, a dispatch logic unit 908, and an interconnect structure 914 are constructed in a first integrated circuit. For example, the first integrated circuit may include a first chip-to-chip gateway, such as... Figure 7 As shown.

[0075] To avoid deadlocks or congestion in the data processing network, DVM nodes can be configured to limit the number of pending listener messages sent to peer DVM nodes. Additionally, DVM nodes can be configured to receive DVM messages for synchronizing DVM operations from multiple requesting nodes in the first DVM domain, but are limited to sending listener messages for synchronizing DVM operations one at a time to both the requesting nodes and peer DVM nodes.

[0076] In this document, relational terms such as first and second, top and bottom, etc., are used only to distinguish one entity or action from another entity or action, and do not necessarily require or imply any actual such relationship or order between such entities or actions. The terms “comprising,” “including,” “containing,” “having,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but may also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element preceded by “comprising…one” does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes that element, unless further limitation is desired.

[0077] Throughout this document, the terms “an embodiment,” “certain embodiments,” “implementation,” “specific implementation,” “aspect,” or similar terms refer to a particular feature, structure, or characteristic described in connection with that embodiment, which is included in at least one embodiment of this disclosure. Therefore, the appearance of such phrases in various places throughout this specification does not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics can be combined in any suitable manner without limitation in one or more embodiments.

[0078] As used herein, the term "or" will be interpreted inclusively, meaning any one or any combination. Therefore, "A, B, or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B, and C." Exceptions to this definition will only occur if the combination of elements, functions, steps, or actions is inherently mutually exclusive in some way.

[0079] As used herein, when applied to a component, the term "configured to" means that the component can be designed or constructed to perform a specified function, or has the necessary structure to enable it to be reconfigured or adapted to perform that function.

[0080] Numerous details are set forth to provide an understanding of the embodiments described herein. These embodiments may be practiced without these details. In other instances, well-known methods, procedures, and components have not been described in detail to avoid obscuring the embodiments described. This disclosure should not be construed as limiting the scope to the embodiments described herein.

[0081] Those skilled in the art will recognize that this disclosure has been described with the aid of examples. This disclosure may be implemented using hardware component equivalents such as dedicated hardware and / or dedicated processors, which are equivalents of the disclosure as described and claimed. Similarly, dedicated processors and / or dedicated hardwired logic may be used to construct alternative equivalent embodiments of this disclosure.

[0082] Specialized or reconfigurable hardware components used to implement the disclosed mechanisms can be described, for example, by instructions in a hardware description language (HDL), such as VHDL, Verilog, or RTL (Register Transfer Language), or by a netlist of components and connectivity. Instructions can be at the functional or logical level, or a combination thereof. Instructions or netlists can be input into automated design or manufacturing processes (sometimes called high-level synthesis), which interpret the instructions and create digital hardware that implements the described functions or logic.

[0083] HDL instructions or netlists may be stored on non-transitory computer-readable media, such as electrically erasable programmable read-only memory (EEPROM); non-volatile memory (NVM); mass storage devices, such as hard disk drives, floppy disk drives, optical disk drives; and, without departing from this disclosure, optical storage elements, magnetic storage elements, magneto-optical storage elements, flash memory, magnetic core memory, and / or other equivalent storage technologies. Such alternative storage devices should be considered equivalents.

[0084] The various embodiments described herein are implemented using dedicated hardware, configurable hardware, or a programmable processor that executes programming instructions, which are broadly described in flowchart form and can be stored on any suitable electronic storage medium or transmitted via any suitable electronic communication medium. Combinations of these elements may be used. Those skilled in the art will understand that the above processes and mechanisms can be implemented in any number of variations without departing from this disclosure. For example, the order of certain operations performed may often be changed, additional operations may be added, or operations may be deleted without departing from this disclosure. Such variations are conceivable and are considered equivalent.

[0085] Various representative embodiments described in detail herein are given by way of example rather than limitation. Those skilled in the art will understand that various changes can be made to the form and details of the described embodiments to obtain equivalent embodiments that remain within the scope of the appended claims.

Claims

1. A method for managing a distributed virtual memory (DVM) in a data processing network, the data processing network having a plurality of requesting nodes and a plurality of DVM nodes, the distributed virtual memory being associated with the local memory of the plurality of requesting nodes, the method comprising: Multiple DVM domains are assigned, and each DVM domain includes a DVM node among the multiple DVM nodes and one or more request nodes among the multiple request nodes. The multiple DVM nodes include a first DVM node in a first DVM domain. The first requesting node in the first DVM domain sends a DVM message to the first DVM node to request a DVM operation. In response to receiving the DVM message, the first DVM node: Send listening messages to all request nodes in the first DVM domain except for the first request node; Send listening messages to peer DVM nodes in DVM domains other than the first DVM domain; and In response to receiving a listening response from the requesting node in the first DVM domain other than the first requesting node, and receiving a listening response from each peer DVM node, a completion message is sent to the first requesting node.

2. The method of claim 1, further comprising each peer DVM node: Send a listening message to the requesting node in the peer DVM node; Collect listening responses from the requesting node in the DVM domain from the peer DVM node; and Send a listening response to the first DVM node.

3. The method according to claim 1, further comprising: In response to receiving a listening message, request nodes other than the first request node invalidate the data in the local memory of the request node.

4. The method according to claim 1, further comprising a request node in the first DVM domain: Receive a listening message from the first DVM node specifying the DVM operation; Perform the DVM operation; For asynchronous DVM operations: Before completing the requested DVM operation, a listening response is sent to the first DVM node; and For synchronous DVM operations: After completing the requested DVM operation, a listening response is sent to the first DVM node.

5. The method of claim 1, wherein the DVM operation comprises: Invalid entry in the address translation table; Invalid entries in the instruction cache; Invalid predictor; or DVM synchronization.

6. The method according to claim 1, further comprising: In response to receiving the DVM message from the first requesting node, the first DVM node sends a buffer identifier message to the first requesting node, the buffer identifier message identifying an available buffer of the first DVM node for storing the DVM message; and The first requesting node sends the target address in the distributed virtual memory to the DVM node. The listening message sent to the requesting node and the listening message sent to each peer DVM node include the target address.

7. The method of claim 1, wherein the first requesting node sending the DVM message includes the first requesting node: The first part of the DVM message is sent in an address segment on the request channel, the first part of the DVM message including the DVM opcode and associated attributes; Receive a message identifying the available message buffer of the first DVM node from the first DVM node; and The second part of the DVM message is transmitted in a data segment on the data channel, the second part of the DVM message including the target address of the DVM operation.

8. The method of claim 1, wherein sending the listening message comprises: The first part of the listening message is sent on the listening channel, the first part including the transaction identifier (TXID), the DVM opcode of the DVM operation, and the first part of the target address of the DVM operation; and The second part of the listening message is sent on the listening channel, the second part including the TXID and a second part of the target address of the DVM operation.

9. A data processing network system, the data processing network system comprising: Multiple request nodes, each request node including local storage, the local storage of the multiple request nodes being accessible as a distributed virtual storage (DVM); Multiple DVM nodes, wherein the local memory of the requesting node can be accessed by other requesting nodes via one or more of the multiple DVM nodes; A dispatch logic component is configured to dispatch one or more of the plurality of request nodes to each of the plurality of DVM nodes to form a DVM domain for each DVM node. and An interconnection structure that operatively couples one or more requesting nodes in a DVM domain to and operatively couples between the DVM nodes in the DVM domain. The first requesting node in the first DVM domain is configured to send a DVM message to the DVM node in the first DVM domain to request a DVM operation, and The DVM nodes in the first DVM domain are configured as follows: In response to receiving the DVM message from the first request node, a listening message is sent to all request nodes in the first DVM domain other than the first request node. In response to receiving the DVM message from the first requesting node, a listening message is sent to one or more peer DVM nodes in a DVM domain other than the first DVM domain; and In response to receiving a listening response from a request node in the first DVM domain other than the first request node, and receiving a listening response from one or more peer DVM nodes, a completion message is sent to the first request node.

10. The data processing network system according to claim 9, wherein the DVM node in the plurality of DVM nodes comprises: A first message buffer for receiving one or more DVM messages from a requesting node in the DVM domain of the DVM node; and A second message buffer used for listening messages received from DVM nodes in other DVM domains. The DVM node is configured to send a buffer identifier message to the first requesting node in response to receiving the DVM message from the first requesting node. The buffer identifier message identifies the available first buffer of the DVM node. The first requesting node is configured to send the target address in the distributed virtual memory to the DVM node in response to receiving the buffer identifier message from the DVM node.

11. The data processing network system of claim 10, wherein the listening message sent to the requesting node in the first DVM domain other than the first requesting node and the listening message sent to the peer DVM node include the target address.

12. The data processing network system of claim 11, wherein the peer-to-peer DVM nodes are configured as follows: Send a listening message to the requesting node in the DVM domain of the peer DVM node; Collect listening responses from the requesting node in the DVM domain from the peer DVM node; and Send a listening response to the DVM node in the first DVM domain.

13. The data processing network system of claim 9, wherein the plurality of request nodes, the plurality of DVM nodes, the dispatch logic component, and the interconnection structure are constructed in a first integrated circuit.

14. The data processing network system of claim 13, wherein the first integrated circuit further comprises a first chip-to-chip gateway operatively coupled to the first DVM node of the plurality of DVM nodes.

15. The data processing network system of claim 14, wherein the DVM node of the first DVM domain of the first integrated circuit is configured as follows: A listening message for the DVM operation is sent to the DVM node of the second integrated circuit, wherein the listening message is sent via the first chip-to-chip gateway of the first integrated circuit and the second chip-to-chip gateway of the second integrated circuit; and The DVM node of the second integrated circuit is received via the second chip-to-chip gateway and the first chip-to-chip gateway to indicate that the DVM operation is completed by the requesting node in the second integrated circuit.

16. The data processing network system of claim 15, further comprising a second integrated circuit, the second integrated circuit comprising: Multiple second request nodes, each second request node including local memory, wherein the multiple request nodes of the first integrated circuit and the local memory of the multiple second request nodes of the second integrated circuit can be accessed as distributed virtual memory; Multiple second DVM nodes; Second dispatch logic unit; The second interconnect structure, and The second chip-to-chip gateway is operatively coupled to a second DVM node among the plurality of second DVM nodes and operatively coupled to the first chip-to-chip gateway of the first integrated circuit. The DVM node in the first DVM domain of the first integrated circuit is configured to send listening messages for DVM operations to the second DVM node of the second integrated circuit via the first chip-to-chip gateway and the second chip-to-chip gateway. The second DVM node of the second integrated circuit is configured as follows: Receive the listening message from the DVM node in the first DVM domain of the first integrated circuit. Send a listening message for the DVM operation to the second request node of the second integrated circuit. Collect the listening response from the second request node of the second integrated circuit, and A single listening response is sent to the DVM node of the first integrated circuit via the second chip-to-chip gateway and the first chip-to-chip gateway to indicate that the DVM operation is completed by the second requesting node in the second integrated circuit.

17. The data processing network system according to claim 9, wherein the request node among the plurality of request nodes includes: Completely identical master equipment; or A master device with consistent input / output.

18. The data processing network system of claim 9, wherein the first DVM node is configured to limit the number of pending listening messages sent to the peer DVM node.

19. The data processing network system according to claim 9, wherein the first DVM node is configured as follows: Receive DVM messages from multiple requesting nodes in the first DVM domain for synchronizing DVM operations.

20. The data processing network system of claim 19, wherein the first DVM node is further configured to: Listening messages for the synchronous DVM operation are sent one at a time to the requesting node and the peer DVM node.

Images