A cancellation and replay protocol scheme to improve sorting bandwidth
By introducing the sorting master and consistent slave mechanism in the PCIe interface, using timers to manage write requests, cancel and playback requests, it solves the bandwidth degradation and deadlock problems caused by waiting for global sorting when writing data, and improves the system data transmission efficiency.
Patent Information
- Application Number
- CN201880088864.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2017-12-28
- Filing Date
- 2018-09-20
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2038-09-20
AI Technical Summary
The existing PCIe interface needs to wait for requests for global sorting when writing data, resulting in significant deterioration of peak sorting bandwidth and increasing the risk of deadlock in the system.
Using the sorting master and consistent slave mechanism, the write request is managed by generating timers, canceling and playing back the write requests to avoid waiting for global sorting, ensuring that the system does not submit new requests before the older requests are globally visible.
It effectively avoids deadlocks, improves the sorting bandwidth, allows reading and writing request processing without waiting for global sorting, and improves the system's data transmission efficiency.
Smart Images

Figure CN111699476B_ABST
Abstract
Description
Background Art
[0001] Description of Related Technology
[0002] Peripheral Component Interconnect Express (PCIe) is a high-speed serial computer expansion bus standard that provides a high-bandwidth interconnect protocol for reliable data transmission. Various types of data (such as memory, input / output (I / O), and configuration data) can be transmitted through the PCIe interface. PCIe bandwidth continues to increase with the new generation of PCIe standards. For example, the extended speed mode (ESM) of PCIe 4.0 can transmit data at a speed of up to 25 gigabits per second (Gbps). In addition, more and more memory channels are needed to maintain higher data rates. PCIe and other standards (such as CPU storage instruction operations) generally require "sorting" writes so that other processors or I / O agents cannot observe newer writes until the processor or I / O agent has observed all older writes. In order to achieve this sorting, switching between memory channels generally requires waiting for requests to be globally sorted to avoid deadlock. Waiting for requests to be globally sorted leads to a significant degradation of peak sorting bandwidth. BRIEF DESCRIPTION OF THE DRAWINGS
[0003] The advantages of the methods and mechanisms described herein may be better understood by referring to the following description in conjunction with the accompanying drawings, in which:
[0004] Figure 1 is a block diagram of one embodiment of a computing system.
[0005] Figure 2 is a block diagram of one embodiment of a core complex.
[0006] Figure 3 is a block diagram of one implementation of a multi-CPU system.
[0007] Figure 4 is a block diagram of one implementation of a master device.
[0008] Figure 5 is a generalized flow chart illustrating one embodiment of a method for implementing a cancel and replay mechanism.
[0009] Figure 6 is a generalized flow chart illustrating another embodiment of a method for implementing a cancel and replay mechanism.
[0010] Figure 7 is a block diagram of one implementation of a deadlock scenario for a computing system. DETAILED DESCRIPTION
[0011] In the following description, numerous specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, it will be appreciated by those of ordinary skill in the art that various embodiments may be practiced without these specific details. In some cases, well-known structures, components, signals, computer program instructions, and techniques are not shown in detail to avoid confusion with the methods described herein. It should be understood that for simplicity and clarity of illustration, the elements shown in the accompanying drawings are not necessarily drawn to scale. For example, the size of some elements may be enlarged relative to other elements.
[0012] Disclosed herein are various systems, devices, methods and computer-readable media for implementing the cancellation and playback mechanism of sorting bandwidth. In one embodiment, a system includes at least a plurality of processing nodes (e.g., central processing unit (CPU)), a sorting master, an interconnection structure, a consistent slave device, a probe filter, a memory controller and a memory. Each processing node includes one or more processing units. The type of one or more processing units (e.g., general purpose processor, graphics processing unit (GPU), application specific integrated circuit (ASIC), field programmable gate array (FPGA), digital signal processor (DSP)) included in each processing node can be different with different implementation schemes and nodes. The sorting master is connected to a CPU or I / O device (such as a PCIe root complex) that needs to sort writes and is responsible for ensuring this sorting in a distributed structure. The consistent slave device is coupled to the sorting master device through the interconnection structure, and the consistent slave device is also coupled to the probe filter and the memory controller.
[0013] In one embodiment, the sorting master generates a write request that is forwarded to the consistent slave on the path to the memory. The consistent slave issues an invalidation probe to the processing node of the copy of the data targeted by the cache, and then when all cached copies of the data targeted by the write request have expired, an indication that the write request is globally visible is sent to the sorting master. In response to receiving the globally visible indication, the sorting master starts a timer. If the timer expires before all older requests are globally visible, the write request is canceled and replayed. In some embodiments, any newer request with an address dependency on the write request will also be canceled and replayed. As used herein, a write request is described as being "globally visible" when all cached copies of the targeted data are invalid.
[0014] Reference now Figure 1, a block diagram of one embodiment of a computing system 100 is shown. In one embodiment, computing system 100 includes at least core complexes 105A to 105N, input / output (I / O) interface 120, bus 125, one or more memory controllers 130, and network interface 135. In other embodiments, computing system 100 may include other components and / or computing system 100 may be arranged in different ways. In one embodiment, each core complex 105A to 105N includes one or more general-purpose processors, such as central processing units (CPUs). It should be noted that a "core complex" may also be referred to as a "processing node" or "CPU" herein. In some embodiments, one or more core complexes 105A to 105N may include a data parallel processor with a highly parallel architecture. Examples of data parallel processors include graphics processing units (GPUs), digital signal processors (DSPs), and the like. Each processor core within core complexes 105A to 105N includes a cache subsystem having one or more levels of cache. In one embodiment, each core complex 105A- 105N includes a cache (eg, a level 3 (L3) cache) that is shared among multiple processor cores.
[0015] The one or more memory controllers 130 represent any number and type of memory controllers accessible by the core complexes 105A to 105N. The one or more memory controllers 130 are coupled to any number and type of memory devices (not shown). For example, the type of memory in the one or more memory devices coupled to the one or more memory controllers 130 may include dynamic random access memory (DRAM), static random access memory (SRAM), NAND flash memory, NOR flash memory, ferroelectric random access memory (FeRAM), etc. The I / O interface 120 represents any number and type of I / O interface (e.g., peripheral component interconnect (PCI) bus, PCI extension (PCI-X), PCIE (PCI Express) bus, Gigabit Ethernet (GBE) bus, universal serial bus (USB)). Various types of peripheral devices may be coupled to the I / O interface 120. Such peripheral devices include (but are not limited to) a display, a keyboard, a mouse, a printer, a scanner, a joystick or other type of game controller, a media recording device, an external storage device, a network interface card, and the like.
[0016] In various embodiments, computing system 100 may be any of a server, a computer, a laptop, a mobile device, a game console, a streaming device, a wearable device, or various other types of computing systems or devices. It should be noted that the number of components of computing system 100 may vary from embodiment to embodiment. Figure 1It should also be noted that computing system 100 may include more or less of each component than shown. Figure 1 In addition, in other embodiments, the computing system 100 may be connected to Figure 1 The different ways of structuring are shown in .
[0017] Now go to Figure 2 , a block diagram of one embodiment of core complex 200 is shown. In one embodiment, core complex 200 includes four processor cores 210A to 210D. In other embodiments, core complex 200 may include other numbers of processor cores. It should be noted that a "core complex" may also be referred to herein as a "processing node" or a "CPU". In one embodiment, the components of core complex 200 are included in ( Figure 1 ) within core complexes 105A to 105N.
[0018] Each processor core 210A to 210D includes a cache subsystem for storing data and instructions retrieved from a memory subsystem (not shown). For example, in one embodiment, each core 210A to 210D includes a corresponding first level (L1) cache 215A to 215D. Each processor core 210A to 210D may include or be coupled to a corresponding second level (L2) cache 220A to 220D. Additionally, in one embodiment, the core complex 200 includes a third level (L3) cache 230 shared by the processor cores 210A to 210D. The L3 cache 230 is coupled to a sorting master to access the structure and memory subsystem. It should be noted that in other embodiments, the core complex 200 may include other types of cache subsystems having other numbers of caches and / or having other configurations with different cache hierarchies.
[0019] Reference now Figure 3 , a block diagram of one embodiment of a multi-CPU system 300 is shown. In one embodiment, the system includes a plurality of CPUs 305A to 305N. The number of CPUs per system may vary depending on the embodiment. Each CPU 305A to 305N may include any number of cores 308A to 308N, respectively, wherein the number of cores varies depending on the embodiment. Each CPU 305A to 305N also includes a corresponding cache subsystem 310A to 310N. Each cache subsystem 310A to 310N may include any number of levels of cache and any type of cache hierarchy.
[0020] In one embodiment, each CPU 305A to 305N is connected to a corresponding sorting master 315A to 315N. As used herein, a "sorting master" is defined as an agent that handles the business flowing on an interconnect structure (e.g., bus / structure 318). In various embodiments, the sorting master can be a CPU-consistent master, an input / output (I / O) master, or a master of any client that requires a fully sorted write memory request. In one embodiment, the sorting master is a consistent agent that manages the consistency of the connected CPUs. In order to manage consistency, the sorting master receives and processes consistency-related messages and probes, and generates consistency-related requests and probes. It should be noted that the "sorting master" may also be referred to as a "sorting master unit" in this article.
[0021] In one embodiment, each CPU 305A to 305N is coupled to a pair of consistent slave devices through corresponding sorting master devices 315A to 315N and bus / structure 318. For example, CPU 305A is coupled to consistent slave devices 320A to 320B through sorting master devices 315A and bus / structure 318. As used herein, "master device" is defined as a component that generates a request, and "slave device" is defined as a component that services a request. Consistent slave device (CS) 320A is coupled to memory controller (MC) 330A, and consistent slave device 320B is coupled to memory controller 330B. Consistent slave device 320A is coupled to probe filter (PF) 325A, wherein probe filter 325A includes entries for memory regions having cache lines for memory accessible by memory controller 330A in system 300. It should be noted that each of probe filter 325A and other probe filters may also be referred to as a "cache directory". Similarly, the consistent slave device 320B is coupled to the probe filter 325B, wherein the probe filter 325B includes entries for memory regions having cache lines for memory accessible by the memory controller 330B cached in the system 300. It should be noted that the example of having two memory controllers per CPU is only indicative of one embodiment. It should be understood that in other embodiments, each CPU 305A to 305N may be connected to a number of memory controllers other than two.
[0022] The bus / fabric 318 is also coupled to a sorting master 315P, which is coupled to endpoints 360 through a root complex 355. The sorting master 315P represents any number of sorting masters that provide connections from input / output (I / O) endpoints to the bus / fabric 318. The root complex 355 is the root of the I / O hierarchy and couples the CPUs 305A to 305N and memory to the I / O system (such as endpoints 360) (through the bus / fabric 318 and the sorting master 315P). The endpoints 360 represent any number and type of peripherals (e.g., I / O devices, network interface controllers, disk controllers) that are coupled to the root complex 355 directly or through a switch (not shown). In one embodiment, the endpoints 360 are coupled to the root complex 355 through a PCIe interconnect link. Any number of other endpoints may also be coupled to the root complex 355, and some embodiments may instantiate multiple root complexes 355 to separate sequencing masters 315P attached to the bus / fabric 318. Additionally, it should be noted that in other embodiments, there may be other connections from the bus / fabric 318 to other components that are not shown to avoid cluttering the drawing. For example, in another embodiment, the bus / fabric 318 includes connections to any number of other I / O interfaces and I / O devices.
[0023] In a configuration similar to CPU 305A, CPU 305N is coupled to consistent slaves 335A to 335B through a sorting master 315N and a bus / structure 318. Consistent slave 335A is coupled to memory through a memory controller 350A, and consistent slave 335A is also coupled to a probe filter 345A to manage the consistency of cache lines corresponding to memory accessible through memory controller 350A. Consistent slave 335B is coupled to a probe filter 345B, and consistent slave 335B is coupled to memory through a memory controller 365B. As used herein, a "consistent slave" is defined as an agent that manages consistency by processing requests and probes received for a corresponding memory controller. It should be noted that a "consistent slave" may also be referred to as a "consistent slave unit" in this article. In addition, as used herein, a "probe" is defined as a message that is passed from a consistency point to one or more caches in a computer system to determine whether the cache has a copy of a data block, and optionally indicates the state in which the cache should put the data block.
[0024] In one embodiment, a given sorting master 315 is configured to receive read and write memory requests from a corresponding CPU 305 or endpoint 360. A "write memory request" may also be referred to herein as a "write request" or "write". Similarly, a "read memory request" may also be referred to herein as a "read request" or "read". When a given sorting master 315 receives a write request from a corresponding CPU 305 or endpoint 360, the given sorting master 315 is configured to transmit the write request without corresponding data to the targeted consistent slave device of the memory controller and memory device. The given sorting master 315 buffers the write data and sends the write request as a write command without data to the targeted consistent slave device.
[0025] When a given sorting master 315 receives an indication from a consistent slave that a write request is globally visible, the given sorting master 315 starts a timer for the write request. In one embodiment, the globally visible indication is a target completion message. If all older outstanding requests queued on the given sorting master 315 are globally visible before the timer expires, the given sorting master 315 sends an indication to the consistent slave that the write request is ready to commit. In one embodiment, the indication that the write request is ready to commit is a source completion (or SrcDone) message.
[0026] If at least one older request is not yet globally visible when the timer expires, the given ordering master 315 cancels the write request. The given ordering master 315 then plays back the write request by reissuing the write request to the consistent slave. This helps prevent deadlock in the system 300 by canceling and playing back the write request when the timer expires (if the older request is still not globally visible). In addition, the cancel and play mechanism allows the ordering masters 315A to 315N to issue read and write requests on the fabric 318 without waiting for the previous request to be globally ordered.
[0027] Now go to Figure 4 , a block diagram of an embodiment of a sorting master device 400 is shown. In one embodiment, ( Figure 3 The ordering master devices 315A to 315N include logic of the ordering master device 400. The ordering master device 400 includes at least a control unit 410, a request queue 420, and a write data buffer 430. The control unit 410 is coupled to the request queue 420, the write data buffer 430, a local CPU (not shown) or one or more endpoints for receiving memory requests and an interconnect structure (not shown) for transmitting memory requests to any number of consistent slave devices. The control unit 410 can be implemented using any suitable combination of software, hardware, and / or firmware.
[0028] In one embodiment, when control unit 410 receives a memory request from a local CPU or an endpoint, control unit 410 creates an entry for the request in request queue 420. In one embodiment, each entry of request queue 420 includes a timer field, a request type field, an address field, a globally visible field, and one or more optional other fields. Control unit 410 is configured to forward the received request to a corresponding consistent slave device. For a write request, control unit 410 is configured to send a write command without data to a consistent slave device. When control unit 410 receives an indication that a write request is globally visible from a consistent slave device, control unit 410 then starts a timer associated with an entry in request queue 420. In one embodiment, a reference clock is used to decrement the timer in a given entry. The clock frequency of the reference clock may vary according to the embodiment.
[0029] In one embodiment, request queue 420 stores requests in the order in which they were received. In other words, the requests in request queue 420 are stored in order from oldest to newest, with the entries wrapping around to the beginning of request queue 420 when the last entry is occupied. In this embodiment, the first pointer may point to the newest entry, and the second pointer may point to the oldest entry. In another embodiment, the entries of request queue 420 may include an age field to indicate a relative age compared to other entries. In other embodiments, other techniques for tracking the relative age of outstanding requests may exist and are contemplated.
[0030] If the timer for a given write request entry expires and there is at least one older request that is not globally visible, then control unit 410 is configured to cancel write request. In one embodiment, control unit 410 cancels write request by sending a source completion (or SrcDone) message with a cancel bit group to a consistent slave device. In other embodiments, control unit 410 can utilize other types of messages or signals to cancel write request. In addition, control unit 410 checks to see whether any newer request has address dependency (that is, for identical addresses) with the write request canceled. In some embodiments, if any newer request has address dependency with the write request canceled, then control unit 410 also cancels these newer requests. After canceling write request (and optionally, any newer slave request), control unit 410 replays write request (and optionally, any newer slave request) by resending write request (and optionally, any newer slave request) to a consistent slave device.
[0031] Reference now Figure 5, showing one embodiment of a method 500 for implementing a cancel and replay mechanism. For discussion purposes, the steps and Figure 6 However, it should be noted that in various embodiments of the described method, one or more of the described elements are performed simultaneously, in a different order than shown, or one or more of the described elements are omitted entirely. Other additional elements are also performed when necessary. Any of the various systems or devices described herein is configured to implement method 500.
[0032] The sorting master transmits the write request without corresponding data to the consistent slave device via the interconnect structure (block 505). In response to receiving the request, the consistent slave device issues an invalidation request to the processing node to invalidate any cached copies of the data targeted by the write request (block 510). As described above, in various embodiments, the probe filter includes entries indicating which nodes or devices are caching copies of the data. In response to receiving a probe response from the processing node (e.g., confirming the invalidation of the cache line and / or returning any modified data), the consistent slave device sends an indication to the sorting master that the write request is now globally visible (block 515). In one embodiment, the indication that the write request is now globally visible is a target completion (or TgtDone) message.
[0033] In response to receiving the indication that the write request is now globally visible from a consistent slave device, the sorting master device starts a timer (frame 520) for the write request. The duration of the timer can vary according to the embodiment. In some embodiments, the duration of the timer is programmable. If all requests older than the write request are globally visible before the timer expires (conditional frame 525, "yes" branch), the sorting master device sends the indication that the write request can be submitted to the consistent slave device (frame 530) together with the data. In one embodiment, this indication is that the source of the data that also includes the write request is completed (or SrcDone) message. Then, the consistent slave device submits the write request (frame 535). As used herein, "submitting" a write request is defined as writing the data of the write request to the location targeted in the memory. In one embodiment, the consistent slave device merges the data of the write request with the data of any modification received from one or more processing nodes by the probe response. After frame 535, method 500 ends.
[0034] If any request older than the write request is not globally visible before the timer expires (conditional block 525, "no" branch), the ordering master cancels the write request (block 540). In one embodiment, the ordering master cancels the write request by sending a source complete (or SrcDone) message with a cancel bit set. Next, the ordering master plays back the write request by resending the write request to the consistent slave device (block 545). In addition, the consistent slave device optionally writes back any modified data received by the probe response (block 550). After block 550, method 500 ends.
[0035] Now go to Figure 6 , another embodiment of the method 600 for implementing the cancel and replay mechanism is shown. A write request is canceled due to a timer expiring before at least one older request has become globally visible (block 605). In response to the write request being canceled, the sorting master checks to see if any newer request has an address dependency with the canceled write request (conditional block 610). If any newer request has an address dependency with the canceled write request (conditional block 610, "yes" branch), the sorting master also cancels one or more newer requests that have an address dependency on the canceled write request (block 615). Next, the sorting master replays the write request and one or more canceled newer requests that have an address dependency on the write request (block 620). After block 620, the method 600 ends. If no newer request has an address dependency with the canceled write request (conditional block 610, "no" branch), the sorting master only replays the write request (block 625). After block 625, the method 600 ends.
[0036] Reference now Figure 7, a block diagram of an embodiment of a deadlock scenario of a computing system 700 is shown. In one embodiment, the ordering master device is required to provide a commit indication for its write in order of age, and the consistent slave device is required to perform an address matching request in order of age. In the context of this embodiment, table 740 shows an instance in which the ordering master device 710 and the ordering master device 715 issue back-to-back writes. As shown in table 740, the ordering master device 710 issues a write A, then issues a write B, and the ordering master device 715 issues a write B, then issues a write A. For the purpose of this discussion, it is assumed that the address A (to which the write of A is directed) belongs to a consistent slave device 725, and the address B (to which the write of B is directed) belongs to a consistent slave device 730. For the purpose of this discussion, it is also assumed that the writes to A and B intersect in structure 720. As used herein, a pair of writes "intersects" means that the newer write in the pair arrives at a consistent slave device earlier than the older write in the pair. For example, in one embodiment, writes may intersect in structure 720 because ordering master 710 is closer to consistent slave 730 , while ordering master 715 is closer to consistent slave 725 .
[0037] If consistent slave 725 and consistent slave 730 receive write requests according to the timing shown in table 745, this will cause a deadlock in system 700. As shown in table 725, consistent slave 725 receives a write to A by sorting master 715, followed by a write to A by sorting master 710. In addition, consistent slave 730 receives a write to B by sorting master 710, followed by a write to B by sorting master 715. Therefore, based on the timing of these requests, consistent slave 725 issues a target completion message for the write to A by sorting master 715, but the write to A is a newer operation in sorting master 715. Sorting master 715 cannot give a commit indication until it receives a target completion message for its write to B, which is blocked due to address dependencies in consistent slave 730. Additionally, consistent slave 730 issues a target completion message for the write to B by ordering master 710, but this write to B is a newer operation in ordering master 710. Ordering master 710 cannot give a commit indication until it receives a target completion message for its write to A, which is blocked due to address dependencies in consistent slave 725. Canceling and replaying the newer transaction in both ordering masters 710 and 715 releases the deadlock caused by this scenario.
[0038] In various embodiments, the program instructions of the software application are used to implement the methods and / or mechanisms described herein. For example, it is envisioned that the program instructions that can be executed by a general-purpose processor or a special-purpose processor. In various embodiments, such program instructions can be represented by a high-level programming language. In other embodiments, the program instructions can be compiled into a binary form, an intermediate form, or other forms from a high-level programming language. Alternatively, program instructions describing the behavior or design of hardware can be written. Such program instructions can be represented by a high-level programming language such as C. Alternatively, a hardware design language (HDL) such as Verilog can be used. In various embodiments, program instructions are stored on any of a variety of non-temporary computer-readable storage media. During use, a computing system can access a storage medium to provide program instructions to a computing system for program execution. Generally speaking, such a computing system at least includes one or more memories and one or more processors configured to execute program instructions.
[0039] It should be emphasized that the above embodiments are only non-limiting examples of implementations. Once the above disclosure is fully understood, many changes and modifications will become apparent to those skilled in the art. It is intended that the appended claims be interpreted as including all such changes and modifications.
Claims
1. A system for cancelling and replaying a protocol scheme, comprising: Sorting master unit; consistent slave units; a memory controller coupled to the consistent slave unit; as well as an interconnect structure coupled to the sequencing master unit and the consistent slave unit; The system is configured as follows: sending a write request without corresponding data from the sorting master unit to the consistent slave unit; In response to receiving an indication from the consistent slave unit that the write request is globally visible, starting a timer by the ordering master unit; In response to the timer expiring while at least one older request is not yet globally visible, canceling the write request; and The write request is played back by resending the write request from the ordering master unit to the consistent slave unit in response to canceling the write request.
2. The system of claim 1, wherein the ordering master unit is configured to cancel the write request by sending a cancel indication identifying the write request to the consistent slave unit.
3. The system of claim 1 , wherein the ordering master unit is further configured to: in response to all older requests being globally visible before the timer expires, send an indication that the older request can be committed to the consistent slave unit along with the data of the write request.
4. The system of claim 1, wherein the sequencing master unit is configured to provide commit indications for write requests in order of age.
5. The system of claim 4, wherein the consistent slave unit is further configured to: perform address matching requests in age order.
6. The system of claim 1, wherein the consistent slave unit is configured to write back any modified data received via a probe response to a memory in response to the write request being cancelled.
7. The system of claim 1, wherein the sequencing master unit is further configured to issue a request to the interconnect structure without waiting for a previous request to be globally sequenced.
8. A method for cancelling and replaying a protocol scheme, comprising: Sending write requests with no corresponding data from the sorting master to the consistent slaves; In response to receiving an indication from the consistent slave unit that the write request is globally visible, starting a timer by the ordering master unit; In response to the timer expiring while at least one older request is not yet globally visible, canceling the write request; as well as The write request is played back by resending the write request from the ordering master unit to the consistent slave unit in response to canceling the write request.
9. The method of claim 8, further comprising: The write request is canceled by sending a cancel indication identifying the write request to the coherent slave unit.
10. The method of claim 8, further comprising: In response to all older requests being globally visible before expiration of the timer, an indication that the older requests can commit is sent to the coherent slave unit along with the data of the write request.
11. The method of claim 8, further comprising: Provides commit indications for write requests in age order.
12. The method of claim 11, further comprising: Perform address matching requests in age order.
13. The method of claim 8, further comprising: In response to the write request being cancelled, any modified data received via the probe response is written back to the memory.
14. The method of claim 8, further comprising: Issue requests to the interconnect fabric without waiting for previous requests to be globally sequenced.
15. An apparatus for cancelling and replaying a protocol scheme, comprising: Sorting master unit; as well as consistent slave units; The device is configured as follows: sending a write request without corresponding data from the sorting master unit to the consistent slave unit; In response to receiving an indication from the consistent slave unit that the write request is globally visible, starting a timer by the ordering master unit; In response to the timer expiring while at least one older request is not yet globally visible, canceling the write request; and The write request is played back by resending the write request from the ordering master unit to the consistent slave unit in response to canceling the write request.
16. The apparatus of claim 15, wherein the ordering master unit is configured to cancel the write request by sending a cancel indication identifying the write request to the consistent slave unit.
17. The apparatus of claim 15, wherein the ordering master unit is further configured to: in response to all older requests being globally visible before expiration of the timer, send an indication that the older request can commit to the consistent slave unit along with the data of the write request.
18. The apparatus of claim 15, wherein the sequencing master unit is configured to provide commit indications for write requests in order of age.
19. The apparatus of claim 18, wherein the consistent slave units are configured to perform address matching requests in age order.
20. The apparatus of claim 15, wherein the sequencing master unit is further configured to issue a request to the interconnect structure without waiting for a previous request to be globally sequenced.
21. The system of claim 1, wherein the write request is globally visible when all cached copies of data targeted by the write request have been invalidated.
22. The method of claim 8, wherein the write request is globally visible when all cached copies of data targeted by the write request have been invalidated.
23. The apparatus of claim 15, wherein the write request is globally visible when all cached copies of data targeted by the write request have been invalidated.
Citation Information
Patent Citations
Wireless architecture for traditional wire based protocol
CN101755431A
Preventing writeback race in multiple core processors
CN101802796A
System, apparatus, and method for modifying the order of memory accesses
CN101861571A
Method and apparatus for implementing a relaxed ordering model in a computer system
US20030145136A1
Cache coherence protocol with speculative writestream
US20070022253A1
