Memory pipeline control in hierarchical memory system

Abstract

The embodiment of the invention relates to memory pipeline control in a hierarchical memory system. In described examples, a processor system includes a processor core that generates memory transactions, a lower level cache memory having a lower memory controller, and a higher level cache memory having a higher memory controller with a memory pipeline. The higher memory controller is connected to the lower memory controller by a bypass path that skips the memory pipeline. The higher memory controller: determines whether a memory transaction is a bypass write, which is a memory write request indicated to not cause a corresponding write to be directed to a higher level cache memory; if the memory transaction is determined to be a bypass write, determining whether the memory transaction that prevents transfer is in a memory pipeline; and if it is determined that there is no transaction in the memory pipeline that prevents transfer, sending the memory transaction to the lower memory controller using a bypass path.

Description

[0001] Related application information of division

[0002] This application is a divisional application of the application patent application with the application date of May 26, 2020, the application number of 202080038514.X, and the invention name of “Memory pipeline control in hierarchical memory system”. TECHNICAL FIELD

[0003] This description relates generally to processing devices that can be formed as part of an integrated circuit such as a system on a chip (SoC). More specifically, this description relates to such systems with improved management of write operations. BACKGROUND

[0004] An SoC is an integrated circuit on a single die with multiple functional blocks such as one or more processor cores, memory, and inputs and outputs.

[0005] Hierarchical memory moves data and instructions between memory blocks with different read / write response times for respective processor cores such as central processing units (CPUs) or digital signal processors (DSPs). For example, memory that is more local to a respective processor core typically has a lower response time. Hierarchical memory includes a cache memory system with multiple levels such as LI and L2, where different levels describe different degrees of locality or different average response times of cache memory to respective processor cores. Here, a more local or lower response time cache memory such as an LI cache is referred to as a higher level cache memory than a less local or higher response time lower level cache memory such as an L2 cache or an L3 cache. Cache associativity refers to cache memory separation, where set-associative divides the cache into a number of sets of storage, and each such set stores a number of (way) blocks, while fully-associative caches are not bound by set restrictions. Thus, for an integer N, each location in main memory (system memory) can reside in any of N possible locations in an N-way associative cache.

[0006] A "victim cache" memory cache stores data (e.g., cache lines) evicted from a cache memory (e.g., an L1 cache). If an L1 cache read results in a miss (data corresponding to a portion of main memory is not stored in the L1 cache), a lookup occurs in the victim cache. If the victim cache lookup results in a hit (data corresponding to the requested memory address exists in the victim cache), the contents of the victim cache location that resulted in the hit are swapped with the contents of the corresponding location in the respective cache (the L1 cache in this example). Some example victim caches are fully-associative. Data corresponding to any location in main memory can be mapped to (stored in) any location in a fully-associative cache. SUMMARY

[0007] In described examples, a processor system includes a processor core that generates a memory transaction, a lower level cache memory having a lower memory controller, and a higher level cache memory having a higher memory controller with a memory pipeline. The higher memory controller is connected to the lower memory controller by a bypass path that skips the memory pipeline. The higher memory controller determines whether a memory transaction is a bypass write, which is a memory write request that is indicated as not resulting in a corresponding write being directed to the higher level cache memory, determines whether a prevent-forwarding memory transaction is in the memory pipeline if the memory transaction is determined to be a bypass write, and sends the memory transaction to the lower memory controller using the bypass path if no prevent-forwarding transaction is determined to be in the memory pipeline. BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 is a block diagram of an example processor that is part of a SoC.

[0009] Figure 2 is a block diagram of an example memory pipeline for Figure 1 a SoC.

[0010] Figure 3 is a process of a memory write operation for Figure 1 a SoC.

[0011] Figure 4 is a block diagram of an example memory pipeline for Figure 1 a SoC.

[0012] Figure 5 is a process of a memory write operation for Figure 1 a SoC. DETAILED DESCRIPTION

[0013] Figure 1 is a block diagram of an example processor 100 that is part of SoC 10. SoC 10 includes a processor core 102 that generates new data, such as a CPU or DSP. Processor 100 can include a clock 103, which can be part of processor core 102, or can be separate from processor core 102 (a separate clock not shown). Processor core 102 also generates memory read requests that request reads from a data memory controller 104 (DMC) and a streaming engine 106, and memory write requests that request writes to data memory controller 104 (DMC) and streaming engine 106. In some embodiments, processor core 102 generates one read request or write request per processor core clock cycle. The memory controller returns a write acknowledgement to processor core 102 to confirm that the requested memory write has been performed. Processor core 102 is also coupled to receive instructions from a program memory controller 108 (PMC). Streaming engine 106 facilitates processor core 102 sending certain memory transactions and other memory related messages that bypass DMC 104 and PMC 108.

[0014] SoC 10 has a hierarchical memory system. Each cache at each level can be unified or divided into separate data and program caches. For example, DMC 104 can be coupled to a level one data cache 110 (L1 D cache) to control data writes to and reads from the L1 D cache 110. Similarly, PMC 108 can be coupled to a level one program cache 112 (L1 P cache) to read instructions for execution by processor core 102 from the L1 P cache 112. (In this example, processor core 102 does not generate writes to the L1 P cache 112.) L1 D cache 110 can have an L1 D victim cache 113. A unified memory controller 114 (UMC) for a level two cache (L2 cache 116, e.g., L2 SRAM) is communicatively coupled to receive read and write memory access requests from DMC 104 and PMC 108, and to receive read requests from streaming engine 106, PMC 108, and a memory management unit 117 (MMU). (Example L2 controller UMC 114 is referred to as a "unified" memory controller in the example system because UMC 114 can store both instructions and data in L2 cache 116.) UMC 114 is communicatively coupled to pass read data and write acknowledgements (from outside the level one caches) to DMC 104, streaming engine 106, and PMC 108, which then pass them to processor core 102. UMC 114 is also coupled to control writes to and reads from L2 cache 116, and to pass memory access requests to a level three cache controller 118 (L3 controller). L3 controller 118 is coupled to control writes to and reads from L3 cache 119. UMC 114 is coupled to receive (via L3 controller 118) write acknowledgements and data read from L2 cache 116 and L3 cache 119. UMC 114 is configured to control pipelining of memory transactions for program content and data content (read and write requests for instructions, data transfers, and write acknowledgements). L3 controller 118 is coupled to control writes to and reads from L3 cache 119, and to mediate transactions using external functions 120 located outside of processor 100, such as other processor cores, peripheral functions of SOC 10, and / or other SoCs (and also to control snoop transactions). Thus, L3 controller 118 is a shared memory controller of SoC 10, and L3 cache 119 is a shared cache memory of SoC 10. Thus, memory transactions related to processor 100 and external functions 120 pass through L3 controller 118.

[0015] Memory transactions are generated by the processor cores 102 and communicated to lower level cache memories, or by the external functions 120 and communicated to higher level cache memories. For example, a victim write transaction can be initiated by the UMC 114 in response to a read transaction from a processor core 102 that generates a miss in the L2 cache 116.

[0016] The MMU 117 provides address translations and memory attribute information to the processor cores 102. It does so by looking up information in a table stored in memory (the connection between the MMU 117 and the UMC 114 enables the MMU 117 to access the memory containing the table using a read request).

[0017] Figure 2 is included in Figure 1 A block diagram of an example memory pipeline 200 within or associated with the UMC 114, and thus for illustration, Figure 2 various blocks from Figure 1 communicating with the UMC 114 are also repeated. The memory pipeline 200 includes an initial dispatch block 202 coupled to an integer number M of pipeline banks 206. Each pipeline bank 206 includes an integer number P of stages 208, and is illustrated as a vertical column below the initial dispatch block 202. The DMC 104 is coupled to the initial dispatch block 202 by a bus 204-1 of number N1 row widths, enabling the DMC 104 to provide a read or write request that transfers number N1 bits of data at a time. The streaming engine 106 is coupled to the initial dispatch block 202 by a bus 204-2 of number N2 row widths, enabling the streaming engine 106 to provide a read request that transfers number N2 bits of data at a time. The PMC 108 is coupled to the initial dispatch block 202 by a bus 204-3 of number N3 row widths, enabling the PMC 108 to provide a read request that transfers number N3 bits of data at a time. The L3 controller 118 is coupled to the initial dispatch block 202 by a bus 204-4 of number N4 row widths, enabling the L3 118 to provide a read or write request that transfers number N4 bits of data at a time. The MMU 117 is coupled to the initial dispatch block 202 by a bus 204-5 of number N5 row widths, enabling the MMU 117 to provide a read request that transfers number N5 bits of data at a time.

[0018] When a memory controller of processor 100 (e.g., DMC 104, streaming engine 106, PMC 108, MMU 117, or L3 controller 118) communicates a request to read from or write to memory mediated by UMC 114 (e.g., L2 cache 116, L3 cache 119, or memory in external function 120) to the UMC 114, initial dispatch block 202 dispatches the request to be handled by the appropriate pipeline library 206 for the particular request. Thus, initial dispatch block 202 performs arbitration for read and write requests. Arbitration determines which pipeline library 206 will receive which memory transaction queued at initial dispatch block 202, and in what order. Often, a read or write request is dispatched to a corresponding one of pipeline libraries 206, depending on, for example, the memory address of the data being written or requested, the request payload of the pipeline library 206, or a pseudo-random function. The initial scheduling block 202 schedules read and write requests received from the DMC 104, streaming engine 106, PMC 108, and L3 controller 118 by selecting in the first stage of the pipeline library 206. Memory transactions requested to be executed on the L3 cache 119 (or external function 120) are dispatched by the L3 cache scheduling block 404 (see the L3 cache scheduling block 404 in the L3 controller 118) after passing through the memory pipeline 200 corresponding to the L2 cache 116 (the pipeline library 206 and possible bus snooping related stages, which are not shown). Figure 4 ) arbitrates and dispatches to the L3 cache pipeline.

[0019] Request scheduling prevents conflicts between read or write requests to be handled by the same pipeline library 206 and maintains memory coherency (described further below). For example, request scheduling maintains order between memory transactions placed into the memory transaction queue (memory access request queue) of the initial scheduling block 202 by different memory controllers of the processor 100 or different buses of the same memory controller.

[0020] Further, pipeline memory transactions (read or write requests) requested by the DMC 104 or the PMC 108 are sent because the memory transaction has passed through the corresponding level-1 cache pipeline (in the DMC 104 for the L1 D cache 110 and in the PMC 108 for the L1 P cache 112) and either for a lower level cache or memory endpoint (or the external function 120), or has resulted in a miss in the respective level-1 cache, or bypassed the L1 D cache 110 because the corresponding data payload for a write request is not cacheable by the L1 D cache 110. Typically, memory transactions directed to the DMC 104 or the PMC 108 that result in a level-1 cache hit result in a write acknowledgement from the L1 D cache 110 or a response with data or instructions read from the L1 D cache 110 or the L1 P cache 112, respectively. Thus, memory transactions that result in a level-1 cache hit typically do not require access to the pipeline library 206 shown in FIG. 1. Figure 2 The pipeline library 206 shown in FIG. 1 is part of the UMC 114. The L1 D cache 110 can hold data generated by the processor core 102. For example, the external function 120 can access data in the L1 D cache 110 by writing data to the L2 cache 116 or the L3 cache 119, or reading or evicting data from the L1 D cache 110 using a snoop transaction controlled by the L3 controller 119 and managed by the UMC 114 (L2 controller, as proxy). Figure 1 ).

[0021] Figure 2 The pipeline library 206 shown in FIG. 1 is part of the UMC 114. The L1 D cache 110 can hold data generated by the processor core 102. For example, the external function 120 can access data in the L1 D cache 110 by writing data to the L2 cache 116 or the L3 cache 119, or reading or evicting data from the L1 D cache 110 using a snoop transaction controlled by the L3 controller 119 and managed by the UMC 114 (L2 controller, as proxy).

[0022] Memory coherency is that the contents of memory (or at least the contents deemed or indicated as valid) in a system is the same as expected by an ordered stream of read and write requests by one or more processors in the system. Writes affecting certain data or certain memory locations are prevented from bypassing earlier-issued writes or reads affecting the same data or the same memory locations. Further, certain types of transactions have priority, such as victim cache transactions and snoop transactions.

[0023] Bus snooping is a scheme whereby a coherence controller (snooper) in a cache monitors or snoops bus transactions to maintain memory coherence in a distributed shared memory system, such as SoC 10. If a transaction that modifies a shared cache block appears on the bus, the snooper checks whether its corresponding cache has an identical copy of the shared block. If the cache has a copy of the shared block, the corresponding snooper performs an action to ensure memory coherence in the cache. For example, depending on the transaction detected on the bus, the action may be to flush the shared block, invalidate the shared block, or update the shared block.

[0024] A "write stream" refers to a stream of write requests issued by a device (e.g., processor core 102), e.g., one write request per cycle, without pauses. For example, a write stream may be interrupted by pauses due to a full buffer or an insufficient number of write request identifiers. The ability to cause write requests to be extracted from the memory transaction queue as quickly as possible facilitates write streaming.

[0025] In order for the processor core 102 to know that a write has completed, it must receive a write acknowledgement. To maintain consistency, the processor core 102 can self-limit itself to a given number of outstanding write requests by limiting write requests that exceed the limit until a write acknowledgement is received for the outstanding write request. Thus, the processor core 102 and the L1D cache 110 can wait for a write acknowledgement (or "handshake") to proceed while pausing the corresponding write flow process within the processor core 102. A pause that interrupts the write flow can also be caused by the processor core 102 or DMC 104 waiting for a write acknowledgement from a previous write request. The processor core 102 can also be configured to pause while waiting for a write acknowledgement for certain operations, such as fence operations. The write acknowledgement (handshake) forwarded by the UMC 114 (Level 2 cache 116 controller) is used by the DMC 104 (Level 1 cache 110 controller) to detect write completion in lower-level caches (e.g., L2 cache 116 or L3 cache 119). However, due to various pipeline requirements including arbitration, ordering, and coherency, writes may take many cycles to complete.

[0026] At the first level of arbitration performed by the initial dispatch block 202, the UMC 114 (L2 cache 116 controller, which includes the initial dispatch block 202) determines whether a memory transaction is allowed to proceed in the memory pipeline 200, and in which pipeline bank 206. Between (1) initial arbitration and dispatch and (2) write completion, writes to the L2 cache 116 typically have few operations. The remaining operations for a dispatched write request can include, for example, checking for errors (e.g., firewall, addressing, and out-of-range errors), read-modify-write actions (updating error check codes for the data payload of the write request), and committing the data payload of the write request to memory. Typically, each pipeline bank 206 is independent, such that write transactions on a pipeline bank 206 (e.g., writing data from the L1 D cache 110 to the L2 cache 116) have no ordering or consistency requirements with respect to write transactions on other pipeline banks 206. Within each pipeline bank, writes to the L2 cache 116 proceed in their dispatched order. For partial writes that initiate read-modify-write transactions, relative ordering is maintained. If a memory transaction causes an addressing hazard or violates ordering requirements, the transaction is stalled and not issued to the pipeline bank 206. (A partial write is a write request with a data payload that is less than the minimum write length of the destination cache memory. A partial write triggers a read-modify-write transaction, in which data is read from the destination cache memory to pad the data payload of the write request to the minimum write length of the destination cache memory, and an updated error correction code (ECC) is generated from the resulting padded data payload and appended to the padded data payload. The padded data payload with the updated ECC is the content that is written to the destination cache memory.)

[0027] Due to these characteristics of the memory pipeline 200, once a write is scheduled in the pipeline bank 206 (e.g., a data write from the L1 D cache 110 to the L2 cache 116), it is guaranteed that the write follows all ordering requirements and does not violate coherency (thus, the conditions required to avoid breaking ordering and coherency are met). It can take a (variable) number of cycles to commit the write operation to memory, but a read issued after this write operation will "see" the write. Thus, if a read requests data or a memory location modified by a write, the read will retrieve the version of the data or the contents of the memory location specified by the write, not the previous version. Write-write ordering is also maintained. L3 cache 119 write requests can also be scheduled by the memory pipeline 200 (by the UMC 114) such that ordered completion of L3 cache 119 write requests is guaranteed. These guarantees mean that write requests scheduled by the memory pipeline 200 (L2 cache pipeline) into the pipeline bank 206 can be guaranteed to meet ordering and coherency requirements and complete within a finite amount of time. In other words, this guarantee is that a write transaction to a particular address and currently being scheduled onto the pipeline bank 206 will have its value "committed" to memory (the write will complete and store the corresponding data payload in memory) after a previously scheduled write transaction to the same address and before a later scheduled write transaction to the same address. This guarantee can be based on the pipeline being inherently "in-order" such that once a command enters the pipeline, it will be written to memory (committed) in the order it was scheduled. In other words, there are no bypass paths within the pipeline. (Bypass paths are handled below such that they do not break the ordering guarantees, e.g., with respect to older transactions targeting the same memory address.)

[0028] "Simultaneous with" is defined herein to mean simultaneous or immediately after. Thus, a first event occurring "simultaneous with" a second event can mean that the two events occur on the same cycle of a system clock.

[0029] The UMC 114 (L2 controller) sends a write acknowledgement of the write of data to the L2 cache 116 or higher level cache (e.g., data from the L1 D cache 110) to the DMC 104 (L1 controller) while the initial dispatch block 202 (first level arbitration) dispatches the corresponding write request. Thus, the write acknowledgement indicating the write completion is sent concurrently with the write request being dispatched, rather than after the memory pipeline 200 completes processing of the write request. This expedited acknowledgement is enabled by the guarantee that the dispatched write request will complete in-order and meet the coherency requirements. The UMC 114 creates the illusion that the write request is completed on the cycle it is dispatched, rather than on the cycle the corresponding data is committed (written) to memory. From the perspective of observability by the processor core 102 or the DMC 104, the L2 cache 116 appears to have completed the write request immediately upon dispatch of the write request. This enables the DMC 104 to lift the stall of the processor core 102 (or prevent the processor core 102 from stalling) more quickly, and to fetch the write request from the queue with lower latency (faster), thereby improving overall performance. The queue is a queue of transactions in the UMC 114 sent from respective "masters" (functional blocks that can send memory transactions to the UMC 114 for queuing), such as the DMC 104, the streaming engine 106, the PMC 108, the MMU 117, and the L3 controller 118. The queue can be implemented as a hold stage in which memory transactions reside while waiting to be arbitrated and dispatched to the pipeline bank 206 by the initial dispatch block 202.

[0030] The processor core 102 is typically configured to read data from memory for processing of the data. This is true for other processor cores 102 of other processors of the SoC 10, such as the processor 100, with respect to memory accessible by those other processors. However, the other processors of the SoC 10 require data produced by the processor core 102 to be available outside the data producing processor 100 in order to be able to access the produced data. This means that the produced data is made accessible outside through the L3 controller 118 either within shared memory (L3 cache 119) or by transmission to external functionality 120.

[0031] Figure 3 is a process 300 for a memory write operation of the SoC 10 of Figure 1 In step 302, the initial dispatch block 202 dispatches a write request directed to the L2 cache 116 to the pipeline bank 206. In step 304, the UMC 114 (L2 cache controller) sends a write acknowledgement to the processor core 102 immediately after dispatching the write request to the pipeline bank. In step 306, the stall of the processor core 102 is lifted in response to the write acknowledgement.

[0032] Figure 4 is for Figure 1 an example memory pipeline 400 of the SoC 10. As Figure 4 shown in FIG. 4, the memory pipeline 400 is similar to the memory pipeline 200 as Figure 2 shown in FIG. 2, but further includes a bypass path 402. The bypass path 402 couples the initial dispatch block 202 to an L3 cache dispatch block 404, thus skipping at least one level of memory pipeline corresponding to a cache. For example, the bypass path 402 enables a bypass write to bypass portions of the memory pipeline 400 associated with the L2 cache 116, thus shortening the overall processing time for a bypass write. Thus, writes to certain memory regions can be written to the L3 cache 119, rather than to the L2 cache 116 (or other lower level cache). Such writes (bypass writes) can be safely performed without accessing the memory pipeline stages associated with the L2 cache 116, including the pipeline bank 206 and associated bus snoop (not shown). This simplifies the coherency requirements for bypass writes.

[0033] To allow bypass writes to use the bypass path 402, memory coherency imposes ordering requirements. For example, a write cannot bypass another write (write-write ordering); a write cannot bypass a read (read-write ordering); a write cannot bypass a victim cache transaction (e.g., a write request from the L1 D victim cache 113 to the L1 D cache 110, which can be handled by the UMC 114 due to a cache miss for the victim cache related write request at the L1 level); and a write cannot bypass a snoop response (e.g., a snoop response corresponding to a controller request for a write from a victim cache to the L1 D cache 110 when the request is not caused by a cache miss). Victim cache transactions and snoop responses have high priority because they constitute memory synchronization events. Furthermore, L1 D cache 110 victims pass through the full pipeline because, for example, they include updating internal state in the UMC 114. However, a bypass write immediately following a victim cache transaction can also be prioritized, ensuring that the bypass write will not be stalled or paused (similar to a sliding stream). This prioritization state applies only to the single bypass write immediately following the victim cache transaction, and ends when the bypass write is sent to the L3 controller 118 for further processing.

[0034] The initial dispatch block 202 can have a designated bypass state (or "bypass mode") in which bypass writes can be dispatched to the bypass path 402 instead of to the full memory pipeline 400, including to the pipeline bank 206. When the initial dispatch block 202 is in bypass mode, bypass writes bypass the entire pipeline of the intermediate cache, including the associated internal arbitration. When the initial dispatch block 202 is not in bypass mode, bypass writes pass through the full memory pipeline 400.

[0035] Figure 5 is for Figure 1 an instance of the process 500 of memory write operations for the SoC 10 of Figure 5 Described is causing the initial dispatch block 202 to enter and remain in bypass mode so that bypass writes are allowed to go along the Figure 4Conditions for bypass path 402 to proceed. In step 502, initial scheduling block 202 determines whether the next memory transaction in the queue is a bypass write, carries data, or is related to a memory location that is guaranteed not to be written to L2 cache 116. (For example, the data payload is too large to fit in L2 cache 116, or contains a data type that is not stored by L2 cache 116.) If not, the next memory transaction is handled using normal (non-bypass) processing (through full memory pipeline 400) according to step 504, and the process returns to step 502. In step 506, initial scheduling block 202 determines whether a write to or from an L1D victim cache is in the pipeline (in pipeline bank 206). If so, the victim cache write is prioritized as a memory synchronization event, and in step 504, the bypass write is not allowed to proceed along bypass path 402. Instead, the bypass write receives normal (non-bypass) processing (albeit prioritized over non-bypass processing), and process 500 returns to step 502. In step 508, initial scheduling block 202 determines whether there is an L1D write (a write from DMC 104; in some embodiments, any write request generated outside of memory pipeline 200) in pipeline bank 206 (to which the bypass write is to be scheduled). If so, in step 510, the bypass write is delayed until the L1D write (all L1D writes in the corresponding pipeline bank 206; in some embodiments, all write requests generated outside of memory pipeline 200) is cleared from the corresponding pipeline bank 206, (e.g., the corresponding write is committed to memory, or reaches L3 cache scheduling block 404 for scheduling into the corresponding memory pipeline of L3 controller 118). Otherwise, in step 512, the bypass write enters bypass mode 514, and the bypass write is sent along bypass path 402 to L3 controller 118 (and from there to L3 cache 119). Next, in step 516, initial scheduling block 202 checks whether the next memory transaction in its queue is a bypass write. If so, process 500 remains in bypass mode by returning to step 512, and the next memory transaction in the queue (a bypass write) is sent along bypass path 402 to L3 cache scheduling 404. Thus, after a bypass write has satisfied the conditions for using bypass path 402, sequentially queued bypass writes can also be sent to L3 cache scheduling 404 via bypass path 402 without having to recheck the conditions handled in steps 506 and 508. If the next memory transaction in the queue of initial scheduling block 202 is not a bypass write, in step 518, initial scheduling block 202 exits bypass mode, the next memory transaction in the queue receives normal (non-bypass) processing, and process 500 returns to step 502.

[0036] In the described embodiments, modifications are possible, and other embodiments are possible within the scope of the claims.

[0037] In some embodiments, the streaming engine passes and returns responses to both read and write requests.

[0038] In some embodiments, the processor can include multiple processor cores (embodiments with multiple processor cores are not shown) with similar functional coupling to the DMC, streaming engine, and PMC as those shown in and described with respect to Figure 1

[0039] In some embodiments, the bus that implements parallel read or write requests can correspond to different types of read or write requests, made for different memory blocks or for different purposes, for example.

[0040] In some embodiments, the streaming engine enables a processor core to communicate directly with a lower level cache (e.g., an L2 cache), skipping a higher level cache (e.g., an L1 cache), to avoid data synchronization issues. This can be used to help maintain memory coherency. In some such embodiments, the streaming engine can be configured to only emit read requests, rather than both read and write requests.

[0041] In some embodiments, an L3 cache or other lower level memory can schedule a write request such that a write acknowledgement can be sent to the DMC (or processor core or another lower level memory controller) at the same time as the write request is scheduled to the corresponding memory pipeline.

[0042] In some embodiments, different memory access pipeline banks can have different numbers of stages.

[0043] In some embodiments, a processor in an external function can access data stored in an L2 cache; in some such embodiments, coherency between the contents stored in the L2 cache and cached in other processors in the external function is not guaranteed.

[0044] In some embodiments, a write can be a bypass write if the included data is too large for a lower level cache (e.g., an L1D cache or an L2 cache).

[0045] In some embodiments, a write can be a bypass write if a page attribute flags the write as corresponding to a device type memory region that is not cached by the UMC (L2 cache controller).

[0046] In some embodiments, an L1D cache (or other lower level cache) can cache the data payload of a bypass write.​

[0047] In some embodiments, the memory coherency rules of the processor prohibit bypassing memory transactions (memory read requests or memory write requests) other than L1D victim cache writes and L1D writes.

[0048] In some embodiments, the bypass path will bypass write jumps to the final arbitration stage before the memory pipeline bank scheduled into the L3 cache (not shown).

[0049] In some embodiments, the guarantee that a memory write will never be directed to the L2 cache (corresponding to a bypass write) includes a guarantee that such a write to the L2 cache is not possible, or a guarantee that this will never change.

[0050] In some embodiments, the guarantee that a memory write will never be directed to the L2 cache includes that the L2 cache does not have a copy or hash of the corresponding data.

[0051] In some embodiments, the guarantee that a memory write will never be directed to the L2 cache can change (if the guarantee has not been in effect recently, then it is initiated). In such embodiments, making this guarantee anew when a line is being written to the L2 cache (e.g., changing the corresponding mode register to make the guarantee) can require flushing the corresponding L2 cache. In this case, the cache flush can avoid cache copies of data payloads that are now guaranteed not to be written to the L2 cache remaining in the L2 cache after the guarantee is made.

[0052] In some embodiments, the cache controller initiates memory transactions only in response to transactions initiated by the processor core 102 or external function 120.

[0053] In some embodiments, only read or write requests can be scheduled into a corresponding one of the pipeline banks 206, e.g., depending on the memory address of the data being written or requested, the request load of the pipeline bank 206, or a pseudo-random function.

[0054] Modifications are possible in the described embodiments, and other embodiments are possible within the scope of the claims.

Claims

1. An integrated circuit device comprising: Processor core; as well as a cache memory hierarchy coupled to the processor core, the cache memory hierarchy comprising; cache memory; as well as a cache memory controller circuit coupled to the cache memory, the cache memory controller circuit comprising: a memory pipeline associated with the cache memory; Pipeline bypass paths; and a scheduler circuit coupled to the memory pipeline and the pipeline bypass path and configured to: Receive write transactions; determining whether the write transaction specifies a write to the cache memory; determining whether an intervening memory transaction inhibits use of the pipeline bypass path when the write transaction does not specify a write to the cache memory; and The write transaction is provided to a subsequent memory via the memory pipeline or the pipeline bypass path based on whether the write transaction specifies a write to the cache memory and whether the intervening memory transaction inhibits use of the pipeline bypass path. 2 . The integrated circuit device of claim 1 , wherein the scheduler circuit is configured to determine that the intervening memory transaction inhibits use of the pipeline bypass path based on the intervening memory transaction being associated with a snoop response.

3. The integrated circuit device according to claim 1 , wherein: The cache memory is a first cache memory; The cache memory hierarchy includes a second cache memory disposed between the first cache memory and the processor core; and The scheduler circuit is configured to determine that the intervening memory transaction inhibits use of the pipeline bypass path based on the intervening memory transaction being associated with a memory synchronization of the second cache memory.

4. The integrated circuit device according to claim 3, wherein: The second cache memory includes a victim cache; and The memory synchronization of the second cache memory is associated with the victim cache.

5. The integrated circuit device according to claim 3, wherein: The first cache memory is an L2 cache memory; and The second cache memory is an L1 cache memory.

6. The integrated circuit device of claim 3 , wherein the scheduler circuit is configured to provide the write transaction for the subsequent memory via the memory pipeline associated with a priority that inhibits blocking and stalling of the write transaction based on the intervening memory transaction being associated with the memory synchronization of the second cache memory.

7. The integrated circuit device according to claim 1, wherein the scheduler circuit is configured to determine whether the intervening memory transaction delays use of the pipeline bypass path.

8. The integrated circuit device according to claim 7, wherein: The cache memory is a first cache memory; The cache memory hierarchy includes a second cache memory disposed between the first cache memory and the processor core; and The scheduler circuit is configured to determine that when the intervening memory transaction is associated with a write to the second cache memory, the intervening memory transaction delays use of the pipeline bypass path.

9. The integrated circuit device according to claim 8, wherein: The first cache memory is an L2 cache memory; and The second cache memory is an L1 cache memory.

10. The integrated circuit device of claim 7, wherein the scheduler circuit is configured to provide the write transaction for the subsequent memory via the pipeline bypass path after the intervening memory transaction completes with respect to the cache memory when the intervening memory transaction delays use of the pipeline bypass path.

11. The integrated circuit device of claim 1, wherein the scheduler circuit is configured to determine, based on a payload size of the write transaction, that the write transaction does not specify a write to the cache memory.

12. The integrated circuit device of claim 1, wherein the scheduler circuit is configured to determine, based on a data type associated with the write transaction, that the write transaction does not specify a write to the cache memory.

13. The integrated circuit device according to claim 1, further comprising the subsequent memory, wherein: The cache memory is an L2 cache memory; and The subsequent memory is an L3 cache memory.

14. An integrated circuit device comprising: Processor core; an L1 cache controller coupled to the processor core; an L1 cache memory coupled to the L1 cache controller; an L2 cache controller coupled to the L1 cache controller; an L2 cache memory coupled to the L2 cache controller; The L2 cache controller comprises: a memory pipeline associated with the L2 cache memory; Pipeline bypass path; as well as a scheduler circuit coupled to the memory pipeline and the pipeline bypass path, the scheduler circuit being configured to: receiving a write transaction from the L1 cache memory; determining whether the write transaction specifies a write to the L2 cache memory; determining whether an intervening memory transaction inhibits use of the pipeline bypass path to service the write transaction when the write transaction does not specify a write to the L2 cache memory; as well as The write transaction is caused to be provided to the memory pipeline or the pipeline bypass path based on whether the write transaction specifies a write to the L2 cache memory and whether the intervening memory transaction inhibits use of the pipeline bypass path to service the write transaction.

15. The integrated circuit device of claim 14, further comprising an L3 cache controller coupled to the L2 cache controller, wherein the scheduler circuit is further configured to cause the write transaction to be provided to the L3 cache controller via the memory pipeline or the pipeline bypass path.

16. The integrated circuit device of claim 14, wherein the scheduler circuit is configured to determine that the intervening memory transaction inhibits use of the pipeline bypass path to service the write transaction based on the intervening memory transaction being associated with a memory synchronization of the L1 cache memory.

17. A method comprising: receiving a memory transaction at a cache controller, the cache controller including a memory pipeline configured for writes to cache memory and including a pipeline bypass path; determining whether the memory transaction is associated with a write to the cache memory; determining whether a previous memory transaction inhibited servicing of the memory transaction via the pipeline bypass path; selecting a resource from the memory pipeline and the pipeline bypass path for servicing the memory transaction based on whether the memory transaction is associated with a write to the cache memory and whether the previous memory transaction inhibits servicing the memory transaction via the pipeline bypass path; as well as The memory transaction is serviced via the resource.

18. The method of claim 17, wherein the selection of the resource causes the memory transaction to be serviced via the memory pipeline when the previous memory transaction is associated with a snoop response.

19. The method of claim 17, wherein: The cache memory is a first cache memory; and Servicing the memory transaction via the resource includes servicing the memory transaction via the memory pipeline associated with a priority that inhibits blocking and stalling of the memory transaction based on the previous memory transaction being associated with a memory synchronization of a second cache memory.

20. The method of claim 17 , further comprising determining whether the previous memory transaction delays servicing the memory transaction via the pipeline bypass path, wherein servicing the memory transaction via the resource comprises: delaying servicing the memory transaction via the pipeline bypass path based on the previous memory transaction, and servicing the memory transaction via the pipeline bypass path after completion of the previous memory transaction.

21. A circuit device comprising: cache memory; as well as a first cache controller coupled to the cache memory, wherein the first cache controller comprises: a first scheduler circuit; a pipeline coupled to the first scheduler circuit and to the cache memory; a second scheduler circuit coupled to the pipeline and configured to couple to a second cache controller; as well as a bypass path coupled between the first scheduler circuit and the second scheduler circuit, wherein the first scheduler circuit is configured to: receiving a first transaction; determining whether the first transaction is associated with writing data to the cache memory; as well as Based on the first transaction not being associated with a write to the cache memory, determining when to provide the first transaction to the second scheduler circuit via the bypass path based on whether a second transaction is in the pipeline.

22. The circuit device of claim 21, wherein the first scheduler circuit is configured to delay providing the first transaction to the bypass path until the second transaction is flushed from the pipeline.

23. The circuit arrangement according to claim 21, wherein: The cache memory is a secondary L2 cache memory; The first cache controller is a level 2 L2 cache controller; and The second transaction is a write associated with a level 1 L1 cache.

24. The circuit arrangement of claim 23, wherein the L1 cache is an L1 data cache.

25. The circuit arrangement of claim 21 , wherein: The cache memory includes a set of libraries; The first cache controller includes a set of pipelines; Each pipeline in the set of pipelines is coupled to a corresponding bank in the set of banks; and The first scheduler circuit is configured to delay providing the first transaction to the bypass path until all transactions of a predetermined type are flushed from the set of pipelines.

26. The circuit device of claim 21 , wherein the first scheduler circuit is configured to determine whether to provide the first transaction to the second scheduler circuit via the bypass path based on whether a third transaction is in the pipeline, based on the first transaction not being associated with a write to the cache memory.

27. The circuit arrangement of claim 26, wherein: The cache memory is a secondary L2 cache memory; The first cache controller is a level 2 L2 cache controller; and The third transaction is a write associated with the first level L1 cache.

28. The circuit arrangement of claim 27, wherein the L1 cache is an L1 victim cache.

29. The circuit arrangement of claim 26, wherein the third transaction is a snoop response.

30. The circuit device of claim 21, wherein the first scheduler circuit is configured to determine whether the first transaction is associated with writing data to the cache memory based on a data payload size associated with the first transaction.

31. The circuit device of claim 21, wherein the first scheduler circuit is configured to determine whether the first transaction is associated with writing data to the cache memory based on a data type associated with the first transaction.

32. The circuit device of claim 21, further comprising a processor core coupled to the first cache controller, wherein the first transaction is responsive to the processor core.

33. A circuit device comprising: Processor core; a level 1 L1 cache controller coupled to the processor core and configured to provide a first transaction; a second level L2 cache controller coupled to the processor core; as well as an L2 cache memory coupled to the L2 cache controller, wherein the L2 cache controller comprises: a scheduler circuit coupled to the L1 cache controller; a pipeline coupled to the scheduler circuit and the L2 cache memory; a bypass path coupled to the scheduler circuit in parallel with the pipeline, wherein the scheduler circuit is configured to: receiving the first transaction; determining whether the first transaction is associated with writing data to the L2 cache memory; as well as Based on the first transaction not being associated with a write to the L2 cache memory, determining when to provide the first transaction to the bypass path is based on whether a second transaction is in the pipeline.

34. The circuit device of claim 33, wherein the scheduler circuit is configured to delay providing the first transaction to the bypass path until the second transaction is flushed from the pipeline.

35. A method comprising: receiving, by a cache controller, a first transaction, wherein the cache controller comprises a pipeline configured to write to memory and a bypass path coupled in parallel with the pipeline; determining whether the first transaction is associated with a write to the memory; determining when to provide the first transaction to the bypass path based on whether a second transaction is in the pipeline, based on the first transaction not being associated with a write to the memory; as well as The first transaction is provided to the bypass path.

36. The method of claim 35, wherein providing the first transaction comprises delaying providing the first transaction to the bypass path until the second transaction is no longer in the pipeline.

37. The method of claim 35, wherein the second transaction is a write associated with a level 1 (L1) cache.

38. The method of claim 37, wherein the L1 cache is an L1 data cache.

39. The method of claim 35, further comprising determining whether to provide the first transaction via the bypass path or the pipeline based on whether a third transaction is in the pipeline.

40. The method of claim 39, wherein the third transaction is a write associated with a level 1 (L1) cache.

41. A system comprising: cache memory; as well as A first cache controller comprising: a pipeline associated with the cache memory; and a bypass path configured to bypass the pipeline; The first cache controller is configured to: receiving a first transaction; determining whether there is a transaction in the pipeline that inhibits use of the bypass path; providing the first transaction to a second cache controller via the bypass path based on determining that no transaction in the pipeline inhibits use of the bypass path; receiving a second transaction subsequent to the first transaction; and The second transaction is provided to the second cache controller via the bypass path without determining whether there is a transaction in the pipeline that inhibits use of the bypass path.

42. The system of claim 41, wherein the first cache controller is configured to: determining that there is a transaction in the pipeline that inhibits use of the bypass path; and Based on determining that there is a transaction in the pipeline that inhibits use of the bypass path, providing the first transaction to the second cache controller via the bypass path is delayed until the transaction is flushed from the pipeline.

43. The system of claim 41 , wherein: The cache memory is a secondary L2 cache memory; The first cache controller is a level 2 L2 cache controller; and The second cache controller is a level 3 L3 cache controller.

44. The system of claim 43, wherein the transaction inhibited from using the bypass path is a write associated with a level 1 (L1) cache.

45. The system of claim 44, wherein the L1 cache is an L1 data cache.

46. ​​The system of claim 44, wherein the L1 cache is an L1 victim cache.

47. The system of claim 41 , wherein to provide the first transaction to the second cache controller via the bypass path, the first cache controller is configured to: in addition to determining that there are no transactions in the pipeline that inhibit use of the bypass path, provide the first transaction to the second cache controller via the bypass path based on determining that the first transaction does not specify a write of data to the cache memory.

48. The system of claim 47, wherein the first cache controller is configured to determine whether the first transaction specifies writing data to the cache memory based on a data payload size associated with the first transaction.

49. The system of claim 41 , wherein: The first cache controller includes a first scheduler circuit; The second cache controller includes a second scheduler circuit; the pipeline being configured to couple the first scheduler circuit to the second scheduler circuit; and The bypass path is configured to couple the first scheduler circuit to the second scheduler circuit.

50. The system of claim 41, wherein the pipeline comprises a set of libraries.

51. A method comprising: receiving a first transaction at a first cache controller, wherein the first cache controller includes a pipeline associated with a cache memory and a bypass path that bypasses the pipeline; using the first cache controller, determining whether there is a transaction in the pipeline that inhibits use of the bypass path; providing the first transaction from the first cache controller to a second cache controller via the bypass path based on determining that no transaction in the pipeline inhibits use of the bypass path; receiving, at the first cache controller, a second transaction subsequent to the first transaction; as well as The second transaction is provided from the first cache controller to the second cache controller via the bypass path without determining whether there is a transaction in the pipeline that inhibits use of the bypass path.

52. The method of claim 51 , comprising: determining that there is a transaction in the pipeline that inhibits use of the bypass path; as well as Based on determining that there is a transaction in the pipeline that inhibits use of the bypass path, providing the first transaction to the second cache controller via the bypass path is delayed until the transaction is flushed from the pipeline.

53. The method of claim 51 , wherein: The cache memory is a secondary L2 cache memory; The first cache controller is a level 2 L2 cache controller; and The second cache controller is a level 3 L3 cache controller.

54. The method of claim 53, wherein the transaction inhibited from using the bypass path is a write associated with a level 1 (L1) cache.

55. The method of claim 54, wherein the L1 cache is an L1 data cache.

56. The method of claim 54, wherein the transaction inhibited from using the bypass path is a write from an L1 victim cache to the L1 cache and is associated with a write miss.

57. The method of claim 54, wherein the transaction inhibited from using the bypass path is a write from an L1 victim cache to the L1 cache and is not associated with a write miss.

58. The method of claim 51 , wherein providing the first transaction to the second cache controller via the bypass path comprises: The first transaction is provided to the second cache controller via the bypass path based on determining that the first transaction does not specify a write of data to the cache memory, in addition to determining that no transactions exist in the pipeline that inhibit use of the bypass path.

59. The method of claim 58, comprising: Based on a data payload size associated with the first transaction, a determination is made as to whether the first transaction specifies writing data to the cache memory.

60. The method of claim 51 , wherein: The first cache controller includes a first scheduler circuit; The second cache controller includes a second scheduler circuit; the pipeline coupling the first scheduler circuit to the second scheduler circuit; and The bypass path couples the first scheduler circuit to the second scheduler circuit.

Images