Multi-port sram system for distributed memory pool

Abstract

Several examples of multi-port SRAM systems and related methods for distributed memory pools are disclosed. In one example, a system includes: a memory bus switch coupled to multiple memory buses; a memory pool switch coupled to one or more memory arrays; and one or more SRAM blocks coupled to the memory bus switch and the memory pool switch to convert a first transaction received on one of the multiple memory buses into a second transaction executed on the one or more memory arrays.

Description

Technical Field

[0001] Several examples of multi-port SRAM systems and related methods for distributed memory pools are disclosed. Background Technology

[0002] To fully realize the potential of high-performance computing (HPC) architectures that involve using multiple processors to solve complex problems, memory pooling techniques have been proposed. Modern system-on-chip (SoC) designs that combine chiplets and chipsets have become critical for increasing the number of parallel concurrent processing units, such as CPUs, neural processing units (NPUs), and graphics processing units (GPUs), in order to maximize parallel processing in generative artificial intelligence applications.

[0003] The proposed memory pool is built on the concept of Unified Memory Access (UMA) with the goal of achieving high performance and scalability in heterogeneous systems comprising multiple main processor clusters (e.g., CPUs) and coprocessor clusters (e.g., GPUs, NPUs). However, silicon size constraints and performance requirements often necessitate a non-unified memory access (NUMA) architecture, especially when dealing with large numbers of main processor and coprocessor clusters. NUMA allocates memory locally for each processing cluster, which can limit scalability, particularly when inter-cluster communication becomes a bottleneck.

[0004] To overcome these challenges, distributed local memory systems must operate as unified memory systems in a broader heterogeneous environment. A key challenge in designing such systems is the diverse behavior of individual memory cells due to the heterogeneous nature of the processing. The main processor and coprocessors have different data flows, requiring independent configuration of each memory pool. Furthermore, variations in memory access latency, especially when using different memory types such as DRAM and NVRAM, introduce further complexity.

[0005] What is needed is a new architecture to bridge the performance gap between different processor types and memory types. Summary of the Invention

[0006] To mitigate latency issues, a high-performance SRAM cache is proposed to bridge the performance gap between memory types. Since the main processor and coprocessor access memory differently, the SRAM cache must support various caching algorithms to prevent cache thrashing and optimize the performance of heterogeneous systems. Furthermore, to support the system disclosed by the same inventors in U.S. Patent Application Publication No. 2023 / 0035610, entitled "Hybrid System Fabric for Enabling Host Operating System and Real-time Operating System Within Chiplet System-on-chip," a portion of the SRAM should be configured as a swap memory between the host operating system and the real-time operating system within the chip-on-chip SoC, which is incorporated herein by reference.

[0007] The proposed invention outlines a multi-port SRAM system for supporting concurrent read and write operations across multiple memory buses using a single-port SRAM cell. This method enhances the concurrency of a system disclosed by the same inventor in U.S. Patent No. 5,206,833, entitled “Pipelined Dual-port RAM,” which is incorporated herein by reference.

[0008] The SRAM system within each memory pool can be flexibly configured to handle various memory cycles, such as cacheable operations, non-cacheable operations, swapping operations, prefetching operations, and write-after operations. This allows the system to function as a unified high-speed memory for various processors, regardless of whether the system consists of DRAM or NVRAM. Distributed memory pools can expand the aggregated SRAM size, enabling individual memory pools to be configured differently within a larger architecture, optimizing unified memory access, and fully leveraging the parallel processing capabilities of CPUs, GPUs, and NPUs.

[0009] In one example, a system includes a memory bus switch coupled to a plurality of memory buses; a memory pool switch coupled to one or more memory arrays; and one or more SRAM blocks coupled to the memory bus switch and the memory pool switch to convert a first transaction received on one of the plurality of memory buses into a second transaction executed on one or more memory arrays.

[0010] In another example, one approach includes: routing transactions from multiple memory buses to one or more SRAM blocks via a memory bus switch; and routing transactions from one or more SRAM blocks to one or more memory arrays via a memory pool switch. Attached Figure Description

[0011] Figure 1 The computing system is described.

[0012] Figure 2 It describes the physical address mapping.

[0013] Figure 3 A configurable multi-port SRAM system is described.

[0014] Figure 4 The bus protocol is described.

[0015] Figure 5 The switching configuration is described.

[0016] Figure 6 The switching configuration is described.

[0017] Figure 7 A multi-port SRAM block is described.

[0018] Figure 8 A timeline diagram was drawn. Detailed Implementation

[0019] Figure 1A computing system 100 is depicted, comprising a main processor unit (CPU) 101 and various coprocessor units such as a PCIe host 102, a neural processing unit (NPU) 103, a graphics processing unit (GPU) 104, and a media processor unit (MPU) 105. These components are interconnected to a distributed memory pool 106, which in this example is configured as a four-module configuration including memory modules 107, 108, 109, and 110. The distributed memory pool 106 also includes an SRAM cache system 118, comprising configurable multi-port SRAM systems 119, 120, 121, and 122, which are respectively part of memory modules 107, 108, 109, and 110. The memory configuration is scalable, allowing the number of memory modules to be adjusted based on specific memory requirements. This flexibility includes options beyond a four-module configuration to increase memory capacity or reduce memory capacity to meet smaller memory requirements. Memory modules 107, 108, 109, and 110 are physically separate from the SoC containing the main processor 101 and coprocessors 102, 103, 104, and 105, and can be monolithic, chip-based, or included in another architecture.

[0020] Interconnect PHY 111 links memory modules 107, 108, 109, and 110 to a distributed memory pool 106. Distributed memory pool 106 supports a Unified Memory Access (UMA) architecture, providing access to all main processors 101 and coprocessors 102, 103, 104, and 105. UMA allows all processors to directly access target physical addresses in memory without memory allocation or deallocation. Distributed memory pool 106 helps prevent memory conflicts between concurrent processing units, resulting in improved performance scaling as the number of processing units increases. To accommodate the diverse bus cycle behaviors of different processors and their varying data consistency algorithms, the memory pool must support a range of bus cycle behaviors.

[0021] A unified memory architecture that allows all processing units to access shared memory without requiring local dedicated memory requires physical memory address mapping. Each memory cell must be combined with swap memory to facilitate data transfer between various processing units, such as CPU-GPU interaction, CPU-NPU interaction, CPU-MPU interaction, MPU-GPU interaction, and NPU-GPU interaction. Figure 1Different memory buses for these tasks are described, including the APB bus (System Memory Mapped Control Register Bus) 112, the system non-cacheable memory bus (swap memory) 113, the host cacheable memory bus 114, the NPU dedicated memory bus 115, the GPU dedicated memory bus 116, and the media processor (e.g., a decoder / encoder for images, videos, or audio) dedicated memory bus 117.

[0022] An ideal memory architecture (referring to the interconnection of memory components within a System-on-a-Chip) should maximize concurrency, allowing all buses in the SoC to access memory simultaneously for both read and write operations. It also requires the memory system to support concurrent access from multiple buses. These characteristics collectively enable efficient data transfer and avoid bottlenecks in the memory system.

[0023] Figure 2 The physical address mapping 200 is shown, which maps the local virtual memory addresses used by the operating systems of processors 101, 102, 103, 104, and 105 to... Figure 1 The physical memory addresses within the distributed memory pool 106. The total memory size in this example configuration includes 32GB of DRAM, 1TB of NVRAM, and a configurable 384MB multi-port SRAM system distributed across four memory pools.

[0024] Physical address mapping 200 will Figure 1 A portion of the physical memory in the distributed memory pool 106 is allocated to host cacheable memory (optionally via...). Figure 1 The host can cache memory bus 114 access), memory-mapped register control (optionally via...) Figure 1 The system local memory is accessed via the APB bus 112, the system non-cacheable memory (optionally accessed via the system non-cacheable memory bus 113), the NPU local memory (optionally accessed via the NPU dedicated memory bus 115), the GPU local memory (optionally accessed via the GPU dedicated memory bus 116), and the media processor local memory (optionally accessed via the media processor dedicated memory bus 117).

[0025] The physical address mapping 200 can be adapted and utilized in various scenarios, including edge devices supporting inference through local training for generative AI applications. Memory configuration is scalable, allowing for adjustments to accommodate different memory requirements. The system can support several terabytes of DRAM, hundreds of terabytes of NVRAM, and tens of gigabytes of SRAM, making it suitable for large-scale generative AI training and inference on supercomputing servers. Memory space allocation can be further dynamically customized to optimize performance for specific target applications.

[0026] It is important to note that any data residing in the SRAM portion that needs to be reallocated should be flushed back to DRAM or NVRAM before customizing the memory space to ensure data consistency.

[0027] Figure 3 A configurable multi-port SRAM system 300 is shown, which can be used to integrate SRAMs into various ports. Figure 1 The memory modules 107, 108, 109, and 110 contain configurable multiport SRAM systems 119, 120, 121, and 122. A multiport SRAM system is a memory architecture that uses underlying single-port SRAM cells and additional circuitry for concurrency between multiple ports to implement multiport SRAM.

[0028] The configurable multi-port SRAM system 300 includes: (1) corresponding to Figure 2 The physical address mappings of the memory buses include: (1) a high-level peripheral bus (APB) 301, a system non-cacheable bus 302, a host cacheable memory bus 303, an NPU memory bus 304, a GPU memory bus 305, and a media processor bus 306; (2) a memory bus switch 307, which is capable of simultaneously routing multiple memory bus requests to n different SRAM blocks 308-1, 308-2, ..., 308-(n-1) and 308-n; (3) multiple memory pool buses, including a DRAM write bus 309, a DRAM read bus 310, an NVRAM write bus 311, and an NVRAM read bus 312; and (4) a memory pool bus switch 313, which routes memory refresh and updates from each SRAM block 308-1, 308-2, ..., 308-(n-1), 308-n to the appropriate memory pool bus 309, 310, 311, or 312.

[0029] SRAM blocks 308-1, 308-2, ..., 308-(n-1), and 308-n respectively include SRAM systems 314-1, 314-2, ..., 314-(n-1), and 314-n, each SRAM system comprising multiple physical SRAM cells. Each SRAM block has four bus ports supporting simultaneous memory write, memory read, memory refresh, and memory update operations.

[0030] SRAM blocks 308-1, 308-2, ..., 308-(n-1) consist of three functional components: (1) control blocks 315-1, 315-2, ..., 315-(n-1), 315-n are programmed respectively, each control block configures the SRAM block for various functions (such as cacheable or non-cacheable operations), and applies different caching algorithms to each independent SRAM block to optimize cache hit rate; (2) cache tag controllers 316-1, 316-2, ..., 316-(n-1), 316-n are used to manage various caching algorithms, cache line size, prefetching algorithms, etc. for each independent SRAM block in the SRAM system of each memory module; and (3) SRAM cells 314-1, 314-2, ..., 314-(n-1), 314-n are used for cacheable or non-cacheable functions respectively.

[0031] In this example, each SRAM block 308 has a size of 16MB. If n=6, SRAM blocks 308-1, 308-2, ..., 308-5 and 308-6 will have a total SRAM size of 96MB. Applying this configuration to configurable multiport SRAM systems 119, 120, 121, and 122 results in... Figure 1 The distributed memory pool 106 in the computing system 100 shown has 384 MB of SRAM.

[0032] Refer again Figure 3 SRAM blocks 308-1, 308-2, ..., 308-(n-1), and 308-n each include four ports: (1) memory write request ports 317-1, 317-2, ..., 317-(n-1), and 317-n; (2) memory read request ports 318-1, 318-2, ..., 318-(n-1), and 318-n; (3) memory refresh ports 319-1, 319-2, ..., 319-(n-1), and 319-n; and (4) memory update ports 320-1, 320-2, ..., 320-(n-1), and 320-n. These four ports operate simultaneously in a four-port configuration, allowing concurrent operations.

[0033] In some configurations, if SRAM block 308 is configured as non-cacheable memory, then SRAM block 308 will disable memory refresh port 319 and memory update port 320, meaning it will not be used to store cached data. SRAM blocks designed for swap memory purposes do not require memory refresh and memory update operations with the memory pool.

[0034] Defining an effective bus protocol for SRAM blocks 308-1, 308-2, ..., 308-(n-1), and 308-n is crucial for achieving high performance in terms of latency and bandwidth. This involves designing a protocol that enables the four-port SRAM blocks 308-1, 308-2, ..., 308-(n-1), and 308-n to interface effectively with the memory bus switch 307 (a switch that manages the data flow between the SRAM blocks and the memory bus) and the memory pool switch 313 (a switch that manages access to the memory pool). By optimizing communication between the memory bus switch 307, the memory pool switch 313, and the multiple SRAM blocks 308, the multi-port SRAM system 300 can be configured to deliver high performance, full-duplex read / write concurrent bandwidth with low latency.

[0035] Figure 4 A bus protocol 400 is illustrated, implemented using four-port SRAM blocks 308-1, 308-2, ..., 308-(n-1), and 308-n. In this example, each of the four ports has an external data width of 256 bits, while the internal data width of each SRAM block is 1024 bits. Each memory read or write cycle requires four 256-bit bursts in one of the ports. The four ports are responsible for memory write, read, refresh, and update cycles, respectively. These ports can operate concurrently to maximize performance and minimize memory conflicts.

[0036] Figure 5 A switching configuration 500 is depicted, wherein the switching configuration 500 comprises memory module 501 and memory module 502 (which are... Figure 1 The memory bus switches 503 and 504 (which are any two of the memory modules 107, 108, 109, and 110 in the distributed memory pool 106) in the distributed memory pool 106 are memory bus switches 503 and 504. Figure 3 The example of memory bus switch 307 in the example scenario couples various buses to various SRAM blocks. Different configurations are feasible for various distributed memory pools with different numbers of memory modules.

[0037] exist Figure 5In this configuration, memory modules 501 and 502 together contain a total of 12 SRAM blocks 308, each SRAM block 308 being 16MB in size. This configuration results in a total configurable SRAM of 192 MB. Each memory module may also have 16GB of DRAM (not shown) and 512GB of NVRAM (not shown). The total memory size visible to the OS kernel is 32GB of DRAM and 1TB of NVRAM. Optionally, the 1TB of NVRAM can be allocated to weights and bias coefficients of large AI models, which are static unless updated by new AI models.

[0038] exist Figure 5 In the example shown, each memory bus switch 503 and 504 can be based on... Figure 2 The physical address mapping is used to configure its SRAM block selection circuit 505 and SRAM block selection circuit 506 to target specific SRAM blocks for the desired application. This allows 12 SRAM blocks in two independent memory modules to work together efficiently, thereby optimizing the performance of the target application.

[0039] The following example illustrates how to dynamically configure SRAM blocks for a typical edge computing generative AI application: 1. Before the generative AI application is invoked to execute: • Block 0, Module 501: Non-cacheable frame buffer (16 MB) • Block 0, Module 502: Non-cacheable frame buffer (16 MB) • Block 1, Module 501: Non-cacheable CPU-GPU swap memory (16MB) • Block 1, Module 502: Non-cacheable CPU-NPU exchange memory (8MB), CPU-MPU exchange memory (8MB) • Block 2, Module 501: Cached GPU Local Memory (16 MB) • Block 2, Module 501: Cacheable MPU local memory (16 MB) • Blocks 3, 4, and 5, Module 501: Cacheable main system memory (48MB) • Blocks 3, 4, and 5, Module 502: Cacheable main system memory (48MB) When generative AI applications are not in the task queue, most of the SRAM can be allocated to the main system cache. 2. Once generative AI applications are in operation: • SRAM blocks 0, 1, and 2 in modules 501 and 502 remain unchanged. • SRAM blocks 3 and 4 in module 501 remain unchanged and are used for cacheable main system memory (32MB). • SRAM block 5 in module 501 is reconfigured for cacheable NPU local memory (16MB). • SRAM block 3 in module 502 remains unchanged and is used to cache main system memory (16 MB). • SRAM blocks 4 and 5 in module 502 are reconfigured for cacheable NPU local memory (32 MB). Depending on whether the DRAM is pre-allocated to AI applications within the OS kernel, it may be necessary to flush and clear block 5 in module 501 and blocks 4 and 5 in module 502 when they are reallocated for different purposes. If the DRAM is pre-allocated like GPU memory, then flushing is not required. Otherwise, the SRAM blocks must be flushed to write dirty data back to DRAM before reconfiguration.

[0040] Allocating SRAM blocks to the frame buffer offers several advantages. The frame buffer is constantly fetched and updated by the GPU and display controller. If DRAM were used for the frame buffer, a dual-buffer scheme (where two buffers alternately use them for reading and writing data) would be required to achieve concurrent read and write operations. However, high-speed DRAM I / O circuitry consumes a significant amount of power. Due to on-chip I / O, integrated four-port SRAM modules not only support concurrent read and write operations but also offer lower power consumption. This is particularly important in low-power, high-performance edge device applications.

[0041] Figure 6 A switching configuration 600 is depicted, wherein the switching configuration 600 comprises memory module 601 and memory module 602 (which are... Figure 1 In the distributed storage pool 106, any two of storage modules 107, 108, 109, and 110, there are storage pool routing switches 601 and 602 (which are... Figure 3 The memory pool switch 313 in the example couples various SRAM blocks to various DRAM and NVRAM cells respectively. Figure 1 Example scenario (part of the distributed storage pool 106 in the example).

[0042] The SRAM blocks in memory modules 601 and 602 will be refreshed to and updated from each heterogeneous memory pool, which is a combination of DRAM and NVRAM. The APB bus can program both the DRAM space and the NVRAM space so that each SRAM block will be refreshed or updated with the correct memory in the distributed memory pool.

[0043] It is important to note that if the SRAM block is programmed as non-cacheable memory, there will be no refresh or update requests from the SRAM block to the distributed storage pool.

[0044] Figure 7 The multi-port SRAM block 700 is shown. The multi-port SRAM block 700 is... Figure 3 The example implementations of SRAM blocks 308-1, 308-2, ..., 308-(n-1) and 308-n are shown in the figure. The multi-port SRAM block 700 includes an SRAM array 701, a configurable cache controller 702, a data input buffer 703, a data output buffer 704, an address decoder 705, and a memory write request port 706 (which is... Figure 3 Example implementations of 317-1, 317-2, ..., 317-(n-1), 317-n; memory read request port 708 (which is...) Figure 3 Example implementations of memory read request ports 318-1, 318-2, ..., 318-(n-1), 318-n; memory refresh port 709 (which is...) Figure 3 The memory refresh ports 319-1, 319-2, ..., 319-(n-1) and 319-n are example implementations, as are memory update ports 707 (which is an example implementation of memory update ports 320-1, 320-2, ..., 320-(n-1) and 320-n), multiplexer 710, data write buffer 711, demultiplexer 712, data read buffer 713, multiplexer 714, memory write request FIFO 716, memory read request FIFO 718, memory refresh FIFO 719 and memory update FIFO 717.

[0045] The SRAM array 701 has a 1024-bit data width for write and read operations. Each external port of the SRAM block has a 256-bit data width. Bus commands always execute a fixed burst length of 4 transfers. This saves a significant number of I / O pins to avoid potential signal routing problems. The internal SRAM array is a typical single-port SRAM to minimize silicon size.

[0046] The configurable cache controller 702 can use a time-based multiplexing scheme to select one of four memory operations (memory write, memory read, memory refresh, and memory update) to be performed on the SRAM array. This allows all four memory operations to access the SRAM array simultaneously. The configurable cache controller 702 provides the multiplexer 714 with memory write address, memory update address, memory read address, and memory refresh address, as well as control signals, to select the address output from the multiplexer 714 at any given time.

[0047] For memory write operations, the data associated with the memory write operation is received by memory write port 706 and then provided to multiplexer 710. Multiplexer 710 then provides the data to data write buffer 711, which writes the data to data input buffer 703 simultaneously with the memory write address being provided to address decoder 705 by multiplexer 714. Address decoder 705 then provides the address to SRAM array 701 and simultaneously writes the data from data input buffer 703 to SRAM array 701 at the address specified by that address.

[0048] For a memory update operation, the data associated with the memory update operation is received by the memory update port 707 and then provided to the multiplexer 710. The multiplexer 710 then provides the data to the data write buffer 711, which writes the data to the data input buffer 703 simultaneously with the memory update address being provided to the address decoder 705 by the multiplexer 714. The address decoder 705 then provides the address to the SRAM array 701 and simultaneously writes the data from the data input buffer 703 to the SRAM array 701 at the address specified by that address.

[0049] For a memory read operation, multiplexer 714 provides the memory read address to address decoder 705, which then applies the address to SRAM 701 during the read operation. The read data is output to a data output buffer, which provides this data to data read buffer 713. Data read buffer 713 then provides this data to demultiplexer 712, which in turn provides the data to memory read port 708.

[0050] For memory refresh operations, multiplexer 714 provides the memory refresh address to address decoder 705, which then applies the address to SRAM 701 during read operations. Read data is output to a data output buffer, which provides this data to data read buffer 713. Data read buffer 713 then provides this data to demultiplexer 712, which in turn provides it to memory refresh port 709.

[0051] Figure 8 Timing diagram 800 is depicted, illustrating how the configurable cache controller 702 generates timings to control... Figure 7The multi-port SRAM block 700 includes the SRAM array 701 and other blocks. The external bus of the SRAM blocks can operate simultaneously for write, read, refresh, and update operations. The cache controller can insert a wait state via an assertion ready signal when a cache miss occurs and an update or refresh operation is required. Considering the different memory access modes and requirements of the host processor or coprocessor, the replacement and prefetch algorithms for each SRAM block can be configured individually via the APB bus for caching by the host processor or coprocessor.

[0052] As shown in time steps t6 to t13, SRAM array 701 is accessed in each clock phase from phase 0 to phase 3. In phase 0 of t6, a 1024-bit memory write to SRAM array 701 occurs. In phase 1 of t7, a 1024-bit memory read from SRAM array 701 occurs. In phase 2 of t8, a 1024-bit memory update to SRAM array 701 occurs. In phase 3 of t9, a 1024-bit memory refresh from SRAM array 701 occurs. Each of the four ports is connected to a ping-pong FIFO (a buffer used for storing and retrieving data in a cyclic, first-in-first-out manner). The ping-pong FIFO can perform 256-bit to 1024-bit demultiplexing and 1024-bit to 256-bit multiplexing, such as... Figure 8 As shown. When there is a long burst of cache updates, such as Figure 8 As shown, the "rdqs_update" signal from t1 to t16 is asserted for a 256-bit long burst of 16 bits, and the cache controller 702 can use a dual ping-pong FIFO to update the cache. This illustrates long data transfers or prefetching that can be used to support NPU data modes. Therefore, the configurable cache controller 702 can be configured to support both short burst lengths for the main processor and long burst lengths for the coprocessor for different cache prefetching algorithms.

[0053] Figures 1 to 8 The operation and design of the embodiments described herein can be summarized as follows. 1. Figure 1 Each individual configurable multiport SRAM system 119, configurable multiport SRAM system 120, configurable multiport SRAM system 121, and configurable multiport SRAM system 122 (connected via interconnect PHY 111 in a chip separate from the on-chip system) within the distributed memory pool 106 shown can be composed of a memory bus switch 307, a memory pool switch 313, and multiple four-port SRAM blocks 308, as shown. Figure 3 As shown. 2. Figure 3The memory bus switch 307 shown can route all memory buses (including the non-cacheable memory bus 302, cacheable memory bus 303, NPU memory bus 304, GPU memory bus 305, and media processor memory bus 306) to Figure 3 The memory write port 317 and memory read port 318 of different SRAM blocks 308 within the SRAM system shown. 3. Figure 5 The SRAM block select circuits 505 and 506 shown can be programmed via APB (Advanced Peripheral Bus) to configure physical address mapping (in... Figure 2 (As shown in the diagram). In this way, the memory size and functionality of each individual SRAM system within a distributed memory pool can be optimized for the target application (such as cacheable memory, non-cacheable memory, swap memory, and local memory for coprocessors). 4. Figure 3 The memory pool switch 313 shown can route all memory refresh requests and memory update requests from each individual SRAM block 308 within the SRAM system to the DRAM system via the DRAM write bus 309 and the DRAM read bus 310, and to the NVRAM via the NVRAM write bus 311 and the NVRAM read bus 312. 5. Figure 6 The memory pool routing switch 603 and memory pool routing switch 604 shown can be programmed via APB (Advanced Peripheral Bus) to configure physical address mapping (in... Figure 2 (As shown in the diagram). This allows for optimization of the size and functionality of the memory pool (DRAM and NVRAM) for each individual within the distributed memory pool for the target application (such as cacheable memory, non-cacheable memory, swap memory, and local memory for coprocessors). 6. Figure 7 The multi-port SRAM block 700, shown as connecting to the memory bus switch and memory pool switch in the SRAM system, consists of the following parts: 1) an SRAM array 701 with a data bit width four times that of the external memory ports. 2) FIFOs 716, 717, 718, and 719 connected to each external memory port 706, 707, 708, and 709, respectively. FIFOs 716, 717, 718, and 719 can multiplex or demultiplex the SRAM array write data bus and read data bus. 3) a configurable cache controller 702 that can synchronize all memory operations from the four external memory ports 706, 707, 708, and 709, such as... Figure 8 The timing sequence is shown in the figure. 7. In Figure 4The bus protocol for a multi-port SRAM block 700 is illustrated. A request for a memory operation begins with a valid signal to each port. If a memory request requires an additional clock cycle to complete, the cache controller can insert a wait state by asserting a ready signal. A typical protocol handshake for memory operations is as follows: Figure 8 As shown. 8. If the SRAM blocks are configured for non-cacheable memory such as frame memory or swap memory, the configurable cache controller should be configured for non-cacheable operations. In non-cacheable mode, the configurable cache controller will only be used to synchronize concurrent memory read and write operations from external memory ports to the SRAM array. Memory refresh and memory update functions will be disabled. 9. If the SRAM block is configured for cacheable memory, it can be either host processor cache memory or coprocessor cache memory. For configurable cache controllers, the caching algorithm is configured differently. 10. If the SRAM block is configured to cache the main processor's system memory, the SRAM block's cache line will be close to the main processor's cache line, and it will have a less aggressive prefetch scheme with shorter burst lengths for memory refresh and memory update. Cache operations will be optimized for memory access latency. 11. If the SRAM block is configured to cache the local memory of a coprocessor (e.g., GPU or NPU), the cache line of the SRAM block will be increased to match the size of block transfers used by the coprocessor's local memory. The prefetch scheme will be aggressive to support long bursts of memory updates and refreshes. Cache operations will be optimized for memory bandwidth.

[0054] The examples of materials, processes, and numerical values ​​described above are merely illustrative and should not be construed as limiting the claims. As used herein, the terms "above" and "on" both include "directly on" (without intermediate materials, elements, or spaces) and "indirectly on" (with intermediate materials, elements, or spaces). Similarly, the term "adjacent" includes "directly adjacent" (without intermediate materials, elements, or spaces) and "indirectly adjacent" (with intermediate materials, elements, or spaces). For example, forming an element "on a substrate" can include forming the element directly on the substrate, where there is no intermediate material / element between the substrate and the element, and forming the element indirectly on the substrate, where there is one or more intermediate materials / elements between the substrate and the element.

Claims

1. A system comprising: A memory bus switch, which is coupled to multiple memory buses; A memory pool switch, the memory pool switch being coupled to one or more memory arrays; as well as One or more SRAM blocks, the one or more SRAM blocks being coupled to the memory bus switch and the memory pool switch, to convert a first transaction received on one of the plurality of memory buses into a second transaction executed on the one or more memory arrays.

2. The system of claim 1, wherein the memory bus switch routes transactions from the plurality of memory buses to the one or more SRAM blocks.

3. The system of claim 2, wherein the memory pool switch routes transactions from the one or more SRAM blocks to the one or more memory arrays.

4. The system of claim 1, wherein the plurality of memory buses includes two or more of an uncacheable bus, a cacheable bus, an NPU bus, a GPU bus, and a media processor bus.

5. The system of claim 1, wherein each of the one or more SRAM blocks comprises: Memory write port used for memory write transactions; The memory read port is used for memory read transactions; The memory refresh port is used for memory refresh transactions; as well as The memory update port is used for memory update transactions.

6. The system of claim 5, wherein each of the one or more SRAM blocks comprises: A memory write port FIFO coupled to the memory write port; A memory read port FIFO coupled to the memory read port; A memory refresh port FIFO coupled to the memory refresh port; as well as A memory update port FIFO coupled to the memory update port.

7. The system of claim 6, comprising: A cache controller is used to implement transactions received through the memory write port, the memory read port, the memory refresh port, and the memory update port.

8. The system of claim 1, wherein the one or more memory arrays include one or more DRAM arrays coupled to a DRAM controller.

9. The system of claim 8, wherein each of the one or more SRAM blocks includes a DRAM write queue and a DRAM read queue coupled to the DRAM controller.

10. The system of claim 1, wherein the one or more memory arrays include one or more NVRAM arrays coupled to an NVRAM controller.

11. The system of claim 10, wherein each of the one or more SRAM blocks includes an NVRAM write queue and an NVRAM read queue coupled to the NVRAM controller.

12. A method comprising: Transactions are routed from multiple memory buses to one or more SRAM blocks via a memory bus switch; as well as Transactions are routed from one or more SRAM blocks to one or more memory arrays via a memory pool switch.

13. The method of claim 12, wherein the plurality of memory buses includes two or more of an uncacheable bus, a cacheable bus, an NPU bus, a GPU bus, and a media processor bus.

14. The method of claim 12, wherein each of the one or more SRAM blocks comprises: Memory write port used for memory write transactions; The memory read port is used for memory read transactions; The memory refresh port is used for memory refresh transactions; as well as The memory update port is used for memory update transactions.

15. The method of claim 14, wherein each of the one or more SRAM blocks comprises: A memory write port FIFO coupled to the memory write port; A memory read port FIFO coupled to the memory read port; A memory refresh port FIFO coupled to the memory refresh port; as well as A memory update port FIFO coupled to the memory update port.

16. The method of claim 12, wherein the one or more memory arrays comprise one or more DRAM arrays coupled to a DRAM controller.

17. The method of claim 16, wherein each of the one or more SRAM blocks includes a DRAM write queue and a DRAM read queue coupled to the DRAM controller.

18. The method of claim 12, wherein the one or more memory arrays comprise one or more NVRAM arrays coupled to an NVRAM controller.

19. The method of claim 18, wherein each of the one or more SRAM blocks includes an NVRAM write queue and an NVRAM read queue coupled to the NVRAM controller.

20. The method of claim 14, wherein the memory write port, the memory read port, the memory refresh port, and the memory update port can receive transactions concurrently.

Images