Method and apparatus for using a storage system as a main memory

Abstract

A data access system includes a processor, a plurality of cache modules for main memory, and a storage drive. The cache module includes an FLC controller and a main memory cache. The plurality of cache modules are used as main memory. The processor sends a read / write request (with a physical address) to the cache module. The cache module includes two or more stages, each stage including an FLC controller and a DRAM (associated with the controller). If the first stage FLC module does not include a physical address, the request is forwarded to the second stage FLC module. If the second stage FLC module does not include a physical address, the request is forwarded to a partition reserved for the main memory by the storage drive. The first stage FLC module has a high-speed, low-power consumption operation, while the second stage FLC is a low-cost implementation. Multiple FLC modules can be connected to the processor in parallel.

Description

[0001] Priority claim

[0002] This application is a continuation of PCT / US2019 / 037787, filed on June 18, 2019, and claims priority to and benefits thereof. This application claims priority to and benefits of U.S. Provisional Patent Application No. 62 / 686,333, filed on June 18, 2018, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates to integrated circuits and computer systems, and more particularly to computer main memory. Background Art

[0004] Computing devices (servers, personal computers, mobile phones, tablets, etc.) typically include a processor or system on a chip (SoC). FIG. 1 shows an example of a device 10 that includes a processor or SoC 12 and a main memory made of one or more dynamic random access memories (DRAMs) 14. DRAM 14 can be implemented as one or more integrated circuits connected to SoC 12 but separate from SoC 12. Device 10 may also include one or more storage drives 16 connected to port 17 of SoC 12. Storage drive 16 may include flash memory, a solid-state drive, a hard disk drive, and / or a hybrid drive. Hybrid drives include solid-state drives with solid-state memory and hard disk drives with rotating storage media.

[0005] The SoC 12 may include one or more image processing devices 20, a system bus 22, and a memory controller 24. Each image processing device 20 may include, for example: a control module 26 having a central processor (or central processing unit (CPU)) 28; a graphics processor (or graphics processing unit (GPU)) 30; a video recorder 32; a camera image signal processor (ISP) 34; an Ethernet interface, such as a Gigabit (Gb) Ethernet interface 36; a serial interface, such as a Universal Serial Bus (USB) interface 38 and a Serial Advanced Technology Attachment (SATA) interface 40; and a Peripheral Component Interconnect Express (PCIe) interface 42. The image processing device 20 accesses the DRAM 14 via the system bus 22 and the memory controller 24. The DRAM 14 is used as a main memory. For example, when accessing a corresponding physical location in one of the DRAMs 14, one of the image processing devices 20 provides a physical address to the memory controller 24. The image processing device 20 may also access the storage drive 16 via the system bus 22.

[0006] The SoC 12 and / or the memory controller 24 may be connected to the DRAM 14 via one or more access ports 44 of the SoC 12. The DRAM 14 stores user data, system data, and / or programs. The SoC 12 may execute a program using first data to generate second data. The first data may be stored in the DRAM 14 before executing the program. The SoC 12 may store the second data in the DRAM 14 during and / or after executing the program. The DRAM 14 may have a high bandwidth interface and a low-cost per-bit memory storage capacity, and may handle a wide range of applications.

[0007] The SoC 12 includes a cache memory, which may include one or more of a level zero (L0) cache, a level one (L1) cache, a level two (L2) cache, or a level three (L3) cache. The L0-L3 caches are arranged on the SoC 12, adjacent to the corresponding image processing device in the image processing device 20. In the example shown, the control module 26 includes a central processor 28 and L1-L3 caches 50. The central processor 28 includes an L0 cache 52. The central processor 28 also includes a memory management unit (MMU) 54, which can control access to the caches 50, 52.

[0008] As the number of cache levels increases, the access latency and storage capacity of the cache also increase. For example, L1 cache typically has less storage capacity than L2 cache and L3 cache. However, L1 cache typically has lower latency than L2 cache and L3 cache.

[0009] The cache within the SoC 12 is typically implemented as static random access memory (SRAM). Since the cache is in close proximity to the image processing device 20, the cache can operate at the same clock frequency as the image processing device 20. Therefore, the cache exhibits a shorter latency period than the DRAM 14.

[0010] The number and size of caches in SoC 12 depends on the application. For example, an entry-level cell phone (or mobile phone) may not include an L3 cache, and may have a smaller L1 cache and L2 cache than a personal computer. Similarly, the number and size of each DRAM 14 depends on the application. For example, mobile phones currently have 4-12 gigabytes (GB) of DRAM, personal computers currently have 8-32GB of DRAM, and servers currently have 32GB-512GB of DRAM. In general, as the number of DRAM chips increases, the cost of large amounts of main memory increases.

[0011] In addition to the cost of DRAM, it is becoming increasingly difficult to reduce the package size of DRAM for the same amount of storage capacity. In addition, as the size and number of DRAMs included in a device increases, the capacitance of the DRAM increases, the number and / or length of conductive elements associated with the DRAM increases, and the buffering associated with the DRAM increases. In addition, as the capacitance of the DRAM increases, the operating frequency of the DRAM decreases and the latency period of the DRAM increases.

[0012] During operation, programs and / or data are transferred from DRAM 14 to the cache in SoC 12 as needed. These transfers have higher latency than data exchanges between (i) the cache and (ii) the corresponding processor and / or image processing device. Therefore, by building the SOC with a larger L3 cache, access to DRAM 14 can be minimized. Despite having larger and larger L3 caches, computing systems still require more and more DRAM (larger main memory) every year. All other things being equal, a computer with a larger main memory will have better performance than a computer with a smaller main memory. Under today's operating systems, a modern PC with 4GB of main memory will actually perform very poorly even if it is equipped with the fastest and best processor. Next, explain why the size of computer main memory continues to increase over time.

[0013] During system startup, programs may be transferred from storage drive 16 to DRAM 14. For example, central processor 28 may transfer programs from storage drive 16 to DRAM 14 during system startup. Only when the program is fully loaded into DRAM can central processor 28 execute the instructions stored in DRAM. If the CPU needs to run one program at a time, and the user is willing to wait for the CPU to terminate the previous program before starting a new program, the computer system does need a very small amount of main memory. However, this will be unacceptable to consumers who are now accustomed to immediate response times when starting a new program and switching between programs on the fly. This is why computers always need more DRAM every year, and this has established the priority of DRAM companies to manufacture larger DRAMs. Summary of the invention

[0014] In order to overcome the shortcomings of the prior art and provide additional benefits, a data storage and access system for use with a processor is disclosed. The processor is configured to generate a data request for data, and the data request may include a physical address. In this embodiment, an FLC cache system is used to virtualize a computer main memory so that the main memory can now be made of a much lower cost memory device such as flash memory, and a computer can be built using a DRAM with a much smaller capacity than previously possible. The FLC system includes a first FLC module having a first FLC controller and a first memory, so that the first FLC module receives a data request from the processor. The second FLC module includes a second FLC controller and a second memory. In response to the first FLC module not having data requested by the processor, the first FLC module receives a data request from the first FLC module. A storage drive is provided and configured with a user file storage portion / partition and a main memory portion / partition. When the second FLC module does not have data requested by the first FLC module, the storage drive receives a data request. By adopting a two-stage FLC system, the first FLC module is now free to primarily serve requests from the processor, while the second FLC module is free to move data into and out of the storage drive without affecting the performance of the first FLC module.

[0015] In one embodiment, the data request includes a physical address, and the first FLC controller includes a lookup table configured to convert the physical address to a first virtual address if the first FLC controller includes a physical address entry. If the lookup table of the first FLC controller does not include a physical address entry, the first FLC controller is configured to forward the data request with the physical address to the second FLC controller. The terms physical address and physical address entry can be used interchangeably in this article. The second FLC controller may include a lookup table configured to convert the physical address to a second virtual address. In addition, if the lookup table of the second FLC controller does not include a physical address, the second FLC controller is configured to forward the data request with the physical address to the storage drive.

[0016] In one embodiment, the first FLC module is optimized for significantly faster access time, lower latency and lower power consumption than the second FLC module, but at the cost of higher cost per bit basis than traditional DRAM main memory. However, because the first FLC module memory capacity is much smaller than the second FLC module, the cost increase is actually negligible, and more importantly, significant cost savings have been achieved by moving the main memory into the storage drive. In one configuration, the first memory is a customized DRAM and includes a first memory controller, and the second memory is made of readily available commodity DRAM and includes a second memory controller. The storage drive preferably uses a reasonable fast solid-state memory because the actual main memory is now physically in the storage drive. When using DRAM, any miss to the second FLC module will be a thousand times slower than traditional main memory access, and its impact must be minimized. This is why the FLC is fully associated and large cache line technology is proposed, and this is crucial to keep the miss rate negligible. The first FLC module and the second FLC module can be additionally configured to perform predictive acquisition of data stored at the address expected to be accessed in the future. In addition, the FLC module can also be configured to get hints (instructions) from the processor or software to perform prefetching of data from the final main memory now residing in the storage drive to the first FLC module and / or the second FLC module. In one embodiment, the software executed on the processing device can know what data (including code) is needed next, and can provide instructions to the FLC system to prefetch the data into the DRAM cache so that the required data exists when needed. It can be cached in a standard speed FLC or a high speed FLC. In one embodiment, thousands of cache lines are prefetched. Accurate prefetching can result in a 100% hit rate.

[0017] A method for operating a data access system is also disclosed herein. In this embodiment, the data access system includes a processor having a processor cache, a storage drive, a first FLC module including a first FLC controller and a first DRAM, and a second FLC module including a second FLC controller and a second DRAM. The method includes using a processor to generate a request for data, the request including a physical address, and providing the request for data to the first FLC module. Using the first FLC module, it is determined whether the first FLC controller includes a physical address entry, and in response to the first FLC controller including a physical address entry, data is retrieved from the first DRAM and provided to the processor. Alternatively, in response to the first FLC controller not including a physical address entry, the request for data and the physical address are forwarded to the second FLC module. At the second FLC module, it is determined whether the second FLC controller includes a physical address entry, and in response to the second FLC controller including a physical address entry, data is retrieved from the second DRAM (in the form of a complete cache line entry related to the requested data), and the requested cache line data (data stored in the cache line) is provided to the first FLC module.

[0018] The method may also include forwarding a request for data (in the form of a cache line corresponding to the requested data) and the physical address to the storage drive in response to the second FLC controller not containing the physical address. The method may also include the storage drive retrieving the data from a portion of the storage drive reserved as additional main memory and then providing the data (cache line) to the second FLC module. In addition, in response to the first FLC controller containing the physical address and providing the data to the processor, the FLC controller updates the status register to reflect the most recent use of the cache line containing the data.

[0019] In one embodiment, retrieving data from the second DRAM and providing the data to the first FLC module also includes storing the physical address in the first FLC controller and storing the data in the first DRAM. If the first DRAM does not have available space, the least recently used data is removed from the first DRAM and the first FLC address lookup entry table to make room for new data. In another embodiment, the FLC module can actively make a portion of DRAM available for future processor requests. The first FLC module can also actively make a portion of its DRAM clean so that the entries stored at the cache line to be cleaned can be erased when necessary to make room for new entries requested by the new processor. This is done by writing the dirty cache line back to the second FLC module. According to the prompt (prefetch request) from the operating system, the program / data that is no longer needed can be further erased from the FLC DRAM. This can save time and power. In addition, this also prevents accidental writing to the SSD whenever the program / data that is no longer used is cleared to the SSD. In one embodiment, the second stage DRAM (referred to as DRAM-SS in this article) is about eight times larger than the first stage DRAM. In one embodiment, a portion of the second stage DRAM is actively maintained as clean (empty) to make room for all or a portion of the first stage DRAM to be moved to the second stage DRAM. In one embodiment, at least ten percent of the second stage DRAM is maintained clean or cleaned on demand for prefetching data or for moving the first stage DRAM contents to the second stage. In one embodiment, at least half of the size of the first stage DRAM is maintained empty or cleanable in the second stage DRAM. In one embodiment, the second stage DRAM is at least four times larger than the first stage DRAM. In one embodiment, the second stage DRAM is at least eight times larger than the first stage DRAM. In one embodiment, the second stage DRAM is at least sixteen times larger than the first stage DRAM.

[0020] Also disclosed herein is a data storage and access system having several elements, including a processor having a processor cache. The processor is configured to generate a first data request for first data. Another part of the system is a first FLC cache system that communicates with the processor. The first FLC cache system is configured to serve as an ultra-fast main memory cache (faster than a second FLC cache system) with an extremely high hit rate (99.9%), receiving a first data request for first data. In one embodiment, the speed of the first FLC cache (HS) is at least twice that of the second FLC cache (SS). In one embodiment, the speed of the first FLC cache (HS) is at least four times that of the second FLC cache (SS). The first FLC cache system is also designed to track the thousands of independent threads that the processor will require. A second FLC cache system is provided, and it communicates with the first FLC, and is configured to serve as an auxiliary (backup) main memory cache of the first FLC, and receives a second data request for second data. The storage drive is configured with a user file storage partition and a main memory partition, so that the main memory partition of the storage drive is used as the actual main memory. Alternatively, a separate and dedicated storage drive can be used as the actual main memory. In addition, preferably, the primary memory partition of the storage drive is configured for high reliability. For Nand flash memory based storage drives, the primary memory partition is preferably configured with single level cells (SLC) rather than traditional triple or quad level cells (TLC / QLC).

[0021] The system may further include a system bus, wherein the processor communicates with multiple independent FLC cache module slices via the system bus. In one configuration, if the first data requested by the processor is not included in the first slice FLC cache system, the first data request is sent to the storage drive to retrieve the first data from the main memory partition of the storage drive, and if the different first data requested by the processor is not included in the second slice FLC cache system, the first data request is sent to the storage drive to retrieve the first data from the main memory partition of the storage drive. It is contemplated that the first slice FLC cache system may include a first-stage FLC module and a second-stage FLC module, and the second slice FLC cache system may include a first-stage FLC module and a second-stage FLC module. In addition, each FLC module should include a dedicated FLC controller, an address lookup table, and a memory.

[0022] Further areas of applicability of the present disclosure will become apparent from the detailed description, claims and drawings.The detailed description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure.

[0023] By examining the following drawings and detailed description, other systems, methods, features and advantages of the present invention will be or will become apparent to those skilled in the art. It is intended that all such additional systems, methods, features and advantages be included in this description, within the scope of the present invention, and protected by the accompanying claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The components in the drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the present invention. In the drawings, like reference numerals designate corresponding parts throughout the different views.

[0025] FIG. 1 is a functional block diagram of a device according to the prior art.

[0026] Figure 2 is a functional block diagram of a data access system according to an embodiment of the present disclosure.

[0027] Figure 3 It is a graphic Figure 2 Functional block diagram of the data access system's DRAM and storage driver entries.

[0028] Figure 4 Graphical operation Figure 2 Methods of accessing the data system.

[0029] Figure 5A is a block diagram of an example embodiment of a cascaded FLC system.

[0030] Figure 5B is a block diagram of an example embodiment of a FLC controller.

[0031] Figure 6 is a block diagram of a cascaded FLC module having two or more FLC modules.

[0032] Figure 7 Yes Figure 5A An operational flow chart of an example operating method of cascaded FLC modules is shown.

[0033] Figure 8 is a block diagram of a split FLC module system having two or more separate FLC modules.

[0034] Fig. 9 Yes Figure 8 An operational flow chart of an example method of operating a split FLC module is shown.

[0035] Fig.10 is an exemplary block diagram of an example embodiment of a cascaded FLC system with a bypass path.

[0036] Fig.11 Yes Figure 8An operational flow chart of an example method of operating a split FLC module is shown.

[0037] Fig.12 is an exemplary block diagram of an example embodiment of a cascaded FLC system having a bypass path and a non-cacheable data path.

[0038] Fig.13 Provided Fig.12 An operational flow chart of an exemplary operating method of an embodiment of the present invention.

[0039] In the drawings, reference numerals may be repeated to identify similar and / or identical elements. DETAILED DESCRIPTION

[0040] At least some of the following examples include a final level cache (FLC) module and a storage drive. The FLC module is used as a main memory cache, and the storage drive is used as a physical storage of user files, and a portion of the storage drive is also partitioned for use by the FLC module as an actual main memory. This is different from a conventional computer, in which the actual main memory is made of DRAM. Data first attempts to read or write from the DRAM of the FLC module to the DRAM of the FLC module, and the main memory portion of the physical storage drive provides a last resort backup in the case of a miss from the FLC module. The lookup table in the FLC module is referred to herein as a content addressable memory (CAM). The FLC controller of the FLC module uses various CAM technologies described below to control access to the memory in the FLC module and the storage drive. CAM technology and other disclosed features reduce the storage capacity required for DRAM in the device while maximizing the memory access rate and minimizing power consumption. The device may be a mobile computing device, a desktop computer, a server, a network device, or a wireless network device. Examples of devices include, but are not limited to, computers, mobile phones, tablet computers, cameras, etc. The DRAM in the following examples is typically not used as a main memory, but as a cache for a much slower main memory, which is now located in a portion of the storage drive. Therefore, the partition of the storage drive is the main memory, and the DRAM is the cache for the main memory.

[0041] Figure 2A data access system 70 is shown, which includes a processing device 72, a system bus 74, an FLC module 76, and a storage drive 78. The data access system 70 can be implemented, for example, in a computer, a mobile phone, a tablet, a server, and / or other devices. The processing device 72 may include, for example: a central processing unit (or central processing unit (CPU)); a graphics processor (or graphics processing unit (GPU)); a video recorder; a camera signal processor (ISP); an Ethernet interface, such as a gigabit (Gb) Ethernet interface; a serial interface, such as a universal serial bus (USB) interface and a serial advanced technology attachment (SATA) interface; and a peripheral component interconnect express (PCIe) interface; and / or other image processing devices. The processing device 72 may be implemented in one or more modules. As an example, the first of the processing modules 72 is shown as including a cache memory, such as one or more of a zero-level (L0) cache, a first-level (L1) cache, a second-level (L2) cache, or a third-level (L3) cache. In the example shown, the first processing device may include a central processing unit 73 and an L1-L3 cache 75. The central processing unit 73 may include an L0 cache 77. The CPU 73 also includes a memory management unit (MMU) 79, which can control access to the caches 75, 77. The MMU 79 can also be considered a memory address translator for the processor caches. The MMU is responsible for translating the CPU virtual addresses into system physical addresses. Most modern CPUs use physical address caches, which means that the L0 / L1 / L2 / L3 caches are physically addressed. Cache misses from the CPU are also sent to the system bus using physical addresses.

[0042] The tasks described below as being performed by a processing device may be performed by, for example, central processor 73 and / or MMU 79 .

[0043] The processing device 72 is connected to the FLC module 76 via the system bus 74. The processing device 72 is connected to the storage drive 78 via the bus and the interface (i) between the processing device 72 and the system bus 74 and the interface (ii) between the system bus 74 and the storage drive 78. The interface may include, for example, an Ethernet interface, a serial interface, a PCIe interface, and / or an embedded multimedia controller (eMMC) interface. The storage drive 78 may be located anywhere in the world away from the processing device 72 and / or the FLC controller 80. The storage drive 78 may communicate with the processing device 72 and / or the FLC controller 80 via one or more networks (e.g., WLAN, an Internet network, or a remote storage network (or cloud)).

[0044] The FLC module 76 includes an FLC controller 80, a DRAM controller 82, and a DRAM IC 84. The terms DRAM IC and DRAM can be used interchangeably. Although referred to as DRAM for the sake of understanding, other types of memory can be used, including any type of RAM, SRAM, DRAM, or any other memory that performs what is described herein but has a different name. The DRAM IC 84 is primarily used as virtual and temporary storage, while the storage drive 78 is used as physical and permanent storage. This means that the location in the DRAM IC does not have a static / fixed relationship with the physical address generated by the processor module in general. The storage drive 78 may include a partition reserved for use as main memory, while the rest of the storage drive is used as a traditional storage drive space to store user files. This is different from the demand paging operation of the prior art that will occur when the computer exceeds the physical main memory space in the DRAM. In this case, large blocks of data / programs from the DRAM will be transferred to and from the hard drive. This also requires the de-allocation and reallocation of physical address assignments, which are done by the MMU and the operating system, which is a slow process because the operating system (OS) has neither sufficient nor accurate information about the relative importance of the data / programs stored in the main memory. The processing device 72 addresses the main memory partitions of the DRAM IC 84 and the storage drive 78 as if they were a single main memory device. The user cannot access and cannot view the data or files stored in the main memory partition of the storage drive, just as the user cannot see the files stored in the RAM during computer operation. When reading and / or writing data, the processing device 72 sends an access request to the FLC controller 80. The FLC controller 80 accesses the DRAM IC 84 and / or accesses the storage drive 78 via the DRAM controller 82. The FLC controller 80 can access the storage drive directly (as shown by the dotted line) or via the system bus 74. From the perspective of the processor and the programmer, access to the storage partition dedicated to the main memory is completed by the processor native load and store operations, rather than as an I / O operation.

[0045] Various examples of the data access system 70 are described herein. In a first example, the FLC module 76 is implemented in a SoC separate from the processing device 72, the system bus 74, and the storage drive 78. In another embodiment, these elements are on different integrated circuits. In a second example, one of the processing devices 72 is a CPU-implemented processing device. One of the processing devices 72 can be implemented in a SoC separate from the FLC module 76 and the storage drive 78. As another example, the processing device 72 and the system bus 74 are implemented in a SoC separate from the FLC module 76 and the storage drive 78. In another example, the processing device 72, the system bus 74, and the FLC module 76 are implemented in a SoC separate from the storage drive 78. Other examples of the data access system 70 are disclosed below.

[0046] DRAM IC 84 may be used as the final level of cache. DRAM IC 84 may have various storage capacities. For example, DRAM IC 84 may have a storage capacity of 1-2 GB for mobile phone applications, a storage capacity of 4-8 GB for personal computer applications, and a storage capacity of 16-64 GB for server applications.

[0047] The storage driver 78 may include a NAND flash SSD or other non-volatile memory, such as a resistive RAM and a phase change memory. The storage driver 78 may have a larger storage capacity than the DRAM IC 84. For example, the storage driver 78 may include 8-16 times more storage than the DRAM IC 84. If low task latency switching time is important, the DRAM IC 84 may include a high-speed DRAM, and even in the future, the storage driver 78 may be made of ultra-low-cost and low-speed DRAM. Ultimately, a new class of high-capacity serial / sequential large-page DRAM (random access capability is limited) may be built for final main memory. Such a serial DRAM device may be at least twice more cost-effective than a conventional DRAM because the die size may be at least twice smaller than a conventional DRAM. In one embodiment, the serial DRAM will have a minimum block (chunk) size that can be retrieved or written at one time, such as a cache line (4KB), but in another embodiment, a minimum block size may be established. Therefore, data is not read or written anywhere, but instead only goes to / from a certain block. In addition, such serial DRAM can be packaged with an ultra-high-speed serial interface to enable high-capacity DRAM to be installed away from the processor device, which will enable the processor to run at its full potential without worrying about overheating. As shown, a portion of the storage drive 78 is partitioned to be used as main memory, and is therefore used by the FLC controller 80 as an extension of the FLC DRAM 84.

[0048] The cache lines stored in the DRAM IC 84 may be the most recently accessed data and / or the data with the highest associated priority. The cache lines stored in the DRAM IC 84 may include cache lines that are locked in. The locked cache lines refer to data that are always kept in the DRAM IC 84. Even if the locked cache lines have not been accessed for a long period of time, the locked cache lines cannot be kicked out by other cache lines. However, the locked cache lines can be updated (written). In one embodiment, defective DRAM cells (and their corresponding cache lines) can be locked out (locked out) (mapped out) from the FLC system by removing the DRAM address entries with (one or more) defective cells to prevent the FLC address lookup engine from assigning cache line entries to the defective DRAM location. Defective DRAM entries are usually found during device manufacturing. However, in another embodiment, the operating system can use the map-out function to place a portion of the DRAM in a temporary state where it cannot be used by the processor for normal operation. Such a function allows the operating system to issue commands to check the health of the mapped DRAM segments one section at a time when the system runs actual applications. If a segment of DRAM is found to have a weak cell, the operating system can then proactively disable the cache line containing the weak cell(s) and take the so-called "weak cache line" out of service. In one embodiment, the FLC engine can include hardware diagnostic functionality to relieve the processor from the burden of performing DRAM diagnostics in software.

[0049] In some example embodiments, the data stored in the DRAM IC 84 does not include software applications, fonts, software codes, alternative codes, and data supporting different spoken languages ​​that are not frequently used (e.g., accessed more than a predetermined number of times within a predetermined time period). This can help minimize the size requirements of the DRAM IC 84. With respect to the FLC, software codes that are rarely used or not used at all can be regarded as "junk code", and they may not be loaded by the FLC in the system startup process, and if they are loaded and only used once, they are cleared by the FLC and will never be loaded again in the future, thus freeing up space in the DRAM IC 84 for truly useful data / programs. As the size of the DRAM IC 84 decreases, DRAM performance increases, and power consumption, capacitance, and buffering decrease. As capacitance and buffering decrease, waiting time decreases. Moreover, by consuming less power, the battery life of the corresponding device increases. Of course, with a larger DRAM IC 84, the performance of the entire system will increase, but this is at the expense of increased cost and power.

[0050] The FLC controller 80 performs a CAM technique in response to receiving a request from the processing device 72. The CAM technique includes converting a first physical address of a request provided by the processing device 72 into a virtual address. These virtual addresses are independent of and different from the virtual addresses originally generated by the processing device 72 and mapped to the first physical address by the processing device 72. The DRAM controller 82 converts (or maps) the virtual address generated by the FLC controller 80 into a DRAM address. If the DRAM address is not in the DRAM IC 84, the FLC controller 80 can (i) obtain data from the storage drive 78, or (ii) can indicate (or signal) a corresponding one of the processing devices 72 that a cache miss has occurred. Obtaining data from the storage drive 78 can include mapping the first physical address received by the FLC controller 80 to a second physical address to access the data in the storage drive 78. When converting the physical address to the virtual address, the FLC controller 80 can detect a cache miss.

[0051] The FLC controller 80 can then signal one of the processing devices 72 that had a cache miss when accessing the storage drive 78 for the data. This can include accessing the data in the storage drive 78 based on the first (original) physical address by mapping the first / original physical address to the storage address and then accessing the storage drive 78 based on the mapped storage address.

[0052] CAM technology is used to map the first physical address to the virtual address in the FLC controller. CAM technology provides a fully associative address conversion. This can include a logical comparison of the processor physical address with all virtual address entries stored in the directory of the FLC controller 80. Set associative address conversion should be avoided because it will lead to a higher miss rate, thereby reducing processor performance. After the initial system startup, the hit rate of the data in the DRAMIC 84 with a fully associative and large cache line architecture (FLC) may be as high as 99.9%, depending on the size of the DRAM IC 84. Typically, the size of the DRAM IC84 should ensure a medium-term (several minutes) average hit rate close to 100% under the minimum idle time of the processor and / or processing device. For example, this can be achieved using 1-2GB DRAM ICs for mobile phone applications, 4-8GB DRAM ICs for personal computer applications, and 16-64GB DRAM ICs for server applications.

[0053] Figure 3 DRAM IC 84 and storage drive 78 entries of data access system 70 are shown. DRAM IC 84 may include DRAM entries 00-XY The storage drive 78 may have a drive entry 00-MNEach DRAM entry 00-XY The address can be mapped to the drive entry 00-MN However, because the size of the DRAM is smaller than the size of the storage device, only a small portion of the storage device can be mapped to a DRAM entry at any given time. If non-conflicting lookup-free processing is used instead of true CAM processing, a portion of the DRAM can also be used for non-cacheable data and to store the complete address lookup table of the FLC controller. 00-XY The data in can include other metadata.

[0054] Each DRAM entry 00-XY It may have a storage capacity of, for example, 4KB. Each drive entry 00-MN It can also have a storage granularity of 4KB. If you want to 00-XY One of the DRAM entries reads data from or writes data to it, and the DRAM entry 00-XY If one of the drive entries is full and / or does not have all the data associated with the request, the drive entry 00-MN The corresponding one is accessed. Therefore, DRAM IC 84 and storage drive 78 are divided into 4KB memory blocks. Each memory block in DRAM IC 84 can have a corresponding one or more memory blocks in storage drive 78. This mapping and division of memory is Figure 2 The processing device 72 may be transparent.

[0055] During operation, one of the processing devices 72 may generate a request signal for a data block (or a portion of the data block). If the data block is not located in the DRAM IC 84, the FLC controller 80 may access the data block in the storage drive 78. When the FLC controller 80 is accessing data from the storage drive 78, the FLC controller 80 may send an alarm signal (such as a bus error signal) back to the processing device requesting the data. The alarm signal may indicate that the FLC controller 80 is in the process of accessing data from a slow storage device, and therefore the system bus 74 is not ready to transfer data to the processing device 72 within a period of time. If a bus error signal is used, the transmission of the bus error signal may be referred to as a "bus abort" from the FLC module 76 to the processing device and / or the SoC of the processing device 72. The processing device 72 may then perform other tasks while waiting for the FLC storage transaction to be ready. Other processor tasks may then proceed by using data already stored in one or more caches (e.g., L0-L3 caches) in, for example, the SoC of the processing device and other data already stored in the FLC DRAM. This also minimizes the idle time of the processor and / or the processing device.

[0056] If sequential access is performed, the FLC controller 80 and / or the DRAM controller 82 can perform predictive acquisition of data stored at an address that is expected to be accessed in the future. This may occur during system startup and / or after system startup. The FLC controller 80 and / or the DRAM controller 82 can: track data and / or software usage; and evaluate the code line that is about to be executed; track memory access patterns; and predict the next address of the data expected to be accessed based on this information. The next address can be the address of the DRAM IC 84 and / or the storage drive 78. As an example, the FLC controller 80 and / or the DRAM controller 82, which is independent of and / or has not previously received a data request, can access the data stored in the storage drive 78 and transfer the data to the DRAM IC 84.

[0057] The above examples can be implemented via servers in a network (which can be referred to as a "cloud"). Each server can include an FLC module (e.g., FLC module 76) and communicate with each other. The servers can share DRAM and / or memory stored in DRAM ICs and storage drives. Each server can access DRAM and / or storage drives in other servers via the network. Each FLC module can be similar to Figure 2The FLC module of one of the servers can be operated, and the DRAM and / or memory in each other server can also be accessed via the cloud. The signal transmitted between the server and the cloud can be encrypted before transmission and decrypted when it reaches the network device of the server and / or the cloud. The server can also share and / or access the memory in the cloud. As an example, the virtual address generated by the FLC controller of one of the servers can correspond to the following physical addresses: the DRAM of the FLC module of the FLC controller; the storage drive of one of the servers; the DRAM of the FLC module of one of the other servers; the storage drive of one of the other servers; or the storage device of the cloud. If a cache miss occurs, the FLC controller and / or processing device of one of the servers can access the DRAM and / or memory in other FLC modules, storage drives and / or storage devices. In short, the storage device can be accessible in the cloud or network. If a storage drive located in the cloud is utilized, this reduces the size and cost of the computing device, and therefore the computing device does not need a storage drive. Although making the storage drive accessible in the cloud or network may be slower than making the storage drive co-located with the DRAM cache and processor, this allows the storage drive to be shared between several different processing devices and DRAM caches. In one example environment, a car may have numerous processors arranged around the vehicle, and each processor may be configured with a DRAM cache system. Instead of each processor also having an SSD drive, a single SSD drive may be shared between all processing devices. In the case of very high hit rates disclosed herein, the SSD drive is rarely accessed. Such an arrangement has the benefits of lower cost, small overall size, and ease of maintenance.

[0058] The above example can also be implemented in a data access system, which includes: a multi-chip module having multiple chips; a switch; and a main chip having a main FLC module. The multi-chip module is connected to the main chip module via the switch. Each FLC module can be similar to Figure 2 The FLC module of one of the chips is operated, but the DRAM and / or memory in each other chip can also be accessed via the switch. As an example, the virtual address generated by the FLC controller of one of the chips can correspond to the following physical addresses: the DRAM of the FLC module of the FLC controller; the storage drive of one of the chips; the DRAM of the FLC module of one of the other chips; the storage drive of one of the other chips; or the storage device of the cloud. If a cache miss occurs, the FLC controller and / or processing device of one of the chips can access the DRAM and / or memory in the other FLC modules, storage drives, and / or storage devices.

[0059] As an example, each auxiliary DRAM in a multi-chip module and the main DRAM in the main chip can have a storage capacity of 1GB. The storage drive in the main chip can have a storage capacity of, for example, 64GB. As another example, a data access system can be used in a car. The main chip can be, for example, a central controller, a module, a processor, an engine control module, a transmission control module, and / or a hybrid control module. The main chip can be used to control corresponding aspects of related systems, such as throttle position, spark timing, fuel timing, conversion between transmission gears, etc. The auxiliary chips in the multi-chip module can each be associated with a specific vehicle system, such as a lighting system, an entertainment system, an air conditioning system, an exhaust system, a navigation system, an audio system, a video system, a braking system, a steering system, etc., and used to control aspects of the corresponding system.

[0060] As another example, the above example can also be implemented in a data access system including a host (or SoC) and a hybrid drive. The host may include a central processor or other processing device and communicate with the hybrid drive via an interface. The interface may be, for example, a GE interface, a USB interface, a SATA interface, a PCIe interface, or other suitable interfaces. The hybrid drive includes a first storage drive and a second storage drive. The first storage drive includes an FLC module (e.g., Figure 2 The FLC controller of the FLC module performs a CAM technique when determining whether to read data from and / or write data to the DRAM and the second storage drive of the FLC module.

[0061] As another example, the above example can also be implemented in a storage system including an SoC, a first high-speed DRAM cache (faster than a second DRAM cache), a second larger DRAM cache (larger than the first DRAM cache), and a non-volatile memory (storage drive). The SoC is separated from the first DRAM, the second DRAM, and the non-volatile memory. The first DRAM can store high priority and / or frequently accessed data. A high percentage of data access requests can point to data stored in the first DRAM. As an example, 99% or more of the data access requests can point to data stored in the first DRAM, and the remaining 0.9% or less of the data access requests can point to data stored in the second DRAM, and less than 0.1% of the data points to the non-volatile memory (the main memory partition in the storage drive). Low priority and / or less frequently accessed data can be stored in the second DRAM and / or the non-volatile memory. For example, a user can open multiple web browsers stored in the first DRAM (high-speed DRAM). On the other hand, the second DRAM has a higher capacity to store a large number of idle applications (such as idle web browser tabs) or applications with low duty cycle operation. Therefore, the second DRAM should be optimized for low cost by using standard commodity DRAM, and therefore, it will only have standard DRAM performance, and will also present a longer waiting time than the first DRAM. Then, the content of the really old application that is not suitable for the second DRAM will be stored in the non-volatile memory. Moreover, before deep sleep, only the dirty cache line content of the first and / or second DRAM can be written to the non-volatile memory. When waking up from deep sleep, only the content needed immediately will be brought back to the second and first FLCDRAM cache. As a result, the wake-up time from deep sleep may be several orders of magnitude faster than the computer using the traditional DRAM main memory solution.

[0062] The SoC may include one or more control modules, interface modules, cache (or FLC) modules, and graphics modules. The cache module may be similar to Figure 2 The control module is operated by the FLC module of the SoC. The control module is connected to the cache module via the interface module. The cache module is configured to access the first DRAM, the second DRAM and the non-volatile memory based on the corresponding hierarchical level. Each control module may include corresponding L1, L2 and L3 caches. Each control module may also include one or more additional caches, such as an L4 cache or other higher-level caches. There may be many signal lines (or conductive elements) between the SoC and the first DRAM. This allows fast parallel and / or serial transmission of data between the SoC and the first DRAM. The data transmission between the SoC and the first DRAM is faster than the data transmission between (i) the SoC and the second DRAM and (ii) the SoC and the non-volatile memory.

[0063] The first DRAM may have a first portion having the same or higher hierarchy level as an L3 cache, an L4 cache, and / or a highest level cache. A second portion of the first DRAM may have the same or lower hierarchy level as a second DRAM and / or a non-volatile memory. The second DRAM may have a higher hierarchy level than the first DRAM. The non-volatile memory may have the same or higher hierarchy level as the second DRAM. The control module may change the hierarchy level of part or all of each of the first DRAM, the second DRAM, and / or the non-volatile memory based on, for example, cache needs.

[0064] The control module, the graphics module connected to the interface module, and / or other devices connected to the interface module (internal or external to the SoC) can send a request signal to the cache module to store and / or access data in the first DRAM, the second DRAM, and / or the non-volatile memory. The cache module can control access to the first DRAM, the second DRAM, and the non-volatile memory. As an example, the control module, the graphics module, and / or other devices connected to the interface module may not know the number and / or size of the DRAMs connected to the SoC.

[0065] The cache module can convert the first processor physical address and / or request received from the control module, the graphics module and / or other devices connected to the interface module into the virtual address of the first DRAM and the second DRAM and / or the storage address of the non-volatile memory. The cache module can store one or more lookup tables (e.g., complete set associative lookup tables) for converting the first processor physical address into the virtual address of the first and second DRAMs and / or converting the first processor physical address into a storage address. As a result, the cache module and one or more of the first DRAM, the second DRAM, and the non-volatile memory (main memory partition of the storage drive) can operate as a single memory (main memory) relative to the control module, the graphics module and / or other devices connected to the interface module. The graphics module can control the output of video data from the control module and / or the SoC to the display and / or another video device.

[0066] The control module may exchange (or transmit) data, data sets, programs and / or portions thereof between (i) a cache module and (ii) an L1 cache, an L2 cache and an L3 cache. The cache module may exchange (or transmit) data, data sets, programs and / or portions thereof between two or more of a first DRAM, a second DRAM and a non-volatile memory. This may be performed independently of the control module and / or performed without receiving a control signal from the control module to perform the transmission. The storage location of the data, data sets, programs and / or portions thereof in one or more of the first DRAM, the second DRAM and the non-volatile memory may be based on corresponding priorities, usage frequencies, access frequencies, and / or other parameters associated with the data, data sets, programs and / or portions thereof. The transmission of data, data sets, programs and / or portions thereof may include transmission data blocks. Each data block may have a predetermined size. As an example, the data exchange from the second DRAM to the first DRAM may include multiple transmission events, wherein each transmission event includes a transmission data block (e.g., 4KB data).

[0067] For best performance, the cache modules of the first DRAM must be fully associative with large cache line sizes (FLC cache solution). However, for applications that can tolerate higher miss rates, a set associative architecture can be used for the first-level DRAM cache only instead. But even then, it still has a large cache line size to reduce the number of cache controller entry tables. As for the second-level DRAM cache, a fully associative and large cache line cache is used, because anything else may shorten the life of the non-volatile main memory.

[0068] The first DRAM may have a first predetermined amount of storage capacity (e.g., 0.25 GB, 0.5 GB, 1 GB, 4 GB, or 8 GB). The 0.5 GB first DRAM is 512 times larger than a typical L2 cache. The second DRAM may have a second predetermined amount of storage capacity (e.g., 2-8 GB or more for a non-server-based system, or 16-64 GB or more for a server-based system). The non-volatile memory may have a third predetermined amount of storage capacity (e.g., 16-256 GB or more). The non-volatile memory may include solid-state memory, such as flash memory or magnetoresistive random access memory (MRAM) and / or rotating magnetic media. The non-volatile memory may include an SSD and a HDD. Although the storage system has a second DRAM and a non-volatile memory (a main memory partition of the storage drive), any one of the second DRAM and the non-volatile memory may not be included in the storage system.

[0069] As another example, the above example can also be implemented in a storage system including a SoC and a DRAM IC. The SoC may include multiple control modules (or processors) that access the DRAM IC via a ring bus. The ring bus may be a bidirectional bus that minimizes access latency. If cost is more important than performance, the ring bus may be a unidirectional bus. An intermediate device may be located between the control module and the ring bus and / or between the ring bus and the DRAM IC. For example, the above cache module may be located between the control module and the ring bus or between the ring bus and the DRAM IC.

[0070] The control modules may share DRAM ICs and / or have designated portions of DRAM ICs. For example, a first portion of a DRAM IC may be allocated as a cache for a first control module. A second portion of a DRAM IC may be allocated as a cache for a second control module. A third portion of a DRAM IC may be allocated as a cache for a third control module. A fourth portion of a DRAM IC may not be allocated as a cache.

[0071] As another example, the above example can also be implemented in a server system. The server system can be referred to as a storage system and includes multiple servers. The server includes corresponding storage systems that communicate with each other via a network (or cloud). One or more storage systems can be located in the cloud. Each storage system can include a corresponding SoC.

[0072] The SoC may have a corresponding first DRAM, a second DRAM, a solid-state non-volatile memory, a non-volatile memory, and an I / O port. The I / O port may communicate with the cloud via a corresponding I / O channel (such as a peripheral component interconnect express (PCIe) channel) and a corresponding network interface (such as a peripheral component interconnect express (PCIe) channel). The I / O port, the I / O channel, and the network interface may be an Ethernet port, a channel, and a network interface, and transmit data at a predetermined speed (e.g., 1 Gigabit per second (Gb / s), 10Gb / s, 50Gb / s, etc.). Some network interfaces may be located in the cloud. The connection of multiple storage systems provides a low-cost, distributed, and scalable server system. Multiple disclosed storage systems and / or server systems may communicate with each other and be included in a network (or cloud).

[0073] Solid-state non-volatile memories may each include, for example, NAND flash memory and / or other solid-state memory. Non-volatile memories may each include solid-state memory and / or rotating magnetic media. Non-volatile memories may each include SSDs and HDDs.

[0074] The architecture of the server system provides DRAM as a cache. DRAM can be allocated as the L4 and / or highest level cache of the corresponding SoC, and has high bandwidth and large storage capacity. The stacked DRAM can include, for example, DDR3 memory, DDR4 memory, low power double data rate type four (LPDDR4) memory, wide I / O2 memory, HMC memory and / or other suitable DRAM. Each SoC can have one or more control modules. The control module communicates with the corresponding DRAM via a corresponding ring bus. The ring bus can be a bidirectional bus. This provides high bandwidth and minimum waiting time between the control module and the corresponding DRAM.

[0075] Each control module can access data and / or programs stored in: a control module in the same or a different SoC; in any DRAM; in any solid-state non-volatile memory; and / or in any non-volatile memory.

[0076] The SoC and / or the port of the SoC may have a media access controller (MAC) address. The control module (or processor) of the SoC may have a corresponding processor cluster address. Each control module may access other control modules in the same SoC or another SoC using the corresponding MAC address and processor cluster address. Each control module of the SoC may access the DRAM. The control module of the first SoC may request data and / or programs stored in the DRAM connected to the second SoC by sending a request signal having the MAC address of the second SoC and the processor cluster address of the second control module in the second SoC.

[0077] Each SoC and / or each control module in the SoC may store one or more address translation tables. The address translation table may include and / or provide translations for: the MAC address of the SoC; the processor cluster address of the control module; the processor physical address of the memory cells in the DRAM, solid-state non-volatile memory, and non-volatile memory; and / or the physical block address of the memory cells in the DRAM, solid-state non-volatile memory, and non-volatile memory. In one embodiment, the DRAM controller generates DRAM row and column address bits that form a virtual address.

[0078] As an example, data and programs can be stored in solid-state non-volatile memory and / or non-volatile memory. Data and programs and / or parts thereof can be distributed to SoC and control modules via a network. Programs and / or data required for execution by the control module can be stored locally in the DRAM, solid-state non-volatile memory and / or non-volatile memory of the SoC where the control module is located. The control module can then access the programs and / or data required for execution from the DRAM, solid-state non-volatile memory and / or non-volatile memory and transfer them to a cache in the control module. Communication between the SoC and the network and / or between SoCs can include wireless communication.

[0079] As another example, the above example can also be implemented in a server system including an SoC. Some SoCs can be incorporated into corresponding servers and can be referred to as server SoCs. Some SoCs (referred to as companion SoCs) can be incorporated into the server of the first SoC or can be separated from the server of the first SoC. The server SoC includes corresponding: a control module (e.g., a central processing module) cluster; an intra-cluster ring bus, an FLC module, a memory control module, an FLC ring bus, and one or more jump buses. The jump bus (i) extends between the server SoC and the companion SoC via an inter-chip bus member and a corresponding port, and (ii) extends through the companion SoC. The jump bus can refer to a bus extending to a jump bus station, an adapter or a node, and corresponding ports of one or more SoCs and extending therefrom. The jump bus can extend through a jump bus station and / or one or more SoCs. A single data transmission to or from a jump bus station can be referred to as a single jump. When transmitting data between a sending device and a receiving device, multiple jumps can be performed. Data can propagate between bus stations at each clock cycle until the data reaches the destination. Each bus station disclosed herein can be implemented as a module and includes logic for transmitting data between devices based on a clock signal. Furthermore, each bus disclosed herein may have any number of lanes for serial and / or parallel transmission of data.

[0080] Each control module cluster has one of the corresponding intra-cluster ring buses. The intra-cluster ring bus is bidirectional and provides communication between the control modules in each cluster. The intra-cluster ring bus may have a ring bus station for the control module to access the data signal transmitted on the intra-cluster ring bus. The ring bus station may be used as a signal repeater and / or access node. The control module may be connected to the intra-cluster ring bus via the ring bus station and access the intra-cluster ring bus. Data may be transmitted around the intra-cluster ring bus from a first control module at a first ring bus station in the ring bus station to a second control module at a second ring bus station in the ring bus station. Each control module may be a central processing unit or a processor.

[0081] Each memory control module can control access to one of the corresponding FLC modules. The FLC modules can be stacked on the server SoC. Each FLC module includes FLC (or DRAM), and can be implemented as any FLC module disclosed herein and any FLC module disclosed herein to operate. The memory control module can access the FLC ring bus at the corresponding ring bus station on the FLC ring bus, and transmit data between the ring bus station and the FLC module. Alternatively, the FLC module can directly access the FLC ring bus at the corresponding ring bus station. Each memory control module may include a memory clock, which generates a memory clock signal for one of the corresponding FLC modules and / or a bus station for a ring bus and / or a jump bus. The bus station can receive a memory clock signal indirectly via the ring bus and / or the jump bus or directly from the memory control module. Data can be circulated through the bus station based on the memory clock signal.

[0082] The FLC ring bus can be a bidirectional bus and has two types of ring bus stations SRB and SRH. Each ring bus station can be used as a signal repeater and / or an access node. The ring bus station SRB is connected to devices other than the jump bus. The device may include: an inter-cluster ring bus 0; an FLC module and / or a memory control module; and a graphics processing module. The inter-cluster ring bus provides (i) connections between clusters, and (ii) connections between cross-ring stations. The cross-ring bus station provides access to the inter-cluster ring bus and can connect the inter-cluster ring bus to a ring bus extension extending between (i) the cluster and (ii) the ring bus station. The ring bus station is located on the FLC ring bus. The inter-cluster ring bus and the cross-ring bus station provide (iii) connections between the first cluster and the ring bus station of the second FLC ring bus, and (iv) connections between the second cluster and the ring bus station of the first FLC ring bus. This allows the control module to access the FLC of the second FLC module, and the control module to access the FLC of the first FLC module.

[0083] The inter-cluster ring bus may include intra-chip traces and inter-chip traces. The intra-chip traces extend inside the server SoC and between (i) one of the ring bus stations and (ii) one of the ports. The inter-chip traces extend outside the server SoC and between corresponding port pairs.

[0084] The ring bus station SRH of each server SoC is connected to the corresponding FLC ring bus and one of the jump buses. Each jump bus has multiple jump bus stations SHB, which provide access to one of the corresponding jump buses for the corresponding interface. The jump bus station SHB can be used as a signal repeater and / or access node.

[0085] The first jump bus, the ring bus station and the first jump bus station provide connections between (i) the FLC ring bus and (ii) the liquid crystal display (LCD) interface in the server SoC and the interface of the collaborative SoC. The LCD interface can be connected to the display and can be controlled via the GPM. The interface of the collaborative SoC includes a serially attached small computer system interface (SAS) interface and a PCIe interface. The interface of the collaborative SoC can be an image processor (IP) interface.

[0086] The interface is connected to a corresponding port, and the port can be connected to a device (such as a peripheral device). The SAS interface and the PCIe interface can be connected to a SAS compatible device and a PCIe compatible device respectively via a port. For example, a storage drive can be connected to a port. The storage drive can be a hard disk drive, a solid state drive, or a hybrid drive. The port can be connected to an image processing device. An example of an image processing device is disclosed above. The fourth SoC can be daisy-chained to the third SoC via an inter-chip bus member (also referred to as a daisy chain member). The inter-chip bus member is a member of the first jump bus. The additional SoC can be daisy-chained to the fourth SoC via a port, which is connected to the first jump bus. The server SoC, the control module, and the FLC module can communicate with the fourth SoC via the FLC ring bus, the first jump bus, and / or the third SoC. As an example, the SoC can be a south bridge chip and controls the communication and interrupted transmission between (i) the server SoC and (ii) the peripheral device connected to the port.

[0087] The second jump bus provides a connection between (i) FLC ring bus and (ii) interface in server SoC via ring bus station and second jump bus station. The interface in server SoC may include Ethernet interface, one or more PCIe interfaces and hybrid (or combination) interface. The Ethernet interface may be a 10GE interface and connected to the network via the first Ethernet bus. The Ethernet interface may communicate with the second SoC via the first Ethernet bus, the network and the second Ethernet bus. The network may be Ethernet, cloud network and / or other Ethernet compatible network. One or more PCIe interfaces may include, for example, third generation PCIe interface PCIe3 and mini PCIe interface (mPCIe). The PCIe interface may be connected to a solid state drive. The hybrid interface may be SATA and PCIe compatible to transmit data to and from a SATA compatible device and / or PCIe compatible device according to SATA and / or PCIe protocols. As an example, the PCIe interface may be connected to a storage drive, such as a solid state drive or a hybrid drive. The interface has a corresponding port for connecting to a device outside the server SoC.

[0088] The third jump bus can be connected to the ring bus via a ring bus station and can be connected to the LCD interface and port via the jump bus station. The LCD interface can be connected to a display and can be controlled via the GPM. The port can be connected to one or more collaborative SoCs. The fourth jump bus can be connected to (i) the ring bus via a ring bus station and to (ii) the interface via a jump bus station. The interface can be Ethernet, PCIe, and a hybrid interface. The interface has a corresponding port.

[0089] The server SoC and / or other server SoCs may communicate with each other via an inter-cluster ring bus.The server SoC and / or other server SoCs may communicate with each other via corresponding Ethernet interfaces and networks.

[0090] The collaborative SoC may include a corresponding control module. The control module may access the interface and / or control access to the interface via the jump bus station. In one embodiment, the control module is not included. The control module may be connected to and communicate with a corresponding jump bus station and / or a corresponding interface in the interface in the jump bus station.

[0091] As another example, the above example can also be implemented in a circuit of a mobile device. The mobile device can be a computer, a cellular phone, or other wireless network device. The circuit includes a SoC. The SoC can be called a mobile SoC. The SoC can be called a collaborative SoC. The mobile SoC includes: a control module cluster; an intra-cluster ring bus, an FLC module, a memory control module, an FLC ring bus, and one or more jump buses. The jump bus (i) extends between the server SoC and the collaborative SoC via an inter-chip bus member and a corresponding port, and (ii) extends through the collaborative SoC.

[0092] The intra-cluster ring bus is bidirectional and provides communication between control modules. The intra-cluster ring bus may have a ring bus station for the control module to access the data signal transmitted on the intra-cluster ring bus. The ring bus station may be used as a signal repeater and / or an access node. The control module may be connected to the intra-cluster ring bus via the ring bus station and access the intra-cluster ring bus. Data may be transmitted around the intra-cluster ring bus from a first control module at a first ring bus station in the ring bus station to a second control module at a second ring bus station in the ring bus station. Data may propagate between bus stations at each clock cycle until the data reaches the destination. Each control module may be a central processing unit or a processor.

[0093] The memory control module can control access to the FLC module. In one embodiment, the memory control module is not included. The FLC module can be stacked on a mobile SoC. The FLC module can be an FLC or a DRAM, and can be implemented as any FLC module disclosed herein and any FLC module similar to that disclosed herein to operate. The memory control module can access the FLC ring bus at a corresponding ring bus station on the FLC ring bus, and transmit data between the ring bus station and the FLC module. Alternatively, the FLC module can directly access the FLC ring bus at a corresponding ring bus station. The memory control module may include a memory clock that generates a memory clock signal for a bus station of the FLC module, the ring bus and / or the jump bus. The bus station may receive a memory clock signal indirectly via the ring bus and / or the jump bus or directly from the memory control module. Data may circulate through the bus station based on the memory clock signal.

[0094] The FLC ring bus can be a bidirectional bus and has two types of ring bus stations SRB and SRH. Each ring bus station can be used as a signal repeater and / or access node. The ring bus station SRB is connected to devices other than the jump bus. The devices may include: a cluster; an FLC module and / or a memory control module; and a graphics processing module.

[0095] The ring bus station SRH of the mobile SoC is connected to the FLC ring bus and one of the corresponding jump buses. Each jump bus has multiple jump bus stations SHB, which provide access to one of the corresponding jump buses for the corresponding interface. The jump bus station SHB can be used as a signal repeater and / or access node.

[0096] The first jump bus, the ring bus station and the first jump bus station are connected between (i) the FLC ring bus and (ii) the liquid crystal display (LCD) interface, the video processing module (VPM) and the interface of the collaborative SoC. The LCD interface is in the server SoC and can be connected to the display and can be controlled via the GPM. The interfaces of the collaborative SoC include a cellular interface, a wireless local area network (WLAN) interface and an image signal processor interface. The cellular interface may include a physical layer device for wireless communication with other mobile and / or wireless devices. The physical layer device may operate and / or send and receive signals according to the long-term evolution (LTE) standard and / or the third generation (3G), fourth generation (4G) and / or fifth generation (5G) mobile telecommunication standards. The WLAN interface may be configured to operate and / or send and receive signals according to the LTE standard and / or the third generation (3G), fourth generation (4G) and / or fifth generation (5G) mobile telecommunication standards. The ISP interface can be connected to an image processing device (or image signal processing device) external to the collaborative SoC, such as a storage drive or other image processing device. The interface can be connected to a device external to the collaborative SoC via a corresponding port. The ISP interface can be connected to a device external to the mobile device.

[0097] The collaborative SoC can be connected to the mobile SoC via an inter-chip bus member. The inter-chip bus member is a member of the first jump bus. The additional SoC can be daisy-chained to the collaborative SoC via a port, which is connected to the first jump bus. The mobile SoC, control module, and FLC module can communicate with the collaborative SoC via the FLC ring bus and the first jump bus.

[0098] The second jump bus provides a connection between (i) the FLC ring bus and (ii) the interface in the mobile SoC via the ring bus station and the second jump bus station. The interface in the mobile SoC may include an Ethernet interface, one or more PCIe interfaces, and a hybrid (or combination) interface. The Ethernet interface may be a 10GE interface and is connected to Ethernet via a port. One or more PCIe interfaces may include, for example, a third-generation PCIe interface PCIe3 and a mini PCIe interface (mPCIe). The PCIe interface may be connected to a solid-state drive. The hybrid interface may be SATA and PCIe compatible to transmit data to and from a SATA compatible device and / or PCIe compatible device according to a SATA and / or PCIe protocol. For example, a PCIe interface may be connected to a storage drive via a port. The storage drive may be a solid-state drive or a hybrid drive. The interface has a corresponding port for connecting to a device outside the mobile SoC.

[0099] The collaborative SoC may include a control module. The control module may access the VPM and the interface via the jump bus station and / or control access to the VPM and the interface. In one embodiment, the control module is not included. The control module may be connected to the jump bus station, the VPM and / or the interface and communicate with it.

[0100] Cache line size

[0101] In this example embodiment, a cache line size of 4KB is selected. In other embodiments, other cache line sizes may be utilized. One benefit of using a cache line of this size is that it matches the size of the memory page size typically assigned to an application or program as the minimum memory allocation size by the operating system. As a result, the 4KB cache line size matches the operating memory allocation size.

[0102] The processor typically reads or writes only 64B (bytes) at a time. Therefore, using 4KB as an example, the FLC cache line size is much larger. As a result, when a write or read request causes a miss at the FLC module, the system first reads a complete 4KB cache line from the storage drive (i.e., the final level of main memory in the storage drive partition). After this happens, the system can write the processor data to the retrieved cache line and store the cache line in DRAM. The cache line is identified by a virtual address. The entire cache line is taken out of the memory at one time. In addition, the entire cache line is forwarded, such as from the FLC-SS module to the FLC-HS module. There may be 100,000 or even 1 million or even more cache lines in the operating system.

[0103] Comparing the FLC module cache to the CPU cache, these elements are separate and distinct caches. As shown, the CPU (processor cache) is part of the processor device and is configured in accordance with the prior art. The FLC module acts as a cache, acting as main memory, and is separate and distinct from the CPU cache. The FLC module cache tracks all data that may be needed in a few minutes of operation, just like the main memory and associated controller. However, the CPU cache only tracks and stores data that the processor needs or will use in the next few microseconds or about a millisecond.

[0104] Fully associative FLC cache

[0105] Fully associative lookup enables a large number of truly random processor tasks / threads to reside semi-permanently (when measured in seconds to minutes) in the FLC cache. This is an essential feature because the thousands of tasks or threads being processed by the processor may otherwise easily discard (destroy) a large number of tasks / threads that should be kept in the FLC cache. However, in view of silicon area, power, or both, fully associative lookup is expensive. Therefore, it is also important to maximize the FLC cache line size to minimize the number of entries in the fully associative lookup table. In fact, it is important that it should be larger than the current 64B CPU cache line size. At the same time, the size of the cache line should not be too large because it will bring unnecessary difficulties to the operating system (OS). Because modern OS typically uses a 4KB page size, in an example embodiment, the FLC cache line size is set to 4KB. If the OS page size is increased to 16KB in the future, the FLC cache line size can also be set to 16KB in theory.

[0106] In order to hide the energy cost of the fully associative address lookup process, in one embodiment, an address cache for the address translation table is included in the FLC controller. It is important to note that the address cache does not cache any processor data. Instead, it only caches the most recently seen address translations and physical address to virtual address translations. Therefore, the optional address cache does not have to be fully associative. For the address cache, a simple set associative cache is sufficient because even a 5% miss rate has reduced the need to perform a fully associative lookup process by at least twenty times. The address cache will also additionally result in lower address translation latency because the simple set associative cache used therein can typically translate addresses within 1 clock cycle. This is about ten to twenty times faster than the fastest multi-stage hashing algorithm that can perform CAM-like address translation operations.

[0107] Storage drive memory allocation

[0108] Storage driver 78 can be a traditional non-volatile storage device, such as a disk drive, solid-state drive, hybrid drive, optical drive or any other type of storage device. The DRAM associated with the FLC module and the partitioned portion of the storage driver are used as main memory. In the embodiment disclosed herein, the amount of DRAM is less than that of the traditional prior art computing system. This provides the benefits of less power consumption, lower system cost and reduced space requirements. If the system operation requires additional main memory, a part of the storage driver 78 is allocated or partitioned (reserved) to be used as additional main memory. Storage driver 78 is understood to have a storage driver controller, and the storage driver controller will process the request for traditional file requests from the processing device 500, and will also process the request for information stored in the partition of the storage driver retained as an extension of the main memory from the FLC module.

[0109] Figure 4 An exemplary method of operation is shown. This is only one possible method of operation, and therefore, other methods are contemplated without departing from the scope of the claims. This exemplary method of operation represents an example of Figure 2 The FLC controller system shown. Although mainly about Figure 2 The examples in describe the following tasks, but the tasks can be applied to other embodiments in the present disclosure. The tasks can be performed iteratively or in parallel.

[0110] The method starts at step 408 where the system may be initialized. At step 412, the FLC controller receives a request for a read or write request from a processing device (processor). The request includes a physical address that the processor uses to identify the location of the data or the location where the data is to be written.

[0111] At decision step 416, it is determined whether the physical address provided by the processor is located in the FLC controller. The memory (SRAM) of the FLC controller stores physical to virtual address mapping data. Physical addresses located in the FLC controller are designated as hits, while physical addresses not located in the FLC controller are designated as misses. The FLC module can satisfy the processor's request for data (with the physical address) only if the FLC controller has a physical address entry in its memory. If the physical address is not stored in the memory of the FLC controller, the request must be forwarded to the storage drive.

[0112] At decision step 416, if the physical address is identified in the FLC controller, the request is considered a hit, and the operation proceeds to step 420. At step 420, the FLC controller converts the physical address to a virtual address based on a lookup operation using a lookup table stored in a memory of the FLC controller or in a memory that is part of the DRAM allocated for use by the FLC controller. The virtual address can be associated with a physical address in the FLC DRAM. The FLC controller can include one or more conversion mapping tables for mapping physical addresses (from the processor) to virtual addresses. Figure 5B The FLC controller and its memory are illustrated in more detail.

[0113] After the physical address is converted to a virtual address, the operation proceeds to decision step 424. If the physical address is not located in the FLC controller at decision step 416, a miss occurs, and the operation proceeds to step 428. At step 428, the FLC controller allocates a new (in this case, empty) cache line in the FLC controller for the data to be read or written, which is not yet in the FLC module (i.e., the DRAM of the FLC module). If there is no other available space, the existing cache line can be overwritten. Step 428 includes updating the memory map to include the physical address provided by the processor, thereby establishing the FLC controller with the physical address. Next, at step 432, the physical address is converted to a storage drive address, which is an address used by the storage drive to retrieve data. In this embodiment, the FLC controller performs this step, but in other embodiments, other devices (such as storage drives) can perform the conversion. The storage drive address is an address used or understood by the storage drive. In one embodiment, the storage drive address is a PCI-e address.

[0114] At step 436, the FLC controller forwards the storage address to a storage drive, such as a PCI-e based device, an NVMe (Non-Volatile Memory Express) type device, a SATTA SSD device, or any other storage drive now known or developed in the future. As described above, the storage drive can be a traditional hard drive, an SSD, or a hybrid drive, and a portion of the storage drive is used in the traditional sense to store files, such as documents, images, videos, etc. A portion of the storage drive is also used and partitioned as main memory to supplement the storage capacity provided by the DRAM of the (multiple) FLC modules.

[0115] Advance to step 440, storage drive controller (not shown) retrieves the cache line at the physical address provided by the processor from the storage drive, and provides the cache line to the FLC controller. The cache line identified by the cache line address stores the requested data or is designated as the location where the data is written. This can occur in a manner known in the art. At step 444, the FLC controller writes the cache line to the FLC DRAM and associates it with the physical address so that the association is maintained in the lookup table in the FLC controller.

[0116] Another part of step 444 is the update to the FLC status register, to specify the cache line or data used most recently. The FLC status register that can be stored in the DRAM or in the separate register is a register that tracks when the cache line or data in the FLC DRAM is continuously used, accessed or written by the processor. As a part of cache mechanism, the cache line used recently is maintained in the cache, so that the recently used data are available at any time for the processor again when requested. The cache line that the processor least recently uses, accesses or writes is overwritten, so as to make room for the cache line / data used more recently. In this arrangement, cache is operated based on the first-in-first-out that is least recently used. After step 444, operation proceeds to step 424.

[0117] At decision step 424, the request from the processor is evaluated as a read request or a write request. If the request is a write request, the operation proceeds to step 448 and a write request with a virtual address is sent to the FLC DRAM controller. Figure 2As shown in and understood in the art, a DRAM device has an associated memory controller to oversee read / write operations to the DRAM. At step 452, the DRAM controller generates DRAM row and column address bits from the virtual address, which are used to write data from the processor (processor data) to the FLC DRAM at step 456. Then, at step 460, the FLC controller updates the FLC status register for the cache line or data to reflect the recent use of the cache line / data just written to the FLC DRAM. Because the physical address is mapped into the FLC controller memory map, the FLC controller now owns the physical address if the processor requests it.

[0118] Alternatively, if it is determined at decision step 424 that the request from the processor is a read request, the operation proceeds to step 464, and the FLC controller sends a read request with a virtual address to the FLC DRAM controller for processing by the DRAM controller. Then at step 468, the DRAM controller generates DRAM row and column address bits from the virtual address, which are used to read (retrieve) data from the FLC DRAM at step 472, so that the data can be provided to the processor. At step 476, the data retrieved from the FLC DRAM is provided to the processor to satisfy the processor read request. Then, at step 480, the FLC controller updates the FLC status register for data (address) to reflect the most recent use of the data read from the FLC DRAM. Because the physical address is mapped into the FLC controller memory map, if the processor requests again, the FLC controller maintains the physical address in the memory map as available at any time.

[0119] Figure 4 The above tasks are intended to be illustrative examples; the tasks may be performed sequentially, in parallel, synchronously, simultaneously, continuously, during overlapping time periods, or in a different order depending on the application. In addition, depending on the example and / or order of events, any task may not be performed or skipped.

[0120] Update FLC status register

[0121] As described above, the status register maintains the state of the cache line stored in the FLC module. It is expected that several aspects about the cache line and the data stored in the cache line can be tracked. One such aspect is the relative importance of different cache lines relative to a preset standard or relative to other cache lines. In one embodiment, the cache line that has been accessed most recently will be marked or defined as the most important, and the cache line that has been used least recently will be marked or defined as the least important. Then, the cache line marked as the least important (such as the cache line that has been used least recently) will be eligible to be kicked out of the FLC or overwritten to allow new cache lines to be created or new data to be stored in the FLC. Those of ordinary skill in the art can understand the steps for this task, and therefore will not be described in detail herein. However, unlike the traditional CPU cache controller, the FLC controller will additionally track the cache lines that have been written by the CPU / GPU. This happens so that when the cache line that has been used only for reading is finally cleared from the FLC, the FLC controller will not accidentally write to the storage drive (such as an SSD). In this case, the FLC controller marks the FLC cache line that has been written as "dirty".

[0122] In one embodiment, certain cache lines may be designed as locked FLC cache lines. Certain cache lines in the FLC may be locked to prevent accidental clearing of such cache lines from the FLC. This may be particularly important for retaining the address of the data in the FLC controller when such address / data cannot tolerate retrieval delays, and will therefore be locked, and therefore maintained in the FLC, even if it is least recently used.

[0123] It is also contemplated that a timeout timer for locked cache lines may be implemented. In this configuration, a cache line may be locked, but only for a certain period of time tracked by a lock timer. The timer may be reset after a period of time from when the lock was created or after the cache line is used. The amount of time may vary based on the cache line, the data stored in the cache line, or the application or program to which the cache line is assigned.

[0124] Additionally, it may be contemplated that a timeout bit may be provided to a locked cache line for the purpose of allowing a locked cache line to be flushed from the FLC after a long period of inactivity, or allowing a locked cache line to be eventually flushed to the next stage or level of the FLC module and simultaneously inherit the locked status bits of the next FLC stage to minimize the time penalty for cache line / data retrieval resulting from flushing a previously locked cache line from a high-speed FLC module.

[0125] Figure 5Ais a block diagram of an example embodiment of a cascaded FLC system. This is only one possible arrangement of a cascaded FLC system. Other embodiments are possible without departing from the scope of the claims. In this embodiment, a processor 500 is provided. The processing device 500 may be generally similar to Figure 2 The processing device 72 is shown. Figure 5A Elements of, merge and repeat Figure 2 5. The processing device 500 may be a central processing unit (CPU), a graphics processing unit (GPU), or any other type of processing system, including, but not limited to, a system on a chip (SoC). The processing device 500 includes a processor 504, which includes various levels of processor cache 512, such as level 0, level 1, level 2, and level 3 cache. A memory management module 508 is also provided to interface the processor 504 with the various levels of processor cache 512 and to interface the processor (e.g., for data requests) with elements external to the processing device 500.

[0126] Figure 5A Another part of the embodiment of is also a storage drive 78. The storage drive 78 is generally similar to Figure 2 554. The storage drive 78 of the present invention is not described in detail. The storage drive 78 may include a hard disk drive, such as a traditional rotating device or a solid-state drive, a combined hybrid drive. The storage drive 78 includes a controller (not shown) to supervise input and output functions. The file input / output path 520 connects the processing device 500 to the storage drive 78 through a multiplexer 554. The file I / O path 520 provides a path and mechanism for the processor to directly access the storage drive 78 for write operations, such as saving files directly to the storage drive, as can occur in traditional systems. The multiplexer 554 is a bidirectional switch that selectively passes inputs from the FLC-SS 536 or the file I / O path 520 in response to a control signal on the control signal input 556.

[0127] In having Figure 2 , 5A, 6, 8, 10 and 12, the storage drive has a segment that is allocated, partitioned or reserved as an extension of main memory (an extension of RAM memory). Therefore, a portion of the storage drive 78 is used for traditional storage of user files, which are such as documents, pictures, videos, music and can be viewed by the user in a traditional folder or directory structure. Another portion of the storage drive 78 is allocated, partitioned or reserved for use by the FLC system as an extension of the DRAM main memory to store active programs and instructions used by the processor, such as operating systems, drivers, application code, and active data being processed by the processing device. Main memory is short-term data storage for a computer system because it stores information that the computer is actively using. The term main memory refers to main memory, primary storage, system memory, or RAM (random access memory). Data that is to be stored in main memory but has been least recently used (operating systems, drivers, application code, and active data) is stored in the main memory partition of the storage drive. In Figure 2 , 5A , 6, 8, 10 and 12, and in other embodiments described herein, the system bus may be located between the processing device and the FLC module, such as Figure 2 shown.

[0128] Although the main memory partition of storage drive 78 is slower than RAM for I / O operation, the hit rate of FLC module is so high (such as 99% or higher) that I / O to the main memory partition in storage drive rarely occurs, and therefore performance will not be reduced. Discussion of storage drive 78 and its main memory partition is applicable to storage drive shown in other figures. In all embodiments shown and described, the content of the main memory partition of storage drive can be encrypted. Encryption may occur to prevent viewing personal information, Internet history, passwords, documents, emails, images stored in the main memory partition of storage drive 78 (non-volatile). Through encryption, if the computing device is discarded, recycled or lost, the sensitive information cannot be read. Different from RAM, which does not maintain stored data after power failure, storage drive will also maintain data even in the event of power failure.

[0129] like Figure 5A 1, there are two final level cache (FLC) modules 540, 542 arranged in a cascade configuration. Each module 540, 542 is referred to as an FLC stage. Although two cascaded stages are shown, more stages can be cascaded. Each FLC stage (module) 540, 542 is generally similar to Figure 2In the FLC module 76 shown in, and therefore, these units are not described in detail herein. In this cascade configuration, the FLC module 540 is a high speed (HS) module, which is configured to operate with a higher bandwidth, lower latency and lower power use than other FLC modules 542 as standard speed modules. Since the FLC-HS module is more frequently utilized compared to the FLC-SS, the benefits realized by the low power and high speed aspects of the FLC-HS module 542 are further increased. It is the main memory used, and has a hit rate greater than 99%, so speed and power saving can be provided in most main memory accesses. The FLC module 542 is called standard speed (SS), and relative operating speed is optimized for lower cost when it is still very fast. Because standard speed DRAM has a larger capacity than high speed DRAM, cost saving is maximized, and in these FLC embodiments, the amount of standard speed DRAM is less than the amount used in the prior art computer that is usually equipped with 8GB or 16GB of RAM. Exemplary FLC system can have 4GB of DRAM and 32GB of storage drive partitions. This will result in a cost saving of approximately $200 for a typical portable computer with 8 to 16GB of RAM. In addition, because most memory accesses are successfully processed by the high-speed FLC module, the standard speed FLC module is usually inactive and therefore does not consume power. The benefits of this configuration are discussed below. It can be expected that the storage capacity of the FLC-HS module 540 is less than the storage capacity of the FLC-SS module 542. In one embodiment, the memory capacity of the FLC-SS module is eight (8) times greater than the memory capacity in the FLC-HS module. However, some applications can even tolerate a capacity ratio of more than 32 times.

[0130] Note that both the FLC-HS controller and the DRAM-HS are optimized for low power consumption, high bandwidth and low latency (high speed). Therefore, both elements provide the above benefits. On the other hand, both the FLC-SS controller and the DRAM-SS are optimized for lower cost. In one configuration, the lookup table of the FLC-HS controller is located in the FLC-HS controller and utilizes SRAM or other high-speed / low-power memory. However, for FLC-SS, the lookup table can be stored in the DRAM-SS. Although having this configuration is slower than having a lookup table stored in the FLC-SS controller, it is more cost-effective to use a small portion of the DRAM-SS partition for the lookup table required for the FLC-SS. In one embodiment, in order to reduce the time loss of accessing the lookup table stored in the DRAM-SS, a small SRAM cache of the DRAM-SS lookup table can be included to cache the address conversions that have been seen (used) the most recently. Such an address cache does not have to be fully associated because only the address conversion table is cached. A set-associated cache (such as the cache used in the CPU L2 and L3 caches) is sufficient because even 5% of misses have reduced the need for address conversion in DRAM by 20 times. This can be achieved by caching only a small percentage of the lookup table entries, such as 1000 out of 64,000. The address cache can also be based on a least recently used / first in first out operation.

[0131] In this embodiment, the FLC module 540 includes a FLC-HS controller 532 and a DRAM-HS memory 528 with an associated memory controller 544. The FLC module 542 includes a FLC-SS controller 536 and a DRAM-SS memory 524 with an associated memory controller 548. The FLC-HS controllers 532 are connected to the processing device 500. As shown, they are also connected to the DRAM-HS 528 and the FLC-SS controllers 536. The output of the FLC-SS controller 536 is connected to the DRAM-SS 524 and also to the storage drive 78.

[0132] The controller 544, 548 of each DRAM 528, 524 operates as understood in the art to direct and control read and write operations to the DRAM, and therefore, these elements and related operations are not described in detail. Although shown as DRAM, it is contemplated that any type of RAM may be utilized. The connection between the controller 544, 548 and the DRAM 528, 524 enables communication between these elements and allows data to be retrieved from and stored to the corresponding DRAM.

[0133] In this example embodiment, the FLC controller 532, 536 includes one or more lookup tables storing physical memory addresses that can be converted to addresses corresponding to locations in the DRAM 528, 524. For example, the physical address can be converted to a virtual address, and the DRAM controller can use the virtual address to generate DRAM row and column address bits. The DRAM 528, 524 is used as a cache memory. In this embodiment, the lookup table is fully associative, so there is a one-to-one mapping, and allows data to be stored in any cache block without causing conflicts between two or more memory addresses mapping to a single cache block.

[0134] like Figure 5A As shown, the standard speed FLC module 542 is not directly connected to the processing device 500. By only connecting the high-speed FLC module 540 to the processing device 500, the standard speed FLC module 542 is dedicated to the high-speed FLC module 540. It is expected that one high-speed FLC module can share one or more standard speed FLC modules. This arrangement does not slow down the processor by having to reroute the miss in the FLC-HS controller 532 back to the processing device 500, and the miss will be routed to the standard speed FLC module 542, which will inevitably consume precious system bus resources and generate additional overhead for the processing device 500.

[0135] Typically, during the operation of a memory read event, a data request having a physical address for the requested data is sent from the processing device 500 to the FLC-HS controller 532. The FLC-HS controller 532 stores one or more memory address tables accessible by the FLC-HS controller 532 in the associated DRAM-HS 528. The FLC-HS controller 532 determines whether its memory table contains the corresponding physical address. If the FLC-HS controller 532 contains the corresponding memory address in its table, a hit occurs, i.e., the FLC-HS controller 532 retrieves the data from the DRAM-HS 528 (via the controller 544), which is then provided back to the processing device 500 by the FLC-HS controller.

[0136] Alternatively, if the FLC-HS controller 532 does not contain a matching physical address, the result is a miss, and the request is forwarded to the FLC-SS controller 536. This process is repeated at the FLC-SS controller 536, so that if the matching physical address is located in the memory address lookup table of the FLC-SS controller 536, the requested data is converted or transformed into a virtual memory address and data fetched from the DRAM-SS 524 via the memory controller 548. The DRAM controller generates DRAM row and column address bits from the virtual address. If the matching physical address is located in the memory address lookup table of the FLC-SS controller 536, the data request and physical address are directed to the storage drive by the FLC-SS controller 536.

[0137] If the requested data is not available in DRAM-HS 528, but is stored and retrieved from DRAM-SS, then when the retrieved data is provided to the processor by being transferred to FLC-SS controller 536 and then to FLC-HS controller and then to processor 500, it will be backfilled in DRAM-HS. When backfilling data, if there is no available space in DRAM-SS or DRAM-HS, the least recently used data or cache line will be removed, or the data therein will be overwritten. In one embodiment, the data removed from the cache is retained in the standard speed cache until additional space is needed in the standard speed cache. It can also be expected that in some cases, data can be stored only in the high-speed FLC module, and not in the standard speed FLC module, and vice versa.

[0138] If the requested data is not available in DRAM-HS 528 and is not available in DRAM-SS 524 and is therefore retrieved from storage drive 78, the retrieved data is backfilled into DRAM-HS, DRAM-SS, or both as it is provided to the processor. Thus, the most recently used data is stored in DRAM 528, 524, and over time, the DRAM contents are dynamically updated with the most recently used data. The least frequently used data is discarded or overwritten from DRAM 528, 524 to make room for more recently used data. These backfill paths are Figure 5A , which are shown as “first-stage cache replacement path” and “second-stage cache replacement path”.

[0139] Figure 5B is a block diagram of an example embodiment of a FLC controller. This is just one configuration of the basic elements of the FLC controller. One of ordinary skill in the art will appreciate that there are additional elements, data paths, and supporting elements in the working system of all embodiments disclosed herein. These elements, data paths, and supporting elements are not shown, but rather focus on the elements that support the disclosed innovations. Figure 5B The FLC controller 532 represents Figure 5A The FLC controller 532 of the present invention or other FLC controllers disclosed herein.

[0140] exist Figure 5B , it is shown to the processor (500, Figure 5A ) of the processor I / O path 564. The processor I / O path 564 is connected to the FLC logic unit state machine (state machine) 560. The state machine 500 may include any device capable of performing as described herein, such as but not limited to an ASIC, a control logic, a state machine, a processor, or any combination of these elements or any other elements. The state machine 560 converts the system physical address to the FLC virtual address. The state machine performs a fully associative lookup process using a hash function of multiple stages. Alternatively, the state machine 560 may be or use a content addressable memory (CAM) to perform the conversion, but this will be expensive.

[0141] The state machine 560 is connected to a memory 576, such as, for example, an SRAM. The memory 576 stores a lookup table containing physical addresses stored in the FLC controller. These physical addresses can be converted or mapped to virtual addresses that identify cache lines accessible by the FLC controller 532. The memory 576 can store address mappings and multiple hash tables. Using multiple hash tables reduces power consumption and reduces operational delays.

[0142] The state machine 560 and memory 576 operate together to convert physical addresses from the processing device into virtual addresses. When a "hit" occurs, the virtual address is provided to the DRAM via the hit I / O line 568. If the state machine 560 determines that its memory 576 does not contain a physical address entry, a miss occurs. If a miss occurs, the FLC logic unit state machine provides a request with the physical address to the miss I / O line 572, which leads to a storage drive or another FLC controller.

[0143] Figure 6 is a block diagram of parallel cascade FLC modules. Figure 5A Compared, the same elements are marked with the same reference numerals and are not described again. Added to this embodiment are one or more additional FLC modules 550, 552. In this example embodiment, high-speed FLC module 550 is generally the same as high-speed FLC module 540, and standard-speed FLC module 552 is generally the same as standard-speed FLC module 542. As shown, high-speed FLC module 550 is connected to processing device 500, and standard-speed FLC module 552 is connected to storage drive 78 via multiplexer 554. Both high-speed FLC modules 540, 550 are connected to processing device 500 such as via a system bus.

[0144] Figure 6 The operation of an embodiment is generally similar to Figure 5A and Figure 8 Operation of an embodiment. Figure 7 Provided Figure 5A Operation flow chart of an embodiment of the present invention. Figure 5A Compared with the single cascade embodiment, Figure 6 The configuration shown has numerous benefits. Although more costly and consuming more space, having multiple cascaded FLC modules arranged in parallel provides the benefits of separating memory addresses to different and dedicated FLC modules and allowing memory operations to be performed in parallel with two or more FLC modules, while still having the above combined Figure 5A The benefits of multi-stage FLC are discussed.

[0145] Figure 7 Yes Figure 5A 10. Operational flow chart of an example method of operation of the cascaded FLC module shown. This is only an example method of operation, and those of ordinary skill in the art will understand other methods of operation. At step 704, a read request with a physical address for data is sent from a processing device (processor) to the FLC-HS module, and in particular to the FLC-HS controller. Then at decision step 708, the FLC-HS controller determines whether the physical address is recognized in the lookup table of the FLC-HS controller. The result of decision step 708 can be a hit or a miss.

[0146] If the physical address is located at step 708, the result is a hit, and the operation proceeds to step 712. At step 712, a read request with the virtual address is sent to the DRAM-HS controller. Figure 2 As shown in and understood in the art, the DRAM device has an associated memory controller to supervise the read / write operations to the DRAM. At step 716, the DRAM controller generates DRAM row and column address bits from the virtual address, which are used to read (retrieve) data or cache lines from the DRAM-HS at step 720. At step 724, the FLC-HS controller provides data to the processor to meet the request. Then, at step 728, the FLC-HS controller updates the FLC status register for the cache line (address or data) to reflect the recent use of the cache line. In one embodiment, data is written to the DRAM-HS and is also written to the FLC-SS module.

[0147] Alternatively, if the physical address is not recognized in the FLC-HS at step 708, the operation proceeds to step 732, and a new (empty) cache line is allocated in the FLC-HS controller, such as a memory lookup table and DRAM-HS. Because the physical address is not recognized in the FLC-HS module, space must be created for the cache line. Then, at step 736, the FLC-HS controller forwards the data request and physical address to the FLC-SS module.

[0148] As occurs in the FLC-HS module, at decision step 740, a determination is made as to whether the physical address is recognized in the FLC-SS. If the physical address is in the FLC-SS module, as revealed by the presence of the physical address in the lookup table of the FLC-SS controller, the operation proceeds to step 744. At step 744, a read request is sent to the DRAM-SS controller along with the virtual address. At step 748, the DRAM-SS controller generates DRAM row and column address bits from the virtual address, which are used to read (retrieve) data from the DRAM-SS at step 752. The virtual address of the FLC-HS is different from the virtual address of the FLC-SS, so a different conversion of the physical address to the virtual address occurs in each FLC controller.

[0149] At step 724, the FLC-HS controller forwards the requested cache line to the FLC-HS controller, which in turn provides the cache line (with data) to the DRAM-HS, which is cached in the FLC-HS module. Ultimately, the data is provided from the FLC-HS to the processor. Then, at step 760, the FLC-HS controller updates the FLC status register for the data (address) to reflect the most recent use of the data provided to the FLC-HS and then to the processor.

[0150] If the physical address is not identified in the FLC-SS at step 740, a miss occurs in the FLC-SS controller, and the operation proceeds to step 764, and a new (empty) cache line is allocated in the FLC-SS controller. Because the physical address is not identified in the FLC-SS controller, space must then be created for the cache line. At step 768, the FLC-SS controller converts the physical address to a storage drive address, such as an address of a PCI-e type. The storage drive address is an address understood by the storage drive or used by it to identify the location of the cache line. Next, at step 772, the storage drive address obtained from the conversion is forwarded to the storage drive, such as PCI-e, NVMe or SATA SSD. At step 776, using the storage drive address, the storage drive controller retrieves the data, and the retrieved data is provided to the FLC-SS controller. At step 780, the FLC-SS controller writes the data to the FLC-SS DRAM and updates the FLC-SS status register. As described above, an update status register occurs to specify the most recently used cache line, thereby preventing it from being overwritten until it becomes the least recently used. Although the tracking of the least recently used status is tracked on a cache line basis, it is contemplated that the least recently used status could be tracked for individual data items within a cache line, but this would add complexity and an additional overhead burden.

[0151] In one embodiment, as shown in steps 764 and 752, cache lines are retrieved from the storage drive. The entire cache line is provided to the FLC-HS controller. The FLC-HS controller stores the entire cache line in the DRAM-HS. The data requested by the processor is stored in the cache line. In order to meet the processor request, the FLC-HS controller extracts data from the cache line and provides the data to the processor. This may occur before or after the cache line is written to the DRAM-HS. In one configuration, only the cache line is provided to the FLC-HS controller from the FLC-SS controller, and then the FLC-HS controller extracts the data requested by the processor from the cache line. In another embodiment, the FLC-SS controller first provides the requested data, and then the cache line is provided to the FLC-HS controller. Then, the FLC-HS controller can provide the data processor, and then or concurrently write the cache line to the FLC-HS. Since the extracted data is first provided to the FLC-HS controller, the speed may be faster.

[0152] As mentioned above, the virtual address of the FLC-HS controller is different from the virtual address of the FLC-SS controller. The lookup table in each FLC controller is different, and there is no relationship between them. As a result, each FLC controller virtual address set is also unique. The virtual address may accidentally have the same bit between them, but the virtual address is different because they are intended to be used in their corresponding DRAM (DRAM-HS and DRAM-SS).

[0153] Figure 8 is a block diagram of a split FLC module system having two or more separate FLC modules. This is only one possible embodiment of a split FLC module system, and it is contemplated that different arrangements are possible without departing from the scope of the claims. Figure 5A In contrast, the same elements are marked with the same reference numerals, and these repeated elements are not described in detail again.

[0154] like Figure 8 As shown, the first (a), second (b) and n-stage FLC modules 802 are provided in parallel to enable parallel processing of memory requests. The value of n can be any integer. With reference to the first FLC module 820A, the FLCa controller 804A is connected to the processing unit 500 or communicates with it to receive a read or write request. The system bus (not shown) can reside between the FLC module 820 and the processing device 500 so that communication and request routing can occur through the system bus. The FLCa controller 804A is also connected to the DRAM memory controller 808A associated with DRAMa 812A. The FLCa controller 804A is also directly connected to the storage drive 78 or communicates with it. Each of the other FLC modules 820B and 820n is similarly configured, wherein each element shares the same reference numeral but has a different identifier letter. For example, the FLC module 820B includes a FLCb controller 804B, a DRAM memory controller 808B and a DRAMb 812B. As shown, FLC module 820B is also connected to or in communication with processing device 500 and storage drive 78. Although a single processing device 500 is shown, it is contemplated that additional processing devices (GPU / audio processing unit / ...) may utilize FLC module 820 as well.

[0155] One or more FLC modules 820 can be configured as high-speed FLC modules with high-speed / low latency / low power DRAM, or the FLC modules can be standard speed modules with standard speed DRAM. This allows different operating speeds for different FLC modules. This in turn adapts the processing module 500 to direct important data read / write requests to high-speed FLC modules, while routing less important read / write requests to standard speed FLC modules.

[0156] In one embodiment, each FLC slice (FLCa, FLCb, FLCc) is connected to the SoC bus, and each FLC slice is assigned an address by a processing device. Each FLC slice is a different element, which helps separate and different memory lookup tables. A bus address lookup table or a hash table can be used to map a memory address to an FLC slice. In one configuration, certain bits in the physical address define which FLC slice is assigned to the address. In another embodiment, a bidirectional multiplexer (not shown) can be provided between the FLC slice and the processing unit 500 to control access to each FLC slice, but the arrangement may produce a bottleneck that slows down the operation.

[0157] It is also expected that Figure 5A and Figure 8 The embodiments of the present invention can be combined so that a system can be assembled, which has one or more FLC modules 820A with a single FLC controller 804A and the like. Figure 5A One or more cascaded FLC modules are shown. The benefit of combining these two different arrangements is to achieve the benefits of both arrangements. There are multiple paths from the processor to the DRAM, thereby increasing system speed and bandwidth, while also providing the benefits of a high-speed two-stage FLC controller to increase speed, bandwidth and reduce power consumption. The combined system can be arranged in any way to customize the system to meet design needs.

[0158] Fig. 9 Yes Figure 8 1. Operational flow chart of an example method of operation of a split FLC module shown. This is only an example method of operation, and those of ordinary skill in the art will understand other methods of operation. Before the method begins, a memory lookup table is provided as part of a processing device or system bus. The lookup table is configured to store an association between an address from a processor and an FLC module. In this embodiment, each FLC module may be referred to as a slice, and each FLC slice may have multiple FLC stages.

[0159] In this embodiment, multiple FLC slices are established to increase FLC capacity and bandwidth. Each FLC slice is assigned a portion of the system bus memory address space (region). In addition, these memory regions are interleaved between FLC slices. The interleaving granularity is set to match the FLC cache line size to prevent unnecessary duplication (by overlapping) of FLC lookup table entries in different FLC controller slices and ultimately maximize the FLC hit rate.

[0160] In one example embodiment, the mapping dispatches FLC cache line sized address blocks to FLC modules in an interleaved order. For example, for an FLC implementation with a cache line size of 4KB and an implementation of four different FLCs (FLCa, FLCb, FLCc, FLCd), the following mapping (dispatching) of memory identified by physical addresses to FLCs is as follows:

[0161] First 4KB – FLCa

[0162] Second 4KB – FLCb

[0163] 3rd 4KB–FLCc

[0164] 4th 4KB–FLCd

[0165] Fifth 4KB–FLCa

[0166] 6th 4KB–FLCb

[0167] 7th 4KB–FLCc

[0168] 8th 4KB–FLCd

[0169] 9th 4KB–FLCa.

[0170] This memory map dispatch scheme continues to follow this pattern. This can be referred to as a memory map with cache line boundaries to separate data to different FLC modules. In this regard, the memory addresses used by the processing device are divided between the FLC slices, thereby creating a parallel arrangement of FLC systems that allow for increased performance without any bottlenecks. This allows multiple different programs to utilize only one FLC module, or to distribute their memory usage between all FLC modules, thereby increasing the speed of operation and reducing bottlenecks.

[0171] In one embodiment, each FLC slice corresponds to a memory address. In this example operating method, there are four FLC slices, defined as FLCa, FLCb, FLCc, and FLCd. Each FLC slice has a unique code that identifies the FLC slice. For example, the following provides an exemplary memory address for FLC slice assignment:

[0172] The address xxxx-00-xxxxx is assigned to FLCa,

[0173] Address xxxx-01-xxxxx is assigned to FLCb,

[0174] The address xxxx-10-xxxxx is assigned to FLCc,

[0175] The address xxxx-11-xxxxx is assigned to FLCd,

[0176] Where x is any combination of "0" and "1". In other embodiments, other addressing mapping schemes may be utilized.

[0177] Any other address block mapping scheme with an integer FLC cache line size can be used. For partial or non-integer block sizes, there may be duplication of lookup table entries in different FLC slices. Although this may not be fatal, it will still result in a smaller number of different address lookup table entries and ultimately affect the FLC cache hit performance.

[0178] return Fig. 9 , at step 912, a memory address is assigned to each FLC module (in this embodiment, FLC1, FLC2, FLC3, but other embodiments may have a greater or lesser number of FLC modules). The assignment may be performed in an interleaved manner as described above. Then, at step 916, the processing device generates a read request for the data stored in the memory. In other embodiments, the request may be a write request. At step 920, the data request from the processing device is analyzed, and based on the memory mapping, the data request (with a physical address) is routed to the appropriate FLC. This may occur in the system bus. Based on the exemplary memory address association provided above, if the physical memory is xxxx-00-xxxxx, then the address is mapped to the FLCa, and the address is routed to the processor bus port assigned to the FLCa. The operation then proceeds to step 924, where a write operation occurs for the data request and the physical address. Figure 4 If the memory address is xxxx-01-xxxxx, then the address will be mapped to FLCb, and the operation proceeds to step 928. If the physical memory address is xxxx-10-xxxxx, then it is mapped to FLCc, and the operation proceeds to step 932, where a request for data and a physical address are generated. Figure 4 If the physical memory address is xxxx-11-xxxxx, then the address is mapped to FLCc and the operation proceeds to step 936 where a request for data and a physical address are generated. Figure 4 method. Figure 4 The method and discussion are incorporated into Fig. 9 under discussion.

[0179] Fig.10 is an exemplary block diagram of an example embodiment of a cascaded FLC system with a bypass path. Figure 5A 1004. In this embodiment, the bypass module 1004 is arranged between the high-speed FLC module 540 and the processing device 500 and is connected to the high-speed FLC module 540 and the processing device 500. The input of the bypass module 1004 receives the request from the processing device 500. The bypass module 1004 can be any type of device that can analyze the request from the processor and classify it as a request to be routed to the bypass path or to the high-speed FLC module 540. The bypass module 1004 can include but is not limited to a state machine, a processor, a control logic, an ASIC, any other similar or equivalent device.

[0180] The first output from the bypass module 1004 is connected to the FLC-HS controller 532. The second output from the bypass module 1004 is connected to the multiplexer 1008. The multiplexer 1008 also receives a control signal on a control input 1012. The multiplexer 1008 can be any type of switch configured to output one of the input signals at a particular time in response to the control signal. The output of the multiplexer 1008 is connected to the standard speed FLC controller 536 of the standard speed FLC module 542.

[0181] Below Fig.11 Discussion and Fig.10 The operation of the bypass module 1004 and multiplexer 1008 in relation to the cascaded FLC modules is shown. In general, the bypass module 1004 analyzes the request from the processing device 500 and determines whether the request qualifies as a request that should be bypassed to the standard speed FLC module 542 or directed to the high speed FLC module 540. If the request is determined to be a bypass type request, the request is redirected by the bypass module 1004 to the multiplexer 1008 where it is selectively switched to the standard speed FLC module 536.

[0182] Fig.11 Yes Figure 8100 is a flowchart of an example method of operation of a split FLC module shown in FIG. This is only an example method of operation, and those of ordinary skill in the art will understand other methods of operation. The method begins at step 1108, where a processing device generates a read request for data from a memory. This step occurs in a conventional case, as a processor requests data from a main memory (such as a RAM). At step 1112, a request from a processing device is provided to a bypass module for processing. The bypass module processes the request to determine whether the request meets or is classified as data that will bypass a high-speed FLC module. Data or certain addresses may be classified to bypass a high-speed FLC module for a variety of different reasons.

[0183] In some embodiments, the bypass data is data that is not used frequently enough from a performance perspective to qualify as being stored in high-speed DRAM. In other embodiments, certain physical addresses from the processing device are designated as bypass addresses that the bypass module routes to the bypass path. This is called fixed address mapping, whereby certain addresses or blocks of addresses are directed to the bypass path. Similarly, the bypass decision can be based on the type of data specified by the processor or other software / hardware functions.

[0184] The bypass designation may also be based on a task ID, which is defined as the importance of the task. The task ID that defines the importance of the task may be set by a fixed set of criteria or may change over time based on the available capacity of the DRAM-HS or other factors. The software engine or algorithm may also specify the task ID. The bypass module may also be configured to reserve space in the DRAM-HS so that only a certain task ID may be placed in the reserved DRAM-HS memory space. To avoid the bypass module control never ending or unnecessarily blocking the cache of the DRAM-HS, the task ID or designation may time out, which means that the bypass designation terminates after a fixed or programmable timer period. The task ID may also be used to define the DRAM-HS cache line allocation capacity on a per-task ID basis. This is to prevent greedy tasks / threads from clearing non-greedy tasks / threads and ultimately make the overall system performance more balanced. The operating system may also change the cache line allocation capacity table over time to reflect the number of concurrent tasks / threads that need to operate simultaneously during a given time period.

[0185] For example, the screen display of active video playback (movie) is shown to have a constantly changing screen display, but when not playing video, the screen display is static. As a result, the bypass module can be configured to bypass the active video display to the bypass path because the video is not redisplayed on the screen more than once or twice. However, for the movie of pause or during the non-video playback when the screen is static, due to the display data can be repeatedly used when refreshing the screen, the display data can be (not bypassed) cached. Therefore, it is best to make data form a static display in the FLC-HS module, because the FLC-HS module has lower power consumption. This can be done in software or hardware to detect whether the screen is repeated screen display.

[0186] In one embodiment, the bypass module includes an algorithm and a machine learning engine that monitors over time which data (rarely used or used only once) should be bypassed from the high-speed FLC module to the standard speed FLC module. Over time, the machine learning capabilities of the bypass module with artificial intelligence determine which data is rarely used or used only once for a particular user, and therefore should be bypassed from the high-speed FLC module. If the user uses the data more often over time, the machine learning aspects of the bypass module will adjust and adapt to the change in behavior to direct the data to the high-speed FLC module for caching to maximize performance.

[0187] In one embodiment, the bypass module does not use machine learning or adapt to the user's behavior, instead, the data or address that is bypassed to other than the high-speed FLC module is fixed, user programmable, or software controlled. This is a less complex approach.

[0188] It is also contemplated that the processing device may designate data as bypass type data. Thus, a request (read or write) from the processing device to the bypass module will include a designation as bypass type data. This provides another mechanism to control which data is stored in the high-speed FLC module, with the flexibility of software control.

[0189] It is also contemplated and disclosed that a bypass designation for data may have a timer function that removes the bypass designation after a period of time, or after a period of time, the bypass designation must be renewed to remain active. This may prevent a bypass designation from being applied to data that no longer has a bypass designation.

[0190] return Fig.11 At decision step 1116, it is determined whether the data is bypass data. If the data is not designated by the bypass module as data that should not be bypassed, the operation proceeds to step 1120. At step 1120, the operation performs the above method. Figure 7 As mentioned above, Figure 7The method steps are not repeated here, but are incorporated into this part of the present application. Figure 7 As explained in , the method progresses at this point like a cascaded FLC system.

[0191] Alternatively, if at decision step 1116 the bypass module determines that the data should be bypassed, then operation proceeds to step 1124 and the data request with the physical address is routed from the bypass module to the bypass multiplexer. In other embodiments, the data request and the physical address may be routed to the bypass multiplexer. The bypass multiplexer (and other multiplexers disclosed herein) is a bypass multiplexer that, in response to a control signal, passes one of its inputs to its output, which in this embodiment is connected to the standard speed FLC module. Another input to the bypass multiplexer comes from a high speed FLC controller, such as Fig.10 shown.

[0192] At step 1128, in response to the control signal to the bypass multiplexer, the bypass multiplexer routes the data request and physical address to the standard speed FLC-SS module. In other embodiments, the data request and physical address from the bypass multiplexer may be transmitted to a different location, such as a different high speed FLC module, or directly to the storage drive. Then, at step 1132, the standard speed FLC-SS module transmits the data request and physical address to the standard speed FLC-SS module. Figure 4 The data request and physical address are processed in the manner described in . Because this data is defined as bypass data, it is not cached in the DRAM-HS or FLC-HS controller. Figure 4 The method incorporates Fig.11 that part of.

[0193] Fig.12 is an exemplary block diagram of an example embodiment of a cascaded FLC system having a bypass path and a non-cacheable data path. Figure 5A and 10 Compared to the above, like elements are labeled with like reference numerals. This example embodiment is only one possible configuration of a system for separately routing non-cacheable data, and thus, other embodiments and arrangements may be derived by one of ordinary skill in the art. Fig.10 In addition to the configuration of the bypass module 1004, a non-cacheable data path 1204 is added in this embodiment, which is connected between the bypass module 1004 and the second multiplexer 1208. The second multiplexer 1208 includes a control signal input 1212 configured to provide a control signal to the multiplexer. The control signal 1212 of the second multiplexer 1208 determines which of the two inputs of the second multiplexer is output to the DRAM-SS 524.

[0194] In this embodiment, a portion of DRAM-SS 524 is partitioned to be reserved as non-cacheable memory. In the non-cacheable data partition of DRAM-SS, non-cacheable data is stored. In this way, the non-cacheable data partition is used as a traditional processor / DRAM. If the processor requests non-cacheable data, such as a video file that is usually viewed once, the file is retrieved from the storage drive 78 by the processor through the file I / O path 520 and provided to the non-cacheable partition of DRAM-SS. The processor can then retrieve the data now stored in DRAM-SS in smaller chunks through the non-cacheable data path. Video files (such as movies) are usually large and are usually viewed only once, and therefore are not cached because caching data that is only used once does not bring performance benefits. Partitioning a portion of memory is understood by those of ordinary skill in the art, and therefore, the process is not described in detail herein. Non-cacheable data can also be stored in the storage drive 78.

[0195] In this embodiment, the bypass module 1004 is also configured to analyze the read request and determine whether the read request is for data classified as non-cacheable data. If so, the data read request from the processing device 500 is routed to the second multiplexer 1208 through the non-cacheable data path 1204. The second multiplexer 1208 determines whether to pass the non-cacheable data read request or the request from the standard speed FLC-SS controller 536 to the DRAM-SS 524 in response to the control signal. Because the data is non-cacheable, after the data is provided to the processor, the data is not cached in the DRAM-HS528 or the DRAM-SS 524, but can be stored in the non-cacheable data partition of the DRAM-SS.

[0196] Fig.13 Provided Fig.12 Flow chart of an exemplary method of operation of an embodiment of the present invention. This is only one example method of operation, and a person of ordinary skill in the art will understand other methods of operation. The method of operation is similar to Fig.11A method for processing uncacheable data, wherein an additional step is directed to processing uncacheable data. At step 1304, a processing device generates a read request for data stored in a memory. The request includes a physical address. Then at step 1308, the request and the physical address are provided to a bypass module to determine whether the request should be routed to a bypass path, or whether the request is a request for uncacheable data and should therefore be routed to an uncacheable data path. At decision step 1312, it is determined whether the data request should be routed to a bypass path. If it is determined that the request is a bypass data type request, the operation proceeds to step 1316, and the bypass module routes the data request and the physical address from the bypass module to a bypass multiplexer. A bypass multiplexer can be any device capable of receiving two or more inputs and selectively routing one of the inputs to an output. A bypass multiplexer is bidirectional, so a signal at a single output of the multiplexer can be routed to any input path. A bypass multiplexer control signal on input 1012 controls the operation of the bypass multiplexer.

[0197] Thereafter, at step 1320, in response to a control signal provided to the bypass multiplexer, a data request with a physical address is routed from the bypass multiplexer to the FLC-SS module. Then at step 1324, the FLC-SS module processes the data request and the physical address, such as Figure 4 shown. Figure 4 The method incorporates Fig.13 .

[0198] Alternatively, if it is determined at decision step 1312 that the bypass criteria are not met, then operation proceeds to decision step 1328 where it is determined whether the request is a cacheable memory request. A cacheable memory request is a request from a processing device for data to be cached in one of the FLC modules, while a non-cacheable memory request is a request for data to be not cached. If the request is for cacheable memory, then operation proceeds to step 1332 and a cache is performed based on the data request and the physical address. Figure 7 processing. Figure 7 The method incorporates Fig.13 .

[0199] Alternatively, if it is determined at step 1328 that the requested data is uncacheable, the operation proceeds to step 1336. At step 1336, the uncacheable data request including the physical address is routed from the bypass module to the second multiplexer. The second multiplexer can be configured and operated generally similar to the bypass multiplexer. At step 1340, in response to the second multiplexer control signal, the data request and the physical address from the second multiplexer are provided to the DRAM-SS controller, which directs the request to the partition of the DRAM-SS reserved for the uncacheable data. At step 1344, the FLC-SS controller retrieves the uncacheable data from the DRAM-SS uncacheable data partition, and at step 1348, the FLC-SS controller provides the uncacheable data to the processing device. The retrieved data is not cached in the DRAM-HS cache or the DRAM-SS cache, but can be maintained in the uncacheable partition of the DRAM-SS. Therefore, it is not accessible through the FLC-SS module, but is accessed through the uncacheable data path.

[0200] It is contemplated and disclosed that any of the above embodiments, elements or variations can be assembled or arranged in any combination to form new embodiments. Figure 6 As shown, parallel FLC module arrangement (FLC slice) can be combined with two or more stage FLC modules. Any of these embodiments can be assembled or claimed using bypass module features and / or non-cacheable data paths. It can also be expected that more than two stages of FLC modules (such as three or four FLC module stages) can be combined with any other element shown or described herein.

[0201] It is also understood that although the flowcharts and methods of operation are shown and discussed with respect to sequential operations, it is understood and disclosed that various operations can occur in parallel. This can increase the speed of operation, bandwidth and reduce latency in the system.

[0202] The wireless communication aspects described in the present disclosure may be performed in whole or in part in accordance with IEEE Standard 802.11-2012, IEEE Standard 802.16-2009, IEEE Standard 802.20-2008, and / or Bluetooth Core Specification v4.0. In various embodiments, Bluetooth Core Specification v4.0 may be modified by one or more of Bluetooth Core Specification Annexes 2, 3, or 4. In various embodiments, IEEE 802.11-2012 may be supplemented by Draft IEEE Standard 802.11ac, Draft IEEE Standard 802.11ad, and / or Draft IEEE Standard 802.11ah.

[0203] Although the terms first, second, third, etc. can be used herein to describe various chips, modules, signals, elements and / or components, these items should not be limited by these terms. These terms can only be used to distinguish one item from another item. Unless the context clearly indicates, the terms such as "first", "second" and other numerical terms used in this article do not imply order or sequence. Therefore, without departing from the teaching of the exemplary example, the first item discussed below can be referred to as the second item.

[0204] Moreover, various terms are used to describe the physical relationship between components. When a first element is referred to as "connected to", "engaged to" or "coupled to" a second element, the first element may be directly connected, engaged, arranged, applied or coupled to the second element, or there may be an intermediate element. On the contrary, when an element is referred to as "directly connected to", "directly engaged to" or "directly coupled to" another element, there may not be an intermediate element. Stating that a first element is "connected to", "engaged to" or "coupled to" a second element means that the first element may be "directly connected to", "directly engaged to" or "directly coupled to" the second element. Other words (e.g., "between" and "directly between", "adjacent" and "directly adjacent", etc.) used to describe the relationship between elements should be interpreted in a similar manner.

[0205] The foregoing description is essentially only illustrative, and is in no way intended to limit the present disclosure, its application or use. The broad teachings of the present disclosure can be implemented in a variety of forms. Therefore, although the present disclosure includes specific examples, the true scope of the present disclosure should not be so limited, because after studying the drawings, the specification and the appended claims, other modifications will become apparent. As used herein, at least one of the phrases A, B and C should be interpreted as meaning the logic (A OR B OR C) using a non-exclusive logical OR, and should not be interpreted as meaning "at least one A, at least one B and at least one C". It should be understood that one or more steps in the method can be performed in different orders (or concurrently) without changing the principles of the present disclosure.

[0206] In this application including the following definitions, the term "module" or the term "controller" may be replaced by the term "circuit". The term "module" and the term "controller" may refer to, belong to or include: an application-specific integrated circuit (ASIC); a digital, analog or mixed analog / digital discrete circuit; a digital, analog or mixed analog / digital integrated circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor circuit (shared, dedicated or group) that executes code; a memory circuit (shared, dedicated or group) that stores code that can be executed by the processor circuit; other suitable hardware components that provide the above functions; or a combination of some or all of the above, such as in a system on a chip.

[0207] A module or controller may include one or more interface circuits. In some examples, the interface circuit may include a wired or wireless interface connected to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functions of any given module or controller of the present disclosure may be distributed between multiple modules and / or controllers connected via interface circuits. For example, multiple modules and / or controllers may allow load balancing. In another example, a server (also referred to as a remote or cloud) module or a (remote or cloud) controller may implement some functions on behalf of a client module and / or a client controller.

[0208] As used above, the term code may include software, firmware, and / or microcode, and may refer to a program, a routine, a function, a class, a data structure, and / or an object. The term shared processor circuit includes a single processor circuit that executes some or all of the code from multiple modules and / or controllers. The term group processor circuit includes a processor circuit that is combined with an additional processor circuit to execute some or all of the code from one or more modules and / or controllers. References to multiple processor circuits include multiple processor circuits on discrete die, multiple processor circuits on a single die, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above. The term shared memory circuit includes a single memory circuit that stores some or all of the code from multiple modules and / or controllers. The term group memory circuit includes a memory circuit that is combined with an additional memory to store some or all of the code from one or more modules and / or controllers.

[0209] The term memory circuit is a subset of the term computer-readable medium. As used herein, the term computer-readable medium does not include transient electrical or electromagnetic signals propagating through a medium (such as on a carrier wave); thus, the term computer-readable medium may be considered to be both tangible and non-transitory. Non-limiting examples of non-transitory tangible computer-readable media are non-volatile memory circuits (such as flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile memory circuits (such as static random access memory circuits or dynamic random access memory circuits), magnetic storage media (such as analog or digital magnetic tape or hard disk drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0210] The apparatus and method described in this application can be partially or completely implemented by a special-purpose computer created by configuring a general-purpose computer to perform one or more specific functions implemented in a computer program. The above-mentioned functional blocks and flow chart elements are used as software specifications, which can be converted into computer programs through routine work of technicians or programmers.

[0211] The computer program includes processor executable instructions stored on at least one non-transitory tangible computer readable medium. The computer program may also include or rely on stored data. The computer program may include a basic input / output system (BIOS) that interacts with the hardware of the special-purpose computer, device drivers that interact with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0212] A computer program may include: (i) descriptive text to be parsed, such as HTML (Hypertext Markup Language) or XML (Extensible Markup Language), (ii) assembly code, (iii) object code generated by a compiler from source code, (iv) source code executed by an interpreter, (v) source code compiled and executed by a just-in-time compiler, etc. By way of example only, source code may be written using syntax from the following languages: C, C++, C#, Objective C, Haskell, Go, SQL, R, Lisp, Fortran, Perl, Pascal, Curl, OCaml, HTML5, Ada, ASP (Active Server Pages), PHP, Scala, Eiffel, Smalltalk, Erlang, Ruby, Lua and

[0213] No element recited in a claim is intended to be a means-plus-function element within the meaning of 35 USC §112(f) unless the element is explicitly recited using the phrase “means for” or, in the case of a method claim, the phrase “for the operation of” or “for the operation of.”

[0214] U.S. Provisional Patent Application No. 62 / 686,333, filed on June 18, 2018, and entitled “Multi-Path or Multi-Stage Cache Improvement,” is incorporated herein by reference in its entirety, and the contents of the incorporated references, including the drawings, should be considered a part of this patent application.

Claims

1. A data storage and access system for use with a processor, comprising: a processor having a processor cache, the processor being configured to generate a data request for data; The FLC cache system is configured to be used as a main memory and receive the data request, and the FLC cache system includes: a first FLC module having a first FLC controller and a first memory, the first FLC module receiving the data request from the processor, wherein the first memory comprises SRAM or DRAM; a second FLC module having a second FLC controller and a second memory, the second FLC module receiving the data request generated by the processor from the first FLC module in response to the first FLC module not having the data requested by the processor, wherein the second memory comprises SRAM or DRAM; a storage drive configured with a user file storage portion and a main memory portion, such that the storage drive receives the data request from the second FLC module in response to the second FLC module not having the data requested by the processor; Wherein, the memory of the first FLC module is a high-speed memory and the memory of the second FLC module is a standard-speed memory, and the memory amount of the first FLC module is smaller than the memory amount of the second FLC module.

2. The system according to claim 1, wherein: The data request includes a physical address, and the first FLC controller includes a lookup table configured to translate the physical address into a first virtual address.

3. The system according to claim 2, wherein: If the lookup table of the first FLC controller does not contain the physical address, the first FLC controller is configured to forward the data request with the physical address to the second FLC controller.

4. The system according to claim 3, wherein: The second FLC controller includes a lookup table configured to translate the physical address into a second virtual address.

5. The system according to claim 4, wherein: If the lookup table of the second FLC controller does not contain the physical address, the second FLC controller is configured to forward the data request with the physical address to the storage drive.

6. The system according to claim 1, wherein: The first FLC module is faster than the second FLC module and has lower power consumption than the second FLC module.

7. The system according to claim 1, wherein: The first memory is a DRAM and includes a first memory controller, and the second memory is a DRAM and includes a second memory controller.

8. The system according to claim 1, wherein: The storage drive is a magnetic disk drive, a solid state memory, or a hybrid drive.

9. The system according to claim 1, wherein: The DRAM is a serial DRAM.

10. The system according to claim 1, wherein: The first FLC module has a speed at least twice that of the second FLC module.

11. The system according to claim 1, wherein: The first FLC module and the second FLC module are configured to perform predictive fetching of data stored at an address predicted to be accessed in the future.

12. A method of operating a data access system, wherein: The data access system includes a processor having a processor cache, a storage drive, a first FLC module including a first FLC controller and a first DRAM, and a second FLC module including a second FLC controller and a second DRAM, and the method includes: generating, using the processor, a request for data, the request for data comprising a physical address; providing the request for data to the first FLC module; determining whether the first FLC controller includes the physical address; in response to the first FLC controller containing the physical address, retrieving the data from the first DRAM and providing the data to the processor; in response to the first FLC controller not including the physical address, forwarding the request for data and the physical address to the second FLC module; determining whether the second FLC controller includes the physical address; and in response to the second FLC controller containing the physical address, retrieving a cache line containing the data from the second DRAM and providing the cache line to the first FLC module, Wherein, the memory of the first FLC module is a high-speed memory and the memory of the second FLC module is a standard-speed memory, and the memory amount of the first FLC module is smaller than the memory amount of the second FLC module.

13. The method according to claim 12, wherein: Determining whether the first FLC controller contains the physical address includes accessing an address cache storing address entries in the first FLC controller to reduce a time taken to make the determination.

14. The method according to claim 12, further comprising: In response to the second FLC controller not including the physical address, forwarding the request for data and the physical address to the storage drive.

15. The method according to claim 14, further comprising: The storage drive retrieves the data from a portion of the storage drive reserved as main memory and provides the data to the second FLC module.

16. The method according to claim 12, further comprising: In response to the first FLC controller including the physical address and providing the data to the processor, a status register is updated to reflect a recent use of a cache line including the data.

17. The method according to claim 12, wherein: Retrieving the data from the second DRAM and providing the data to the first FLC module further includes storing the physical address in the first FLC controller and storing the data in the first DRAM.

18. The method according to claim 17, wherein: If the first DRAM does not have available space, the least recently used data is removed from the first DRAM to make room for the data.

19. A data storage and access system for use with a processor, comprising: a processor having a processor cache, the processor configured to generate a first data request for first data and a second data request for second data; a first FLC cache system in communication with the processor, configured to function as a main memory cache and receive the first data request for the first data; a second FLC cache system in communication with the processor, configured to function as a main memory cache and receive the second data request for second data; a storage drive configured with a user file storage portion and a main memory partition such that the main memory partition of the storage drive is used as a main memory; and The memory of the first FLC cache system is a high-speed memory and the memory of the second FLC cache system is a standard-speed memory, and the memory amount of the first FLC cache system is smaller than the memory amount of the second FLC cache system.

20. The system of claim 19, further comprising a system bus, and the processor communicates with the first FLC cache system and the second FLC cache system through the system bus such that addresses assigned to each FLC cache system are interleaved.

21. The system of claim 19, wherein: If the first data is not contained in the first FLC cache system, the first data request is sent to the storage drive to retrieve the first data from a primary storage partition of the storage drive; and If the second data is not contained in the second FLC cache system, the second data request is sent to the storage drive to retrieve the second data from a primary storage partition of the storage drive.

22. The system of claim 19, wherein: The first FLC cache system includes a first FLC module and a second FLC module, and the second FLC cache system includes a third FLC module and a fourth FLC module.

23. The system of claim 22, wherein: Each of the FLC modules includes a FLC controller and a memory.

24. The system of claim 23, wherein: Each of the FLC modules maintains a dedicated and unique lookup table.

25. A data storage and access system for use with a processor, comprising: The FLC cache system is configured to be used as a main memory cache and receive a data request from a processing device, the processor device having a processor cache, the FLC cache system comprising: a first FLC module having a first FLC controller and a first cache memory, the first FLC module receiving the data request from the processor; A second FLC module having a second FLC controller and a second cache memory, wherein the second FLC module is configured as follows: In response to the first FLC module not having the data requested by the processor, the second FLC module receiving the data request from the first FLC module; and In response to the second FLC module not having data requested by the processor, the second FLC module sending the data request to a storage drive, the storage drive being configured with a portion serving as a main memory; wherein the first FLC controller comprises a lookup table configured to convert a physical address into a first virtual address, and wherein, if the lookup table of the first FLC controller does not contain the physical address, the first FLC controller is configured to forward the data request having the physical address to the second FLC controller; wherein the second FLC controller includes a lookup table configured to convert the physical address to a second virtual address, and wherein, if the lookup table of the second FLC controller does not contain the physical address, the second FLC controller is configured to forward the data request having the physical address to the storage drive; and Wherein, the memory of the first FLC module is a high-speed memory and the memory of the second FLC module is a standard-speed memory, and the memory amount of the first FLC module is smaller than the memory amount of the second FLC module.

26. The system of claim 25, further comprising a storage drive configured with a user file storage portion and a main memory portion, such that the storage drive receives the data request in response to the second FLC module and provides the requested data to the second FLC module.

27. The system of claim 25, wherein: The storage drive is accessed via a network connection.

28. The system of claim 25, wherein: The capacity of the second FLC module is at least eight times greater than the capacity of the first FLC module.

29. The system of claim 25, wherein: The first FLC module and the second FLC module store data in a cache line, and a size of the cache line is the same as an operating system page size.

30. The system of claim 25, wherein: The lookup table of the first FLC controller is arranged to be associated with a cache line size of at least 4 KB, and the lookup table of the second FLC controller is fully associative.

31. The system of claim 25, wherein: The first FLC module includes a first DRAM, and the second FLC module includes a second DRAM, and the first DRAM and the second DRAM are single-level cell type DRAMs.

32. The system of claim 31, wherein: The memory of the first FLC module is a DRAM and includes a first memory controller, and the memory of the second FLC module is a DRAM and includes a second memory controller.

33. The system of claim 25, wherein: The first FLC module and the second FLC module are configured to perform predictive fetching of data stored at an address predicted to be accessed in the future.

34. The system of claim 25, wherein: The first cache memory and the second cache memory include DRAM and are configured to cause one or more failed sections to be locked out from use.

35. The system of claim 25, wherein: The first cache memory and the second cache memory include DRAM and are configured to cause one or more sections of memory to be temporarily placed in an unavailable state during a health check of the remaining sections of memory.

Images