Memory device, method of operating the same, and method performed by compression

Abstract

A memory device, a method of operating the same, and a method performed by a compressed memory device are disclosed. The memory device performs at least one of determining whether a read request is hit in a cache in response to the read request received from a host, and performing operations for determining data corresponding to the read request in parallel in a memory processing unit. The memory device returns data obtained from the memory module by the memory processing unit to the host in a case where the cache is missed, and returns data hit in the cache to the host in a case where the cache is hit.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2024-0161477, filed on November 13, 2024, with the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field

[0002] The following description relates to memory devices and operating methods with prefetch acceleration technology. Background Technology

[0003] With the widespread adoption of applications requiring large amounts of storage, such as big data and / or machine learning, data centers increasingly need larger-capacity storage devices. To meet this need, research has been actively seeking devices that can scale main memory; however, expanding storage capacity by adding storage devices can increase the total cost of ownership (TCO) of data centers due to device purchase costs and additional power consumption. Compressed storage technology has attracted attention as a method to effectively reduce the TCO of data centers by increasing the overall storage density and utilization of storage devices. While compressed storage technology can improve the effective capacity of storage devices by compressing data and storing it, it may suffer from retrieval latency. Summary of the Invention

[0004] This summary is provided to present, in a simplified form, the selection of concepts further described below in the detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0005] In one general aspect, a memory device includes: a cache configured to make a cache hit determination in response to a read request from a host, the cache hit determination determining whether the read request results in a cache hit; a pre-request manager configured to provide a pre-request to a memory processing unit before completing the determination of whether the read request results in a cache hit, the pre-request being used to perform at least one of operations for determining data corresponding to the read request, wherein the pre-request corresponds to the read request or is a copy of the read request; and a memory processing unit configured to perform the at least one operation in response to receiving the pre-request, wherein, if the cache hit determination determines that a cache miss has occurred, data obtained from a memory module by the memory processing unit based on the pre-request is returned to the host, and if the cache hit determination determines that a cache hit has occurred, the data hit in the cache is returned to the host.

[0006] The pre-request manager can be configured to: in response to a cache hit determination that a cache hit has occurred for the read request, instruct the memory processing unit to abort operations following at least one of the operations.

[0007] The pre-request manager can be configured to: in response to a cache hit determination for the read request, provide a read request including an abort flag value to the memory processing unit, and the memory processing unit can be configured to: remove the response corresponding to the read request including the abort flag value from the memory response queue without further processing of the response to the read request including the abort flag value.

[0008] The pre-request manager can be configured to: in response to a cache hit determination for the read request, notify the memory processing unit to abort the read request, and the memory processing unit can be configured to: remove the read request from the memory request queue or invalidate the read request in the memory request queue.

[0009] The memory processing unit may be configured to perform operations following the at least one operation in response to a cache miss determination for the read request.

[0010] The at least one operation may include a data loading operation corresponding to the read request, the data loading operation loading data from the memory module, and the memory processing unit may be configured to start the data loading operation before the cache hit determination is completed.

[0011] The memory processing unit can be configured to: receive data corresponding to the read request from the memory module before a cache hit determination is completed; and, in response to a cache hit determination that a cache miss occurs for the read request, obtain data to be returned to the host based on the data loaded from the memory module.

[0012] The at least one operation may include an address translation operation for the read request, and the memory processing unit may be configured to initiate the address translation operation for the read request before a cache hit determination is completed.

[0013] The memory processing unit can be configured to: perform address translation for the read request before a cache hit determination is completed; and in response to a cache hit determination that a cache miss occurs for the read request, load data from the memory module using the translated address and use the loaded data to obtain data to be returned to the host.

[0014] The memory device may be configured to update the cache with data to be returned to the host in response to a cache hit determination that a cache miss has occurred for the read request.

[0015] In another general aspect, a method of operating a memory device includes: in response to a read request received from a host, making a cache hit determination for the read request; prior to completing the cache hit determination for the read request, providing a pre-request to a memory processing unit, the pre-request being used to perform at least one of operations for determining data corresponding to the read request, wherein the pre-request corresponds to the read request or is a copy of the read request; performing the at least one operation in response to receiving the pre-request; and, if the cache hit determination determines a cache miss, returning data obtained from a memory module by the memory processing unit to the host, and if the cache hit determination determines a cache hit, returning the data hit in the cache to the host.

[0016] The step of returning the data to the host may include: in response to a cache hit determination that a cache hit has occurred for the read request, instructing the memory processing unit to abort operations following the at least one operation.

[0017] The step of returning the data to the host may include: in response to a cache hit determination that the read request has a cache hit, providing a read request including an abort flag value to the memory processing unit; and removing the response corresponding to the read request including the abort flag value from the memory response queue without further processing of the response to the read request including the abort flag value.

[0018] The step of returning the data to the host may include: in response to a cache hit determination that a cache hit has occurred for the read request, notifying the memory processing unit to abort the read request; and removing the read request from the memory request queue of the memory processing unit or invalidating the read request in the memory request queue.

[0019] The step of returning the data to the host may include: in response to a cache miss determination that the cache for the read request was missed, performing operations following the at least one operation via the memory processing unit.

[0020] The at least one operation may include a data loading operation corresponding to the read request. The data loading operation may include loading data corresponding to the read request from the memory module. The step of providing a pre-request to the memory processing unit may include: starting the data loading operation through the memory processing unit before completing the cache hit determination.

[0021] The step of performing at least one operation in response to receiving a pre-request may include: receiving data corresponding to the read request from the memory module before completing a cache hit determination, and returning the data to the host. The step of returning the data to the host may include: in response to determining that a cache miss occurs for the read request in a cache hit determination, obtaining data to be returned to the host based on the data loaded from the memory module.

[0022] The at least one operation may include an address translation operation for the read request, and the step of providing a pre-request to the memory processing unit may include: initiating an address translation operation for the read request by the memory processing unit before completing a cache hit determination.

[0023] The steps of performing at least one operation in response to receiving a pre-request may include: performing address translation for the read request before completing a cache hit determination, and returning the data to the host. The steps of performing the at least one operation in response to a cache hit determination determining that the cache for the read request is not hit, loading data from the memory module using the translated address, and using the loaded data to obtain data to be returned to the host.

[0024] The step of returning the data to the host may include: in response to a cache hit determination that the cache for the read request was not hit, updating the cache with the data to be returned to the host.

[0025] In another general aspect, a method is performed by a compressed memory device including a cache and a memory module storing compressed data, and the method includes: receiving a read request for data from a host, the read request including a host address; determining, based on the read request, that a cache hit has occurred in the cache for the host address; initiating an operation to retrieve the data from the memory module based on receiving the read request and before completing the determination of the cache hit; and aborting or not completing the operation to retrieve the data from the memory module based on the determination that a cache hit has occurred.

[0026] The operation may include translating a host address into a memory address in a memory module.

[0027] The operation may include decompressing the requested data from the memory module.

[0028] Other features and aspects will become clear from the following detailed description, drawings and claims. Attached Figure Description

[0029] Figure 1 A computing system according to one or more embodiments is shown.

[0030] Figure 2 A memory device according to one or more embodiments is shown.

[0031] Figure 3 Examples of methods for operating a memory device according to one or more embodiments are shown.

[0032] Figure 4 Examples of operations in the case of a cache hit and in the case of a cache miss are shown according to one or more embodiments.

[0033] Figure 5 Examples of memory devices, wherein the memory processing unit is a compressor, are shown according to one or more embodiments.

[0034] Figure 6 and Figure 7 An example is shown of a memory device including a compressor, according to one or more embodiments, processing a response from a memory module based on whether a cache hit occurs.

[0035] Figure 8 and Figure 9 An example is shown of a memory device including a compressor, according to one or more embodiments, processing a request for a memory module based on whether a cache hit occurs.

[0036] Throughout the accompanying drawings and detailed embodiments, unless otherwise described or provided, the same or similar reference numerals will be understood to refer to the same or similar elements, features, and structures. The drawings may not be to scale, and for clarity, illustration, and convenience, the relative dimensions, scale, and depiction of elements in the drawings may be exaggerated. Detailed Implementation

[0037] The following detailed description is provided to assist the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become clear upon understanding the disclosure of this application. For example, the order of operations described herein is merely illustrative and is not limited to those orders of operations set forth herein, but may be changed as will become clear upon understanding the disclosure of this application, except for operations that must occur in a specific order. Furthermore, for greater clarity and conciseness, descriptions of features known upon understanding the disclosure of this application may be omitted.

[0038] The features described herein may be embodied in different forms and should not be construed as limited to the examples described herein. Rather, the examples provided herein are merely to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein that will become clear upon understanding the disclosure of this application.

[0039] The terminology used herein is for the purpose of describing various examples only and is not intended to limit this disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. As used herein, the term “and / or” includes any one of the associated listed items and any combination of any two or more. As a non-limiting example, the terms “comprising,” “including,” and “having” indicate the presence of the features, quantities, operations, components, elements, and / or combinations thereof stated, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0040] Throughout this specification, when a component or element is described as being "connected to," "joined to," or "attached to" another component or element, the component or element may be directly "connected to," "joined to," or "attached to" the other component or element, or one or more other components or elements may reasonably be present in between. When a component or element is described as being "directly connected to," "directly joined to," or "directly attached to" another component or element, there may be no other components or elements present in between. Similarly, expressions such as "between" and "immediately between," as well as "adjacent to" and "closely adjacent to," may also be interpreted as described above.

[0041] Although terms such as “first,” “second,” and “third,” or A, B, (a), (b), etc., may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts are not limited by these terms. Each of these terms is not used to define, for example, the nature, order, or sequence of the corresponding component, assembly, region, layer, or part, but only to distinguish the corresponding component, assembly, region, layer, or part from other components, assemblies, regions, layers, or parts. Therefore, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as the first component, first assembly, first region, first layer, or first part may also be referred to as the second component, second assembly, second region, second layer, or second part.

[0042] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains and based on the understanding of the disclosure of this application. Unless expressly defined herein, terms (such as those defined in common dictionaries) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant art and in the disclosure of this application, and shall not be interpreted in an idealized or overly formal sense. The use of the term “may” herein with respect to examples or embodiments (e.g., regarding what an example or embodiment may include or implement) indicates the existence of at least one example or embodiment that includes or implements such a feature, while not all examples are limited thereto.

[0043] Figure 1 A computing system according to one or more embodiments is shown.

[0044] The computing system 100 according to the example may include a host 150 and a memory device 110. For example, the memory device 110 may be decoupled from the host processor, for example via a bus, memory channel, memory-over-IP fabric, etc.

[0045] Host 150 (e.g., host central processing unit (CPU)) is the main management entity of computing system 100 (e.g., electronic device) and may be implemented as a host processor or a server. The host processor may include, for example, a host CPU. For instance, the host processor may include processor core 151 and memory controller (MC) 155. MC 155 may control memory device 110 (e.g., by executing a memory management unit, dedicated memory controller, etc.). MC 155 may send instructions to memory device 110. Furthermore, the host processor may use processor core 151 to process data received from memory device 110.

[0046] Memory device 110 can process data in a memory region (e.g., in memory module 113) as described below, in cooperation with a host processor. Memory device 110 can control the memory region in response to instructions from the host processor. Memory device 110 can process data based on instructions received from the host processor (e.g., read instructions or write instructions). According to the instructions, memory device 110 can read data from the memory region or write data to the memory region. To facilitate these operations, memory device 110 may include memory processing unit 111 and memory module 113.

[0047] Memory module 113 can store data. Memory module 113 may include block-based means that internally manage memory blocks forming memory regions. Memory regions may be physical regions that store data (in blocks) and can read data from and write data to the memory chips of physical memory device 110. Memory regions may be located in (or in the form of) memory dies (or core dies) of memory device 110. Memory blocks may reside on a portion or all of the memory chips of memory device 110. For example, each memory block may correspond to a memory bank, and memory blocks may be grouped by memory rank and / or memory channel. For example, a memory rank may be a group of memory chips (e.g., dynamic random access memory (DRAM) chips) connected to the same chip select and thus accessible simultaneously. A memory channel may be a group of memory chips accessible via the same channel (e.g., memory channel).

[0048] The memory processing unit 111 can perform specific operations that primarily serve to "return data corresponding to instructions received by the memory device 110 from the host 150 back to the host 150." The memory processing unit 111 can access the memory (e.g., memory blocks) of the memory module 113 and use values ​​recorded on the accessed memory blocks to perform specified operations (e.g., decompression or quantization). The memory processing unit 111 can be a component manufactured and / or implemented to include logic elements (e.g., logic circuits) that include logic for specific operations.

[0049] The host processor oversees the entire operation, while the memory device 110 can perform specific operations independently of the host. For example, when the memory device 110 receives a read request for compressed data from the host 150, a decompression operation may be required (performed by the memory device) to allow the memory device 110 to return the value requested by the read request. The memory processing unit 111 can perform the decompression operation as the aforementioned specific operation, and the memory device 110 can return the decompression result (e.g., decompressed data or decompressed value) to the host 150.

[0050] When the memory processing unit 111 is implemented as a near-memory processing (PNM, or near-memory processing) unit, a specific operation may be referred to as a PNM operation, and when the memory processing unit 111 is implemented as an in-memory processing (PIM, or in-memory processing) unit, a specific operation may be referred to as a PIM operation. A memory device 110 including PNM units may also be referred to as a PNM memory device, and a memory device 110 including PIM units may also be referred to as a PIM memory device. PNM memory devices or PIM memory devices may include, for example, accelerated dual in-line memory modules (AXDIMM), compute fast link-AXDIMM (CXL-AXDIMM), and CXL-disaggregated memory pools (DMP).

[0051] This description primarily focuses on an embodiment where the memory processing unit 111 is mounted on the memory device 110, but the example is not limited thereto. The memory processing unit 111 may be integrated with a host processor, memory device 110, or other devices. Furthermore, although Figure 1 The example shows an MC 155 included on the host 150 side, but the example is not limited thereto. Memory module 113 may include an MC (not shown), and host 150 may include a cache. Memory processing unit 111 (e.g., a PNM unit) may be located between host 150 and the MC (not shown) of memory module 113. The following... Figures 5 to 9 The memory processing unit described herein is an example of a compression device; however, the example is not limited to this. For example, the memory processing unit could be a quantization unit or an acceleration unit. Furthermore, although memory device 110 is primarily described herein, the techniques described with respect to memory device 110 can be applied to various types of peripheral devices including memory module 113 and memory processing unit 111. Memory processing unit 111 can be implemented integrated with a CXL switch, MC 155, and interface unit. (Refer to...) Figure 2 A more detailed description of the structure of the memory device 110 is provided, and reference is made to... Figure 3 Describe the operation of memory device 110.

[0052] As large-scale artificial intelligence (AI) applications become more widespread, the amount of memory required is rapidly increasing. When using distributed server systems to secure large amounts of memory, the total cost of ownership (TCO) can increase, and the performance of AI inference can degrade. Changing the size of memory capacity on individual servers is unavoidable, and current data centers spend a significant proportion (e.g., approximately 33%) of their total server construction cost on securing memory capacity. To construct low-cost, high-capacity memory in hyperscale vendors, CXL memory modules-DRAM (CMM-D) products can be used. In such CMM-D products, memory compression techniques can be introduced to ensure additional memory capacity for various applications, including, for example, the Open Compute Project (OCP). While memory compression techniques advantageously extend capacity, they also introduce additional latency due to the compression and decompression processing required for memory access. The implementation of the memory device 110 described herein can take measures in parallel with the operation of determining whether a cache hit has occurred for a host request to obtain decompressed data for the host request. Therefore, even when the requested data is compressed at a high compression ratio, the overall system latency can be reduced (that is, the latency caused by decompressing highly compressed data can be mitigated).

[0053] For example, the data in the memory itself may be stored in a compressed format, and the time required for decompression may delay memory access time. Decompression latency may include: (i) the cost of accessing a compression transformation table directory (e.g., metadata) containing information needed to decompress a specific data segment (the stored data segment may be compressed according to different compression parameters); (ii) the cost of retrieving compressed data from memory; and (iii) the cost of decompressing the retrieved data. Some architectures can hide the aforementioned decompression latency costs by setting up a cache in the middle. Three known memory mapping methods for using caches include: (i) direct mapping for a one-to-one mapping of memory addresses to cache blocks; (ii) set associative cache, in which cache blocks are configured as sets and the set of memory blocks is determined; and (iii) fully associative cache, in which memory blocks can be allocated in any set. Specifically, set-associative caches and fully associative caches can have significant latency in determining cache hits or cache misses. When caches are used in the manner described above in a mass storage system, there may be latency (e.g., tag matching latency) in determining cache hits or cache misses. The implementation of the memory device 110 described herein reduces tag matching latency.

[0054] Figure 2 A memory device according to one or more embodiments is shown.

[0055] In one example implementation, memory device 200 may receive an access instruction (e.g., a read request) from host 150. Memory device 200 may perform operations to return data (or a value) corresponding to the received access instruction. For example, memory device 200 may include memory processing unit 210, memory module 230, pre-request manager 250, and cache 270. Note that the received access instruction may be sent to both cache 270 and pre-request manager 250, as described below, which can reduce latency in fulfilling the access instruction (e.g., a read request). Cache 270 may be of any type mentioned above.

[0056] In parallel with the processing of access instructions by the pre-request manager 250, the cache 270 may determine whether a read instruction received from the host 150 is a cache hit in response to a read request.

[0057] Memory module 230 can be referenced as above. Figure 1 Data is stored as described in memory module 113. Memory module 230 may store data, for example, in compressed form.

[0058] The pre-request manager 250 may provide the memory processing unit 210 with a pre-request for performing at least one of the operations "typically to be performed to determine the data corresponding to the read request." The step of providing the pre-request to the memory processing unit 210 may be completed before "Cache 270 determines whether the read request has a cache hit." The pre-request may be a request provided to the memory processing unit 210 in advance before determining whether the read request results in a cache hit or cache miss in Cache 270. The pre-request may be, but is not limited to, the same instructions as, the read request received by Cache 270 from the host. According to an example, multiple operations may be associated with determining the data corresponding to the read request, and the memory processing unit receiving the pre-request may perform at least one of these operations (e.g., a partial operation), which may be performed independently of whether the same read request processed by Cache 270 results in a hit or a miss. At least one operation corresponding to the pre-request (among the operations associated with the read request) may also be referred to as a preliminary operation. Operations following the preliminary operation may also be referred to as subsequent operations. As described later, in the event of a corresponding cache hit in cache 270 processing a corresponding / double read request, subsequent operations are aborted, and this can be based on a memory request queue managed by the memory processing unit (e.g., Figure 6 The timing of read requests (e.g., the time when a request is sent to a memory module (e.g., memory module 230) or the time when a response is received from a memory module) in the memory request queue 663 shown in the diagram distinguishes between initial and subsequent operations. Which timing distinction is relevant may depend on the implementation or embodiment, as described below.

[0059] The pre-request manager 250 may include a queue for storing requests (e.g., Figure 6 The diagram shows a request queue 654 and registers for managing the request queue, including, for example, aborting requests in the memory request queue. The request queue may store requests for memory module 230. The control logic may include, for example, one or more logic circuits (e.g., request management logic) for managing (e.g., adding and / or removing) requests received from host 150, and one or more logic circuits (e.g., request abort logic) for aborting memory requests determined to correspond to cache hits. The request management logic and request abort logic may be implemented integratedly or separately.

[0060] The memory processing unit 210 may perform at least one operation in response to receiving a pre-request. As described above, the memory processing unit 210 may perform a portion of an operation to determine the data to be returned to the host 150 in response to a pre-request (e.g., a read request sent and received from the host before determining whether a cache miss has occurred). For example, one or more subsequent operations following at least one operation (e.g., a preliminary operation) corresponding to the pre-request may depend on the preliminary operation. To perform subsequent operations, it may be necessary to base them on the result data (or result value) of the preliminary operation.

[0061] For example, for the same data (or value), the address managed by host 150 (e.g., host address) and the address managed by memory device 200 (e.g., device address) may be different. Memory device 200 can translate the host address to a device address to access the actual location of the stored data in memory module 230 and load the data. For example, the device address may indicate the location of stored metadata within memory module 230, and memory device 200 can use the metadata to access the location of stored compressed data. Metadata may include information for obtaining requested data. For example, when requested data is compressed, metadata required to obtain the compressed data (e.g., compressed size and location of the compressed data, or compression parameters) may be requested. Memory device 200 may obtain the metadata for obtaining the compressed data before loading the compressed data.

[0062] As described later, the memory processing unit 210 may perform at least one operation in response to a pre-request: (i) address translation and metadata acquisition (see...) Figure 6 and Figure 7 ); or in another embodiment, (ii) only address translation (see Figure 8 and Figure 9 ).

[0063] The memory device 200 can return data obtained from the memory module 230 via the memory processing unit 210 to the host 150 in the event of a cache miss, and can return data that is hit in the cache 270 to the host 150 in the event of a cache hit. For example, in the event of a cache miss, the memory device 200 can obtain the (missed) requested data by performing a subsequent operation via the memory processing unit 210 after at least one operation corresponding to the pre-request has been performed (this pre-execution of data retrieval can reduce latency in the event of a cache miss). In this case, the memory device 200 can return data originating from the memory module 230 as requested data to the host 150. In the event of a cache hit, the memory device 200 can return data already included in the cache 270 (or data retrieved from the cache 270) to the host 150. Furthermore, in the event of a cache hit, the additional processing in the memory processing unit 210 becomes unnecessary, and therefore, operations following the operation corresponding to the pre-request (which would otherwise complete or assist in completing the requested data from the memory module 230) can be aborted.

[0064] For reference, the memory processing unit 210 may execute at least one operation according to a pre-requested operation in parallel and / or independently with the operation of determining whether a cache hit or cache miss has occurred in the cache 270. The at least one operation according to the pre-requested operation may be completed before the "cache hit determination operation" is completed, but is not limited thereto; according to an embodiment, the at least one operation according to the pre-requested operation may be completed after the "cache hit determination operation" is completed.

[0065] Pre-request management allows memory device 200 to minimize the tag matching latency described above with respect to host 150. For example, in memory device 200, memory processing unit 210 may receive access instructions in advance via pre-request manager 250 (the access instructions are also given to cache 270). When a request for memory module 230 needs to be sent to return requested data to host 150, according to the example memory device 200, at least one request may be sent in advance, or preparation for sending the request may be performed before the corresponding determination of cache miss (or cache hit) is completed. For example, when decompression is required to return data requested by host 150, address translation, obtaining metadata for decompression, obtaining compressed data using metadata, and / or decompression of compressed data may be performed as described above before cache hit / miss determination. Since memory device 200 can perform the determination of whether a cache hit has occurred in parallel with address translation and metadata acquisition (since many operations for decompressed data may have already been completed when cache hit / miss determination is performed), the latency required to return decompressed data can be significantly reduced. On the other hand, in existing memory systems (specifically, compressed memory systems), address translation and the like may only begin after a cache miss is determined to have occurred, and host 150 may have to wait a considerable amount of time until data is returned from memory.

[0066] Figure 3 Examples of methods for operating a memory device according to one or more embodiments are shown. Figure 4 Examples of operations in the case of a cache hit and in the case of a cache miss are shown according to one or more embodiments.

[0067] In operation 310, the memory device (e.g., memory device 200) may receive a read request from the host (e.g., host 150).

[0068] In operation 330, the memory device can determine whether a cache hit has occurred. For example, the memory device can search for the requested block (n greater than or equal to 1) in any of n cache lines within the cache memory of the cache. The memory device can compare the tag of the read request received from the host with the tags of the cache lines in the cache memory. The memory device can search for a cache line with a tag that matches the tag of the read request among the tags of the various cache lines. When a cache line with a tag that matches the tag of the read request is found, the memory device can check whether the data in that cache line is valid. When the data in the cache line with the matching tag is valid, the memory device (e.g., cache hit determination logic in the cache) can determine that a cache hit has occurred. When the tag does not match or when the data in the cache line with the matching tag is invalid, the memory device can determine that a cache miss has occurred.

[0069] In operation 350, the memory device may provide a pre-request corresponding to the read request of operation 310 to the memory processing unit (in some embodiments, the pre-request may be a copy or near copy of the read request received from the host in operation 310). In operation 370, the memory processing unit of the memory device may perform at least one operation in response to the pre-request. As described above, the at least one operation corresponding to the pre-request may be a subset of several operations that would normally be performed in the event of a cache miss to return the requested data to the host. That is, at least one operation may be a subset of operations that are normally performed by the memory device to serve the read request. Operations 350 and 370 described above may be performed independently of and / or in parallel with the operation in operation 330 determining whether a cache hit has occurred. The pre-request manager may provide the pre-request to the memory processing unit before completing the determination of whether a cache hit has occurred (in some types of caching schemes, the time for determining a cache hit / miss may be long).

[0070] In operation 390, the memory device can return data in a manner dependent on whether a cache hit occurred in operation 330. If a cache hit occurs, the memory device returns the requested data obtained from the cache. If a cache miss occurs in operation 330, data to be returned to the host is obtained from the memory module through preprocessing by the memory processing unit. For example, in operation 491 ( Figure 4 In this context, the memory device can determine whether a cache hit has occurred. As described below, the way the memory device returns the requested data can differ between cache hit and cache miss scenarios.

[0071] In operation 493, in the event of a cache hit, the memory device may abort pre-requesting. Specifically, when a cache hit occurs for a read request, the pre-request manager may command the memory processing unit to abort operations following at least one operation. Furthermore, in response to a cache hit, in operation 495, the memory device may return the requested data obtained from the cache.

[0072] In operation 499, when a cache miss occurs, the memory device may return the requested data obtained from the memory module or the processing result thereof to the host. For example, the data obtained from the memory module may be compressed data loaded from memory, and the processing result of the loaded compressed data may be uncompressed data or decompressed data. Furthermore, in response to a cache miss, the memory processing unit may perform other operations following at least one operation to obtain the requested data from the memory module. Additionally, in response to a cache miss, the memory device may update the cache with the data to be returned to the host.

[0073] Figure 5 Examples of memory devices, wherein the memory processing unit is a compressor, are shown according to one or more embodiments.

[0074] Figure 5 The memory device 500 shown may include, as Figure 2 The compression device 510 is an example of the memory processing unit 210 shown in the figure. Figure 2 The cache 270 can be implemented as a compressed memory cache 570.

[0075] Compression device 510 may include compressor 511, decompressor 513, and compression memory manager 515. In some embodiments, compressor 511 and decompressor 513 may be configured to implement a variety of compression algorithms.

[0076] Compressor 511 can compress data requested to be written. Decompressor 513 can decompress compressed data. Compressed memory manager 515 can request MC 535 to write compressed data to a specific address or read compressed data from a specific address. However, the example is not limited to this, and the configuration of compression device 510 is not limited to this, and may include components of logic elements (e.g., logic circuits) that implement at least one of decoding logic, decompression logic, encoding logic, and compression logic. The logic circuits included in the memory processing unit (e.g., compression device 510) may be implemented integrated with the buffer unit (e.g., buffer chip) of memory device 500.

[0077] MC 535 can, in response to a request from compression device 510, send data (or values) to be written to a specified address to memory module 230, or send data (or values) read from a specified address from memory module 230.

[0078] When the memory device 500 receives a read request for compressed data from the host, the compressed memory cache 570 can determine whether a cache hit has occurred. As described above, a pre-request manager 250 can send a pre-request to the compression device 510. While determining whether a cache hit has occurred in the compressed memory cache 570, the compression device 510 can perform preliminary operations in parallel (i.e., it can begin operations or preprocessing as if no cache hit will occur). For example, the compressed memory manager 515 can request metadata from the MC 535 for address translation and / or decompression. For example, the metadata may include the location and size of the compressed data.

[0079] When a cache hit occurs, the memory device 500 can return the data obtained from the compressed memory cache 570 to the host.

[0080] However, when a cache miss occurs, memory device 500 can perform subsequent operations to obtain decompressed data from memory module 230. When address translation and metadata acquisition are performed as preliminary operations, compression device 510 can use the metadata to obtain compressed data and perform decompression on the obtained data. When address translation is performed as a preliminary operation, compression device 510 can perform metadata acquisition, use the metadata to obtain compressed data, and decompress the obtained data. Memory device 500 can update the cache by inserting the decompressed data into a cache (e.g., compressed memory cache 570) and return the decompressed data to the host.

[0081] Figure 6 and Figure 7 An example is shown of a memory device including a compressor, according to one or more embodiments, processing a response from a memory module (e.g., memory module 230) based on whether a cache hit occurs.

[0082] In the event of a cache hit, the memory device may send a pre-request (e.g., a pre-received read request) to the memory module 230 to receive data, and then abort the request in the compression device 510 (the pre-request may be sent independently of the cache hit).

[0083] The compressed memory manager 515 may include a memory address translator 610, a memory manager, and a memory queue manager 660. The memory queue manager 660 manages a memory request queue 663 and a memory response queue 665.

[0084] The memory address converter 610 can translate the address of a read instruction. For example, the memory address converter 610 can translate the address of a read instruction (e.g., a host address) into an address indicating the location of the corresponding metadata stored within the memory module 230 (e.g., a memory address within the memory module 230).

[0085] As noted, the memory queue manager 660 manages memory queues. Memory queues may include a memory request queue 663 and a memory response queue 665. The memory request queue 663 may include an identifier (ID) and address (ADDR or ADDRESS) for each request. In addition to the ID and address corresponding to each request, the memory response queue 665 may also include data (or values) received from the memory module 230 as a response to a request.

[0086] The abort flag field 653 may be included in each request in the memory request queue 663 (e.g., response queue) and / or each response in the memory response queue 665 managed by the memory processing unit (e.g., compression device 510 (e.g., compressed memory manager 515)). The abort flag field 653 is a field whose value indicates whether the request to which it belongs is aborted, and may include a 1-bit value as an example. Figure 6 In the example shown, in addition to ID, address, and data, the memory response queue 665 may also include an abort flag field 653 in each of its entries.

[0087] The pre-request manager 250 may include request management logic 651, request abort logic 657, and request queue 654. For example, when a read request is received from the host, the request management logic 651 may update the request queue 654 with information about the read request. For example, the request management logic 651 may update the request queue 654 with the ID (REQUEST_ID) and address (REQUEST_ADDR) of the read request. The request queue 654, managed by the pre-request manager 250, may include an abort flag field 653 containing the corresponding request. The request abort logic 657 may receive a signal (TAG_MISS / HIT) indicating a cache hit or cache miss from the compressed memory cache 570. For example, in the compressed memory cache 570, cache hit determination logic 673 may determine whether the data (or value) corresponding to the read request received from the host is hit in the cache memory 671. The request abort logic 657 can set the bit value of the abort flag field 653 for each request in the request queue based on the corresponding signal indicating a cache hit or cache miss. For example, in the case of a cache hit, the bit value of the abort flag field 653 (e.g., the abort flag value) can be set to 1, and in the case of a cache miss, the bit value of the abort flag field 653 can be set to 0.

[0088] The pre-request manager 250 can be notified whether a read request has been hit or missed in cache memory 671. For example, a memory processing unit (e.g., compression device 510) can send a read request to memory module 230 and receive a response from memory module 230 in which data is added to the read request (or data is provided for the read request). The pre-request manager 250 can provide the memory processing unit with a flag value indicating whether the read request has been hit. The memory processing unit can set the flag value indicating whether the read request has been hit in the corresponding response in memory response queue 665. The queue management logic 661 of memory queue manager 660 can set the value of abort flag field 653 in the response entry with the corresponding ID and address in memory response queue 665 based on the ID and address of each request in the request queue of pre-request manager 250. As described above, for each response, a value of 1 can be set in abort flag field 653 when the corresponding read request cache hits, and a value of 0 can be set in abort flag field 653 when the corresponding read request cache misses. Figure 6 and Figure 7 The operation is described in an example where an entry in the response of the memory queue has an abort flag field 653.

[0089] For example, in operation 751, the pre-request manager 250 may send a read request to the compression device 510. As described above, pre-requests may be sent to the compression device 510 in parallel with (or independently of) determining whether a cache hit has occurred according to operation 330 (e.g., simultaneously with receiving a read instruction from the host or within a threshold time period after receiving the read instruction).

[0090] According to the example, the aforementioned "at least one operation corresponding to the pre-request" may include a data loading operation (corresponding to the read request) that loads data from memory module 230. The memory processing unit may begin the data loading operation before determining whether a cache hit has occurred (e.g., simultaneously with receiving a read instruction from the host). The memory processing unit may receive data (corresponding to the read request) from memory module 230 until the determination of whether a cache hit has occurred is completed. For example, as part of the data receiving step, in operation 771, compression device 510 may perform address translation. In operation 773, compression device 510 may send a read request to MC 535. In operation 775, MC 535 may request memory module 230 to read the corresponding address (e.g., an address indicating the location of metadata for the read request). In operation 777, MC 535 may load data (e.g., metadata).

[0091] In operation 493, the memory device may abort the request (or in other words, abort the activities related to the execution of the pre-request). When a read request triggers a cache hit, the operation following the pre-request may be aborted according to the example memory processing unit (e.g., compression device 510). For example, the pre-request manager 250 may provide the memory processing unit with a read request including an abort flag value (e.g., a bit value of 1 set / recorded in the abort flag field 653), and may do so in response to a cache hit that occurs for the read request. The memory processing unit may remove or ignore (in the memory response queue 665) the response corresponding to the read request including the abort flag value without further processing of the response corresponding to the read request including the abort flag value in the memory response queue 665. The memory queue manager 660 may, for example, abort the response when the value of the abort flag field 653 of any response in the memory response queue 665 indicates a value corresponding to abort (e.g., an abort flag value of 1).

[0092] For example, memory queue manager 660 may, in response to receiving / detecting a value (e.g., an abort flag value) from pre-request manager 250 corresponding to a cache hit for any request, remove the response corresponding to the abort flag value indicated by abort flag field 653 from memory response queue 665. Since the response is removed from memory response queue 665, any subsequent operations that would otherwise be performed associated with it may be aborted and not performed. In another example, memory queue manager 660 may not provide other devices and / or modules with a response that "has a value (e.g., an abort flag value) in abort flag field 653 corresponding to a cache hit." Since the transmission / processing of the response (e.g., data included in the response) is omitted, any subsequent operations that would otherwise be performed after the response may instead be aborted (not performed).

[0093] When a cache miss is determined for a read request, the memory processing unit can obtain the data to be returned to the host (or continue obtaining data) based on the data loaded from memory module 230. When a read request cache miss occurs, the memory processing unit can continue reading data from memory module 230 according to a queue (e.g., a response queue). When the abort flag of any response indicates another value (e.g., 0), the memory queue manager 660 can maintain the response so that subsequent operations are performed. For example, in operation 791, as part of a subsequent operation, compression device 510 can obtain metadata for decompression. In operation 793, compression device 510 can load compressed data using the metadata. The memory processing unit (e.g., compression device 510) can obtain compressed data by sending a read request to MC 535 based on an address (included in the metadata) indicating the storage location of the compressed data. In operation 795, compression device 510 can decompress the compressed data. In operation 797, a cache (e.g., compressed memory cache 570) can update a cache line. The cache can update the cache line with decompressed data. In Operation 799, the memory device can return the data corresponding to the read request to the host.

[0094] Figure 8 and Figure 9 An example is shown of a memory device including a compressor, according to one or more embodiments, processing a request for a memory module based on whether a cache hit occurs.

[0095] According to the example memory device, address translation may be performed only for pre-requested (e.g., pre-received read requests), and requests in compression device 510 may be aborted in the event of a cache hit. That is, the only operation that can be performed in advance, independent of cache hit determination, may be related to address translation (decompression, etc., may not be performed as part of processing pre-requests).

[0096] Figure 8 Most of the blocks of the compression device 510 shown in the figure are similar to those referenced above. Figure 6 Those blocks described. However, Figure 8 The memory queue manager 860 in the compressed memory manager 515 shown may differ from the one referenced above. Figure 6 The memory queue manager 660 is described. Figure 6 In the memory response queue 665, there is an abort flag field 653; however, in Figure 8 In contrast, the memory request queue 863 includes an abort flag field. For example, in the memory queue of the memory queue manager 860, each entry in the memory request queue 863 may include an ID, an address, and an abort flag field 653. The memory response queue 865 may include an ID, an address, and data (or values) received from the memory module 230 as a response to a request, but does not include an abort flag field. In some embodiments, both the memory request queue and the memory response queue 865 may include an abort flag; however, the memory device may operate in different modes; in one mode, the abort flag functions only in the memory response queue, while in another mode, the abort flag functions only in the memory request queue.

[0097] For example, in operation 951, the pre-request manager 250 may send a read request to the compression device 510. As described above, pre-requests can be sent to the compression device 510 in parallel with determining whether a cache hit has occurred according to operation 330.

[0098] According to the example, the aforementioned "at least one operation corresponding to the pre-request" may include an address translation operation for a read request. The memory processing unit may begin the address translation operation for the read request before (e.g., simultaneously with receiving a read instruction from the host) completing the determination of whether a cache hit has occurred. The memory processing unit may perform address translation corresponding to the read request until (or independently of) completing the determination of whether a cache hit has occurred. For example, in operation 971, the compression device 510 may perform address translation. The memory processing unit (e.g., compression device 510) may wait (or have an available) translated address (e.g., an address indicating the location of metadata) until a cache hit determination or a cache miss determination is made. The compression device 510 may maintain the transmission of the request to MC 535 until the determination of whether a cache hit has occurred is completed.

[0099] In operation 493, in the event of a cache hit, the memory device may abort the request (or pre-request). The pre-request manager 250, according to the example, may abort the read request in response to a cache hit occurring for a read request.

[0100] For example, the pre-request manager 250 may provide a read request, including an abort flag value, to the memory processing unit in response to a cache hit for a read request. The memory queue manager may abort a request when the abort flag field of any request in the memory request queue 863 has a value indicating abort. The memory queue manager may include queue management logic 861, which is used to process requests with an abort flag value in response to receiving a value (e.g., an abort flag value) from the pre-request manager 250 indicating a cache hit for any request. The queue management logic 861 may record the value of the abort flag field 653 in the memory request queue 863 for the request with the corresponding ID and address, based on the ID and address of each request in the request queue of the pre-request manager 250.

[0101] However, the examples are not limited to this. When the order of the memory request queue 863 of the compression device 510 (e.g., memory request order) is the same as the order in which the determination of whether a cache hit has occurred is completed, requests in the memory request queue 863 can be aborted without using the request queue of the pre-request manager. For example, the memory processing unit (e.g., the compression device 510) can abort or maintain each request in the memory request queue 863 based on the determination results of whether a cache hit has occurred received sequentially from the pre-request manager 250. When the queue management logic 861 receives the determination results indicating a cache hit in any order, the queue management logic 861 can abort the requests in the corresponding order in the memory request queue 863.

[0102] When it is determined that any read request should be aborted based on a cache hit, the memory processing unit may remove the read request from the memory request queue 863 (or deactivate the read request in the memory request queue 863). However, the abort method is not limited to this, and the memory processing unit may effectively invalidate the request by recording an abort flag value (e.g., 1) in the abort flag field of the request corresponding to the cache hit in the memory request queue 863. The memory processing unit may also abort the request by not sending a request with an abort flag value to MC 535.

[0103] When a cache miss is determined for a read request, the memory processing unit (e.g., a compression device) can load data (e.g., metadata) from memory module 230 using a translated address. The memory processing unit can use the loaded data (e.g., metadata) to obtain data to be returned to the host. The memory processing unit can use the metadata to load compressed data and obtain the data to be returned by decompressing the compressed data. For example, in operation 991, compression device 510 can send a read request to MC 535. In operation 992, MC 535 can request a read from memory module 230. As described above, the translated address can indicate the location of the metadata. In operation 993, MC 535 can load the metadata. As described above, the metadata can be used to obtain compressed data and can include an address indicating the location where the compressed data is stored. In operation 994, compression device 510 can load compressed data using the metadata. In operation 995, compression device 510 can decompress the compressed data. In operation 996, a cache (e.g., compressed memory cache 570) can use the decompressed data to update a cache line. In operation 997, the memory device may return the data corresponding to the read request (e.g., decompressed data) to the host.

[0104] As a reference, it has already been referenced. Figure 6 and Figure 7 This describes aborting in the memory response queue in the event of a cache hit, and has been referenced. Figure 8 and Figure 9 Aborts in the memory request queue are described; however, aborts in the memory response queue and in the memory request queue can be selectively performed depending on the design. For example, in memory queue managers 660, 860, memory request queues 663, 863 and memory response queues 665, 865 may each include an abort flag field. A memory processing unit (e.g., compression device 510) may selectively abort (e.g., remove) each request in memory request queues 663, 863 or memory response queues 665, 865. For example, the memory processing unit may pre-fetch data for requests requiring less latency and then abort (e.g., remove / invalidate) requests in memory response queues 665, 865 in response to a cache hit. The memory processing unit may perform address translation only for requests requiring relatively less stringent latency and then abort (e.g., remove) requests in memory request queues 663, 863 in response to a cache hit.

[0105] As a non-limiting example, the memory device according to the example can be integrated into a CXL system (e.g., CMM-D). For example, in a compressed memory system used in CMM-D, access instructions can be sent from the host to the memory device via a CXL interface. The latency consumed in the compressed memory cache 570 and the compression device 510 can average approximately 50% of the latency required to process a read instruction for compressed data and return the data. (Referring to the above...) Figures 1 to 9 The described operation reduces the time spent by the memory device preparing decompressed data (decompression itself is reduced by approximately 10%). Therefore, the total time can be reduced by an average of approximately 5%. In a compressed memory system, the effective capacity can be increased by an average of more than three times, while latency is effectively reduced. A memory device according to the example described herein can provide a compressed memory system with reduced latency and high compression efficiency for reading without introducing complex hardware.

[0106] The examples described herein can be implemented using hardware components, software components, and / or combinations thereof. The processing device can be implemented using one or more general-purpose or special-purpose computers, such as processors, controllers and arithmetic logic units (ALUs), DSPs, microcomputers, FPGAs, programmable logic units (PLUs), microprocessors, or any other device capable of responding to and executing instructions in a defined manner. The processing device can run an operating system and one or more software applications running on the operating system. The processing device can also access, store, manipulate, process, and create data in response to the execution of the software. For simplicity, the description of the processing device is used as the singular; however, those skilled in the art will understand that the processing device can include multiple processing elements and / or various types of processing elements. For example, the processing device can include multiple processors, or a single processor and a single controller. Furthermore, different processing configurations are feasible (such as parallel processors).

[0107] Software may include computer programs, code segments (e.g., microcode), instructions, or combinations thereof, to independently or uniformly command or configure a processing device to operate as needed. Software and data may be stored in any type of machine, component, physical or virtual device, or computer storage medium or device capable of providing instructions or data to or interpreting the processing device. Software may also be distributed across networked computer systems, enabling it to be stored and executed in a distributed manner. Software and data may be stored on one or more non-transitory computer-readable recording media.

[0108] The methods described in the examples above can be recorded in a non-transitory computer-readable medium containing program instructions to implement the various operations described above. The medium may also include data files, data structures, etc., alone or in combination with the program instructions. The program instructions recorded on the medium may be program instructions specifically designed and constructed for the purposes of the examples, or they may be of types known and available to those skilled in the art of computer software. Examples of non-transitory computer-readable media include magnetic media (such as hard disks, floppy disks, and magnetic tapes); optical media (such as CD-ROMs and / or DVDs); magneto-optical media (such as optical discs); and hardware devices specifically configured to store and execute program instructions (such as read-only memory (ROM), random access memory (RAM), flash memory, etc. (but not the signals themselves)). Examples of program instructions include both machine code generated by a compiler and files containing high-level code that can be executed by a computer using an interpreter.

[0109] The aforementioned hardware device can be configured to act as one or more software modules to perform the operations described above, or vice versa.

[0110] Regarding Figures 1 to 9The described computing devices, electronic devices, processors, memories, controllers, information output systems and hardware, storage devices, and other devices, apparatuses, units, modules, and components are implemented by or represent hardware components. Examples of hardware components that can be used to perform the operations described in this application include, where appropriate, controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more of the hardware components performing the operations described in this application are implemented by computing hardware (e.g., by one or more processors or computers). A processor or computer may be implemented by one or more processing elements, such as logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field-programmable gate arrays, programmable logic arrays, microprocessors, or any other means or combination of means configured to respond to and execute instructions in a defined manner to achieve a desired result. In one example, the processor or computer includes or is connected to one or more memories storing instructions or software executed by the processor or computer. Hardware components implemented by a processor or computer can execute instructions or software (such as an operating system (OS) and one or more software applications running on the OS) for performing the operations described in this application. The hardware component can also access, manipulate, process, create, and store data in response to the execution of instructions or software. For simplicity, the singular terms "processor" or "computer" may be used in the description of the examples described in this application; however, in other examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements, or multiple types of processing elements, or both. For example, a single hardware component or two or more hardware components may be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors, or another processor and another controller. One or more processors or a processor and a controller may implement a single hardware component, or two or more hardware components. The hardware components can have any one or more different processing configurations, examples of which include a single processor, a discrete processor, a parallel processor, a single instruction single data (SISD) multiprocessing, a single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.

[0111] Figures 1 to 9The methods for performing the operations described in this application, as shown, are executed by computing hardware (e.g., one or more processors or a computer), which is implemented to execute instructions or software as described above to perform the operations performed by the methods described in this application. For example, a single operation or two or more operations may be performed by a single processor, or two or more processors, or a processor and a controller. One or more operations may be performed by one or more processors, or a processor and a controller, and one or more other operations may be performed by one or more other processors, or another processor and another controller. One or more processors or a processor and a controller may perform a single operation, or two or more operations.

[0112] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above may be written as computer programs, code segments, instructions, or any combination thereof to individually or collectively instruct or configure one or more processors or computers to operate as a machine or special-purpose computer to perform operations performed by the hardware components and methods described above. In one example, the instructions or software include machine code (such as machine code generated by a compiler) that is directly executed by one or more processors or computers. In another example, the instructions or software include high-level code that is executed by one or more processors or computers using an interpreter. The instructions or software may be written using any programming language based on the block diagrams and flowcharts shown in the accompanying drawings and the corresponding description herein, which disclose algorithms for performing operations performed by the hardware components and methods described above.

[0113] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above, as well as any associated data, data files, and data structures, may be recorded, stored, or fixed in, or may be recorded, stored, or fixed on, one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage devices, hard disk drives (HDDs), solid-state drives (SSDs), flash memory, card storage (such as multimedia cards or microcards (e.g., Secure Digital (SD) or Extreme Digital (XD))), magnetic tape, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid-state drives, and any other means configured to store instructions or software and any associated data, data files, and data structures in a non-transitory manner and to provide instructions or software and any associated data, data files, and data structures to one or more processors or computers such that one or more processors or computers can execute the instructions. In one example, the instructions or software and any associated data, data files, and data structures are distributed across a networked computer system, such that the instructions and software and any associated data, data files, and data structures are stored, accessed, and executed by one or more processors or computers in a distributed manner.

[0114] While this disclosure includes specific examples, it will be clear upon understanding this disclosure that various changes in form and detail may be made in these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein are to be considered descriptive only and not for limiting purposes. The description of features or aspects in each example is to be considered applicable to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order, and / or if components in the described system, architecture, apparatus, or circuit are combined in a different manner, and / or replaced or supplemented by other components or their equivalents.

[0115] Therefore, in addition to the above disclosure, the scope of this disclosure may also be defined by the claims and their equivalents, and all variations within the scope of the claims and their equivalents should be interpreted as being included in this disclosure.

Claims

1. A memory device, comprising: The cache is configured to make a cache hit determination in response to a read request from the host, the cache hit determination determining whether the read request results in a cache hit; A pre-request manager is configured to provide a pre-request to the memory processing unit before determining whether a read request results in a cache hit. The pre-request is used to perform at least one of the operations for determining data corresponding to the read request, wherein the pre-request corresponds to or is a copy of the read request; and The memory processing unit is configured to perform at least one of the operations in response to receiving a pre-request. In cases where a cache miss has occurred, data obtained from the memory module via the memory processing unit based on a pre-request is returned to the host. In cases where a cache miss has occurred, the data in the cache is returned to the host.

2. The memory device according to claim 1, wherein, The pre-request manager is configured to: in response to a cache hit determination for the read request, instruct the memory processing unit to suspend operations following the at least one operation.

3. The memory device according to claim 1, wherein, The pre-request manager is configured to: in response to a cache hit determination for the read request, provide a read request including an abort flag value to the memory processing unit, and The memory processing unit is configured to remove responses from the memory response queue that correspond to read requests including a stop flag value, without requiring further processing of the responses to read requests including a stop flag value.

4. The memory device according to claim 1, wherein, The pre-request manager is configured to: in response to a cache hit determination for the read request, notify the memory processing unit to abort the read request, and The memory processing unit is configured to either remove the read request from the memory request queue or invalidate the read request in the memory request queue.

5. The memory device according to claim 1, wherein, The memory processing unit is configured to: in response to a cache miss determination for the read request, perform operations following the at least one operation.

6. The memory device according to claim 1, wherein, The at least one operation includes a data loading operation corresponding to the read request, the data loading operation loading data from the memory module, and The memory processing unit is configured to begin the data loading operation before a cache hit determination is completed.

7. The memory device according to claim 1, wherein, The memory processing unit is configured as follows: Before a cache hit is determined, data corresponding to the read request is received from the memory module; and In response to a cache hit determination that a cache miss has occurred for the read request, data to be returned to the host is obtained based on the data loaded from the memory module.

8. The memory device according to claim 1, wherein, The at least one operation includes an address translation operation for the read request, and The memory processing unit is configured to initiate an address translation operation for the read request before a cache hit determination is completed.

9. The memory device according to claim 1, wherein, The memory processing unit is configured as follows: Before a cache hit is determined, the address translation corresponding to the read request is performed; and In response to a cache miss determination that the cache for the read request was not found, data is loaded from the memory module using the translated address and the loaded data is used to obtain the data to be returned to the host.

10. The memory device according to claim 1, wherein, In response to a cache hit determination that the cache for the read request was not hit, the cache will update the data to be returned to the host in the cache.

11. A method of operating a memory device, the method comprising: In response to a read request received from the host, a cache hit determination is made for the read request; Before completing the cache hit determination for the read request, a pre-request is provided to the memory processing unit. The pre-request is used to perform at least one of the operations for determining the data corresponding to the read request, wherein the pre-request corresponds to the read request or is a copy of the read request. Perform the at least one operation in response to receiving a pre-request; and If a cache miss is confirmed, the data obtained from the memory module through the memory processing unit will be returned to the host. If a cache miss is confirmed, the data in the cache will be returned to the host.

12. The method according to claim 11, wherein, The step of returning data that has been hit in the cache to the host includes: in response to a cache hit determination that a cache hit has been determined for the read request, instructing the memory processing unit to suspend operations following the at least one operation.

13. The method according to claim 11, wherein, The steps to return data that has been hit in the cache to the host include: In response to a cache hit determination that a cache hit has occurred for the read request, a read request including an abort flag value is provided to the memory processing unit; and Remove the response corresponding to the read request that includes the abort flag value from the memory response queue without further processing of the response corresponding to the read request that includes the abort flag value.

14. The method according to claim 11, wherein, The steps to return data that has been hit in the cache to the host include: In response to a cache hit determination that a cache hit has occurred for the read request, the memory processing unit is notified to abort the read request; and Remove the read request from the memory request queue of the memory processing unit or invalidate the read request in the memory request queue.

15. The method according to claim 11, wherein, The steps for returning data obtained from the memory module to the host include: In response to a cache miss determination that the cache was not hit for the read request, the operation following the at least one operation is performed by the memory processing unit.

16. The method according to claim 11, wherein, The at least one operation includes a data loading operation corresponding to the read request, the data loading operation including loading data corresponding to the read request from the memory module, and The steps of providing a pre-request to the memory processing unit include: initiating a data loading operation through the memory processing unit before a cache hit determination is completed.

17. The method according to claim 11, wherein, The step of performing at least one operation in response to receiving a pre-request includes: receiving data corresponding to the read request from the memory module before completing a cache hit determination, and The steps of returning data obtained from the memory module to the host include: in response to a cache hit determination that a cache miss occurred for the read request, obtaining data to be returned to the host based on the data loaded from the memory module.

18. The method according to claim 11, wherein, The at least one operation includes an address translation operation for the read request, and The step of providing a pre-request to the memory processing unit includes: initiating an address translation operation for the read request via the memory processing unit before a cache hit determination is completed.

19. The method according to claim 11, wherein, The step of performing at least one operation in response to receiving a pre-request includes: performing address translation corresponding to the read request before completing cache hit determination, and The steps for returning data obtained from the memory module to the host include: in response to a cache hit determination that the cache miss for the read request is determined, loading data from the memory module using the translated address, and using the loaded data to obtain the data to be returned to the host.

20. The method according to claim 11, wherein, The steps of returning data obtained from the memory module to the host include: in response to a cache miss determination that the cache for the read request was missed, updating the cache with the data to be returned to the host.

21. A method performed by a compression storage device, the compression storage device including a cache and a memory module storing compressed data, the method comprising: Receive a read request for data from the host, the read request including the host address; Based on the read request, a determination is made that a cache hit has occurred for the host address in the cache. Based on receiving the read request and before confirming the cache hit, an operation to retrieve the data from the memory module is initiated. as well as Based on the determination that a cache hit has occurred, the operation of retrieving the data from the memory module may be aborted or not completed.

22. The method according to claim 21, wherein, The operation includes translating the host address into a memory address in the memory module.

23. The method according to claim 21, wherein, The operation includes decompressing the requested data from the memory module.

Images