Systems, methods and apparatus for eliminating copies and value redundancy in computer memory

Abstract

The computer memory compression method involves analyzing (1210) computer memory content with respect to the presence of duplicate memory objects and value redundancy of data values ​​in unique memory objects. The computer memory content is encoded (1220) by eliminating duplicate memory objects and compressing each remaining unique memory object by exploiting the data value locality of the data values ​​of each remaining unique memory object. Metadata (500) representing memory objects of the encoded computer memory content is provided (1230). The metadata reflects the eliminated duplicate memory objects, the remaining unique memory objects, and the type of compression used to compress each remaining unique memory object. The memory objects are located (1240) in the encoded computer memory content using the metadata (500).

Description

Technical Field

[0001] The subject matter relates generally to the field of data compression in memory in electronic computers. Background Art

[0002] Data compression is a general technique for storing and transmitting data more efficiently by encoding frequent data sets more efficiently than infrequent data sets. For some reasons, it is interesting to store and transmit data more efficiently in general. In a computer memory, for example, a memory that retains data and computer instructions for processing device operations, such as a main memory or cache memory, it is interesting to store the data more efficiently, for example, K times, because it can then potentially reduce the size of the memory by K times, use the communication capacity that is potentially reduced by K times to transmit data between one memory and another memory, and potentially reduce the energy consumption of storing and transmitting the data within or between computer systems and / or between memories by K times. Alternatively, people can store data that is potentially increased by K times in available computer memory compared to without data compression. This is interesting for achieving a computer's potentially K times higher performance without having to add more memory, which may be expensive or may just be less than ideal due to resource limitations. As another example, the size and weight of a smartphone, tablet, laptop / desktop, or set-top box may be limited because a larger or heavier smartphone, tablet, laptop / desktop, or set-top box has less value to the end user; thus potentially reducing the market value of such products. However, more memory capacity or higher memory communication bandwidth can potentially increase the market value of the product because more memory capacity or memory communication bandwidth can result in higher performance, thereby increasing the utility of the product.

[0003] In summary, in the overall landscape of computerized products including standalone devices or interconnected devices, data compression can potentially improve performance, reduce energy consumption, increase memory communication bandwidth, or reduce the cost and area consumed by memory. Thus, data compression has broad utility in a wide range of computerized products other than those mentioned herein.

[0004] Prior art compressed memory systems typically compress a memory page as it is created, either by reading it from disk or by memory allocation. Compression can be accomplished using a variety of well-known methods, either by software routines or by hardware accelerators. When a processor requests data from memory, the data must typically first be decompressed before servicing the requesting processor. Since such requests may end up on a critical memory access path, decompression is typically hardware accelerated to exert less impact on memory access time.

[0005] In a compression method called deduplication, the idea is to identify identical memory objects. For example, let's assume that the memory contains five identical instances of the same page. Then, only one of them needs to be saved, and the remaining four can refer to this unique instance; thus, a five-fold compression ratio is provided. Deduplication known in the prior art has been applied to fixed-size objects of a range of granularities, such as memory pages that are typically about a few kilobytes to tens of kilobytes (KB) or even more in size, and memory blocks that are typically tens of bytes, such as 64 bytes (64B) in size. Other prior art considers variable granular sizes, such as variable-sized storage files. In any case, the limitation of deduplication is that it is only based on removing copies (duplications) of the same memory object.

[0006] When removing identical objects, the removed object must establish a reference to the unique object that is identical to it. In terms of pointers, the reference points to the only copy of the memory object, and this consumes memory space. Therefore, deduplication will cause a lot of compression metadata overhead. For example, let's assume that deduplication is applied to 1Terabyte=2 40 64B (=2 6 Then, a reference pointer of (40-6=)34 bits is needed to point to the unique copy of the deduplicated memory block.

[0007] Alternative compression methods known in the prior art take advantage of value redundancy (e.g., 32 or 64 bits in terms of single words). For example, a memory object that is more common than another memory object will be encoded using fewer bits than a less common memory object. As an example, entropy-based compression techniques abound in the prior art, including, for example, Huffman coding and arithmetic coding. Other compression techniques include Base-Delta-Immediate compression, which takes advantage of the fact that the values ​​stored in data objects, such as memory pages and memory blocks, are numerically close to each other and densely encodes the differences between them.

[0008] Importantly, deduplication that removes duplicates and compression techniques that exploit value locality that removes value redundancy, such as entropy-based compression and base-delta on-the-fly compression, are complementary in many respects. Consider, for example, page-based deduplication, where a single copy of the same page is stored and reference pointers are provided from the copy to reference the unique copy. However, such deduplication schemes do not exploit value redundancy that exists at a finer granularity, such as value redundancy at the word level (e.g., 32 or 64 bit entities) that exists within a page. By combining deduplication with a compression scheme that reduces value redundancy, duplicates can be eliminated and the remaining unique copies can be stored more densely by encoding each data value within the unique copy based on its statistical value properties. This document is intended to disclose an invention that provides devices, systems, and methods for a family of compression techniques applied to computer memory to eliminate duplicates and value redundancy.

[0009] Combining deduplication with value locality-based compression presents a number of technical challenges. The first challenge is how to find an encoding that provides a combined gain in compressibility by removing duplicates and compressing items in the remaining unique copies using a value locality-based approach. Using a combined approach of deduplication and value locality-based compression to efficiently locate memory blocks in a compressed memory presents the challenge of keeping the amount of metadata low and allowing compression and decompression devices to impose a low memory latency overhead. Therefore, the second challenge is to propose compression and decompression methods, devices, and systems that can keep the amount of metadata low and impose a low memory latency overhead. In operation, data objects will change in response to processor writes. This causes the properties of duplicate blocks and unique blocks to change; both in terms of the number of copies and the statistical properties of the value locality of the remaining unique copies. The third challenge is to provide methods, devices, and systems that can maintain high compressibility based on this dynamic effect. The disclosed invention is intended to address all of these and other challenges. Summary of the invention

[0010] A first aspect of the present invention is a method for compressing computer memory. The method includes analyzing computer memory contents with respect to the presence of duplicate memory objects and value redundancy of data values ​​in unique memory objects. The method also includes encoding the computer memory contents by eliminating the duplicate memory objects and compressing each remaining unique memory object by exploiting the data value locality of the data values ​​of each remaining unique memory object. The method also includes providing metadata of the memory object representing the encoded computer memory contents. The metadata reflects the duplicate memory objects eliminated, the remaining unique memory objects, and the compression type used to compress each remaining unique memory object. The method also includes locating the memory object in the encoded computer memory contents using the metadata.

[0011] A second aspect of the present invention is a computer memory compression device. The device includes an analyzer unit configured to analyze computer memory content with respect to the occurrence of replica memory objects and value redundancy of data values ​​in unique memory objects. The device also includes an encoder unit configured to encode the computer memory content by eliminating the replica memory objects and compressing each remaining unique memory object by utilizing the data value locality of the data value of each remaining unique memory object. The encoder unit is also configured to provide metadata of the memory object representing the encoded computer memory content. The metadata reflects the eliminated replica memory objects, the remaining unique memory objects, and the compression type used to compress each remaining unique memory object. The device also includes a locator unit configured to locate the memory object in the encoded computer memory content using the metadata.

[0012] Further aspects as well as objects, features and advantages of the disclosed embodiments will appear from the following detailed disclosure, from the attached dependent claims as well as from the accompanying drawings.

[0013] In general, compression by exploiting locality of data values ​​as described in this document may involve entropy-based encoding, delta encoding, dictionary-based encoding, or pattern-based encoding, without limitation.

[0014] Generally, all terms used in the claims should be interpreted according to their ordinary meaning in the technical field, unless otherwise explicitly defined herein. All references to "a / an / the [element, device, part, means, step, etc.]" should be openly interpreted as referring to at least one instance of the element, device, part, means, step, etc., unless otherwise explicitly stated. Unless explicitly stated, the steps of any method disclosed herein do not have to be performed in the exact order disclosed. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A computer system is described that includes a microprocessor chip having one or more processing units, an exemplary three-level cache hierarchy, and one or more memory controllers connected to one or more off-chip memories.

[0016] Figure 2 A computer system configured to compress data and instructions in memory, such as a Figure 1 Computer systems in.

[0017] Figure 3 Exemplary memory page layouts before and after application of deduplication and value redundancy removal compression methods are described.

[0018] Figure 4An exemplary layout of a tree-based data structure for keeping track of unique blocks within a page is described.

[0019] Figure 5 Examples of metadata structures and their use for compressing memory pages using a combination of deduplication and other compression methods are described.

[0020] Figure 6 A flowchart describing the deduplication process.

[0021] Figure 7 A flow chart of a value redundancy removal compression method is described.

[0022] Figure 8 Changes in an exemplary memory in response to modifications of a unique memory block are described.

[0023] Fig. 9 Changes in an exemplary memory in response to modifications of a deduplicated memory block are described.

[0024] Fig.10 An address translation unit is described for determining the location of a memory block using a deduplication and value redundancy removal compression method.

[0025] Fig.11 A decompressor unit is described for decompressing a memory block using deduplication and value redundancy removal compression.

[0026] Fig.12 A general flow chart of a computer memory compression method is described. DETAILED DESCRIPTION

[0027] This document discloses systems, methods, and apparatus for compressing data in computer memory using a series of compression methods that eliminate copies and redundancy of values ​​in computer memory.

[0028] Figure 1 An exemplary embodiment of a computer system 100 is described in 1 151.M 2 152 to M K The microprocessor chip may be a discrete system or integrated on a system on chip (SoC) using any available technology. The microprocessor 110 includes a processor, which is represented by P 1 131.P 2 132 to P N 133. One or more processing units, sometimes referred to as CPUs or cores, and a memory hierarchy. On the other hand, the memory hierarchy includes several cache levels, such as Figure 1100 and are exemplarily shown in and denoted as C1, C2, and C3. These tiers may be implemented with the same or different memory technologies such as SRAM, DRAM, or any type of non-volatile technology including, for example, phase change memory (PCM). In different embodiments, the number of cache tiers may vary, and exemplary embodiment 100 depicts three tiers, with the last cache tier being C3 120. The tiers are connected using some interconnection means such as a bus or any other interconnection network. In the exemplary embodiment, tiers C1 and C2 are connected to a memory tier denoted as P i The corresponding processing unit i (e.g. Figure 1 P 1 ) is private and can only be accessed by it. It is well known to those skilled in the art that alternative embodiments may have any number of private cache levels, or alternatively, all cache levels are shared, such as Figure 1 120 in the third level C3 120. Any implementation is possible with respect to including data in the cache hierarchy and can be understood by those skilled in the art. For example, C1 can be included in C2, while C2 can be excluded from level C3. Those skilled in the art can understand alternative implementations. Figure 1 The computer system 100 includes a MCTRL 1 141.MCTRL 2 142 to MCTRL L One or more memory controllers of 143. The last cache level ( Figure 1 The memory controller is connected to the memory module. The memory controller can be integrated on the microprocessor chip 110, or it can be implemented outside the microprocessor chip. Finally, the computer system runs one or more tasks. A task can be any software application or part thereof that can be run on a specific system.

[0029] by Figure 1 As an example of the implementation method in the embodiment, the computer system may be subject to M 1 151 to M K 153 due to the limited capacity of the memory and the limited capacity of the cache memory, no matter which level (e.g. Figure 1C1, C2 and C3 in ). Limited cache capacity can manifest itself as a higher proportion of memory requests that must be served at the next level in the memory hierarchy, resulting in performance loss or higher energy consumption. To alleviate this problem, we can consider increasing the cache capacity, thereby reducing the number of requests that need to be served by the next level of the memory hierarchy. Increasing the capacity of the cache hierarchy on a microprocessor chip will cause some problems. First, the cache access request time may increase, resulting in performance loss. Second, the energy consumed by access requests to larger caches can be potentially higher. Third, using more silicon or equivalent materials on the microprocessor chip to implement a larger cache hierarchy may have to be exchanged for less processing power. Therefore, it is expected to achieve more cache capacity without the above problems. Limited memory capacity has similar problems, and it can manifest itself as more memory requests that must be served at the next level of the memory hierarchy, which is usually implemented as a storage level of the memory hierarchy. Access to this storage level is slower and may cause huge losses in performance and energy consumption. Increasing memory capacity can alleviate these shortcomings. However, more memory capacity increases the cost of the computer system, both at the component level and in terms of energy consumption. In addition, more memory consumes more space, which may limit the usefulness of the computer system, especially in products with limited form factors, such as mobile computers (e.g., tablets, smartphones, wearable devices, and small computerized devices connected to the Internet).

[0030] The present disclosure contemplates several implementations that differ in which level of the exemplary memory hierarchy described above the compression is applied. The first implementation contemplates the invented compression method being applied at the main memory. However, other implementations may be understood by those skilled in the art. Such implementations are intended to be contemplated, but are not explicitly covered in this patent disclosure.

[0031] For the first disclosed embodiment, we consider the problem of limited main memory capacity. Figure 1 The exemplary system in may be configured to allow data and instructions to be compressed in main memory. Figure 2 An example of such a computer system 200 is shown. What is added is a computer memory compression device 205 on a microprocessor chip 210. The computer memory compression device 205 includes four functional blocks. These blocks include a locator (address translation) unit 211, an encoder (compressor) unit 212, a decompressor unit 213, and an analyzer unit 214.

[0032] As will be explained in detail below, the analyzer unit 214 is configured to analyze computer memory content with respect to the presence of duplicate memory objects and value redundancy of data values ​​in unique memory objects. In this regard, data values ​​will generally be finer in granularity than memory objects, and memory objects will generally be finer in granularity than computer memory content. Computer memory content may generally be pages of computer memory, memory objects may generally be memory blocks, and each memory block may generally include multiple data values, such as memory words.

[0033] The encoder unit 212 is configured to encode the computer memory content by eliminating duplicate memory objects and compressing the remaining unique memory objects by exploiting the data value locality of the data values ​​of each remaining unique memory object. The encoder unit 212 is also configured to provide metadata representing the memory objects of the encoded computer memory content. The metadata reflects the duplicate memory objects that were eliminated, the remaining unique memory objects, and the type of compression used to compress each remaining unique memory object. For example, an example of such metadata is found at Figure 5 At 500 in . The locator unit 211 is configured to locate the memory object in the encoded computer memory content using the metadata.

[0034] Fig.12 1200. The method involves analyzing 1210 computer memory content with respect to occurrences of duplicate memory objects and value redundancy of data values ​​in unique memory objects. The method further involves encoding 1220 the computer memory content by eliminating duplicate memory objects and compressing each remaining unique memory object by exploiting data value locality of the data values ​​of each remaining unique memory object, the method further involves providing 1230 metadata 500 representing the memory objects of the encoded computer memory content, and the method further involves locating 1240 the memory objects in the encoded computer memory content using the metadata 500. The computer memory compression method 1200 may generally include functional steps corresponding to functions performed by any or all structural features and embodiments of the computer memory compression device 205 as described throughout this document.

[0035] The computer memory compression device 205 is connected to the memory controller on one side and to the cache C3 of the last level on the other side. The purpose of the address translation unit 211 is to convert the conventional physical address PA into a compressed address CA to locate the memory block in the compressed memory. Those skilled in the art will appreciate that this address translation is necessary because in the compressed memory, conventional memory pages (e.g., 4KB) can be compressed to any size. The purpose of the encoder (compressor) unit 212 is to compress the memory blocks that have been modified and evicted from the cache of the last level. In order to make the impact on the performance of the memory system negligible, compression must be very fast and is usually accelerated by a dedicated compressor unit. Similarly, when a memory block is requested by a processor and is not available in any cache level, such as C1, C2, and C3 in the exemplary embodiment, the memory block must be requested from the memory. The address translation unit 211 will locate the block, but before it is installed in the cache hierarchy, such as in C1, it must be decompressed. The purpose of the decompressor unit 213 is to accelerate the process so that the impact on the performance of the memory system is negligible.

[0036] Those skilled in the art will appreciate that the functionality of the compressor and decompressor units depends on the type of compression algorithm being used. In one embodiment, incremental encoding (such as base value-incremental on-the-fly encoding) can be used, where the difference between the value and the base value is stored, rather than the value itself. In another embodiment, entropy-based encoding (such as Huffman encoding) can be used, where values ​​that are more frequent than other values ​​use more intensive encoding. In a third embodiment, deduplication can be used, where only unique blocks are stored in a memory. The present disclosure is intended to cover all compression algorithms for the purpose of removing value redundancy.

[0037] In view of the Figure 2 The remainder of this document follows the structure of providing devices, methods, and systems using a combined compression method based on deduplication and value redundancy removal compression as described below: how to encode computer memory contents (e.g., memory pages); how to locate memory objects (e.g., memory blocks) in compressed memory; how to compress and decompress individual memory objects (e.g., memory blocks); and finally, how to recompress computer memory contents (e.g., memory pages) to remove additional copies and value redundancy.

[0038] Data analysis and encoding of memory pages

[0039] We are Figure 3An exemplary overview of how to use deduplication in conjunction with entropy-based compression to compress memory pages is provided in . However, note that other value redundancy removal compression methods such as delta encoding (e.g., base-delta on-the-fly encoding) are possible. The exemplary layout 300 of a page includes N blocks, which are as follows Figure 3 As shown at (A) on the left, they are numbered 1, 2, ..., N and appear at consecutive memory block addresses in a conventional uncompressed computer memory, where 310 represents the first memory block and 320 represents the last memory block. Deduplication aims to remove duplicate blocks so that only unique blocks belong to a memory page. Figure 3 As shown in the middle (B), in the exemplary layout 300 of memory blocks in a page, the memory blocks represented by 340 and 350 are copies. Instead of storing duplicate data in memory, and as will be explained in detail later, they only store references, such as pointers, to unique blocks. Memory block 350 uses a pointer labeled 370 to reference memory block 2 (330). Similarly, memory block 340 uses a pointer labeled 360 to reference memory block 1. Therefore, after the deduplication process, the memory page only contains memory blocks with different content, which are called unique memory blocks.

[0040] Prior art also includes compression methods that encode frequently used data more densely than infrequently used data, such as Huffman coding, or compression methods that exploit numerical similarity, such as delta coding (e.g., base-delta on-the-fly coding). These compression methods are referred to as value redundancy removal compression methods. In order to compress pages using value redundancy removal compression methods, people typically analyze all individual data items at some granularity, such as at the word level (e.g., 64 bits). The value frequency distribution will capture the relative occurrences of different values ​​in a page. However, before deduplication, the presence of copies can greatly change the value distribution when trivially applied to the original contents of the memory page. For this reason, the proposed implementation first applies deduplication to remove copies and then determines the value distribution of the remaining unique memory blocks. Figure 3 The rightmost exemplary layout seen in (C) shows the result after applying value redundancy removal compression. The size of the unique memory blocks denoted as 1 (380) and 2 (390) can now be smaller because value redundancy removal compression can manage to encode more frequent data items more densely than less frequent data items.

[0041] Now let's turn our attention to how to implement Figure 3 This will combine Figures 4 to 7As explained in the previous paragraphs, in order to realize the full potential of value redundancy removal compression in combination with deduplication, it is important to perform deduplication first. In one embodiment, deduplication can be achieved by sequentially analyzing all memory blocks in a single page, not excluding the possibility of considering all memory blocks in multiple pages, with the goal of determining whether it is unique. For this purpose, the first memory block to be analyzed will always be classified as unique. Turning our attention to Figure 4 , an exemplary tree data structure 400 records all unique blocks. Before inserting a block, a block signature known from the prior art is created. The signature is a dense representation of the value of the block, with the property that if the signatures of two memory blocks are different, they are not identical. However, the opposite may not be true.

[0042] To determine if a memory block is unique and must be inserted into the tree-based data structure, its signature is first compared to the signature of the top node 410 in the tree data structure 400. If they are identical, a second test is performed to compare the contents of the two memory blocks. If the memory blocks are indeed identical, a duplicate block has been detected. This same operation is performed at each node in the tree-based data structure. However, if the signatures are identical, but the two blocks are not identical, a new block with the same signature must be inserted. This may involve the following additional tests to handle false positives. When the created signature S matches 650 with the signature represented in the tree data structure 400:

[0043] determining whether the separate memory object is identical to the unique memory block represented by the matching signature; and

[0044] If the individual memory object and the unique memory block represented by the matching signature are not identical:

[0045] Inserting a node in the tree data structure 400;

[0046] Enter the created signature S in the inserted node; and

[0047] Metadata 500 is generated for the separate memory object, the metadata having: information 510 indicating that it is a unique memory object; and a unique memory object reference 530, U_PTR, to the separate memory object.

[0048] On the other hand, when there is a signature mismatch, if the signature is less than the signature of the top node 410, a search is performed on the left branch of the tree according to the test at 460. If the signature is greater than the signature of the top node, a search is performed on the right branch of the tree according to the test (box 470). Therefore, all nodes 410, 420, 430, 440 and 450 are organized in descending order (left branch) and ascending order (right branch) to make the search time logarithmic rather than linear. Since the copies will be removed in this process, the memory blocks will not reside at the same address as the conventional uncompressed pages. For this reason, the new location of the block will be recorded in a tree-based data structure, as described by "block location-BL" in each node.

[0049] The end result of the deduplication process is that all duplicate memory blocks are eliminated. For this reason, and as in Figure 3 As explained in , the location of a memory block in a deduplicated page may not be the same as the location in an originally non-deduplicated page. For this reason, in order to help locate memory blocks in compressed memory, metadata must be generated during the deduplication process as will be described later.

[0050] Figure 5 At (A) on the left, exemplary metadata 500 generated for each memory block in the process is shown. Metadata 500 includes four parts: type (510), block encoding - ENC (520), U_PTR (530) and D_PTR (540). The type describes whether the block is unique or a copy. The block encoding, represented as ENC, indicates the type of compression algorithm applied to the contents of the memory block, for example, Huffman or arithmetic coding or other. If the memory block is unique, a pointer is required to locate it. This is represented by U_PTR (530). On the other hand, if the block is a copy, a pointer to the unique copy is required. As with the unique copy, U_PTR (530) points to the unique copy in the compressed memory.

[0051] Figure 5The rightmost portion (B) depicts an exemplary layout of three selected memory blocks in a page, where memory block 570 is unique, and memory blocks 580 and 590 are copies of memory block 570. Here, the U_PTR of the copy points to the unique memory block 570. However, during operation, it may happen that the unique block is modified. Then, according to boxes 580 and 590, it may not contain the same content as the duplicated block. For this reason, it is necessary to track the copies from the perspective of the unique memory blocks. For this purpose, pointers denoted as D_PTR are provided. They establish a link data structure that links all duplicate memory blocks with the corresponding unique memory blocks. In this example, the D_PTR (550) of the unique memory block 570 points to the duplicate memory block 580. The D_PTR of the duplicate memory block box 580 points to other duplicate memory blocks 590.

[0052] Thus, in summary, for each memory object of the encoded computer memory content, the metadata 500 advantageously includes:

[0053] Information 510 indicating whether the memory object is a deleted duplicate memory object or a remaining unique memory object;

[0054] ·When the memory object is a unique memory object, information 520 indicating the type of compression used and a unique memory object reference 530, U_PTR, to the unique memory object; and ·When the memory object is a replica memory object, a unique memory object reference 530, U_PTR, to the unique memory object, whose uncompressed content is the same as that of the replica memory object.

[0055] Advantageously, for each memory object that is a unique memory object, the metadata 500 also includes a replica memory object reference 540, D_PTR, to the eliminated replica memory object, whose uncompressed content is the same as the unique memory object.

[0056] Now let's build the whole process of memory block deduplication by analyzing all memory blocks within a page (other granularity such as multiple pages are also applicable). Figure 6 As described in the flowchart of Figure 4 As mentioned, the process 600 of deduplicating memory blocks uses a tree-based data structure 400 that will eventually contain all unique memory blocks. The process 600 will analyze all memory blocks of a page in a sequential manner. The first decision step 620 checks whether all memory blocks of the page have been accessed. If not, the next memory block is considered. Figure 4As explained, a signature for the block is created according to some method known in the art, and the signature is compared to all nodes in the tree as long as no match is detected. Finally, there are two outcomes. If the block is unique, as tested in decision step 640, it will be inserted at 660. The memory block location is then recorded along with the memory block number within the page. If the block is not unique, it is deduplicated at 650 and the block is removed according to Figure 5 (See 530), the pointer of the block location of the unique block already recorded in the tree-based data structure will be assigned to the U_PTR pointer of the metadata. Regardless of the result, the next action is to analyze the next memory block of the page, which returns us to decision step 620.

[0057] according to Figures 4 to 6 It will be appreciated from the description that the analyzer unit 214 and the encoder unit 212 of the computer memory compression device 205 are therefore configured to process 600 each individual memory object in the computer memory content by:

[0058] Create a signature S, which is a dense representation of the data value of the memory object;

[0059] Traversing the tree data structure 400 to compare the created signature S with the signatures already represented in the tree data structure 400;

[0060] If the created signature S does not match 660 any signature represented in the tree data structure 400:

[0061] Inserting a node in the tree data structure 400;

[0062] Entering the created signature S in the inserted node; and

[0063] Generate metadata 500 for the separate memory object, the metadata having: information 510 indicating that it is a unique memory object; and a unique memory object reference 530, U_PTR, to the separate memory object; and

[0064] If the created signature S matches 650 the signature represented in the tree data structure 400:

[0065] generating metadata 500 for the separate memory object, the metadata having: information 510 indicating that it is a copy memory object; and a unique memory object reference 530, U_PTR, to the unique memory block represented by the matching signature in the tree data structure 400; and

[0066] The metadata 500 for the unique memory block represented by the matching signature in the tree data structure 400 is updated to introduce a duplicate memory object reference 540, D_PTR, to the separate memory object.

[0067] As already noted, it is important to apply deduplication before any compression method that aims to exploit the value locality of a single data item, for example at the word level, because the duplicates will not correctly represent the value frequency distribution. To this end, a process is needed to establish the value frequency distribution of unique blocks. Figure 7 Such a process 700 is described in . The entire process 700 is intended to analyze the data recorded in Figure 4 The process 700 starts at step 710. As long as all the unique memory blocks have not been accessed (step 720), the process 700 records the value frequency of each unique block that has been accessed. This can be done by recording the frequency of each individual value in the hash table at step 730. Finally, the value frequency of all unique memory blocks can be established, and an encoding based on any compression method based on value locality, such as Huffman or arithmetic encoding or other encoding, can be established in step 740.

[0068] Thus, the analyzer unit 214 and the encoder unit 212 of the computer memory compression device 205 may be configured to, when all memory objects in the computer memory content have been processed 600:

[0069] traversing the tree data structure 400 to generate a value-frequency table for the data values ​​of the unique memory objects represented by the nodes of the tree data structure 400; and

[0070] • Each unique memory object is compressed using the generated value frequency table through an entropy-based compression scheme.

[0071] In one such embodiment, the analyzer unit may implement a hash table to record the frequency of each value for later analysis, perhaps using a software routine to create an encoding using, for example, Huffman encoding or some other entropy-based encoding technique.

[0072] In an alternative embodiment, using delta encoding (e.g., base value-delta on-the-fly encoding), the values ​​remaining after removing the duplicates can be used to select one or more base values. In one approach, after the duplicates have been removed, clustering techniques can be used to analyze which base value is closest to all values ​​in the unique copies in the page.

[0073] Therefore, alternatively, the analyzer unit 214 and the encoder unit 212 of the computer memory compression device 205 may be configured to, when all memory objects in the computer memory content have been processed 600:

[0074] traversing the tree data structure 400 to generate a value frequency table for the data values ​​of the unique memory objects represented by the nodes of the tree data structure 400; and

[0075] • Each unique memory object is compressed using the generated value frequency table through an entropy-based compression scheme.

[0076] Locating a memory block in compressed memory

[0077] We now turn our attention to how to locate and decompress memory blocks in compressed memory using a combined deduplication and value redundancy removal compression technique. Figure 2 , recall that the task of the locator (address translation) unit 211 is to translate regular physical addresses into addresses in the compressed address space. Fig.10 Shows Figure 2 The detailed layout 1000 of the locator (address translation) unit 211 of FIG. 1000 includes a number of page entries, where each entry 1010 includes a number of fields: a physical address (PA) 1011 and a compressed address (CA) 1012, which correspond to the physical address and compressed address of the page, respectively. Figure 5 As shown at 1021 to 1023 in FIG. 1021, each page entry is associated with metadata MD 1...MD N for each block. For details on the metadata, refer to Figure 5 The leftmost part (A) of is as described above. Figure 5 As shown, whether the memory block is unique or deduplicated, U_PTR acts as an offset to the location of the block relative to the compressed address CA. Adder 1030 forms the block address 1040 by establishing the sum of U_PTR and compressed address CA 1012 .

[0078] when Figure 2 The processor in (denoted as P 1 To P N ) can not be cached by any level (C 1 , C 2 or C 3 ) satisfies the read request, it will be Figure 2 The locator (address translation) unit 211 / 1000 in / 10 is processed and the physical address PA is translated according to the process explained above using the compressed address CA associated with PA and the metadata of the requested block. However, if the write request cannot be satisfied by any level of cache, the write can affect which blocks are unique and which blocks are deduplicated. Consider Figure 8Here, the unique block is represented by 830. The two deduplicated blocks represented by 810 and 820 have the same content, and their U_PTRs point to the unique block 830. The deduplicated memory blocks 810, 820 are linked to the unique block 830 by having the D_PTR of the unique block point to the memory block 820 and the D_PTR of the same block point to the memory block 810.

[0079] Now assume that the write request is destined for unique memory block 830, and let us turn our attention to Figure 8 The scenario at (B) on the far right. Since there is a copy of the unique memory block 830, it is desirable to preserve the memory block. This is done using a dedicated free memory area to copy the old contents of the unique memory block 830, which is located at Figure 8 840, which may but is not necessarily within a page. The current starting address 805 of the free memory area is pointed to by a reference pointer, such as a dedicated pointer, denoted as F_PTR. The old contents of the unique memory block 830 are copied to the free area 840, as seen at 870, and the U_PTRs of the deduplicated memory blocks 810, 820 now point to the old unique memory block, now located at 870. Additionally, at 870, the D_PTR is changed to link the deduplicated memory blocks 810, 820 to the old unique memory block. Alternatively, only the D_PTR of block 870 is updated to point to block 820, while the rest of the D_PTR remains as is. Figure 8 As seen in the leftmost portion (A) - that is, only the head of the linked list is updated. The unique memory block at the original location, 830, can now be updated with new content according to the write request. The F_PTR is also updated to reflect the change in the starting address of the free memory area, as shown at 880.

[0080] Processing Figure 8 An alternative way of specifying a write request to a unique memory block 830 in is as follows. However, please note that Figure 8 The details of are not reflected in the following description. By using the signature S of the unique memory block 830, in the tree data structure 400 (refer to Figure 4 ), the deduplicated memory block 820 is promoted to unique. Based on the write request, the updated content of the unique memory block 830 is written to the dedicated free memory area 840. The U_PTR of 830 is updated to point to the location of the new block in the free memory area, and the D_PTR of 830 is deleted or at least updated to not point to anywhere, because block 830 is a unique block with no detected duplicates. The F_PTR is updated to reflect the new starting address after the write.

[0081] Now let's consider the scenario where write requests are destined for deduplicated chunks and turn our attention to Fig. 9 's leftmost scenario. Here, there are two deduplicated memory blocks 920 and 930 whose contents are identical to the unique memory block 910, and the U_PTR and D_PTR are set to reflect this situation, as explained in the previous paragraph. Now, assume that a write request is assigned to the deduplicated block 920 of the leftmost scenario. Since the contents of the deduplicated block will now change, its contents will not be identical to the unique memory block 910. Instead, the updated deduplicated block will now be placed in the free area at 980 and is represented as D_U. The U_PTR of block 920 is now changed to point to the new unique block 980, and the D_PTR of the block points to the deduplicated block 920. Additionally, the D_PTR that linked the previous deduplicated block to the unique block 910 is now changed to point to another deduplicated block 930, and the 930 U_PTR of the block points to the unique block 910.

[0082] Please note that in Figure 8 and Fig. 9 In the scenario of , write operations will create new chunk contents that may or may not already exist. In the process described, no attempt is made to analyze whether new chunks are identical to other already existing chunks. Therefore, they do not need to be inserted into the tree structure that keeps track of unique chunks. However, in a later analysis phase, we can revisit the contents of all chunks, including newly created chunks, to find more opportunities for deduplication.

[0083] Compression, decompression, and recompression of memory blocks

[0084] Figure 5 The metadata of the block contains the compression algorithm used for encoding (ENC at 520). After the block is retrieved from memory, it will be decompressed. This Fig.11 1 is exemplified by a decompressor unit 1110 which takes as input the type of compression algorithm (ENC) and the block to be decompressed from the memory. The type of compression algorithm may be deduplication, entropy-based compression, delta encoding (e.g. base-delta on-the-fly encoding), or any other compression method that aims to remove value redundancy.

[0085] In the block Figure 2 In the case of a C3 replacement at the last level of cache of the exemplary embodiment of the present invention, the block may have been modified and then a write-back request may be generated. Then it must be compressed before being written back to the memory. Figure 2 The locator (address translation) unit 211 of is queried. If it has an entry for the metadata of the page to which the block belongs, the metadata will encode the kind of compression algorithm that will be used to compress the block. Back to Figure 8 and Fig. 9 In the scenario of Figure 8 ) or the deduplicated block ( Fig. 9 ). In these cases, the address translation unit and related logic will assist in changing the metadata to help reduce the number of copies made in the process.

[0086] Figures 8 to 11 The description can be summarized and generalized as follows. The computer memory compression device 205 can be advantageously configured to:

[0087] · receiving a read request for a memory block in a memory page having a physical memory page address PA;

[0088] Determine the compressed memory page address CA from the lookup table 1010;

[0089] Retrieve metadata 1020 for a memory block;

[0090] Calculate the compressed memory block address 1040 from the compressed memory page address CA and the unique memory object reference 530, U_PTR, of the retrieved metadata;

[0091] · retrieving a compressed memory block 1105 at the calculated compressed memory block address 1040; and· decompressing 1110 the retrieved compressed memory block 1105 by the decompressor unit 213 using the information 520 indicating the compression type and available in the retrieved metadata of the memory block; 1120, ENC.

[0092] As reference Figure 8 As explained, the computer memory compression device 205 may advantageously also be configured to:

[0093] · receiving a write-back request involving an update of a unique memory block 830 at an original memory location;

[0094] · copying the unique memory block 830 to a new memory location 870 in a dedicated free memory area 840 of computer memory contents before updating;

[0095] Updating the metadata of the replica memory blocks 810, 820 linked to the unique memory block 830 so that their replica memory object references 540, D_PTRs are redirected to new memory locations 870 in the dedicated free memory region 840; and

[0096] • Based on the write-back request, the unique memory block at its original memory location 830 is updated.

[0097] In addition, the computer memory compression device 205 may advantageously also be configured to:

[0098] providing metadata including a reference F_PTR to the start address 805 of the dedicated free memory area 840; and

[0099] After the unique memory block 830 is copied to the new memory location 870 in the dedicated free memory area 840 , the reference F_PTR is updated to reflect the new starting address 880 .

[0100] As Figure 8 As explained in the alternative, the computer memory compression device 205 may advantageously also be configured to:

[0101] · receiving a write-back request involving an update of a unique memory block 830 at an original memory location;

[0102] The deduplicated memory block 820 is found to be a duplicate of the unique memory block 830;

[0103] Promote the discovered deduplicated memory block 820 to a unique memory block in the tree data structure 400 by using the signature S of the unique memory block 830

[0104] · writing the updated contents of the unique memory block 830 to a new memory location 870 in the dedicated free memory area 840 in response to the write-back request; and

[0105] • The metadata of the unique memory block 830 is updated so that its unique memory object reference 530, U_PTR is redirected to a new memory location 870 in the dedicated free memory area 840, while any of its duplicate memory object references 540, D_PTR are removed.

[0106] In addition, the computer memory compression device 205 may advantageously also be configured to:

[0107] providing metadata including a reference F_PTR to the start address 805 of the dedicated free memory area 840; and

[0108] After writing the updated contents of the unique memory block 830 to the new memory location 870 in the dedicated free memory area 840 according to the write-back request, the reference F_PTR is updated to reflect the new starting address 880.

[0109] As reference Fig. 9 As explained, the computer memory compression device 205 may advantageously also be configured to:

[0110] · receiving a write-back request involving an update of the replica memory block 920;

[0111] Storing the contents of the updated copy memory block as a new unique memory block 980 in the dedicated free memory area 940; and

[0112] Updating the metadata of a unique memory block 910 that was previously linked to a replica memory block 920 to reflect that the unique memory block 910 is no longer linked to the replica memory block 920 , while maintaining any links between the unique memory block 910 and other replica memory blocks 930 .

[0113] As a result of the write-back request, the unique copy and the deduplicated copy will be updated and will eventually appear in the free area used to avoid unnecessary duplication, as with Figure 8 and Fig. 9 However, over time, this may have the effect of deteriorating the compression ratio. In one embodiment, the compression ratio is constantly monitored. When it exceeds a pre-set threshold, processes are initiated that are designed to analyze the memory page for the presence of duplicates and redundancy values ​​to perform recompression on the page. These processes can be related to Figure 6 and Figure 7 In another embodiment, the page is periodically visited to assess whether recompression is necessary. When it is deemed necessary, Figure 6 and Figure 7 The process described in and explained in the previous paragraphs is activated.

[0114] Therefore, the computer memory compression device 205 may advantageously also be configured to:

[0115] Monitoring the compression ratio of memory pages over time; and

[0116] If the compression ratio does not meet the given criteria, recompress the memory page by executing the functionality of the computer memory compression method 1200 described in this document.

[0117] Alternatively, or in addition, the computer memory compression device 205 may be further configured to periodically perform recompression of memory pages to improve the compression ratio by executing the functions of the computer memory compression method 1200 described in this document.

[0118] While inventive aspects have been described in this document by referring to exemplary embodiments, the inventive aspects are not limited to the disclosed embodiments but instead encompass alternative embodiments that may be implemented by those skilled in the art.

[0119] An alternative inventive aspect may be viewed as a system for analyzing computer memory data for the purpose of compressing the data by eliminating copies and value redundancies of data items, the system comprising: means for eliminating copies and value redundancies; means for locating data items after removing copies and value redundancies; means for compressing and decompressing data items using said compression method; and means for recompressing data items.

[0120] Another alternative inventive aspect may be viewed as a method for analyzing computer memory data for the purpose of compressing the data by eliminating copies and value redundancies of data items, the method comprising the steps of: eliminating copies and value redundancies; locating data items after removing copies and value redundancies; compressing and decompressing data items using the compression method; and recompressing data items.

[0121] Yet another alternative inventive aspect may be viewed as a device for analyzing computer memory data for the purpose of compressing data by eliminating copies and value redundancy of data items, the device being configured to: eliminate copies and value redundancy; locate data items after eliminating copies and value redundancy; compress and decompress data items using the compression method; and recompress data items.

[0122] Another alternative inventive aspect may be viewed as the disclosed invention comprising a system for data analysis having means for analyzing the contents of pages in main memory with respect to the presence of copies of memory blocks and the presence of value redundancy with respect to remaining unique memory blocks. The disclosed invention also includes a system having means for removing copies and value redundancy of memory. In addition, the disclosed invention also includes the following system having: means for locating a single memory block after the copies and value redundancy have been removed; and means for compressing and decompressing the memory block using it. Finally, the disclosed invention includes a system having means for recompressing memory pages.

[0123] Further alternative inventive aspects may be viewed as methods of analyzing the contents of pages in main memory with respect to the presence of duplicates of memory blocks and with respect to the relative frequency of values ​​in the remaining unique memory blocks; methods for encoding memory blocks taking into account both deduplication and value locality-based encoding methods; and methods for locating a single memory block in a compressed memory for a family of combined deduplication and value locality-based compression techniques, and methods for compressing and decompressing memory blocks using the same. Finally, the disclosed invention includes methods for recompressing memory pages.

[0124] Other alternative inventive aspects may be viewed as: a data analyzer device configured to analyze page contents in a main memory with respect to the presence of copies of a memory block and with respect to value redundancy of remaining unique memory blocks; a data encoder device configured to encode memory blocks taking into account the removal of copies and value redundancy of remaining unique blocks; a memory block locator device configured to locate a single memory block in a compressed memory of a series for combined deduplication and value locality-based compression techniques, and a device configured to compress and decompress memory blocks using the same; and a device configured to recompress memory pages.

Claims

1. A computer memory compression method (1200), comprising: analyzing (1210) computer memory contents with respect to occurrences of duplicate memory objects and value redundancy of data values ​​in unique memory objects; encoding the computer memory contents by eliminating the duplicate memory objects and compressing each remaining unique memory object by exploiting data value locality of the data values ​​of each remaining unique memory object (1220); providing (1230) metadata (500) for memory objects representing the encoded computer memory contents, wherein the metadata (500) reflects the eliminated duplicate memory objects, the remaining unique memory objects, and the type of compression used to compress each remaining unique memory object; and locating (1240) a memory object in the encoded computer memory content using the metadata (500), Wherein, for each memory object of the encoded computer memory content, the metadata (500) includes: Information (510) indicating that the memory object is a duplicate memory object that is eliminated or a remaining unique memory object; When the memory object is a unique memory object, information indicating the type of compression used (520) and a unique memory object reference (530, U_PTR) to the unique memory object; and When the memory object is a replica memory object, a unique memory object reference (530, U_PTR) to a unique memory object whose uncompressed content is the same as the replica memory object, Wherein, for each memory object that is a unique memory object, the metadata (500) further includes a copy memory object reference (540, D_PTR) to the deleted copy memory object, whose uncompressed content is the same as that of the unique memory object.

2. The method of claim 1 , further comprising processing ( 600 ) each individual memory object in the computer memory contents by: creating a signature (S), the signature being a dense representation of the data value of the memory object; traversing the tree data structure (400) to compare the created signature (S) with signatures already represented in the tree data structure (400); If the created signature (S) does not match (660) any of the signatures represented in the tree data structure (400): inserting a node into the tree data structure (400); Enter the created signature (S) in the inserted node; as well as Generating the metadata (500) for the separate memory object, the metadata having: information indicating that it is a unique memory object (510); and the unique memory object reference (530, U_PTR) to the separate memory object; as well as If the created signature (S) matches (650) the signature represented in the tree data structure (400): Generating the metadata (500) for the separate storage object, the metadata having: information indicating that it is a replica storage object (510); and said unique memory object reference (530, U_PTR) to a unique memory block represented by a matching signature in said tree data structure (400); as well as The metadata (500) for the unique memory block represented by the matching signature in the tree data structure (400) is updated to introduce a duplicate memory object reference (540, D_PTR) to the separate memory object.

3. The method of claim 2, further comprising, when the created signature (S) matches (650) the signature represented in the tree data structure (400): determining whether the separate memory object is identical to the unique memory block represented by the matching signature; and If the separate memory object and the unique memory block represented by the matching signature are not identical: inserting a node into the tree data structure (400); Enter the created signature (S) in the inserted node; as well as The metadata (500) is generated for the separate memory object, the metadata having: information indicating that it is a unique memory object (510); and the unique memory object reference (530, U_PTR) to the separate memory object.

4. The method of claim 2, further comprising, when all memory objects in the computer memory contents have been processed (600): traversing the tree data structure (400) to generate a value frequency table for data values ​​of unique memory objects represented by nodes of the tree data structure (400); and The generated value frequency table is used to compress each unique memory object through an entropy-based compression scheme.

5. The method of claim 2, further comprising, when all memory objects in the computer memory contents have been processed (600): traversing the tree data structure (400) by examining data values ​​of unique memory objects represented by nodes of the tree data structure (400) and determining one or more base values; and Each unique memory object is compressed using the determined one or more base values ​​through a delta encoding based compression scheme.

6. The method according to claim 1, wherein: The data values ​​have a finer granularity than the memory objects, and the memory objects have a finer granularity than the computer memory contents.

7. The method according to claim 6, wherein: The computer memory contents are pages of computer memory, the memory objects are memory blocks, and each memory block includes a plurality of data values.

8. The method according to claim 7, in, For each memory object of the encoded computer memory contents, the metadata includes: information indicating that the memory object is a duplicate memory object that is eliminated or a remaining unique memory object; When the memory object is a unique memory object, information indicating the type of compression used and a unique memory object reference to the unique memory object; and When the memory object is a replica memory object, a unique memory object reference to a unique memory object, the uncompressed content of which is the same as that of the replica memory object, and wherein the method further comprises: receiving a read request for a memory block in a memory page having a physical memory page address (PA); determining a compressed memory page address (CA) from a lookup table (1010); retrieving metadata for the memory block (1020); Calculating a compressed memory block address (1040) based on the compressed memory page address (CA) and the unique memory object reference (530, U_PTR) of the retrieved metadata; retrieving a compressed memory block (1105) at the calculated compressed memory block address (1040); and Using information indicating the type of compression from the retrieved metadata of the memory block (520; 1120, ENC), to decompress (1110) the retrieved compressed memory block (1105).

9. The method according to claim 7, in, For each memory object of the encoded computer memory contents, the metadata includes: information indicating that the memory object is a duplicate memory object that is eliminated or a remaining unique memory object; When the memory object is a unique memory object, information indicating the type of compression used and a unique memory object reference to the unique memory object; and when the memory object is a replica memory object, a unique memory object reference to a unique memory object whose uncompressed content is the same as the replica memory object, wherein, for each memory object that is a unique memory object, the metadata further includes a copy memory object reference to the deleted copy memory object, whose uncompressed content is the same as that of the unique memory object, and Wherein, the method further comprises: receiving a write-back request involving an update of a unique memory block (830) at an original memory location; copying the unique memory block (830) to a new memory location (870) in a dedicated free memory area (840) of the computer memory contents before updating; updating metadata of a replica memory block (810, 820) linked to the unique memory block (830) so that its replica memory object reference (540, D_PTR) is redirected to a new memory location (870) in the dedicated free memory region (840); and Based on the write-back request, the unique memory block at its original memory location (830) is updated.

10. The method according to claim 9, further comprising: providing metadata including a reference (F_PTR) to a starting address (805) of said dedicated free memory area (840); as well as After the unique memory block (830) is copied to the new memory location (870) in the dedicated free memory area (840), the reference (F_PTR) is updated to reflect the new starting address (880).

11. The method according to claim 7, in, For each memory object of the encoded computer memory contents, the metadata includes: information indicating that the memory object is a duplicate memory object that is eliminated or a remaining unique memory object; When the memory object is a unique memory object, information indicating the type of compression used and a unique memory object reference to the unique memory object; and when the memory object is a replica memory object, a unique memory object reference to a unique memory object whose uncompressed content is the same as the replica memory object, wherein, for each memory object that is a unique memory object, the metadata further includes a copy memory object reference to the deleted copy memory object, whose uncompressed content is the same as that of the unique memory object, The method further comprises processing each individual memory object in the computer memory contents by: creating a signature, the signature being a dense representation of the data value of the memory object; traversing the tree data structure to compare the created signature with signatures already represented in the tree data structure; If the created signature does not match any of the signatures represented in the tree data structure: inserting a node into the tree data structure; Entering the created signature in the inserted node; and generating the metadata for the separate memory object, the metadata having: information indicating that it is a unique memory object; and the unique memory object reference to the separate memory object; and If the created signature matches the signature represented in the tree data structure: generating the metadata for the separate memory object, the metadata having: information indicating that it is a replica memory object; and the unique memory object reference to the unique memory block represented by the matching signature in the tree data structure; and updating the metadata for the unique memory block represented by the matching signature in the tree data structure to introduce a duplicate memory object reference to the separate memory object, and Wherein, the method further comprises: receiving a write-back request involving an update of a unique memory block (830) at an original memory location; discovering that the deduplicated memory block (820) is a duplicate of the unique memory block (830); promoting the discovered deduplicated memory block (820) to be unique in the tree data structure (400) by using the signature (S) of the unique memory block (830); writing the updated contents of the unique memory block (830) to a new memory location (870) in a dedicated free memory area (840) in accordance with the write-back request; and The metadata of the unique memory block (830) is updated so that its unique memory object reference (530, U_PTR) is redirected to the new memory location (870) in the dedicated free memory area (840) and any of its copy memory object references (540, D_PTR) are removed.

12. The method according to claim 11, further comprising: providing metadata including a reference (F_PTR) to a starting address (805) of said dedicated free memory area (840); as well as After writing the updated contents of the unique memory block (830) to the new memory location (870) in the dedicated free memory area (840) in accordance with the write-back request, the reference (F_PTR) is updated to reflect the new starting address (880).

13. The method according to claim 7, further comprising: receiving a write-back request involving an update of a replica memory block (920); storing the contents of the updated replica memory block as a new unique memory block (980) in a dedicated free memory area (940); as well as The metadata of a unique memory block (910) previously linked to the replica memory block (920) is updated to reflect that the unique memory block (910) is no longer linked to the replica memory block (920), while maintaining any links between the unique memory block (910) and other replica memory blocks (930).

14. The method according to claim 9, further comprising: monitoring the compression ratio of memory pages over time; as well as If the compression ratio does not meet a given standard, recompression is performed on the memory page by executing the functions of the method according to claim 1 .

15. The method according to claim 13, further comprising: By executing the functions of the method according to claim 1, recompression of memory pages is periodically performed to improve the compression ratio.

16. The method according to claim 1, wherein: Such compression by exploiting data value locality involves one of the following: Entropy-based coding; Incremental encoding; Dictionary-based encoding; and Pattern-based encoding.

17. A computer memory compression device (205), comprising: an analyzer unit (214) configured to analyze computer memory contents with respect to occurrences of duplicate memory objects and value redundancy of data values ​​in unique memory objects; an encoder unit (212) configured to encode the computer memory content by eliminating the duplicate memory objects and compressing each remaining unique memory object by exploiting data value locality of the data values ​​of each remaining unique memory object, the encoder unit (212) further configured to provide metadata (500) of the memory object representing the encoded computer memory content, wherein the metadata (500) reflects the eliminated duplicate memory objects, the remaining unique memory objects, and the compression type used to compress each remaining unique memory object; and a locator unit (211) configured to locate a memory object in the encoded computer memory content using the metadata (500), Wherein, for each memory object of the encoded computer memory content, the metadata (500) includes: Information (510) indicating that the memory object is a duplicate memory object that is eliminated or a remaining unique memory object; When the memory object is a unique memory object, information indicating the type of compression used (520) and a unique memory object reference (530, U_PTR) to the unique memory object; and When the memory object is a replica memory object, a unique memory object reference (530, U_PTR) to a unique memory object whose uncompressed content is the same as the replica memory object, Wherein, for each memory object that is a unique memory object, the metadata (500) further includes a copy memory object reference (540, D_PTR) to the deleted copy memory object, whose uncompressed content is the same as that of the unique memory object.

18. The apparatus according to claim 17, wherein: The analyzer unit (214) and the encoder unit (212) are configured to process (600) each individual memory object in the computer memory content by: creating a signature (S), the signature being a dense representation of the data value of the memory object; traversing the tree data structure (400) to compare the created signature (S) with signatures already represented in the tree data structure (400); If the created signature (S) does not match (660) any of the signatures represented in the tree data structure (400): inserting a node into the tree data structure (400); Enter the created signature (S) in the inserted node; as well as Generating the metadata (500) for the separate memory object, the metadata having: information indicating that it is a unique memory object (510); and the unique memory object reference (530, U_PTR) to the separate memory object; as well as If the created signature (S) matches (650) the signature represented in the tree data structure (400): Generating the metadata (500) for the separate storage object, the metadata having: information indicating that it is a replica storage object (510); and said unique memory object reference (530, U_PTR) to a unique memory block represented by a matching signature in said tree data structure (400); as well as The metadata (500) for the unique memory block represented by the matching signature in the tree data structure (400) is updated to introduce a duplicate memory object reference (540, D_PTR) to the separate memory object.

19. The apparatus according to claim 18, wherein: The analyzer unit (214) and the encoder unit (212) are further configured to, when the created signature (S) matches (650) the signature represented in the tree data structure (400): determining whether the separate memory object is identical to the unique memory block represented by the matching signature; and If the separate memory object and the unique memory block represented by the matching signature are not identical: inserting a node into the tree data structure (400); Enter the created signature (S) in the inserted node; as well as The metadata (500) is generated for the separate memory object, the metadata having: information indicating that it is a unique memory object (510); and the unique memory object reference (530, U_PTR) to the separate memory object.

20. The apparatus of claim 19, wherein: The analyzer unit (214) and the encoder unit (212) are further configured to, when all memory objects in the computer memory content have been processed (600): traversing the tree data structure (400) to generate a value frequency table for data values ​​of unique memory objects represented by nodes of the tree data structure (400); as well as The generated value frequency table is used to compress each unique memory object through an entropy-based compression scheme.

21. The apparatus of claim 19, wherein: The analyzer unit (214) and the encoder unit (212) are further configured to, when all memory objects in the computer memory content have been processed (600): traversing the tree data structure (400) by examining data values ​​of unique memory objects represented by nodes of the tree data structure (400) and determining one or more base values; as well as Each unique memory object is compressed using the determined one or more base values ​​through a delta encoding based compression scheme.

22. The apparatus of claim 17, wherein: The data values ​​have a finer granularity than the memory objects, and the memory objects have a finer granularity than the computer memory contents.

23. The apparatus of claim 22, wherein: The computer memory contents are pages of computer memory, the memory objects are memory blocks, and each memory block includes a plurality of data values.

24. The device according to claim 23, in, For each memory object of the encoded computer memory contents, the metadata includes: information indicating that the memory object is a duplicate memory object that is eliminated or a remaining unique memory object; When the memory object is a unique memory object, information indicating the type of compression used and a unique memory object reference to the unique memory object; and When the memory object is a replica memory object, a unique memory object reference to a unique memory object, the uncompressed content of which is the same as that of the replica memory object, and wherein the apparatus further comprises a decompression unit (213; 1110) and is configured to: receiving a read request for a memory block in a memory page having a physical memory page address (PA); determining a compressed memory page address (CA) from a lookup table (1010); retrieving metadata for the memory block (1020); Calculating a compressed memory block address (1040) based on the compressed memory page address (CA) and the unique memory object reference (530, U_PTR) of the retrieved metadata; retrieving a compressed memory block (1105) at the calculated compressed memory block address (1040); and The retrieved compressed memory block (1105) is decompressed by a decompressor unit (213) using information (520; 1120, ENC) from the retrieved metadata of the memory block indicating the type of compression.

25. The apparatus according to claim 23, in, For each memory object of the encoded computer memory contents, the metadata includes: information indicating that the memory object is a duplicate memory object that is eliminated or a remaining unique memory object; When the memory object is a unique memory object, information indicating the type of compression used and a unique memory object reference to the unique memory object; and when the memory object is a replica memory object, a unique memory object reference to a unique memory object whose uncompressed content is the same as the replica memory object, wherein, for each memory object that is a unique memory object, the metadata further includes a copy memory object reference to the deleted copy memory object, whose uncompressed content is the same as that of the unique memory object, and Wherein, the device is further configured as: receiving a write-back request involving an update of a unique memory block (830) at an original memory location; copying the unique memory block (830) to a new memory location (870) in a dedicated free memory area (840) of the computer memory contents before updating; updating metadata of a replica memory block (810, 820) linked to the unique memory block (830) so that its replica memory object reference (540, D_PTR) is redirected to a new memory location (870) in the dedicated free memory region (840); and Based on the write-back request, the unique memory block at its original memory location (830) is updated.

26. The device according to claim 25, further configured to: providing metadata including a reference (F_PTR) to a starting address (805) of said dedicated free memory area (840); and After the unique memory block (830) is copied to the new memory location (870) in the dedicated free memory area (840), the reference (F_PTR) is updated to reflect the new starting address (880).

27. The apparatus according to claim 23, in, For each memory object of the encoded computer memory contents, the metadata includes: information indicating that the memory object is a duplicate memory object that is eliminated or a remaining unique memory object; When the memory object is a unique memory object, information indicating the type of compression used and a unique memory object reference to the unique memory object; and when the memory object is a replica memory object, a unique memory object reference to a unique memory object whose uncompressed content is the same as the replica memory object, wherein, for each memory object that is a unique memory object, the metadata further includes a copy memory object reference to the deleted copy memory object, whose uncompressed content is the same as that of the unique memory object, Wherein, the analyzer unit and the encoder unit are further configured to process each individual memory object in the computer memory content by: creating a signature, the signature being a dense representation of the data value of the memory object; traversing the tree data structure to compare the created signature with signatures already represented in the tree data structure; If the created signature does not match any of the signatures represented in the tree data structure: inserting a node into the tree data structure; Entering the created signature in the inserted node; and generating the metadata for the separate memory object, the metadata having: information indicating that it is a unique memory object; and the unique memory object reference to the separate memory object; and If the created signature matches the signature represented in the tree data structure: generating the metadata for the separate memory object, the metadata having: information indicating that it is a replica memory object; and the unique memory object reference to the unique memory block represented by the matching signature in the tree data structure; and updating the metadata for the unique memory block represented by the matching signature in the tree data structure to introduce a duplicate memory object reference to the separate memory object, and Wherein, the device is further configured as: receiving a write-back request involving an update of a unique memory block (830) at an original memory location; discovering that the deduplicated memory block (820) is a duplicate of the unique memory block (830); promoting the discovered deduplicated memory block (820) to be unique in the tree data structure (400) by using the signature (S) of the unique memory block (830); writing the updated contents of the unique memory block (830) to a new memory location (870) in a dedicated free memory area (840) in accordance with the write-back request; and The metadata of the unique memory block (830) is updated so that its unique memory object reference (530, U_PTR) is redirected to the new memory location (870) in the dedicated free memory area (840) and any of its copy memory object references (540, D_PTR) are removed.

28. The device according to claim 27, further configured to: providing metadata including a reference (F_PTR) to a starting address (805) of said dedicated free memory area (840); and After writing the updated contents of the unique memory block (830) to the new memory location (870) in the dedicated free memory area (840) in accordance with the write-back request, the reference (F_PTR) is updated to reflect the new starting address (880).

29. The device of claim 23, further configured to: receiving a write-back request involving an update of a replica memory block (920); storing the contents of the updated copy memory block as a new unique memory block (980) in the dedicated free memory area (940); and The metadata of a unique memory block (910) previously linked to the replica memory block (920) is updated to reflect that the unique memory block (910) is no longer linked to the replica memory block (920), while maintaining any links between the unique memory block (910) and other replica memory blocks (930).

30. The device of claim 28, further configured to: monitoring compression ratios of memory pages over time; and If the compression ratio does not meet a given standard, recompression is performed on the memory page by executing the functions of the method according to claim 1 .

31. The device of claim 28, further configured to: By executing the functions of the method according to claim 1, recompression of memory pages is periodically performed to improve the compression ratio.

32. The apparatus of claim 17, wherein: The encoder unit (212) is configured to exploit data value locality for compression by applying one of the following: Entropy-based compression; and Base value - incremental compression on the fly.

33. A computer system (200), comprising: One or more processors (P1…P N ); One or more computer memories (M1-M k ; C1-C3); and The computer memory compression device (205) of claim 17.

Images