Memory optimization system for data deduplication under memory overload

Abstract

The invention discloses a memory optimization system for data deduplication under memory overload. The memory optimization system comprises a data page screening module, a read and write request processing module, an all-zero data page filtering module, a duplicated data page deleting module and a data management module, wherein the data page screening module is used for searching stable data pages which are suitable for deduplication in the memory; the read and write request processing module is used for implementing address translation and read / write classification; the all-zero data page filtering module is used for filtering the screened data pages which are subjected to deduplication, and removing the all-zero data pages to save the memory overhead; the duplicated data page deleting module is used for implementing deduplication operation on the remaining pages of the all-zero data pages after being filtered; and the data management module is used for organizing and managing metadata and the memory data pages subjected to deduplication. According to the memory optimization system disclosed by the invention, the utilization rate of the memory can be improved by screening the all-zero pages of the memory and deleting the duplicated data pages, the slow external memory exchange operations under memory overload can be avoided, and the overall performance of the system can be effectively improved.

Description

technical field [0001] The invention relates to the technical field of computer memory optimization, in particular to a memory optimization system for deleting duplicated data under memory overload. Background technique [0002] As the price of main memory continues to decrease, the main memory capacity of a typical computer continues to increase. At the same time, the functions, types and required storage space of system software and applications are also rapidly expanding, and computing memory resources are still insufficient for many applications. Program requirements, especially the massive application of virtualization technology, the maximum number of virtual machines running on a physical machine is usually limited by the memory of the physical machine, and memory resources are still scarce resources of the computer system. When the memory resources cannot meet the needs of the application, the system will use the swap area (swap area) to swap out the memory pages tha...

AI Technical Summary

Problems solved by technology

However, this method only has a good deduplication effect on relatively static memory pages, and it takes a long time to periodically scan anonymous pages to find new merge opportunities

Apply data deduplication technology to system swap operations (such as MSDD), detect and delete duplicate data pages before swapping pages from memory to disk swap area to reduce disk I / O, however, on the one hand, there is no all-zero Page filtering increases deduplication calculation and storage overhead. On the other hand, deduplication data needs to be written to the external storage swap area. No matter how high or low the deduplication rate is, slow external storage read and write cannot be avoided.

Images