Dynamic Memory Expansion via Transparent Data Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge faced by enterprises is the increasing demand for memory capacity to process large volumes of data, which is not efficiently met by traditional methods due to limitations in physical memory expansion, leading to high total cost of ownership (TCO) and performance issues with existing memory compression techniques.
Innovation Solution
A dynamic memory expansion system using transparent data compression, implemented through a controller engine that compresses data in the main memory without altering the operating system, providing a shadow address space for mapping virtual to physical addresses, thereby increasing effective memory capacity with minimal performance overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional physical memory expansion is used to meet increasing data processing needs, then memory capacity is improved, but total cost of ownership increases and performance issues arise
Solution Approach 1:
The patent changes the state of data in memory from uncompressed to compressed form, transforming the density parameter of stored information. By applying compression algorithms to data pages in main memory, the system increases the effective memory capacity without adding physical memory components, thereby avoiding increased TCO while meeting growing data processing demands
Solution Approach 2:
The patent creates a shadow address space that is a virtual copy of the physical address space. This shadow copy allows the system to present a larger memory capacity to applications than physically exists, as multiple virtual addresses can map to the same or fewer physical memory locations through compression, effectively multiplying the usable memory capacity without additional hardware
2Quantity of substance
If memory compression techniques are used to increase effective memory capacity, then memory capacity is improved, but latency increases and performance deteriorates
Solution Approach 1:
The patent performs compression and decompression operations in advance and in parallel with normal processing operations. By pre-compressing data pages before they are needed and pre-decompressing them when anticipated, the system minimizes the time penalty during actual data access, thereby reducing latency while still achieving memory capacity expansion
Solution Approach 2:
The patent introduces a shadow address space as an intermediary layer between the application and the compressed physical memory. This shadow layer handles address translation and compression management transparently, allowing applications to access compressed data without experiencing the full impact of compression overhead, thereby reducing perceived latency while maintaining effective memory capacity expansion
Data Source
AI summary
Dynamic memory expansion based on data compression is described. Data represented in at least one page to be written to a main memory of a computing device is received. The data is compressed in the at least one page to generate at least one compressed physical page and a metadata entry corresponding to each page of the at least one compressed physical page. The metadata entry is cached in a metadata cache including metadata entries and pointers to the uncompressed region of the at least one compressed physical page.