Hierarchical Cache Line Tracker for Virtualized Memory

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing methods for tracking modified cache lines in computing systems are inefficient, especially when only a small part of a page is modified, as they require writing back the entire page to persistent storage, which is wasteful in terms of memory and bandwidth due to the need to track cache lines at the page level.

Innovation Solution

Implementing a hierarchical data structure in an FPGA to track cache line states, using a cache line tracker that monitors cache coherence events and updates a hierarchical bitmap structure based on cache line addresses, allowing for efficient identification and management of modified cache lines without writing back entire pages.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If page-level tracking is used to monitor modified data in memory, then the implementation is simple using existing page tables, but the memory and bandwidth requirements increase significantly because entire pages must be written back to persistent storage even when only small portions are modified

Engineering Contradiction:
Improveease of implementationVSAvoidmemory and bandwidth requirements
Core Design Contradiction:
Ease of manufactureVSQuantity of substance

Solution Approach 1:

The patent divides the memory tracking granularity from page-level to cache line-level by implementing a cache line tracker that monitors individual cache lines. The tracker uses a hierarchical bitmap data structure where each bit represents the dirty state of a specific cache line, enabling selective tracking of only modified cache lines rather than entire pages. This segmentation allows the system to write back only the necessary cache lines to persistent storage, reducing memory and bandwidth requirements while maintaining implementation feasibility through the structured bitmap approach.

Inventive Principle:
Principle #1Segmentation

2Quantity of substance

If cache line-level tracking is implemented using a hierarchical bitmap data structure, then memory and bandwidth requirements are reduced by tracking only modified cache lines, but the device complexity increases due to the need for monitoring hardware and coherence event tracking

Engineering Contradiction:
Improvememory and bandwidth requirementsVSAvoidtracking system complexity
Core Design Contradiction:
Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent introduces a cache line tracker as an intermediary component that sits between the processing hardware and persistent storage. This tracker includes monitoring hardware that intercepts cache coherence events (such as write requests) on the coherence interconnect, determines which cache lines are modified, and updates the hierarchical bitmap accordingly. By placing this intermediary layer, the system achieves fine-grained cache line tracking without requiring complex modifications to the existing memory hierarchy or storage subsystem, as the tracker handles all the complexity of monitoring and tracking internally.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Reliability

If the entire page is written back to persistent storage when any cache line is modified, then data consistency is maintained, but the write-back efficiency decreases due to unnecessary data transfer of unmodified cache lines

Engineering Contradiction:
Improvedata consistencyVSAvoidwrite-back efficiency
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

The patent applies partial action by implementing selective write-back of only the dirty cache lines identified in the hierarchical bitmap, rather than writing back entire pages. The monitoring hardware tracks which specific cache lines have been modified by detecting cache coherence events, and the write-back operation is triggered only for those marked as dirty in the bitmap. This partial approach maintains data consistency for modified cache lines while avoiding unnecessary data transfer of clean cache lines, thereby significantly improving write-back efficiency without compromising reliability.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS11880309B2Method and system for tracking state of cache lines
Publication Date: 2024.01.23 VMWARE INC
  • US11880309B2 patent drawing
  • US11880309B2 patent drawing
  • US11880309B2 patent drawing

AI summary

The state of cache lines transferred into an out of caches of processing hardware is tracked by monitoring hardware. The method of tracking includes monitoring the processing hardware for cache coherence events on a coherence interconnect between the processing hardware and monitoring hardware, determining that the state of a cache line has changed, and updating a hierarchical data structure to indicate the change in the state of said cache line. The hierarchical data structure includes a first level data structure including first bits, and a second level data structure including second bits, each of the first bits associated with a group of second bits. The step of updating includes setting one of the first bits and one of the second bits in the group corresponding to the first bit that is being set, according to an address of said cache line.