Hardware-Software Cache Management for Pipeline Throughput

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing cache management systems implemented by CPU software in external storage systems suffer from long operation times, delays, and reduced throughput due to serial processing and mutual exclusion locks, affecting real-time performance and throughput rates.

Innovation Solution

A cache management method and apparatus that combines hardware and software approaches, processing high-real-time requests in a pipeline manner by a hardware cache manager and handling lower-real-time tasks with a software cache manager, utilizing a pipeline architecture to parallelize operations and reduce processing delays.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If CPU software is used for cache management, then flexibility and adaptability are improved, but operation time increases and throughput decreases

Engineering Contradiction:

Improvecache management flexibilityVSAvoidthroughput rate

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The cache management system is segmented into two parts: a hardware cache manager for high-speed operations and a software cache manager for complex management tasks. The hardware cache manager handles time-critical operations like cache line allocation, validity checks, and data transfer using dedicated circuits and FIFO queues, while the software cache manager handles policy decisions and replacement algorithms, allowing each component to operate at its optimal speed

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A hardware cache manager acts as an intermediary between the storage device and the CPU software cache manager. It receives requests from the storage device, performs preliminary processing in hardware, and only passes completed operations to the software cache manager, thereby reducing the software cache manager's workload and improving overall throughput

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If serial processing with mutual exclusion locks is used, then data consistency is improved, but operation delay increases

Engineering Contradiction:

Improvedata consistencyVSAvoidoperation delay

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The cache management operations are segmented into independent parallel streams handled by multiple FIFO queues in the hardware cache manager. Each queue can process requests independently without requiring mutual exclusion locks, as the hardware architecture ensures atomic operations through dedicated circuitry. This parallel processing eliminates serialization delays while maintaining consistency through hardware-enforced operation ordering

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The software-based mutual exclusion locking mechanism is replaced with a hardware-based arbitration system. The hardware cache manager uses dedicated circuits and control logic to manage concurrent access to cache resources, eliminating the need for software locks and their associated delays. The hardware naturally handles concurrent operations through its parallel architecture and atomic instruction support

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Speed

If hardware pipeline processing is implemented, then real-time performance is improved, but device complexity increases

Engineering Contradiction:

Improvereal-time performanceVSAvoidhardware structure complexity

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

The hardware cache manager is segmented into distinct functional modules: request reception units, FIFO queue structures, hash calculation units, validity check modules, and data transfer units. Each module performs a specific function in the processing pipeline, allowing for modular design and implementation. This segmentation makes the complex hardware system more manageable and maintainable while enabling parallel processing across modules

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The hardware cache manager is designed with multi-functional components that can handle different cache operations (allocation, validation, eviction, data transfer) using the same basic pipeline structure. The FIFO queues and processing units are configured to handle various request types uniformly, reducing the need for separate dedicated hardware for each operation type and thereby controlling complexity

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12423239B2Cache management method, apparatus and system, device and medium

Publication Date: 2025.09.23 INSPUR SUZHOU INTELLIGENT TECH CO LTD

AI summary

A cache management method, apparatus and system, a device, and a nonvolatile readable storage medium. The method includes: receiving, by a storage system cache manager, a target operation request sent by a target user; processing the target operation request in a pipeline manner, so as to execute a processing operation corresponding to the target operation request on a cache line using the target operation request; and feeding back a processing result of the target operation request to the target user, and executing a cache maintenance operation corresponding to the processing result according to the processing result by means of a software cache manager.