Pattern Detection System for High-Availability Storage Cache Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-availability storage systems face inefficiencies in managing and accessing electronic content due to the lack of effective methods for duplicative content detection and cache memory optimization, leading to unnecessary data writes and storage overhead.
Innovation Solution
A method that compares incoming content to static and dynamic databases within a high-availability storage system, writing it to the cache memory only if unique, and utilizing a temporal database for non-matching content with a defined retention period, while periodically reviewing and managing entries based on hit rates to determine deletion or promotion to the dynamic database.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If content is written to cache memory system without duplication detection, then write speed is improved, but storage efficiency deteriorates due to unnecessary duplicate writes
Solution Approach 1:
The system performs preliminary duplication detection by comparing incoming content against static and dynamic databases before writing to cache memory. This preliminary check prevents unnecessary duplicate writes, resolving the contradiction by filtering out duplicates in advance while allowing unique content to be written at full speed.
Solution Approach 2:
The patent introduces intermediary databases (static database and dynamic database) that act as mediators between incoming content and the cache memory system. These databases store content fingerprints and enable rapid duplication detection without blocking the main write path for unique content, thus maintaining write speed while improving storage efficiency.
2Device complexity
If duplication detection is performed using only static database, then system complexity is reduced, but detection accuracy deteriorates due to inability to detect recent duplicates
Solution Approach 1:
The detection system is segmented into two distinct databases: a static database for historical content fingerprints and a dynamic database for recent content fingerprints. This segmentation allows each database to serve a specific purpose - the static database provides persistent detection capability while the dynamic database captures recent duplicates, together achieving high detection accuracy without excessive complexity.
Solution Approach 2:
The system employs a dynamic database that is continuously updated with recent content fingerprints, making the detection system adaptive to changing content patterns. This dynamic component complements the static database, enabling the system to detect both historical and recent duplicates with high accuracy while maintaining manageable complexity through structured organization.
3Measurement precision
If temporal database retains entries for long period, then detection accuracy is improved, but memory usage increases
Solution Approach 1:
The system implements periodic review of the temporal database to manage its size. Entries are periodically evaluated based on their hit rates, and those with low relevance are removed after a defined retention period. This periodic maintenance ensures the temporal database maintains high detection accuracy for relevant content while preventing unbounded memory growth.
Solution Approach 2:
The temporal database implements a policy of discarding old or low-value entries after a defined retention period while recovering and promoting high-value entries to the dynamic database. This selective discarding and recovering mechanism maintains detection accuracy by preserving important content fingerprints while freeing memory space by removing redundant or expired entries.
Data Source
AI summary
A method, computer program product, and computing system for receiving content on a high-availability storage system. The content is compared to one or more entries in a static database associated with a cache memory system of the high-availability storage system. If the content does not match the one or more entries in the static database, the content is compared to one or more entries in a dynamic database associated with the cache memory system. If the content does not match the one or more entries in the dynamic database: the content is written to the cache memory system and a representation of the content is written to a temporal database associated with the cache memory system and maintained for a defined period of time.