Shared Metadata Cache Access for Duplicate Request Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud native architectures, frequent metadata requests to a metadata server lead to inefficient resource utilization due to duplicate metadata caching and invalidation, particularly in PostgreSQL caching solutions, resulting in resource waste and increased transmission demands.
Innovation Solution
Implement a metadata access method utilizing a three-layer architecture with a global cache file, cache description array, and memory block to efficiently manage metadata caching, reducing duplicate metadata caches and optimizing resource utilization through layered access and partition version information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a metadata cache is maintained in the computing cluster to reduce metadata request frequency, then the number of metadata requests to the metadata server is reduced, but duplicate metadata caching and invalidation occur leading to resource waste
Solution Approach 1:
The patent merges the metadata caching functionality into a shared memory space that is common across all computing nodes. By establishing a shared metadata cache in the memory space rather than allowing each node to maintain its own independent cache, the system eliminates duplicate caching of the same metadata across different nodes, thereby reducing resource waste while maintaining high request efficiency.
Solution Approach 2:
The shared memory space serves multiple functions: it acts as a universal metadata cache for all computing nodes, provides a common invalidation mechanism, and enables coordinated access to metadata across the entire computing cluster. This universal structure ensures that all nodes access the same metadata copy, eliminating duplication and improving resource utilization.
2Speed
If each computing cluster maintains its own metadata cache, then local metadata access speed is improved, but transmission resources are increased due to duplicate caching and invalidation
Solution Approach 1:
The patent combines multiple local metadata caches into a single shared memory-based metadata cache that is accessible by all computing nodes. This unified cache structure maintains fast local access speeds while eliminating the need for multiple copies of metadata across different nodes, thereby reducing transmission resources spent on synchronizing duplicate data and invalidation messages.
Solution Approach 2:
The shared memory space acts as an intermediary between the metadata server and individual computing nodes. Instead of each node directly querying the metadata server or maintaining separate caches, all nodes access the unified shared cache through this intermediary, reducing the overall transmission burden and eliminating resource waste from duplicate caching operations.
3Loss of energy
If a global cache management system is implemented to reduce duplicate metadata requests, then resource efficiency is improved, but system complexity increases
Solution Approach 1:
The shared memory space provides a universal cache management mechanism that handles multiple functions simultaneously: caching metadata, coordinating invalidation across nodes, and managing cache consistency. By consolidating these functions into a single universal structure rather than requiring separate management systems for each node, the patent improves resource efficiency while keeping the overall system complexity manageable through unified rather than fragmented management.
Data Source
AI summary
A metadata access method, apparatus, and computer-readable storage medium for efficient metadata retrieval through cache management. The method receives metadata query requests including target index information from processes and performs matching operations on a global cache file containing records with index information and slot identifiers. Each cache description array corresponds to memory blocks caching metadata. Upon successful matching, the target cache description array is accessed using the target slot identifier. The data state of target metadata is determined from the cache description array, and target address information indicating the location of the target memory block in shared memory is obtained and returned to the requesting process, enabling efficient shared memory-based metadata access.