Distributed Metadata Management via Global Process Lookup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large parallel and distributed systems, managing process metadata across multiple processors leads to performance issues due to centralization, where a single entity becomes a bottleneck as the number of processes increases, making it difficult to efficiently locate and access metadata.
Innovation Solution
Implementing a distributed approach using a global process look-up data structure (GPLD) where each process is associated with a home processor, which stores and updates metadata, allowing processors to efficiently locate and retrieve metadata using process IDs, and enabling metadata replication for improved performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a centralized entity is used to manage process metadata, then metadata access is simplified, but performance deteriorates due to bottlenecks as the number of processes increases
Solution Approach 1:
The patent divides the centralized metadata management into distributed home processors, where each processor manages metadata for specific processes. This segmentation eliminates the single-point bottleneck by distributing the metadata management workload across multiple processors, allowing simultaneous access without performance degradation.
Solution Approach 2:
The patent introduces a global process lookup table (GLPT) as an intermediary data structure that maps process IDs to home processors. This intermediary enables efficient routing of metadata requests to the correct home processor without requiring a centralized management entity, thus maintaining ease of access while improving overall system performance.
2Quantity of substance
If the number of processes in the multiprocessor system increases, then system capability is improved, but metadata management difficulty increases
Solution Approach 1:
By segmenting metadata management responsibilities across multiple home processors based on process ID ranges or hashing, the system can handle increased process quantities without proportionally increasing management complexity. Each home processor manages a subset of processes independently.
Solution Approach 2:
The patent uses the global process lookup table as a copied indexing structure that replicates process-to-home-processor mappings across the system. This copying mechanism allows any processor to quickly determine which home processor manages any given process, maintaining management efficiency even as process numbers grow.
3Ease of operation
If a centralized entity manages all process metadata, then metadata location is simplified, but access speed deteriorates due to search difficulties
Solution Approach 1:
The global process lookup table serves as an intermediary indexing structure that provides O(1) or O(log n) lookup of home processor identities based on process ID. This eliminates linear search through centralized metadata structures, dramatically improving access speed while maintaining simplified location logic through the lookup table.
Solution Approach 2:
The system performs preliminary organization of process metadata by assigning each process to a home processor and pre-storing this mapping in the global process lookup table. When metadata is needed, the system first performs a quick preliminary lookup to identify the home processor, then directly accesses the pre-located metadata without searching.
Data Source
AI summary
A method for metadata management for scalable processes, involving creating a process by a first home processor, wherein the process is associated with a process identification (ID), storing the processor ID and information identifying the first home processor in a global process look-up data structure (GPLD), requesting metadata associated with the process, searching the GPLD to obtain the first home processor of the process using the process ID, and retrieving the metadata associated with the process from the first home processor.