Mobile State Object for Cluster Metadata Scalability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cluster computing systems face scalability issues due to centralized database architectures, which lead to performance bottlenecks and increased load on resources, and require significant setup and management, restricting resource utilization and flexibility.
Innovation Solution
Implementing a mobile state object that uses a database as an envelope for job metadata, enabling real-time tracking and event-driven resource management at a sub-computer level, allowing for the decomposition and distribution of jobs at an atomic level to available resources without the need for agents or polling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a centralized database architecture is used to store job metadata, then metadata storage and access are simplified, but the system scalability is reduced and performance bottlenecks occur
Solution Approach 1:
The patent divides the centralized metadata storage into distributed state objects that are segmented across multiple compute nodes. Each state object contains a portion of the metadata and can be independently accessed, eliminating the single-point bottleneck of centralized databases while maintaining operational simplicity through standardized access interfaces.
Solution Approach 2:
The patent transitions from a single-dimensional centralized storage model to a multi-dimensional distributed architecture where metadata is organized across spatial (multiple nodes) and hierarchical (state objects within nodes) dimensions. This dimensional expansion enables parallel access paths and improves scalability without sacrificing ease of operation.
2Ease of operation
If a centralized database architecture is used to store job metadata, then metadata management is simplified, but the load on database resources increases
Solution Approach 1:
The patent segments the metadata storage load across multiple compute nodes, with each node maintaining local state objects. This distribution prevents any single database resource from becoming overloaded, as the metadata management workload is parallelized across the cluster while maintaining simplified access through standardized state object interfaces.
Solution Approach 2:
Each compute node in the patent maintains its own state objects locally, enabling self-service metadata management without requiring centralized database queries. Nodes can independently access and update their local metadata, eliminating the need for external database resources and reducing overall system load.
3Adaptability or versatility
If protocol messages are used to transport metadata between nodes, then standardized communication is achieved, but CPU processing time increases due to serialization and parsing
Solution Approach 1:
The patent extracts the metadata from protocol message envelopes and places it directly into state objects that can be transferred between nodes. This eliminates the serialization and parsing overhead of protocol messages while maintaining standardized communication through defined state object interfaces, significantly reducing CPU processing time.
Solution Approach 2:
Instead of embedding metadata within protocol messages (traditional approach), the patent inverts the approach by making state objects the primary transport mechanism. This reversal eliminates unnecessary protocol overhead while maintaining communication standards through standardized state object schemas and access methods.
4Reliability
If agents are installed on managed nodes for resource management, then resource status tracking is improved, but device complexity and setup requirements increase
Solution Approach 1:
The patent enables compute nodes to autonomously publish their resource status and capabilities through standardized state objects without requiring external agents. This self-service approach maintains reliable resource tracking while eliminating the complexity of agent installation, configuration, and maintenance across the cluster.
Solution Approach 2:
The state object architecture in the patent serves multiple functions simultaneously: it stores job metadata, tracks resource status, and enables communication between nodes. This multi-functionality replaces the need for separate agents while maintaining comprehensive resource monitoring and management capabilities.
5Reliability
If polling techniques are used to gather resource information, then resource status is monitored, but network traffic and system load increase
Solution Approach 1:
The patent replaces periodic polling with event-driven updates where state objects are published and updated only when changes occur. This eliminates unnecessary periodic network traffic while maintaining reliable resource status monitoring through real-time event notification mechanisms.
Solution Approach 2:
The patent implements a feedback mechanism where compute nodes automatically publish state object updates when their status changes. This feedback-driven approach replaces passive polling with active notification, reducing network traffic to only when necessary while maintaining continuous and reliable resource monitoring.
Data Source
AI summary
Embodiments of a mobile state object for storing and transporting job metadata on a cluster computing system may use a database as an envelope for the metadata. A state object may include a database that stores the job metadata and wrapper methods. A small database engine may be employed. Since the entire database exists within a single file, complex, extensible applications may be created on the same base state object, and the state object can be sent across the network with the state intact, along with history of the object. An SQLite technology database engine, or alternatively other single file relational database engine technologies, may be used as the database engine. To support the database engine, compute nodes on the cluster may be configured with a runtime library for the database engine via which applications or other entities may access the state file database.