Data Fabric Node Architecture for Efficient Caching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large-scale distributed business applications face significant resource drain due to the need to manage and maintain large numbers of real-time streaming objects, which is inefficient in existing data replication and caching systems.
Innovation Solution
A data fabric node architecture that includes data fabric nodes, content adapters, and an orchestration service, which applies policies to manage content caching, partitions content into shards, and publishes assignment metadata, enabling efficient data replication and caching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If explicit removal or refresh operations are performed on Objects in key-value store solutions, then Objects can be kept alive and in cache, but large computing resource drain occurs due to the large numbers of Objects that must be attended to
Solution Approach 1:
The patent implements automatic Object lifecycle management where the data fabric system autonomously handles Object creation, replication, caching, and removal without requiring explicit user operations. The system self-monitors Object state and automatically performs necessary actions to maintain cache availability, eliminating the need for manual refresh or removal operations that consume computing resources.
Solution Approach 2:
The system employs continuous monitoring and feedback mechanisms to track Object state, cache performance, and system resources. Based on this feedback, the data fabric dynamically adjusts Object replication, caching strategies, and removal timing to maintain reliability while optimizing resource consumption. The system responds to changing conditions automatically without requiring external intervention.
2Reliability
If data is replicated across multiple nodes in a distributed data store, then data availability and fault tolerance are improved, but system complexity and resource management overhead increase
Solution Approach 1:
The patent segments the distributed data store into independent data fabric nodes, each capable of autonomous operation. Objects are divided into replicable units that can be independently managed across nodes. This segmentation allows the system to achieve fault tolerance through replication while keeping each node's complexity manageable through modular design and independent operation.
Solution Approach 2:
The data fabric nodes are designed as universal, multi-functional units that can perform data storage, replication, caching, and lifecycle management operations. Each node can handle multiple Object types and workloads, reducing overall system complexity by eliminating the need for specialized components for different functions. The universal node design simplifies deployment and management while maintaining distributed reliability.
Data Source
AI summary
Embodiments operate a data fabric node architecture comprising one or more data fabric nodes, one or more content adapters and an orchestration service. Embodiments, at the orchestration service, receive content orchestration requests, response content bindings and content metadata from the data fabric nodes. Embodiments apply policies to choose assignments of content to caches and partition a result content as a whole into shards associated with orchestration targets. Embodiments then publish assignment metadata to the data fabric node architecture.