Ruleset Engine for Homogeneous Data Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex computing systems with diverse components and autonomous data sources present challenges in data collection due to heterogeneity and complexity, requiring a method to aggregate data into homogenous sets and provide relative availability scores without requiring high expertise or monolithic development.
Innovation Solution
A ruleset engine is used to transform heterogeneous data from diverse sources into standardized, flattened data sets, enabling abstraction and reducing the need for hardcoded data processing, and an application server generates relative availability scores by defining failure conditions and aggregating metrics across interconnected systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is collected from diverse autonomous systems with heterogeneous data structures, then data coverage and system visibility are improved, but data processing complexity and client code complexity increase
Solution Approach 1:
The patent introduces an intermediary layer (data collection service and normalization service) between the diverse data sources and the client code. This intermediary automatically collects data from multiple autonomous systems with heterogeneous structures, normalizes it to a unified format, and presents it to clients in a consistent manner, thereby eliminating the need for clients to handle diverse data structures directly
Solution Approach 2:
The patent segments the data collection and processing functionality into separate, independent services (data collection service, normalization service, scoring service). Each service handles a specific aspect of data processing, allowing them to be developed, deployed, and maintained independently. This segmentation reduces the complexity burden on client code by isolating processing logic in dedicated services
2Adaptability or versatility
If autonomous systems are allowed to change independently without centralized control, then system autonomy and flexibility are improved, but data structure consistency and integration difficulty worsen
Solution Approach 1:
The patent applies parameter changes by transforming diverse data structures into a unified normalized format through the normalization service. This service automatically adapts to different data structures from various autonomous systems and converts them to a consistent parameter set, allowing systems to maintain their autonomy while ensuring data consistency at the integration layer
Solution Approach 2:
The patent creates a universal data interface that can handle multiple different data structures from various autonomous systems. The normalization service provides multi-functional capability to process different data formats (JSON, nested structures, flat structures) and convert them to a unified format, enabling consistent data integration across autonomous systems without requiring them to adopt a specific structure
3Measurement precision
If comprehensive metrics are collected to determine system health, then measurement accuracy and system visibility are improved, but processing time and expertise requirements increase
Solution Approach 1:
The patent implements preliminary action by pre-defining failure conditions, metrics, and normalization rules in the system architecture. The normalization service pre-processes data into a standardized format, and the scoring service pre-configures health metrics and failure conditions, so that when data arrives, it can be quickly evaluated against predefined criteria without requiring extensive real-time processing or expert analysis
Solution Approach 2:
The patent enables self-service through automated data collection, normalization, and scoring processes. The system automatically collects data from autonomous systems, normalizes it without human intervention, evaluates it against predefined failure conditions, and generates health scores. This automation reduces both processing time and the need for expert analysis while maintaining comprehensive measurement accuracy
Data Source
AI summary
A method, performed by a server, for aggregating data in from devices within a complex network including receiving, at a network interface, an end data from a data source wherein the end data includes a source field having a source value, determining, by a processor, a subset of a ruleset for the data source in response to a source stanza within the ruleset, generating, by the processor, a results data structure in response to the end data and the subset of the ruleset wherein the results data structure includes a destination field generated in response to the source field and a transform instruction in the subset of the ruleset and a destination value generated in response to the source value and the transform instruction in the subset of the ruleset, and storing, within a memory, a destination value and the destination field as a key value pair.