Metadata-Driven Data Collector for Scalable System Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data collection methods in management software are inefficient, cumbersome, and lack scalability due to hard-coded data collectors that require extensive customization and updates for each component, limiting flexibility and scalability.
Innovation Solution
Implementing metadata-driven data collection methods that use metadata to configure and manage data collectors, allowing for dynamic data collection across various protocols and components, enabling flexible data collection and correlation without the need for extensive software updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If hard-coded data collectors are used for each managed component, then data collection can be performed for specific components, but the system becomes inefficient, cumbersome, and error-prone requiring extensive customization for each component
Solution Approach 1:
The patent implements a universal data collection framework where a single data collector can handle multiple component types through configuration files. Instead of having separate hard-coded collectors for each component (storage, network, CPU, etc.), the system uses a unified collector that adapts its behavior based on configuration parameters, making one collector serve multiple functions across diverse component types.
Solution Approach 2:
The patent merges previously separate data collection functionalities into a single unified data collector. Configuration files consolidate the logic for collecting data from different component types (storage devices, network interfaces, CPU metrics) into one structured format, eliminating the need for multiple specialized collectors and reducing system complexity.
2Adaptability or versatility
If hard-coded software agents and drivers are written for each component, then data collection can be customized for specific components, but software updates and code changes are required to support new data or metrics
Solution Approach 1:
The patent introduces dynamic configuration files that allow data collection behavior to be changed without modifying the underlying software code. Configuration files can be updated to collect new metrics, change collection intervals, or adapt to new component types dynamically, making the system flexible and easy to maintain without requiring software updates for each change.
Solution Approach 2:
The data collector automatically adapts to different component types by reading configuration files that specify component characteristics, data paths, and collection parameters. The system serves itself by using configuration metadata to guide data collection behavior, eliminating the need for manual code customization for each component or metric type.
3Reliability
If separate software agents are created for different components supporting the same protocol, then each component can be managed specifically, but the system becomes inefficient and lacks scalability
Solution Approach 1:
The patent creates a universal data collector that can handle multiple component types (storage devices, network interfaces, CPU metrics) through a single unified implementation. The collector uses configuration files to adapt its behavior to different component types and protocols, achieving both component-specific reliability and overall system efficiency without requiring separate agents for each component.
Data Source
AI summary
Methods, systems, and computer readable mediums for performing metadata-driven data collection are disclosed. According to one embodiment, a method for performing metadata-driven data collection includes receiving a request for system related data, wherein the request includes metadata indicating a protocol and identifying information for obtaining the system related data. The method also includes configuring, using the metadata, at least one data collector for obtaining the system related data. The method further includes obtaining, by the at least one data collector, the system related data and storing the system related data in a memory.