Polling-Based Data Retrieval for Distributed Network Management

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional storage area network management systems face challenges with scalability and security due to the push model of data retrieval, which leads to bottlenecks and overloading of data retrieval components, and requires multiple security holes in firewalls, making it inefficient and prone to downtime.

Innovation Solution

Implementing a polling-based data retrieval process where data collection agents communicate with a management server to schedule data retrieval, allowing for intelligent polling that balances network bandwidth and processor load, reducing the need for constant firewall openings and enabling real-time data collection without overloading the system.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If a push model is used for data retrieval, then real-time data collection is achieved, but data retrieval components become overloaded and bottlenecks occur

Engineering Contradiction:

Improvereal-time data collection capabilityVSAvoidsystem availability

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent inverts the conventional push model by implementing a pull model where data retrieval components initiate requests to data collection agents rather than agents pushing data continuously. This inversion eliminates the overload on retrieval components while maintaining real-time data collection through scheduled polling mechanisms.

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The patent implements periodic polling where data retrieval components schedule regular intervals to request data from agents. This periodic action maintains real-time data availability while preventing system overload by spacing out data retrieval operations and allowing agents to prepare data batches efficiently.

Inventive Principle:

Principle #19Periodic action

2Ease of operation

If multiple security holes are opened in firewalls for agent communication, then data collection from remote agents is enabled, but system security is compromised

Engineering Contradiction:

Improveremote data collection capabilityVSAvoidfirewall security vulnerability

Core Design Contradiction:

Ease of operationVSObject-affected harmful factors

Solution Approach 1:

The patent extracts the security requirement from the communication model by eliminating the need for agents to initiate outbound connections. Only the central server requires firewall access, reducing security holes from multiple agent connections to a single server connection, thereby maintaining remote data collection while significantly improving security.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent introduces a centralized server as an intermediary between the external network and data collection agents. All communication funnels through this single secured point rather than requiring direct firewall access from multiple agents, acting as a mediator that maintains security while enabling distributed data collection.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If data collection agents execute remotely on host systems, then distributed data gathering is achieved, but network bandwidth consumption increases

Engineering Contradiction:

Improvedistributed deployment capabilityVSAvoidnetwork bandwidth consumption

Core Design Contradiction:

Adaptability or versatilityVSLoss of energy

Solution Approach 1:

The patent implements preliminary action by having data collection agents prepare and stage data locally before retrieval is requested. Agents collect and buffer data in batches during idle periods, so when the central server polls for data, it is immediately available. This preliminary preparation reduces the frequency and intensity of network transmissions while maintaining distributed deployment.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS9418110B1Intelligent, scalable, low-overhead mechanism for data retrieval in a distributed network environment

Publication Date: 2016.08.16 EMC IP HLDG CO LLC

AI summary

Techniques disclosed herein describe a data retrieval process for storing management data from a network environment using an efficient polling-based approach without sacrificing the real-time aspect of data retrieval that a push model provides. The process is highly scalable and reliable, and is useful in Network Address Translation (NAT) environments because the number of holes in a NAT firewall is minimized. The data retrieval component uses intelligent polling to retrieve data and store the data in a central database without overloading the system. A data retrieval component polls a data collection agent for a dataset. The data collection agent transfers the data set and a poll indication of when a subsequent data set will be ready for transfer. Subsequent polling is based on a processor load at a data retrieval component.