Network Device Audit Logs with Pointer-Based Configuration Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network management systems face challenges in efficiently storing and managing large volumes of network events due to high storage requirements and limitations in event logs, leading to the loss of critical event data and inconsistent availability, which hampers network diagnosis and debugging.
Innovation Solution
A network management system (NMS) stores event information with pointers to network information, reducing duplicate storage and utilizing separate data stores for configuration and telemetry data, allowing for extended event logging and improved query capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If network management systems store complete event information including network information in event stores, then event data availability is improved, but storage requirements increase significantly
Solution Approach 1:
The system segments event information into two separate stores: an event store containing only event metadata (event ID, timestamp, source) and a network information store containing actual network configuration data. This segmentation allows the event store to remain lightweight while network information is preserved separately, resolving the contradiction between maintaining event data availability and reducing storage requirements.
Solution Approach 2:
The system introduces a network information store as an intermediary component that holds network configuration data. Instead of duplicating this data in every event record, the event store contains references (pointers) to the actual network information stored in this intermediary store, thereby reducing storage redundancy while maintaining data accessibility.
2Reliability
If network management systems increase event log retention period, then diagnosis capability is improved, but storage space consumption increases
Solution Approach 1:
By segmenting event metadata from network information into separate stores, the system enables independent retention policies. The event store can retain metadata for extended periods to support diagnosis, while the network information store manages its own retention based on actual data needs, optimizing storage space utilization.
Solution Approach 2:
The system stores only references (pointers) to network information in the event store rather than copying the actual network data. This allows the event store to grow to support long-term retention for diagnosis purposes without proportionally increasing storage space consumption, as the same network information is referenced rather than duplicated.
3Ease of operation
If network management systems store duplicate network information in event stores, then query processing is simplified, but resource utilization decreases
Solution Approach 1:
The network information store serves as an intermediary that centralizes network configuration data. The event store contains simple references to this intermediary store, allowing query processing to access network information through the reference rather than requiring duplicate data to be stored and processed, thereby improving resource utilization.
Solution Approach 2:
The system extracts network information from the event store and places it in a separate network information store. The event store is then stripped of redundant network data, retaining only essential event metadata. This extraction reduces the data volume that requires processing while query operations can still access network information through the separated store via references.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system includes computer-readable media configured to store an event store associated with a network. Processing circuitry, in response to a determination that an event associated with the network device has occurred stores, at a time series store, configuration information for a network device in the network and stores, at the event store, event information comprising a pointer to the configuration information and an indication of a time that the event occurred. In response to receiving a request for data corresponding to the event, the processing circuitry determines, using the event information in the event store, the pointer to the network information for the event, obtains, from the time series store, the configuration information for the event using the pointer, generates a response to the request based on the configuration information obtained from the time series store, and outputs the response to the request.