Network Log Time Sanitization for Distributed Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In geographically distributed computer systems, relying on device timestamps for log events can lead to incorrect analysis due to varying clock configurations across different devices, especially when geographic locations are not accurately reflected in time zones, and documentation about device configurations is scarce.
Innovation Solution
The solution involves translating timestamps from log messages into UTC time using either an active approach, where probes are sent to determine clock configurations, or a passive approach that relies solely on log message contents without additional configuration information, ensuring consistent timestamp representation across devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If device timestamps are used for log events, then local time accuracy is improved, but consistency across geographically distributed devices deteriorates
Solution Approach 1:
The system changes the time zone parameter of each logging device to UTC, transforming local timestamps into a standardized format. This parameter change ensures that all devices report times in the same reference frame, resolving the consistency issue while preserving the precision of each device's local clock through mathematical adjustment.
Solution Approach 2:
The patent creates a temporal equipotential state by setting all devices to UTC time zone. This eliminates time zone differences between devices, ensuring that timestamps from geographically distributed devices are directly comparable without requiring complex time zone conversions or adjustments.
2Reliability
If NTP synchronization is implemented, then clock synchronization is improved, but time zone configuration complexity increases
Solution Approach 1:
The patent extracts the time zone configuration step entirely from the synchronization process by setting all devices to UTC. This removes the complexity of configuring different time zones for different devices while maintaining NTP's clock synchronization functionality, simplifying the overall configuration process.
Solution Approach 2:
Instead of allowing devices to use their local time zones and then converting timestamps, the patent inverts the approach by setting all devices to UTC from the beginning. This eliminates the need for time zone conversion logic and simplifies both configuration and processing.
3Adaptability or versatility
If local time zones are used, then geographic relevance is improved, but analysis accuracy deteriorates
Solution Approach 1:
The patent introduces UTC as an intermediary time reference that mediates between local device times and centralized analysis requirements. By converting all timestamps to UTC, the system maintains the ability to track events accurately while providing a common reference frame that eliminates geographic time zone variations from the analysis.
Data Source
AI summary
Time correction records are created for correcting timestamps of network logs to identify timing of network events in a predetermined time reference frame, the network logs being created by logging devices generating the timestamps in device time reference frames. For each logging device, one or more network events are generated or identified at respective event times in the predetermined time reference frame, each network event having a corresponding event-related network log from the logging device and a respective timestamp in a device time reference frame. For each network event, a respective difference value is calculated as a difference between the event time and a respective timestamp from a network log. For each logging device, a selection function is applied to the difference values to calculate a correction value, and the correction value is stored along with an identifier of the logging device in a time correction record.