Real-Time Web Analytics System with Low-Latency Data Overlay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current web site analytics systems suffer from high latency, making it difficult to monitor sudden traffic spikes or drops and track link propagation on social networking sites, and delaying publishers' ability to view web traffic data and detect errors in analytics tracking code deployment.
Innovation Solution
A multi-tiered real-time analytics system that collects and processes web site traffic data with low latency by routing event tracking data through a collecting tier, logging tier, persistent tier, and real-time analytics tier, enabling concurrent display of traffic data with the web page and allowing for real-time reporting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing web site analytics data systems are used to process high volume web traffic data, then interactive reports over substantial periods of time with high fidelity are provided, but high latency from hit occurrence to hit reporting occurs and fine-granular time basis segmentation is not allowed
Solution Approach 1:
The patent segments the analytics system into multiple specialized components: a high-speed data collection layer using memory-mapped files for immediate data capture, a separate data processing layer, and a reporting layer. This segmentation allows different parts of the system to operate at different speeds and granularities, enabling fine-grained time basis segmentation without requiring the entire system to process data at maximum speed, thus reducing overall latency while maintaining measurement precision.
Solution Approach 2:
The patent introduces memory-mapped files as an intermediary mechanism between data collection and data processing. This intermediary allows data to be captured and stored in real-time with minimal processing overhead, enabling the system to achieve fine-grained time segmentation without the latency that would result from immediate complex processing. The memory-mapped files act as a buffer that decouples data capture from data analysis.
2Productivity
If existing web site analytics reporting systems are used, then web traffic data is reported to publishers, but the high latency makes it difficult to monitor sudden spikes or drops in traffic and track link propagation
Solution Approach 1:
The patent performs preliminary actions by pre-configuring memory-mapped files and data collection mechanisms before traffic events occur. The system is pre-positioned to capture data immediately when hits occur, with data collection structures already in place and ready to receive information. This preliminary preparation eliminates setup delays and enables immediate monitoring of sudden traffic spikes or drops without requiring system reconfiguration.
Solution Approach 2:
The patent implements continuous data collection and reporting operations that run without interruption. The memory-mapped files continuously capture traffic data as it occurs, and the system maintains continuous connections for real-time reporting to publishers. This continuity ensures that no traffic changes are missed and enables immediate detection and reporting of sudden spikes or drops in traffic, eliminating gaps in monitoring.
3Reliability
If existing analytics systems are deployed, then web traffic data is collected and reported, but the high latency causes publishers to fail to notice recently occurring changes or erroneous deployment of tracking code
Solution Approach 1:
The patent implements immediate feedback loops where collected traffic data is rapidly processed and returned to publishers in real-time. The system continuously monitors incoming data and provides feedback about traffic patterns, changes, and potential errors without delay. This rapid feedback mechanism enables publishers to immediately detect erroneous deployments of tracking code or unusual traffic changes, allowing for quick correction and improving overall system reliability.
Data Source
AI summary
Machine-implemented methods for providing information to a graphical user interface for displaying web site traffic information are provided. In one aspect, a method includes acquiring low latency web site traffic data for a web page, and providing the acquired low latency web site traffic data to a graphical user interface for display. The acquired low latency web site traffic data is configured to be displayed concurrently with a display of at least a portion of the web page. Systems, graphical user interfaces, and machine-readable media are also provided.