Low Latency Web Analytics Service for Real-Time Traffic Monitoring

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing web site analytics systems suffer from high latency, making it difficult to monitor sudden traffic spikes or drops and track link propagation on social networking sites, and fail to provide timely analytics data, which hinders publishers' ability to notice changes or errors in web site analytics deployment.

Innovation Solution

A low latency web site traffic analytics and monitoring service that receives analytics tracking requests, sends requests to load web pages, and outputs notifications to users about successful registration, utilizing a multi-tiered system for real-time data processing and reporting, including a collecting tier, logging tier, persistent tier, real-time analytics tier, and front-end tier to reduce latency and improve reporting speed.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If existing web site analytics data systems process high volume of web traffic data, then they can provide interactive reports with high fidelity, but they incur high latency from hit occurrence to hit reporting

Engineering Contradiction:

Improvereport fidelityVSAvoidlatency

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system segments the analytics processing into multiple tiers: a collecting tier that receives hits immediately, a logging tier that stores raw data, a persistent tier for long-term storage, and a real-time analytics tier that processes data. This segmentation allows each tier to handle specific functions independently, enabling low-latency reporting without sacrificing the ability to process high volumes of data with high fidelity.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The collecting tier performs preliminary actions by immediately receiving and validating analytics tracking requests as they occur, before the data needs to be processed or reported. This preliminary receipt of data enables the system to provide low-latency responses while still maintaining the ability to process comprehensive data volumes through the subsequent logging and analytics tiers.

Inventive Principle:

Principle #10Preliminary action

2Duration of action of stationary object

If existing web site analytics systems use traditional processing approaches, then they can handle substantial periods of time with high fidelity, but they do not allow reports to segment in fine and granular time basis

Engineering Contradiction:

Improvereporting periodVSAvoidtime segmentation capability

Core Design Contradiction:

Duration of action of stationary objectVSDevice complexity

Solution Approach 1:

The real-time analytics tier dynamically segments reports into fine and granular time intervals based on the specific needs of analytics queries. Rather than using fixed aggregation periods, the system can dynamically create time segments at any granularity level, enabling detailed analysis of traffic patterns within minutes while maintaining the ability to aggregate over substantial time periods when needed.

Inventive Principle:

Principle #15Dynamics

3Quantity of substance

If publishers wait for traditional analytics reporting, then they can receive comprehensive data, but they are unable to view web traffic data in a timely manner

Engineering Contradiction:

Improvedata completenessVSAvoiddata availability time

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The system establishes continuous data flow from the collecting tier through the logging tier to the real-time analytics tier. Hits are continuously received, logged, and processed without interruption, ensuring that comprehensive data is always available immediately upon occurrence. This continuous processing eliminates the need to wait for batch processing cycles while maintaining data completeness.

Inventive Principle:

Principle #20Continuity of useful action

4Measurement precision

If traditional analytics reporting is used, then systems can process data with high fidelity, but publishers fail to notice recently occurring changes in web traffic or erroneous deployment of tracking code

Engineering Contradiction:

Improveanalytics fidelityVSAvoidreal-time detection capability

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The real-time analytics tier provides immediate feedback to publishers about web traffic changes and tracking code deployment status. The system continuously monitors analytics data and can promptly alert publishers to recently occurring changes or errors in tracking code, enabling timely response while maintaining high-fidelity data processing through the multi-tier architecture.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS9628355B1System for validating site configuration based on real-time analytics data

Publication Date: 2017.04.18 GOOGLE LLC

AI summary

Machine-implemented methods for validating a web site analytics configuration are provided. In one aspect, a method includes receiving, with a low latency web site traffic analytics and monitoring service, a registration for a web site, and sending an initial transmission to the registered web site, the transmission configured to generate a response transmission of web site traffic data to the analytics service. The method also includes outputting a notification indicating that the web site has been successfully registered with the analytics service when the traffic data is received by the analytics service, and outputting a notification indicating that the web site has not been successfully registered with the analytics service when the traffic data is not received by the analytics service. Systems, graphical user interfaces, and machine-readable media are also provided.