Session Based Web Usage Reporter Real-Time Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current web analytics systems face significant delays in processing and presenting tracking data due to the time-consuming nature of data validation and storage, particularly in eliminating duplicates and detecting click fraud, which hampers real-time analysis and decision-making for website owners.
Innovation Solution
The system groups user click data into sessions and processes it in RAM using session transformers, enabling real-time analysis and presentation before database storage, and utilizes a multithreaded, massively parallel model to handle large volumes of sessions, with data organized using session IDs and hashing algorithms for efficient data routing and analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data is validated and stored in a database before analysis, then data accuracy is improved, but processing time increases significantly
Solution Approach 1:
The patent applies preliminary action by performing data validation, deduplication, and fraud detection in advance during the data collection phase, before data is stored in the database. The system validates data formats, detects duplicate clicks, and identifies fraudulent patterns as data arrives at the analytics server, so that when data is later retrieved for analysis, it is already clean and ready for immediate processing.
Solution Approach 2:
The patent segments the data processing workflow into distinct phases: data collection with initial validation, data storage, and data analysis. By separating validation operations from storage and analysis operations, the system can validate data in the background without blocking the analysis pipeline, thus reducing the perceived processing time while maintaining data accuracy.
2Reliability
If comprehensive data validation and fraud detection is performed, then data reliability is improved, but processing speed decreases
Solution Approach 1:
The patent applies partial action by implementing selective validation and fraud detection based on risk levels. The system performs basic validation on all data points but applies more intensive fraud detection algorithms only to suspicious patterns or high-value transactions. This layered approach maintains data reliability for critical operations while preserving processing speed for routine operations.
Solution Approach 2:
The system implements self-service fraud detection by using machine learning models that automatically learn from historical data and independently identify fraudulent patterns without requiring manual review of every data point. The fraud detection system serves itself by continuously improving its algorithms based on detected patterns, maintaining high reliability while minimizing processing overhead.
3Loss of information
If data is processed and stored in a database before presentation, then data completeness is improved, but real-time analysis capability is reduced
Solution Approach 1:
The patent extracts and pre-processes essential analytical features from the raw data during the data collection phase, storing only the processed results in the database. Instead of storing all raw clickstream data and processing it later, the system extracts key metrics such as session duration, page view counts, and conversion events during data collection, storing these aggregated features for immediate analysis while maintaining data completeness.
Solution Approach 2:
The patent introduces an intermediary data layer between raw data collection and database storage. This intermediary layer performs real-time data transformation, aggregation, and validation, converting raw clickstream data into structured analytical metrics before storage. This intermediary processing enables both complete data capture and rapid analysis by pre-computing analytical features.
Data Source
AI summary
A system groups the data into sessions to allow tracking and evaluation of individual user behavior. By grouping clicks of a user in a session, the pattern of clicks can be observed, such as which path or pattern of clicks leads to a purchase. In particular, the session data is organized by session, using session transformers or “sessionizers,” before it is provided for database storage, enabling real-time session based analytics.