Query-Time Sessionization Using Sampled Event Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently processing and analyzing large datasets for query-time sessionization and analysis, leading to significant computational burdens and slowdowns in real-time data analytics due to the complexity of understanding data models and the need for extensive computing resources.
Innovation Solution
A query mechanism that dynamically performs sessionization and analysis at query-time by executing queries against a table of independent events, limiting the number of sessions analyzed based on specified criteria, and using unbiased random sampling to reconstruct sessions, thereby reducing the need for extensive computing resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large datasets are processed for query-time sessionization and analysis, then analysis completeness is improved, but processing time and computational resources increase significantly
Solution Approach 1:
The patent applies partial action by using random sampling to analyze only a subset of sessions rather than processing the entire large dataset. The sampling framework allows users to specify the number of sessions to analyze, providing sufficient statistical representation without the computational burden of processing all data, thus resolving the contradiction between analysis completeness and processing time.
Solution Approach 2:
The patent changes the parameter of data volume by introducing a sampling rate or sample size parameter that can be adjusted. This allows the system to transform the processing task from analyzing the full dataset to analyzing a controlled sample, reducing processing time while maintaining analytical validity through statistical sampling theory.
2Measurement precision
If large datasets are processed for query-time sessionization and analysis, then analysis completeness is improved, but computational resources and system complexity increase
Solution Approach 1:
The patent reduces computational resource requirements by applying partial action through random sampling. Instead of processing the entire large dataset which would demand extensive computational resources, the system analyzes only a statistically valid sample, significantly reducing the computational burden while maintaining analysis quality.
Solution Approach 2:
The patent creates a representative copy of the data through random sampling. This sampled copy preserves the essential statistical properties of the original large dataset at a much smaller size, allowing analysis to be performed on the copy rather than the full data, thereby reducing computational resource requirements.
3Productivity
If query-time sessionization is implemented, then real-time analytics speed is improved, but data model complexity increases
Solution Approach 1:
The patent segments the data processing task into independent sampling operations rather than requiring complex real-time sessionization of the entire dataset. By sampling sessions after they are formed and using simplified analysis operations on the samples, the system achieves real-time analytics speed without the complexity of processing all data through complex data models.
Data Source
AI summary
A system and method for implementing an iterative query mechanism to facilitate query-time sessionization and analysis of data is disclosed. At least, the method includes determining a table of independent events, determining a query, the query including a parameter specifying a size limit on a sample of sessions, executing the query against the table of independent events, at the time of query execution, processing the table of independent events to reconstruct the sample of sessions approaching the size limit, and at the time of query execution, analyzing the sample of reconstructed sessions to generate a result.