Partitioned Batch Processing for Usage Analysis Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing number of mobile applications and vast amount of usage events generated daily pose challenges for mobile application providers in analyzing user behavior effectively, as existing systems struggle to process and store this data in real-time, leading to potential latency and resource overload.
Innovation Solution
A usage analysis system that partitions and processes session events using unique application instance IDs and application IDs, allowing for batch processing and decoupling of session event processing from application usage update processing, thereby reducing the load on storage and computation resources and enabling real-time data aggregation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If real-time processing of all usage events is implemented, then measurement precision is improved, but use of energy and computational resources worsen
Solution Approach 1:
The patent segments usage events into two categories: session events (grouped by application instance) and application events (aggregated by application). Session events are processed in batches to update application instance records, while application events are aggregated and processed separately to update application records. This segmentation allows the system to maintain accurate usage analysis while reducing the frequency and intensity of computational operations, thereby lowering resource consumption.
2Measurement precision
If all usage events are processed and stored individually, then measurement precision is improved, but device complexity worsens
Solution Approach 1:
The patent merges multiple usage events into aggregated batches for processing. Application instance usage updates are merged into batches that are processed together, and application usage updates are merged from multiple application instances. This merging reduces the number of individual processing operations and storage operations required, simplifying the system architecture while maintaining complete usage information through proper batch management and aggregation.
3Productivity
If real-time processing of millions of usage events is implemented, then productivity is improved, but loss of time due to processing latency worsens
Solution Approach 1:
The patent implements periodic batch processing where session events are collected over time and processed in periodic batches to update application instance records. Similarly, application usage updates are processed in periodic batches. This periodic action allows the system to accumulate and process large volumes of events efficiently, achieving high throughput while the batch intervals are optimized to minimize latency for time-sensitive operations.
4Measurement precision
If detailed tracking of each usage event is maintained, then measurement precision is improved, but loss of information due to data overload worsens
Solution Approach 1:
The patent performs preliminary grouping and aggregation of usage events into structured batches before final processing and storage. Session events are preliminarily grouped by application instance, and application events are preliminarily aggregated by application. This preliminary action organizes the data in advance, ensuring that no usage information is lost during processing while making the data more manageable and reducing the risk of information loss due to overload or processing errors.
Data Source
AI summary
A usage analysis service that obtains session events from application instances, updates usage metrics for the application instances according to the session events, and stores or updates aggregated usage metrics for each application according to the session events. Updates for the instance usage metrics may be partitioned according to instance identifier, collected in batches, and applied to respective instance records in an instances table by one or more instance update nodes. Updates for the aggregated usage metrics may be partitioned according to application identifier, collected in batches, processed to combine multiple updates, and applied to respective aggregate records in an application usage table by multiple application usage update nodes. The system may provide an interface that retrieves aggregated usage metrics from the application usage table and presents application usage information for the applications to consumers.