Hash-Based Activity Log Pattern Detection for Tabular Data Automation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Tabular data management systems face challenges in automatically detecting and creating automation for repetitive tasks due to the complexity and scale of activity logs from multiple users, making it difficult to efficiently compare sequences of actions for automation opportunities.
Innovation Solution
A tabular data management system that uses a pattern detection engine to hash and normalize activity log records, allowing for the detection of duplicate sequences and creation of automated action sequences by generating hash values and identifying patterns within these records.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If activity log records are compared directly to detect duplicate sequences, then detection accuracy is maintained, but processing time increases significantly
Solution Approach 1:
The activity log records are segmented into fixed-size windows, and each window is transformed into a hash value. This segmentation allows the system to compare condensed representations rather than full records, dramatically reducing processing time while maintaining detection accuracy through the properties of hash functions.
Solution Approach 2:
Hash values serve as intermediary representations of the original activity log records. By comparing hash values instead of raw records, the system achieves efficient processing while the hash function ensures that duplicate sequences are accurately identified. The intermediary hash layer enables scalable processing of large volumes of activity data.
2Adaptability or versatility
If all activity log records from multiple users are processed to find automation opportunities, then comprehensive detection is achieved, but system complexity and computational resources increase
Solution Approach 1:
The system segments the comprehensive analysis task into individual window-based hash computations for each user's activity log. This allows parallel processing of multiple users' data independently, reducing overall system complexity while maintaining comprehensive detection capabilities across the entire user base.
Solution Approach 2:
Instead of processing and storing all raw activity log records, the system creates compact hash value copies that represent the essential patterns. These hash copies enable comprehensive comparison across users without the computational burden of handling full activity records, simplifying the system architecture.
3Measurement precision
If detailed activity log records are stored and analyzed, then pattern detection precision is improved, but storage requirements and processing overhead increase
Solution Approach 1:
The system extracts the essential pattern information from detailed activity log records by computing hash values over fixed-size windows. This extraction process removes redundant data while preserving the critical information needed for pattern detection, significantly reducing data volume while maintaining detection precision.
Solution Approach 2:
The system changes the parameter representation of activity logs from detailed raw records to condensed hash values. This parameter transformation maintains the ability to detect duplicate sequences accurately while dramatically reducing the quantity of data that must be stored and processed.
Data Source
AI summary
In some embodiments, techniques are provided for improving the detection of repeated actions during the manipulation of tabular data in order to enable the creation of automated action sequences. Activity log records are created that represent actions taken with respect to a set of tabular data. One or more normalized versions of the activity log records may be created. The activity log records (and, in some embodiments, the normalized versions of the activity log records) may be grouped and hashed together, such that groups of activity log records may be compared to each other by comparing the resulting hashes. Grouping and hashing may allow large numbers of comparisons to be made efficiently enough to support detection of repeated actions on large scale, multi-user, and multi-tenant systems.