Overlapping Shingles for Repetitive UI Action Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems fail to accurately detect and automate repetitive user interface actions in software applications, leading to inefficiencies in user interaction.
Innovation Solution
A method and system that define overlapping shingles from sequences of user interface events, group them into clusters based on similarity metrics, and select the cluster with the maximum intra-shingle similarity to identify repetitive operations, enabling automation of these actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional methods are used to detect user interface actions, then the system is simple to implement, but the detection accuracy of repetitive patterns is low
Solution Approach 1:
The user interface event sequence is segmented into overlapping shingles of fixed size, where each shingle represents a contiguous subsequence of events. This segmentation transforms the detection problem into identifying repeated shingle patterns, thereby improving detection accuracy through structured analysis of UI interaction sequences.
Solution Approach 2:
Shingles serve as intermediary elements that bridge raw user interface events and repetitive pattern detection. By introducing shingles as intermediate structures that capture local event sequences, the system achieves higher detection accuracy without directly analyzing the entire event sequence, thus managing complexity effectively.
2Measurement precision
If overlapping shingles are defined and clustered to detect repetitive patterns, then the detection accuracy improves, but the computational complexity increases
Solution Approach 1:
The event sequence is divided into overlapping shingles of fixed length k, creating a structured representation that enables efficient pattern matching. This segmentation allows the system to process UI sequences in manageable units while maintaining detection accuracy through the overlap property that ensures no pattern is missed.
Solution Approach 2:
Instead of analyzing the entire event sequence directly, the system creates multiple copies (shingles) of subsequences and clusters them to identify repetitive patterns. This copying approach transforms the computational problem into one of finding repeated structures, improving accuracy while managing computational complexity through efficient clustering algorithms.
3Extent of automation
If shingle clustering is performed to identify repetitive operations, then automation capability is enhanced, but the processing time increases
Solution Approach 1:
The system performs preliminary actions by pre-defining shingles and organizing them into clusters before actual pattern detection. This preliminary structuring of UI events into reusable shingle units enables faster identification of repetitive operations during execution, thereby enhancing automation capability while reducing real-time processing time.
Solution Approach 2:
The system changes parameters by adjusting shingle size and overlap characteristics to optimize the balance between detection accuracy and processing time. By tuning these parameters, the system can enhance automation capability for specific UI patterns while controlling computational overhead and processing time requirements.
Data Source
AI summary
An example method of detecting repetitive patterns of user interface actions comprises: defining a set of overlapping shingles on a sequence of user interface events; grouping the shingles into a plurality of shingle clusters based on a chosen shingle similarity metric; selecting a shingle cluster having a maximum, among the plurality of shingle clusters, value of a chosen intra-shingle similarity metric; and identifying a repetitive user interface operation represented by the selected shingle cluster.