Simulating User Interface Testing with ML Interaction Traces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
User interface testing is costly and inefficient due to the need for human testers and the lack of comprehensive testing, as human testers are not as thorough as machine learning models trained on large amounts of historical data.
Innovation Solution
A method using a previously trained machine learning model to generate simulated user interactions, encoded as input traces, for automated testing of user interfaces, incorporating timestamps, actions, metadata, eye gaze data, and user roles to simulate realistic user behavior across multiple interfaces within a common domain.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If human testers are used to test user interfaces, then comprehensive testing can be achieved, but the cost and time consumption increase significantly
Solution Approach 1:
The patent creates simulated user interactions that copy and replicate real human user behavior patterns. These synthetic interactions are generated by training a machine learning model on historical user interaction data, then using the model to produce realistic test cases that mimic actual user workflows, click patterns, and navigation sequences without requiring actual human testers.
Solution Approach 2:
The patent replaces the mechanical system of human testers with an automated machine learning-based system. The machine learning model processes historical interaction data and generates simulated user interactions automatically, substituting the manual, time-consuming human testing process with an automated computational approach that operates continuously without fatigue or scheduling constraints.
2Reliability
If human testers are used to test user interfaces, then testing can be performed, but the cost increases due to expensive human tester time
Solution Approach 1:
The patent creates simulated user interactions that copy and replicate real human user behavior patterns. These synthetic interactions are generated by training a machine learning model on historical user interaction data, then using the model to produce realistic test cases that mimic actual user workflows, click patterns, and navigation sequences without requiring actual human testers.
Solution Approach 2:
The patent uses computationally generated simulated interactions instead of expensive human testers. These synthetic test cases are essentially disposable digital artifacts that can be generated, executed, and discarded at minimal computational cost, replacing the expensive, finite resource of human tester time with inexpensive automated computational processes.
3Adaptability or versatility
If traditional testing methods are used, then some testing coverage is achieved, but comprehensive testing of all user interface scenarios cannot be accomplished
Solution Approach 1:
The patent performs preliminary action by training the machine learning model on historical user interaction data before actual testing begins. This pre-training phase captures diverse user behavior patterns, navigation paths, and interaction sequences that will later be replicated in the simulated interactions, ensuring comprehensive test coverage is built into the system before deployment.
Solution Approach 2:
The patent introduces dynamics by using a machine learning model that can adaptively generate diverse simulated interactions based on learned patterns. The system dynamically produces varied test scenarios including different user roles, permission levels, and interaction sequences, allowing comprehensive coverage of edge cases and rare scenarios that static traditional testing methods would miss.
Data Source
AI summary
A method, computer program, and computer system is provided for testing a user interface. A previously trained machine learning model trained with traces of interactions between one or more users and a user interface is accessed. The interactions include one or more timestamps of user interactions with the user interface, actions by each user associated with the user interface, and metadata associated with user interactions. A simulated interaction of a simulated agent utilizing the user interface is generated using the previously trained machine learning model. The simulated interaction is encoded as an input trace to a user interface. The encoded simulated interaction is input into the user interface for automated testing of the user interface. Results of the automated testing of the user interface are received.