Autonomous Workflow Testing with ML Prioritization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing systems lack the ability to automatically and autonomously run performance testing on user interface workflows, leading to inefficiencies in identifying failures, errors, defects, and vulnerable code.
Innovation Solution
A system that combines randomization functionalities with machine-learning prioritization of workflows for performance testing, generating test workflows by randomizing the sequence of steps in a primary workflow while maintaining locked positions, and identifying subsets that meet or exceed a matching threshold to historical data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual performance testing is performed on user interface workflows, then testing can be conducted, but the process is inefficient and time-consuming
Solution Approach 1:
The system performs autonomous testing where the testing engine automatically executes test workflows, identifies failures, and generates reports without continuous human intervention. The machine learning model autonomously prioritizes test workflows based on historical data, enabling the system to serve itself in the testing process.
Solution Approach 2:
Manual testing operations are replaced by an automated testing engine that uses machine learning models to prioritize and execute test workflows. The system substitutes human-driven mechanical testing processes with algorithm-driven automated testing, significantly improving efficiency and reducing time loss.
2Reliability
If all possible test workflows are generated through randomization, then comprehensive testing coverage is achieved, but the complexity and number of test cases increases significantly
Solution Approach 1:
The system extracts only the most relevant test workflows from the randomly generated set by using machine learning models to prioritize based on historical data patterns. Instead of executing all possible randomized test workflows, the system identifies and executes only those that meet a matching threshold, reducing complexity while maintaining reliability.
Solution Approach 2:
The system changes the parameter of test workflow selection by introducing a matching threshold parameter. Test workflows are filtered based on how well they match historical failure patterns, transforming the approach from exhaustive testing to targeted testing based on parameter-based filtering.
3Measurement precision
If test workflows are prioritized based on historical data patterns, then testing accuracy is improved, but the system requires access to and processing of historical data
Solution Approach 1:
The system performs preliminary analysis by training machine learning models on historical data before actual testing. The models are pre-trained to recognize failure patterns, so when new test workflows are generated, the prioritization can quickly identify relevant cases without requiring real-time processing of all historical data during execution.
Data Source
AI summary
Methods, systems, and computer-readable media are disclosed herein combine randomization functionalities with the machine-learning prioritization of workflows for performance testing. In aspects, a primary workflow having a sequence of user interface steps is input. Testing workflows are generated that represent each variable position of unlocked steps in the sequence of the primary workflow while maintaining the sequential position of any locked steps. These testing workflows are then ingested to a machine learning model that identifies as subset of the testing workflows to prioritize over other. Specifically, testing workflows are prioritized that at least partially match sequence patterns in historical workflow data that is associated with vulnerable computer code. The subset is output and tested by testing engine to generate a report of any vulnerable computer code.