RPA Process Identification from User Interaction Recordings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional robotic process automation (RPA) systems face challenges in identifying repeatable user-initiated processes that can be automated, as these processes vary widely and are difficult to discover effectively.
Innovation Solution
The system acquires recordings of user interactions with application programs, processes user-triggered events and UI screen images to identify UI controls and metadata, and selects candidate UI screen images that indicate start or stop points of repeatable user-driven processes, which can then be converted into software robots.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If conventional RPA systems attempt to identify user-initiated processes for automation, then automation capability is provided, but the ability to effectively discover repeatable processes deteriorates due to wide variation in user processes
Solution Approach 1:
The system records user interactions with applications by capturing screen images and user-triggered events, creating copies of the actual user workflow. These recordings serve as templates that can be analyzed to identify repeatable processes, eliminating the need to directly observe and analyze varied user behaviors in real-time.
Solution Approach 2:
The system analyzes recorded user interactions to provide feedback about process repeatability and automation suitability. By examining captured events and screen images, the system identifies patterns and feeds this information back to determine which processes are suitable for automation, resolving the difficulty of discovering repeatable processes amidst variation.
2Measurement precision
If the system captures detailed UI screen images and user-triggered events for analysis, then process identification accuracy is improved, but data processing complexity and time increase
Solution Approach 1:
The system extracts only the essential elements from captured screen images - specifically user-triggered events and associated UI controls. Rather than processing entire screen images in detail, the system identifies and extracts key interaction points and control elements, maintaining identification accuracy while reducing processing complexity.
Solution Approach 2:
The captured data is segmented into discrete user-triggered events, each associated with specific UI controls and screen images. This segmentation allows the system to process individual events independently, identifying repeatable processes through pattern matching across segmented event sequences rather than analyzing entire recording sessions as monolithic data blocks.
Data Source
AI summary
Systems and methods for locating repeatable user-driven processes in recordings of users interacting with application programs. The processing, such as by a Robotic Process Automation (RPA) system, to locate repeatable user-driven processes can, for example, initially acquire various recordings of one or more users interacting with one or more application programs operating on one or more user devices. The recordings can include at least user-triggered events as well as a User Interface (UI) screen image captured for each of the user-triggered events. By processing the captured UI screens from the recordings, those of the screen images that are more likely (or predictably) used to start or stop a repeatable user-driven process can be selected. Thereafter, repeatable user-driven processes within the recordings can be identified based on the selected screen images. Advantageously, the repeatable user-driven processes found within the recordings can thereafter be converted into software robots.