Generative AI RPA Robot Generation From User Interaction Sequences

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing UI automation technologies face challenges in determining user interactions and generating robotic process automation (RPA) robots due to limited system-level access to kernel hook information and the variability of user interactions, requiring extensive driver and application-level functionality.

Innovation Solution

Utilizing generative AI/ML models to analyze recorded user interactions, identify related sequences, and generate RPA workflows by training models to recognize sequences through n-grams over a sliding window, determining optimal n-values, and associating interactions with RPA activities.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Extent of automation

If driver and application-level interactions are used to perform UI automation, then automation functionality can be implemented, but system-level access to kernel hook information is not available and extensive driver level functionality is required

Engineering Contradiction:

ImproveUI automation capabilityVSAvoiddriver and application-level functionality

Core Design Contradiction:

Extent of automationVSDevice complexity

Solution Approach 1:

The patent introduces an intermediary AI/ML model that sits between the user interactions and the RPA workflow generation process. This model analyzes recorded user interactions and automatically determines automation sequences without requiring direct driver-level access or extensive application-level functionality, thereby reducing system complexity while maintaining automation capability

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the traditional mechanical approach of using driver and application-level hooks with an AI/ML-based analytical system. Instead of relying on kernel-level information capture through drivers, the system uses machine learning models to infer automation opportunities from recorded user behavior, substituting mechanical interception with intelligent analysis

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Measurement precision

If the way users interact with computing systems is not known, then automation cannot be accurately determined, but extensive monitoring and recording of user interactions increases system complexity

Engineering Contradiction:

Improveuser interaction analysis accuracyVSAvoidinteraction monitoring system

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent implements preliminary action by recording user interactions in advance before automation determination. The system captures user behavior data during normal operations, then uses this pre-collected data to train and execute AI/ML models that identify automation patterns, eliminating the need for real-time complex monitoring during automation execution

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent uses copying by creating simplified representations of user interactions through recorded data. Instead of continuously monitoring and analyzing complex real-time user behavior, the system captures copies of interaction sequences and uses these recordings as input for AI/ML analysis, reducing the complexity of real-time interaction tracking while maintaining analysis accuracy

Inventive Principle:

Principle #26Copying

3Measurement precision

If n-grams with higher values of n are used to find related sequences, then more comprehensive task recognition is achieved, but the computational complexity and training requirements increase

Engineering Contradiction:

Improvetask sequence recognition accuracyVSAvoidmodel training time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent applies partial action by using multiple n-gram values (including n=1, n=2, n=3, etc.) rather than requiring only high-order n-grams for accurate task recognition. This approach achieves comprehensive task understanding through a combination of shorter and longer sequence patterns, reducing training complexity and time while maintaining recognition accuracy

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The patent segments the task recognition process by analyzing user interactions at multiple n-gram levels rather than using a single comprehensive high-order n-gram. This segmentation allows the system to identify patterns at different granularities (individual actions, pairs of actions, triples, etc.), achieving thorough task understanding through modular analysis that reduces overall computational burden and training time

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS12591441B2Determining sequences of interactions, process extraction, and robot generation using generative artificial intelligence / machine learning models

Publication Date: 2026.03.31 UIPATH INC

AI summary

Use of generative artificial intelligence (AI)/machine learning (ML) models is disclosed to determine sequences of user interactions with computing systems, extract common processes, and generate robotic process automation (RPA) robots. The generative AI/ML model may be trained to recognize matching n-grams of user interactions and/or a beneficial end state. Recorded real user interactions may be analyzed, and matching sequences may be implemented as corresponding activities in an RPA workflow.