RPA Task Recording With External Desktop Object Capture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing RPA technologies face challenges in accurately recording and executing actions involving objects within external desktops due to the lack of detailed information about selected objects, limiting the automation of repetitive tasks.
Innovation Solution
A method for recording and executing RPA tasks that involves receiving object information from an external desktop agent, determining the execution environment, and selecting the appropriate device (computing or external) for action execution, using object and execution environment information to ensure accurate task execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If external desktop access technology is used to allow users to perform tasks through external computing devices, then user accessibility and remote task execution are improved, but the ability to accurately record and execute actions on external desktop objects is lost
Solution Approach 1:
An agent is introduced as an intermediary component installed on the external desktop device. This agent captures object information (such as object identifiers, coordinates, and properties) and transmits it to the RPA system. The agent acts as a bridge that enables the RPA system to access and record external desktop objects without direct interaction, thus preserving object information while maintaining user accessibility through external desktop access.
2Extent of automation
If RPA task recording is performed on external desktop windows, then automation capability is improved, but detailed object information becomes unavailable making action recording difficult
Solution Approach 1:
The agent performs preliminary actions by pre-capturing and transmitting object information to the RPA system before the actual RPA task execution. This includes obtaining object identifiers, coordinates, and properties in advance, so that when the RPA system needs to interact with external desktop objects, the precise information is already available for accurate action recording and execution.
3Adaptability or versatility
If the RPA task is executed on a different desktop environment than where it was recorded, then task portability is improved, but execution accuracy decreases due to environment mismatches
Solution Approach 1:
The system creates a digital copy of the execution environment information (including desktop configuration, application states, and object properties) from the recording environment. This environment information is stored with the RPA task specification and used during execution to replicate or adapt to the target environment, ensuring that the task executes accurately even when deployed to different desktop environments.
Data Source
AI summary
A method for recording robotics process automation (RPA) and executing RPA is provided. The method for recording RPA task may comprise receiving object information on an object selected by a user input for selecting the object inside a window region of an external desktop from an agent of the external desktop in response to the user input in a recording situation of the RPA task and adding action information to specification data of the RPA task by using the received object information.