Relative Path Object Identification for Cross-Screen RPA
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional object identification methods in Robotic Process Automation (RPA) rely on physical positions of objects on the screen, which are unreliable when screen environments differ between recording and execution stages due to changes in resolution or layout, leading to inaccurate automation processes.
Innovation Solution
Identify objects using relative paths within a hierarchical structure, such as a tree structure, to ensure consistent identification across different screen environments, utilizing methods like lazy loading to optimize navigation performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If physical position-based object identification is used, then object identification is simple and direct, but identification accuracy deteriorates when screen resolution or layout changes
Solution Approach 1:
The patent transitions from two-dimensional screen coordinate-based identification to a hierarchical tree-structure-based identification. Instead of relying on x,y coordinates that change with resolution, the system uses parent-child relationships and path identifiers that remain invariant across different screen configurations, effectively adding a hierarchical dimension to the identification approach.
Solution Approach 2:
The patent changes the identification parameters from absolute screen coordinates (x, y positions) to relative hierarchical parameters (parent object identifiers, path sequences, and positional relationships within the tree structure). This parameter transformation allows the same object to be identified consistently regardless of screen resolution or layout variations.
2Reliability
If multiple scripts are created for various screen environments, then identification accuracy improves for each environment, but system complexity and maintenance difficulty increase
Solution Approach 1:
The patent creates a universal object identification system that works across all screen environments using a single script. The tree-structure-based identification method with relative path calculation serves multiple functions: it identifies objects accurately, adapts to different resolutions, and maintains consistency across layouts, eliminating the need for environment-specific scripts.
Solution Approach 2:
The system dynamically calculates relative paths between objects based on the current screen environment while maintaining a consistent hierarchical structure. The identification process adapts to different resolutions and layouts by computing relative positions within the tree structure rather than using fixed coordinates, making the system flexible and environment-independent.
3Measurement precision
If absolute position-based identification is used, then object location is precise in the recording environment, but the method becomes sensitive to environmental changes
Solution Approach 1:
The patent segments the screen into a hierarchical tree structure where objects are organized in parent-child relationships. Instead of treating the screen as a continuous coordinate space, the system divides it into discrete hierarchical levels and uses path-based navigation, making the identification process resistant to environmental variations while maintaining precision.
Solution Approach 2:
The patent introduces a tree structure as an intermediary layer between the physical screen and the object identification process. This hierarchical structure acts as a mediator that translates physical positions into relative path relationships, allowing precise object location while adapting to different screen environments through the intermediary hierarchical representation.
Data Source
AI summary
A method and a system for relative path-based object identification are provided. The method according to some embodiments may include receiving, by a first computing device, a selection of a first object and a second object from among a plurality of objects present on a first desktop screen from a user of the first computing device; identifying, by the first computing device, a relative path in a tree structure between the first and second objects; storing, by the first computing device, the identified relative path; and identifying, by the first computing device or a second computing device, the second object from among a plurality of objects present on a second desktop screen, using the first object and the stored relative path, wherein the first desktop screen is generated by the first computing device, and the second desktop screen is generated by the first computing device or the second computing device.