Grid Format Object Identification in RPA via Hierarchical Tree Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional RPA solutions struggle to recognize grid-format objects when they are displayed using non-standard UI control schemes, leading to difficulties in automating work processes on information with similar structures.
Innovation Solution
A method and system for identifying grid-format objects in an RPA environment by utilizing logical hierarchical structure information, even when the implemented UI control does not provide automation characteristic information. This involves searching for ancestor and sibling objects on a hierarchical structure tree and determining their relationships to identify grid-format objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If standard UI control scheme is used to display grid-format information, then the operating system can provide automation characteristic information, but the RPA solution cannot recognize grid-format objects displayed by non-standard UI control schemes
Solution Approach 1:
The patent introduces an intermediary analysis mechanism that acts as a mediator between the RPA solution and non-standard UI control schemes. This intermediary analyzes the hierarchical structure tree and visual characteristics of UI objects to infer grid-format structure, enabling the RPA solution to recognize grid objects regardless of the underlying UI control scheme used.
Solution Approach 2:
The patent replaces the mechanical dependency on OS-provided automation characteristics with a computational analysis system. Instead of relying on the operating system to provide grid-format information, the RPA solution uses image processing and hierarchical structure analysis to substitute and derive this information independently, enabling compatibility with any UI control scheme.
2Productivity
If RPA solution relies on operating system to provide grid-format structure information, then work automation can be achieved for standard UI controls, but automation cannot be realized for information with similar structure displayed by non-standard UI controls
Solution Approach 1:
The patent implements a universal grid-format recognition mechanism that can handle multiple UI control schemes simultaneously. The system analyzes hierarchical structure trees and visual characteristics to universally identify grid-format objects regardless of whether they are rendered by standard or non-standard UI controls, making the RPA solution adaptable to diverse programming environments.
Solution Approach 2:
The RPA solution performs self-service by independently analyzing and identifying grid-format structures without relying on the operating system to provide this information. The system uses its own image processing and hierarchical structure analysis capabilities to extract grid information, enabling it to automate work on any UI control scheme without external assistance.
3Adaptability or versatility
If RPA solution uses hierarchical structure tree analysis to identify grid-format objects, then recognition capability is extended to nonstandard UI controls, but the complexity of object identification increases
Solution Approach 1:
The patent segments the complex task of grid-format object identification into manageable components: hierarchical structure tree analysis, visual characteristic analysis, and relationship inference. By dividing the complex identification process into these segmented steps, the system can handle non-standard UI controls while maintaining manageable complexity through systematic analysis.
Data Source
AI summary
A method for identifying a grid-format object in a robotic process automation (RPA) environment, the method being performed by a computing device, is provided. The method may comprise receiving a search reference object, searching for an ancestor object of the search reference object on a hierarchical structure tree and determining the searched ancestor object as a first row candidate object, determining a sibling object of the first row candidate object on the hierarchical structure tree as a second row candidate object, determining whether a child object of the first row candidate object and a child object of the second row candidate object satisfy a column relationship with each other on the hierarchical structure tree and identifying each of the first row candidate object and the second row candidate object as a grid-format object, based on the determination result.