Software Robot Change Detection via Screen Fingerprinting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Robotic process automation (RPA) systems face challenges when interacting with software applications through user interfaces, as changes to these interfaces can cause software robots to fail in execution, necessitating improved methods for detecting changes to ensure reliability and flexibility.
Innovation Solution
A system and method for detecting changes in application programs used by software robots, involving the generation of design-time and execution-time fingerprints of application screens, with comparison to determine if changes have occurred, and providing notifications for potential updates or recreations of software robots.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If software robots interact with software applications through user interfaces, then automation of repetitive tasks is enabled, but execution fails when user interfaces are changed
Solution Approach 1:
The system performs preliminary actions by generating fingerprints of application screens during design time and storing them for later comparison. This advance preparation enables the system to detect interface changes before they cause execution failures, allowing proactive adaptation rather than reactive failure.
Solution Approach 2:
The system implements feedback mechanisms by continuously comparing execution-time fingerprints with stored design-time fingerprints to detect changes in application interfaces. This feedback loop enables the system to monitor for changes and trigger notifications or updates, maintaining reliability despite interface modifications.
2Ease of manufacture
If software robots are created with fixed user interface interactions, then initial execution is simple, but updates are required when interfaces change
Solution Approach 1:
The system creates a digital copy of the application interface state in the form of a fingerprint during design time. This copy serves as a reference model that can be compared against future states, enabling the system to detect changes without requiring complete redesign of the robot when interfaces are modified.
Solution Approach 2:
The system transitions from a static robot design to a dynamic system that can adapt to interface changes. By implementing change detection through fingerprint comparison, the robot can be automatically updated or notified when its underlying interface changes, providing versatility without sacrificing ease of initial creation.
3Reliability
If change detection is implemented through fingerprint comparison, then reliability is improved, but system complexity increases
Solution Approach 1:
The system extracts the essential characteristics of the application interface into a condensed fingerprint representation. By taking out only the critical identifying features rather than analyzing the entire interface, the system achieves reliable change detection with minimal complexity and computational overhead.
Solution Approach 2:
The system uses simple, lightweight fingerprint objects that are inexpensive to generate and compare. These disposable-like fingerprint structures provide sufficient information for change detection without requiring complex, resource-intensive analysis mechanisms, maintaining simplicity while improving reliability.
Data Source
AI summary
Systems and methods for evaluating whether software robots need to be updated due to changes in underlying application programs upon which the software robots operate. According to one embodiment, during creation of a software robot, a fingerprint for a screen of an application program being utilized by the software robot can be generated and stored. Then, later during execution of the software robot, a fingerprint for the screen of the application program can be again generated and compared with the stored fingerprint. If the fingerprints do not match, then the screen of the application program can be determined to have changed. When one or more of the screens of the application program have changed, the software robot may no longer execute correctly with the application program. In such case, the system and method can recommend that that the software robot be recreated.