Cross-Device RPA Using Screenshot Analysis and Control Signals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Robotic Process Automation (RPA) devices are limited to automating processes for a single device and cannot integrate software or operating systems across different devices, especially in industrial production environments where internet connectivity is restricted, making multi-machine automation impossible.
Innovation Solution
A device and method for RPA that uses a processor connected to storage and a communication interface to analyze screenshot images of multiple devices, generate control signals, and transmit them to perform actions on other devices, enabling cross-device automation through script-based event determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional RPA devices are used for single-device automation, then device-specific automation is achieved, but multi-device integration and cross-device operation are impossible
Solution Approach 1:
The RPA device is designed with universal communication interfaces that can connect to multiple different electronic devices simultaneously. The processor can execute scripts to perform operations on different device types (first device and second device) through a unified control system, enabling one RPA device to automate processes across multiple devices rather than requiring dedicated RPA devices for each target device.
Solution Approach 2:
The communication interface acts as an intermediary between the RPA device and multiple target devices. It receives screenshot images from the first device and transmits control signals to the second device, mediating the interaction between different devices without requiring them to be directly connected or have complex peer-to-peer integration.
2Reliability
If internet connectivity is restricted for information security, then security is improved, but multi-machine automation becomes impossible
Solution Approach 1:
The communication interface serves as a secure intermediary that enables device communication without requiring internet connectivity. It allows the RPA device to receive screenshot images from and send control signals to multiple target devices through a local or private communication channel, maintaining information security while enabling multi-device automation.
Solution Approach 2:
The system replaces internet-based communication with a direct communication interface connection between the RPA device and target devices. This substitution eliminates the need for internet connectivity while maintaining the ability to transmit control signals and receive screenshot images, thus preserving both security and automation capability.
3Ease of operation
If screenshot image analysis is used for event determination, then non-invasive monitoring is achieved, but processing complexity increases
Solution Approach 1:
The system replaces direct interaction with target devices (invasive monitoring) with screenshot image analysis (non-invasive monitoring). The processor analyzes visual information from screenshot images to determine events and trigger corresponding actions, eliminating the need for direct device interaction while maintaining monitoring capability.
Solution Approach 2:
Instead of directly accessing or interacting with the target devices, the system uses copies of their display state (screenshot images) for analysis. The communication interface captures visual representations of device states, and the processor determines events based on these image copies, enabling non-invasive monitoring without increasing physical system complexity.
Data Source
AI summary
A device and a method for robotic process automation of multiple electronic computing devices are disclosed. The device creates and stores a script that records at least an event relating to a first device and at least one action corresponding to the event. The device continuously receives a current screenshot image of a first device, and continuously analyzes the current screenshot image of the first device according to the script so as to determine whether the event has occurred. When determining that the event has occurred, the device generates at least one control signal corresponding to the at least one action according to the script, and transmits the at least one control signal to a second device, so as to perform the at least one action for the second device.