Host-Based Video Automation for Air-Gapped Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automation technologies require installation on the target operating system and are prone to errors due to interface changes, necessitating extensive programming knowledge and potential data compromise.
Innovation Solution
A method and system that automates tasks on a target machine from a host without installing software on the target computer, using a video feed to simulate keystrokes via a host computer connected via USB, ensuring data protection and compatibility with air-gapped systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If software is installed on the target operating system to perform automation, then automation functionality is achieved, but system complexity increases and data security is compromised
Solution Approach 1:
The patent introduces a camera as an intermediary device to capture the target system's display output. Instead of installing software on the target system, the camera captures visual information which is then processed by external automation software on a host system. This intermediary approach enables automation functionality while keeping the target system simple and secure.
Solution Approach 2:
The patent creates a visual copy of the target system's display through camera capture. The automation software operates on this copied visual representation rather than directly interacting with the target system's operating system. This copying mechanism allows automation to function on the target system without installing any software there, thereby reducing system complexity.
2Extent of automation
If traditional automation software is installed on the target system, then automation tasks can be performed, but data security is compromised
Solution Approach 1:
The camera serves as a secure intermediary that allows the host system to observe and control the target system without direct software installation. The automation software on the host system processes camera images to determine keystrokes and mouse actions, enabling automation capability while maintaining data security by avoiding direct access to the target system's data.
Solution Approach 2:
By operating on a visual copy of the display rather than the actual system data, the automation software prevents direct access to sensitive information on the target system. The copied visual interface allows automation tasks to be performed without compromising the security of the underlying data.
3Productivity
If automation software interacts directly with the target operating system, then automation efficiency is high, but compatibility with air-gapped systems is lost
Solution Approach 1:
The camera acts as a mediator that bridges the gap between air-gapped systems and automation capabilities. Since the camera captures visual output without requiring network connectivity or software installation on the target system, it enables automation on air-gapped systems while maintaining their isolation properties.
Solution Approach 2:
The visual copy approach allows automation to function on air-gapped systems by operating on captured images rather than requiring direct OS interaction. This copying mechanism preserves compatibility with air-gapped environments while still achieving automation efficiency through external processing.
4Ease of operation
If automation scripts interact with graphical user interface elements, then ease of operation is improved, but reliability decreases due to interface changes
Solution Approach 1:
The patent implements dynamic image processing that adapts to visual changes in the user interface. Instead of relying on fixed coordinates or element identifiers that break with UI updates, the system continuously processes camera images to dynamically identify and locate interface elements, maintaining reliability despite interface changes.
Solution Approach 2:
The system uses feedback from continuous camera capture and image processing to adapt to UI changes. By constantly monitoring the visual output and adjusting element locations based on current frame analysis, the automation maintains reliability even when the interface layout changes, while still providing ease of operation through visual element interaction.
Data Source
AI summary
A robotic process automation compares an incoming digital video feed with a dictionary of saved images. A host computer may process the digital video output of a target computer, for example, using a video capture card. An incoming video frame as received from the target computer may be matched, by the host computer, to an image stored in the host computer's dictionary. Average hashes of the incoming frame and the image in the dictionary may be calculated. If the difference between the two hashes falls within a predefined range set by the user, a keystroke may be sent.