Secure HMI Screen Mirroring via Sniffed Display Commands
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing HMI screens in industrial settings are often installed in resource-constrained or remote locations, making it difficult and hazardous for operators to access and modify parameters, necessitating a secure method for mirroring the HMI screen to a second device for remote assistance.
Innovation Solution
A system and method for securely mirroring HMI screen from a first HMI device to a second HMI device using a sniffing technique to intercept display commands, serialize them into slugs, encrypt and transmit them, and decrypt for display on the second HMI device, which can be virtual, enabling remote access and control.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If HMI device is installed in remote or constrained location, then system automation coverage is improved, but operator accessibility and safety deteriorate
Solution Approach 1:
The patent creates a virtual copy of the HMI interface that can be accessed remotely. The virtual HMI device replicates the functionality of the physical HMI installed in remote locations, allowing operators to interact with the system from anywhere in the world without physically visiting the constrained location.
Solution Approach 2:
The patent introduces a communication network as an intermediary between the operator and the remote HMI device. This network-mediated connection allows operators to access the HMI interface remotely through a virtual interface, eliminating the need for direct physical access to the remote location.
2Ease of operation
If HMI screen is mirrored to second device, then remote access capability is improved, but security risk from unauthorized access increases
Solution Approach 1:
The patent introduces multiple intermediary layers including encryption modules and authentication mechanisms between the virtual HMI device and the physical HMI device. These intermediaries secure the communication channel and prevent unauthorized access while allowing legitimate remote operations.
Solution Approach 2:
The patent transforms the HMI interface into a digital format that can be transmitted and displayed on remote devices. By converting the visual interface into transmittable data while maintaining functional equivalence, the system enables remote access without exposing the physical HMI device to security risks.
3Speed
If display commands are transmitted in real-time, then mirroring responsiveness is improved, but network bandwidth consumption increases
Solution Approach 1:
The patent extracts only the essential display commands from the complete HMI interface and transmits them to the virtual HMI device. By filtering and transmitting only the necessary visual elements and interactions rather than the entire interface state, the system maintains responsiveness while reducing network bandwidth consumption.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention provides a system (100), apparatus (110) and method (400) for securely mirroring HMI screen from a first HMI device to a second HMI device for remote assistance in a technical installation, the method comprising initiating, by a processing unit, a sniffing technique when a display command is sent to a display driver to draw one or more elements on the HMI screen of the first HMI device, sniffing display commands from the transmission between the display controller and the display driver of the first HMI device, converting the display commands to slugs by serializing the display commands of the first HMI device, transmitting the slugs from the first HMI device to the second HMI device, deserializing the slugs into display commands for the second HMI device, and displaying the one or more elements in the second HMI device based on the received display commands.