An HMI touch screen remote Web adaptive interaction implementation method based on BS architecture

Abstract

The application relates to the technical field of industrial control, remote interaction and Web application, in particular to a remote Web self-adaptive interaction implementation method of an HMI touch screen based on a BS architecture, a three-layer hierarchical architecture is constructed, a user terminal can access operation only by relying on a general browser without needing to additionally install a program, a communication adaptation layer relies on a basic network protocol to complete encrypted data transmission and is matched with a network optimization and line break continuous connection mechanism, a back-end service layer completes standardized processing of multi-type device communication data and operation instruction verification and forwarding, a front-end interaction layer completes lightweight real-time rendering of a device picture, multi-type operation collection and screen self-adaptive layout adjustment, effectively solves defects such as terminal dependence of a traditional scheme, inability of remote visual interaction, multi-terminal interface display abnormity and the like, the whole scheme does not need to transform on-site hardware, can be adapted to various industrial network environments, can complete remote device fault elimination and process parameter adjustment, and significantly reduces remote management and control deployment and long-term operation and maintenance cost.

Description

Technical Field

[0001] This invention relates to the fields of industrial control, remote interaction and web application technology, and in particular to a method for implementing remote web adaptive interaction of HMI touch screen based on BS architecture. Background Technology

[0002] Currently, remote operation of HMI device screens mainly relies on two traditional methods: 1. Dedicated Host Computer Control Software: Users must install dedicated HMI control software on a specific computer (usually a PC). This type of software often connects to the HMI device via proprietary communication protocols. It is not only bloated and cumbersome to install and deploy, but also typically only supports use within a local area network or on a fixed terminal, lacking cross-platform (such as mobile phones, tablets, etc.) access and control capabilities. Its flexibility is extremely poor, failing to meet the needs of remote, off-site control. Furthermore, for different mobile operating systems such as Android and iOS, separate adaptation versions or specific drivers need to be developed, increasing the cost of multi-device deployment and making it difficult to unify the technology stack, significantly increasing maintenance complexity.

[0003] 2. Simple Embedded Web Pages: While some HMI devices have built-in web servers providing basic remote access functionality, the web pages they display are often static HTML parameter lists or simple text tables. This type of solution can only statically display relevant HMI device parameters and cannot convert parameter data into intuitive HMI device screen information. For example, it cannot render the HMI device's operating interface, button states, indicator light displays, or other visual elements in real time. It also cannot enable remote interactive operations such as clicking and switching on the HMI device screen. In remote scenarios, users still need to rely on on-site personnel for feedback and cannot directly complete core operations such as troubleshooting and parameter adjustments, significantly reducing its practicality.

[0004] In addition, most existing simple web pages that support remote access are designed with a fixed size and do not have responsive adaptation. On small-screen mobile devices such as mobile phones, problems such as interface misalignment, button obstruction, and interaction failure are likely to occur, which cannot achieve a consistent remote operation experience across multiple terminals and limit their practicality.

[0005] In summary, the existing technology has two major drawbacks: first, it is highly dependent on terminals and has poor cross-platform compatibility, making it impossible to achieve real-time remote two-way interaction of the entire HMI screen; second, the web page lacks adaptive capabilities, and the multi-terminal operation interface displays abnormally and the interaction fails; making it difficult to meet the usage needs of remote mobile operation and maintenance and remote emergency handling in industrial scenarios.

[0006] Therefore, a new technical solution is urgently needed to solve the above-mentioned technical problems. Summary of the Invention

[0007] The purpose of this invention is to overcome the problems of the prior art and provide a remote web adaptive interaction implementation method for HMI touch screens based on a BS architecture. This method addresses the technical problems of existing HMI remote control methods, which either rely on dedicated host computer software, limiting their use to fixed LAN terminals and making cross-mobile adaptation difficult and costly, or use simple static web pages that can only display text parameters and cannot fully render the device screen or support remote interactive operations. Furthermore, traditional remote web pages use a fixed-size design, which can easily lead to interface errors and interaction failures when accessed by terminals of different sizes. These methods fail to simultaneously meet the technical requirements of cross-platform, installation-free access, real-time bidirectional control of the device screen, and unified operation experience across multiple terminals, thus failing to meet the needs of remote operation and maintenance in industrial settings.

[0008] The above objectives are achieved through the following technical solutions: A method for implementing remote Web adaptive interaction of HMI touchscreen based on B / S architecture includes the following steps: Step S1: Build a three-tier BS architecture consisting of a communication adaptation layer, a backend service layer, and a frontend interaction layer; the communication adaptation layer establishes bidirectional data interaction with the HMI device, the backend service layer connects the communication adaptation layer and the frontend interaction layer respectively, and the frontend interaction layer is accessible to the user terminal's general browser. Step S2: The communication adaptation layer completes encrypted data transmission between the server and HMI device, and between the browser terminal and the server based on the TCP / IP protocol, adapts to different network environments, and configures a disconnection reconnection mechanism. Step S3: The backend service layer adapts and parses the communication data of different HMI device models through the protocol parsing module, and sends it to the frontend interaction layer after standardization processing by the data aggregation module. At the same time, it receives the frontend interaction command through the command forwarding module, verifies and encrypts it, sends it to the HMI device, and sends back the device feedback data. Step S4: The front-end interaction layer uses the screen rendering module to render the complete visual screen of the HMI in a lightweight and real-time manner, collects terminal operation commands through the interaction operation module and uploads them to the back-end service layer, and uses the responsive adaptation module to adaptively adjust the interface layout and interaction mode according to the terminal screen size to achieve unified interactive display across multiple terminals.

[0009] Furthermore, in step S1, the backend service layer integrates a permission management module, which assigns differentiated remote operation permissions based on the logged-in user's role and intercepts unauthorized access and unauthorized operations.

[0010] Furthermore, in step S2, the communication adaptation layer is also configured with a dynamic network adjustment mechanism, which monitors the network transmission rate in real time, automatically optimizes the data transmission bandwidth, and reduces the latency of screen rendering and command transmission.

[0011] Furthermore, in step S3, the protocol parsing module supports parsing and adapting to various mainstream HMI device communication protocols, and identifies the status of HMI device screen elements, button commands, and real-time parameter values; the data aggregation module unifies the raw HMI data into a standardized data stream, providing standardized data support for front-end screen rendering.

[0012] Furthermore, in step S3, the instruction forwarding module performs a validity check on the remote operation instructions uploaded by the front end, such as click, screen switching, and parameter modification. After the verification is passed, the instructions are encrypted and then sent to the corresponding HMI device. At the same time, the module receives the operation feedback data returned by the HMI device and synchronously sends it back to the front-end interaction layer to update the interface status.

[0013] Furthermore, in step S4, the front-end interaction layer is developed and constructed based on HTML5, CSS3, and JavaScript components; the screen rendering module synchronously renders all visual elements of the HMI interface layout, button status, indicator lights, and parameter values, realizing WYSIWYG for the remote screen.

[0014] Furthermore, in step S4, the interactive operation module is compatible with both mouse and touch screen operation methods, accurately collects click, swipe, and parameter input operations on the browser end, and synchronizes them to the backend service layer in real time, while also receiving device feedback to update the front-end screen.

[0015] Furthermore, in step S4, the responsive adaptation module incorporates an adaptive layout algorithm; the adaptive layout algorithm identifies the size of PC large screens, tablet medium screens, and mobile phone small screen terminals, automatically scales the size of interface elements, reconstructs the page layout, matches the corresponding interaction methods, and eliminates interface misalignment and button obstruction problems.

[0016] Furthermore, the three-tier BS architecture deployment method is as follows: the communication adaptation layer and the backend service layer are deployed on the server side, and the frontend interaction layer runs on the user terminal browser. The user terminal does not need to install dedicated control software or drivers, and can complete remote access and control only through a general browser.

[0017] Furthermore, this method is compatible with both local area networks (LANs) and wide area networks (WANs), and can be used to complete the access without hardware modification of the on-site HMI equipment. It can be used for remote mobile operation and maintenance, remote emergency fault handling in industrial sites, and remote parameter adjustment.

[0018] This invention provides a remote Web adaptive interaction method for HMI touchscreens based on a B / S architecture. It achieves installation-free access via a universal browser for various terminals through a three-tier B / S architecture, overcoming the limitations of dedicated software and local area networks. Layered data processing in the backend ensures compatibility with multiple HMI device models and secure bidirectional command transmission. Lightweight rendering in the frontend enables real-time, synchronized visualization of the complete device screen. A responsive adaptation module automatically adapts the interface to different screen terminals. An optimized communication layer mechanism ensures stable, low-latency data interaction across multiple network environments. Overall, this method reduces the cost of multi-terminal deployment and maintenance, and improves the convenience and practicality of remote operation and maintenance. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a remote Web adaptive interaction implementation method for an HMI touchscreen based on a B / S architecture, as described in this invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. The described embodiments are merely some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] like Figure 1 As shown, this solution provides a method for implementing remote web adaptive interaction of HMI touchscreen based on a B / S architecture. It establishes a three-layer B / S architecture consisting of a communication adaptation layer, a backend service layer, and a frontend interaction layer. Relying on layered collaboration, it achieves installation-free cross-terminal access, real-time full-screen HMI rendering, two-way remote interaction, multi-terminal adaptive layout, and secure and stable communication transmission. The overall implementation steps are as follows: Step S1: Build a three-tier BS architecture consisting of a communication adaptation layer, a backend service layer, and a frontend interaction layer; the communication adaptation layer establishes bidirectional data interaction with the HMI device, the backend service layer connects the communication adaptation layer and the frontend interaction layer respectively, and the frontend interaction layer is accessible to the user terminal's general browser. The three-tier BS architecture deployment distinguishes between the server and the user terminal: the communication adaptation layer and the backend service layer are deployed on the server, while the frontend interaction layer runs on the user terminal's browser. The user terminal does not need to install dedicated control software or drivers; it can access the system simply through the general browser that comes with a PC, mobile phone, or tablet.

[0022] The backend service layer integrates a protocol parsing module, a data aggregation module, an instruction forwarding module, and a permission management module. The permission management module assigns differentiated remote operation permissions based on the logged-in user's role, intercepts unauthorized access and unauthorized operations, and ensures the security of remote operations.

[0023] Step S2: The communication adaptation layer completes encrypted data transmission between the server and HMI device, and between the browser terminal and the server based on the TCP / IP protocol, adapts to different network environments, and configures a disconnection reconnection mechanism. The communication adaptation layer is additionally configured with a dynamic network adjustment mechanism; the dynamic network adjustment mechanism monitors the network transmission rate in real time, automatically optimizes the data transmission bandwidth, and reduces the delay of screen rendering and command transmission; the disconnection reconnection mechanism automatically synchronizes the current complete screen and operating parameters of the HMI device after the network interruption is restored, avoiding abnormal operation interruption; this layer is compatible with both local area network and wide area network industrial network scenarios, and can be accessed without modifying the on-site HMI hardware.

[0024] Step S3: The backend service layer adapts and parses the communication data of different HMI device models through the protocol parsing module, and sends it to the frontend interaction layer after standardization processing by the data aggregation module. At the same time, it receives the frontend interaction command through the command forwarding module, verifies and encrypts it, sends it to the HMI device, and sends back the device feedback data. The protocol parsing module supports parsing and adaptation of various mainstream HMI device communication protocols, and identifies the status of HMI device screen elements, button commands, and real-time parameter values; the data aggregation module unifies the raw HMI data into a standardized data stream, providing standardized data support for front-end screen rendering. The instruction forwarding module performs a validity check on remote operation instructions such as clicks, screen switching, and parameter modification uploaded from the front end. After the verification is successful, the instructions are encrypted and then sent to the corresponding HMI device. At the same time, it receives operation feedback data returned by the HMI device and synchronously sends it back to the front-end interaction layer to update the interface status, forming a two-way real-time data interaction link.

[0025] Step S4: The front-end interaction layer uses the screen rendering module to render the complete visual screen of the HMI in a lightweight and real-time manner, collects terminal operation commands through the interaction operation module and uploads them to the back-end service layer, and uses the responsive adaptation module to adaptively adjust the interface layout and interaction mode according to the terminal screen size to achieve unified interactive display across multiple terminals. The front-end interaction layer is built on HTML5, CSS3, and JavaScript components; the screen rendering module synchronously renders all visual elements of the HMI interface layout, button status, indicator lights, and parameter values, realizing a WYSIWYG (What You See Is What You Get) experience for remote screens. The interactive operation module is compatible with both mouse and touch screen operation methods. It accurately collects click, swipe and parameter input operations on the browser and synchronizes them to the backend service layer in real time. It also receives device feedback and updates the front-end screen. The responsive adaptation module incorporates an adaptive layout algorithm; this algorithm identifies the screen sizes of large PC screens, medium tablet screens, and small mobile phone screens, automatically scales the size of interface elements, reconstructs the page layout, matches the corresponding interaction methods, and eliminates interface misalignment and button obstruction issues.

[0026] As a specific embodiment of this solution, this embodiment combines Figure 1 The implementation process of this invention is fully described, and the logical link in the attached diagram is as follows: user terminal (PC / mobile phone / tablet) → general browser → front-end interaction layer → back-end service layer → communication adaptation layer → HMI device. The reverse data synchronization link is: HMI device → communication adaptation layer → back-end service layer → front-end interaction layer → browser terminal.

[0027] Implementation Step 1: Building and Deploying a Three-Tier B / S Architecture Staff deploy the communication adaptation layer and backend service layer on the industrial service server; the front-end interaction layer is developed based on HTML5, CSS3, and JavaScript components. It does not need to be deployed to the terminal and can be published directly through the Web service. User terminals (PC computers, Android / iOS mobile phones, tablets) only need to open the built-in general browsers such as Chrome, Edge, and Safari, and enter the server access address to access the front-end interaction layer. No HMI-specific control software or drivers need to be installed throughout the process.

[0028] The backend service layer has four core modules: protocol parsing module, data aggregation module, command forwarding module, and permission management module. Administrators pre-enter operation and maintenance personnel accounts in the permission management module and assign permissions according to their positions. For example, on-site inspection personnel are only allowed to view the screen, process engineers are allowed to modify parameters and switch screens, and equipment administrators have full operation permissions. When an unauthorized account logs in or an account performs an unauthorized operation, the permission management module directly intercepts the command and returns an alarm message to the front-end browser.

[0029] The communication adaptation layer establishes a network connection with one or more HMI touch screen devices in the workshop through TCP / IP network lines, and at the same time completes the data transmission link between the server and the browser terminal, supporting two network modes: local access via LAN and remote access via WAN.

[0030] Implementation Step 2: Secure and Stable Bidirectional Data Transmission at the Communication Adaptation Layer The communication adaptation layer uses TCP / IP as the basic communication protocol and performs encryption processing on the screen data, operation instructions and device feedback data transmitted between all servers, HMI devices and browser terminals to prevent industrial data leakage and instruction tampering. Enable dynamic network adjustment mechanism: The communication adaptation layer collects the current network uplink and downlink transmission rates in real time. If the network bandwidth is insufficient, it automatically reduces the refresh frequency of non-core elements on the screen and prioritizes the data transmission of key controls such as buttons and parameters to reduce command response latency. Enable the reconnection function after disconnection: continuously monitor the network link status, record the complete HMI screen and device operating parameters before the network disconnection when the network is disconnected; automatically initiate a reconnection request after the network is restored, and synchronize the real-time status of all devices at once after successful reconnection, without requiring users to manually refresh the page, thus avoiding interruption of remote operation.

[0031] Implementation Step 3: Backend Service Layer Data Parsing, Aggregation, and Command Forwarding The device uplink data processing flow is as follows: The HMI device pushes raw data such as screen layout, button status, indicator light on / off status, and real-time process parameters to the communication adaptation layer in real time. The communication adaptation layer forwards the raw data to the backend service layer. The protocol parsing module identifies the communication protocol of the currently accessing HMI device, completes protocol decoding, and extracts all visualization element data and device operating parameters. After parsing, the data is handed over to the data aggregation module, which converts the raw data of different formats and models of HMI into a standardized data stream and pushes it to the front-end interaction layer for screen rendering.

[0032] Terminal downlink command processing flow: Maintenance personnel perform operations such as clicking buttons, switching screens, and modifying process parameters on the browser page. The front-end interaction layer collects the operation commands and uploads them to the back-end service layer. The command forwarding module first verifies the legality of the command and determines whether the account permissions match the operation content. After verification, the command is encrypted and sent to the corresponding HMI device through the communication adaptation layer. After the HMI device executes the operation, it generates operation feedback data and sends it back to the back-end service layer along the uplink. The back-end synchronously pushes the feedback data to the front-end and updates the HMI screen status in the browser.

[0033] Implementation Step 4: Front-end interaction layer rendering, interaction capture, and adaptive adaptation Lightweight real-time rendering: The rendering module receives the standardized data stream from the backend and uses lightweight graphics rendering technology to completely restore the original operation interface of the HMI device. It synchronously renders all visual elements, including the interface background, function buttons, indicator lights, digital parameters, and pop-up controls, so as to realize real-time synchronization between the browser screen and the on-site touch screen screen and achieve a WYSIWYG effect.

[0034] Multi-mode interactive data acquisition: The interactive operation module supports mouse click and drag operations on PC, and finger touch and swipe operations on mobile phones / tablets; it accurately captures all operation behaviors in the browser, encapsulates them into standardized operation instructions and uploads them to the backend service layer; it synchronously receives device feedback data returned from the backend, refreshes the page control status in real time, and ensures that remote operation and on-site device actions are completely synchronized.

[0035] Responsive Adaptive Layout: The responsive adaptation module has a built-in adaptive layout algorithm that automatically obtains the terminal screen resolution and physical size parameters when the browser loads the page. When the access device is a large-screen PC terminal, the algorithm maintains the original HMI screen ratio, fully displays all controls, and enables mouse interaction logic by default. When the access device is a tablet mid-screen terminal, the algorithm proportionally reduces the overall screen size and fine-tunes the spacing between buttons and controls to balance viewing completeness and touch convenience. When the access device is a small-screen mobile phone terminal, the algorithm automatically scales all interactive buttons, enlarges the touch click area, reconstructs the horizontal and vertical layout of the page, and eliminates problems such as interface misalignment, button obstruction, and click failure, adapting to one-handed touch operation. When the terminal switches between portrait and landscape modes, the adaptive layout algorithm recalculates the page layout in real time, dynamically adjusting the size and arrangement of elements to ensure that the interface remains fully usable regardless of size or usage posture.

[0036] The full-process application scenarios of this solution are as follows: Remote maintenance engineers access the server via the internet using a mobile browser. After the access control module verifies account permissions, the front-end page is accessed. The communication adaptation layer encrypts and transmits real-time HMI data, and the back-end parses and aggregates device screen data before pushing it to the front-end. The responsive adaptation module automatically scales the interface buttons according to the small screen size of the mobile phone, and the lightweight rendering module fully displays the entire screen of the workshop HMI touchscreen. Engineers issue operation commands by clicking the corresponding button for the fault with their fingers. The back-end verifies and encrypts the commands before transmitting them to the on-site HMI device. After the device executes the operation, it provides feedback on its status, and the front-end updates indicator lights and parameter values ​​in real time. Network fluctuations trigger a dynamic network adjustment mechanism to ensure low latency of commands. After the network is disconnected and restored, it automatically reconnects and synchronizes the screen. Engineers can complete troubleshooting and parameter adjustment remotely without going to the workshop site.

[0037] The above description is merely illustrative of the embodiments of the present invention and is not intended to limit the present invention. For those skilled in the art, any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for implementing remote Web adaptive interaction of HMI touchscreen based on a B / S architecture, characterized in that, Includes the following steps: Step S1: Build a three-tier BS architecture consisting of a communication adaptation layer, a backend service layer, and a frontend interaction layer; the communication adaptation layer establishes bidirectional data interaction with the HMI device, the backend service layer connects the communication adaptation layer and the frontend interaction layer respectively, and the frontend interaction layer is accessible to the user terminal's general browser. Step S2: The communication adaptation layer completes encrypted data transmission between the server and HMI device, and between the browser terminal and the server based on the TCP / IP protocol, adapts to different network environments, and configures a disconnection reconnection mechanism. Step S3: The backend service layer adapts and parses the communication data of different HMI device models through the protocol parsing module, and sends it to the frontend interaction layer after standardization processing by the data aggregation module. At the same time, it receives the frontend interaction command through the command forwarding module, verifies and encrypts it, sends it to the HMI device, and sends back the device feedback data. Step S4: The front-end interaction layer uses the screen rendering module to render the complete visual screen of the HMI in a lightweight and real-time manner, collects terminal operation commands through the interaction operation module and uploads them to the back-end service layer, and uses the responsive adaptation module to adaptively adjust the interface layout and interaction mode according to the terminal screen size to achieve unified interactive display across multiple terminals.

2. The method for implementing remote Web adaptive interaction of HMI touchscreen based on BS architecture according to claim 1, characterized in that, In step S1, the backend service layer integrates a permission management module, which assigns differentiated remote operation permissions based on the logged-in user's role and intercepts unauthorized access and unauthorized operations.

3. The method for implementing remote Web adaptive interaction of HMI touchscreen based on BS architecture according to claim 1, characterized in that, In step S2, the communication adaptation layer is also configured with a dynamic network adjustment mechanism, which monitors the network transmission rate in real time, automatically optimizes the data transmission bandwidth, and reduces the latency of screen rendering and command transmission.

4. The method for implementing remote Web adaptive interaction of HMI touchscreen based on BS architecture according to claim 1, characterized in that, In step S3, the protocol parsing module supports parsing and adapting to various mainstream HMI device communication protocols, and identifies the status of HMI device screen elements, button commands, and real-time parameter values; the data aggregation module organizes the raw HMI data into a standardized data stream, providing standardized data support for front-end screen rendering.

5. The method for implementing remote Web adaptive interaction of HMI touchscreen based on BS architecture according to claim 1, characterized in that, In step S3, the instruction forwarding module performs a validity check on the remote operation instructions uploaded by the front end, such as click, screen switching, and parameter modification. After the verification is successful, the instructions are encrypted and then sent to the corresponding HMI device. At the same time, the module receives the operation feedback data returned by the HMI device and synchronously sends it back to the front-end interaction layer to update the interface status.

6. The method for implementing remote Web adaptive interaction of HMI touchscreen based on BS architecture according to claim 1, characterized in that, In step S4, the front-end interaction layer is developed and constructed based on HTML5, CSS3, and JavaScript components; the screen rendering module synchronously renders all visual elements of the HMI interface layout, button status, indicator lights, and parameter values.

7. The method for implementing remote Web adaptive interaction of HMI touchscreen based on BS architecture according to claim 1, characterized in that, In step S4, the interactive operation module is compatible with both mouse and touch screen operation methods, accurately collects click, swipe, and parameter input operations on the browser end, and synchronizes them to the backend service layer in real time, while also receiving device feedback to update the front-end screen.

8. The method for implementing remote Web adaptive interaction of HMI touchscreen based on BS architecture according to claim 1, characterized in that, In step S4, the responsive adaptation module incorporates an adaptive layout algorithm; the adaptive layout algorithm identifies the sizes of large PC screens, medium tablet screens, and small mobile phone screens, and automatically scales the size of interface elements and reconstructs the page layout.

9. The method for implementing remote Web adaptive interaction of HMI touch screen based on BS architecture according to claim 1, characterized in that, The three-tier BS architecture deployment method is as follows: the communication adaptation layer and the backend service layer are deployed on the server side, and the frontend interaction layer runs on the user terminal browser. The user terminal does not need to install dedicated control software or drivers, and can complete remote access and control only through a general browser.

Images