Human-machine interface

Abstract

Embodiments of the present invention generally relate to a human-machine interface, a distributed human-machine interface system (100) for a distributed control system or integrated operation management, the system comprising: a set of view modules (102) as individually isolated micro-front-end applications (104), each implemented by a network component (106), each micro-front-end application includes a back-end application (108) housed in a respective separate container (110), the back-end application communicating with their view modules using a network-based protocol, a common HMI framework (112) for all view modules, the common HMI framework including at least infrastructure functionality and a user experience design.

Description

Technical Field

[0001] The present invention generally relates to a human-machine interface, to a collection of control units for implementing a human-machine interface, to a system, and to a computer program product. Background Art

[0002] The current generation of industrial HMIs is aging because they were developed based on concepts and use cases that have changed significantly, and they are based on specific deployments for a specific set of users. Today's customers are looking for greater flexibility in all the software systems they use, and the same should be valid for industrial HMIs. This document describes how we envision an architecture that brings traditional functionality (such as industrial human-machine interfaces) to a different level.

[0003] Compared to more modern systems, current human machine interfaces (HMIs) were designed decades ago with different assumptions and different conditions. Typically, current HMIs are designed for a specific operating system (OS) or for a generic OS like Windows. The spread of standard protocols such as OPC Classic has increased the efficiency of distributed control systems (DCS) and in particular HMI design, but has also increased OS dependency.

[0004] Furthermore, traditional HMI designs are often monolithic in that they are rarely redesigned from their original configuration, often due to the costs associated with refactoring. Furthermore, the lifecycle of monolithic systems deployed on a specific operating system is complex to manage and often a challenge to evolve. Summary of the invention

[0005] In view of the above and other shortcomings of the prior art, an object of the present invention is to provide a human-machine interface which at least partially alleviates the shortcomings of the prior art.

[0006] According to a first aspect of the present invention, a distributed human-machine interface system is provided, the system comprising: a collection of view modules, which are separately isolated micro-frontend applications, the collection of view modules are each implemented by a network component, each micro-frontend application includes a backend application contained in a corresponding separate container, the backend application communicates with the view module of the backend application using a network-based protocol, and a common HMI framework for all view modules, the common HMI framework includes at least infrastructure functions and user experience design.

[0007] The present invention is based at least in part on the implementation of deploying HMI as micro-frontend applications, where independent backend applications are in separate containers. This allows micro-frontend applications to execute in isolation from each other, thereby simplifying the software life cycle, and performing updates and other management services on the isolated micro-frontends.

[0008] Distributed human-machine interface systems can be applied or integrated into distributed control systems or integrated operations management applications.

[0009] Containers can be managed by an external service or orchestrator (e.g., Kubernetes, Docker Swarm, etc.), which will improve the availability of individual core services of the micro-frontend application and allow better load balancing.

[0010] A container is a stand-alone, executable software package that includes everything needed to run an application: code, runtime, system tools, system libraries, and settings.

[0011] Example containers include, for example, Docker, Podman, and ContainerD.

[0012] Web-based protocols package micro-frontend applications as web applications, which advantageously allows a web-based workplace for operators.

[0013] The web-based protocol may be, for example, HTTPS or WSS.

[0014] Infrastructure functionality includes, for example, authentication, logging, messaging, licensing, and layout management, to name a few.

[0015] In an embodiment, the web component may be HTML5.

[0016] The isolation given by the micro frontend in the container advantageously allows software engineers to choose their preferred network framework to improve development efficiency. In addition, the life cycle of individual modules is independent and allows for easier system maintenance and evolution. In addition, the backend of the micro frontend is hosted in a container to strengthen service isolation, modularity, and ease of deployment.

[0017] In an embodiment, the common HMI framework can be configured to provide coordinated control of all view modules in the workplace (i.e., the front end). The use of the common HMI framework advantageously supports the generation of a unique platform for exchanging messages and states across different micro-frontends, and simplifies the integration of other applications. More specifically, in one embodiment, the common HMI framework can be configured to exchange messages and states across different micro-frontends. In addition, the HMI framework can be configured to provide token-based authentication for user logins to prevent tampering with distributed systems.

[0018] In an embodiment, the backend application may use an HTTPS REST API or a web socket to communicate with the view module of the backend application.

[0019] In an embodiment, the distributed human-machine interface system may include at least one control unit, the at least one control unit being configured to each run at least one micro-frontend. Thus, the distributed human-machine interface system may be geographically distributed across more than one control unit of the industrial system.

[0020] In an embodiment, the micro-frontend application may include at least a process graphics module and a workplace module, wherein the process graphics module may be configured to provide data indicating characteristics of an object with which a user interacts, and the workplace module is configured to find micro-frontend applications related to the object. In addition, the workplace module is configured to provide overall navigation, such as a context menu, and arrange views into layouts, such as a single view, two views next to each other, a tabbed view, etc.

[0021] Thus, the distributed human-machine interface system advantageously allows for a wide range of applications, particularly being configured for industrial systems.

[0022] The distributed human-machine interface system may further include an emissions reporting module as an extended application, the module being configured to change its content associated with the object based on changes made to the object in the process graphic module.

[0023] The micro frontend application may advantageously further include at least one of an alarm list module, a trend view module, and an object browser module.

[0024] In a preferred embodiment, the micro-frontend application can provide at least two types of applications. That is, the distributed human-machine interface system advantageously provides coordination of more than one independent application in a distributed control system.

[0025] In an embodiment, the distributed human-machine interface system may include a viewer application configured to collect visual content from the micro-frontend application and from at least one external application, and provide the visual content to a computer display. In other words, the viewer application provides for integrating external applications with the micro-frontend application, or making, for example, real-time operational data available for visualization, analysis, and action. An example platform that can host the extension application is ABB Ability Edgenius.

[0026] In an embodiment, the containers may be geographically distributed on separate control units of a distributed control system.

[0027] In an embodiment, the micro frontend application may include at least one of a process graphics module, an alarm module, and a trend view module.

[0028] According to a second aspect of the present invention, there is provided a set of control units configured to implement a human-machine interface system according to any one of the methods disclosed herein.

[0029] Further effects and features of the second aspect of the invention are largely similar to those described above in conjunction with the first aspect of the invention.

[0030] According to a third aspect of the invention there is provided a system comprising a computer display and a set of control units according to the second aspect.

[0031] Further effects and features of the third aspect of the present invention are largely similar to those described above in conjunction with the first and second aspects of the present invention.

[0032] According to a fourth aspect of the present invention, there is provided a computer program product comprising a computer readable medium having computer program means stored thereon to implement a human-machine interface system according to any one of the embodiments disclosed herein.

[0033] The computer readable medium may be a non-transitory computer readable storage medium.

[0034] Further effects and features of the fourth aspect of the invention are largely similar to those described above in conjunction with the first, second and third aspects of the invention.

[0035] Further features and advantages of the present invention will become apparent when studying the appended claims and the following description.The skilled person realizes that different features of the present invention can be combined to create embodiments other than those described in the following, without departing from the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] These and other aspects of the present invention will now be described in more detail with reference to the accompanying drawings, which show example embodiments of the present invention, in which:

[0037] Figure 1 A distributed human-machine interface system according to an embodiment of the present invention is shown;

[0038] Figure 2 A distributed human-machine interface system according to an embodiment of the present invention is shown;

[0039] Figure 3 An industrial system with two clients and a distributed human-machine interface system according to an embodiment of the present invention is shown;

[0040] Figure 4 Schematically illustrates the deployment of current technology with new technology including a distributed human-machine interface system according to an embodiment of the present invention;

[0041] Figure 5 Provided Figure 4 A concrete example of the high-level architecture described. DETAILED DESCRIPTION

[0042] In the present detailed description, various embodiments of the present invention are described herein with reference to specific implementations. When describing the embodiments, specific terms are used for clarity. However, the present invention is not intended to be limited to the specific terms so selected. Although specific exemplary embodiments are discussed, it should be understood that this is only for illustrative purposes. Those skilled in the relevant art will recognize that other components and configurations may be used without departing from the scope of the present invention.

[0043] Figure 1 A distributed human-machine interface system 100 according to an embodiment of the present invention is shown. The distributed human-machine interface system 100 includes a set 102 of view modules, which are individually isolated micro-frontend applications 104, each of which is implemented by a network component 106. In a preferred embodiment, the network component is HTML5, so the view module is an HTML5-hosted network component.

[0044] Each micro frontend application 104 includes a backend application 108, which is housed in a corresponding separate Docker container 110 along with its view module 102. The backend applications communicate with their view modules using a network-based protocol such as an HTTPS REST API or a web socket.

[0045] The backend application is the program code developed, for example, in C#.NET 6, which allows the operation of the micro frontend and is usually not accessible to users. The backend application supports the operation of the frontend. The view module is the part of the HMI with which the user interacts via a web component hosted, for example, in HTML5.

[0046] The distributed human-machine interface system 100 also includes a common HMI framework 112 for all view modules 102 , which includes at least infrastructure functions and user experience functions, such as layout, navigation, common header with user information, etc. Thus, the HMI framework 112 provides a user experience across multiple view modules.

[0047] The common HMI framework 112 is a platform configured to provide coordinated control of all view modules 102 in the workplace. For example, the common HMI framework 112 can be a JavaScript, TypeScript, or web assembly that provides common functionality across different micro-frontends 104. Such common functionality includes, for example, exchanging messages and states across different micro-frontends, or providing token-based authentication for user login. Other typical common functionalities can be user session management, localization, storage of user preferences, navigation, context-sensitive online help, seamless upgrade experience, logging, licensing, etc.

[0048] The backend application 108 communicates with the data source 111 (e.g., device, controller, input / output device on the device side) via the standard OPC UA protocol, which is a known machine-to-machine communication protocol for industrial automation. Other envisioned communication protocols include accessing data from a database, or using MQTT, or interacting with another service using an HTTPS REST API.

[0049] Figure 2 An example of a common HMI framework 112 that seamlessly integrates different types of applications in the same HMI workplace even if running on different application servers 118a - c is shown.

[0050] More specifically, Figure 2 Two view modules 102a and 102b are shown, each with a separate backend application 108a and 108b. A common HMI framework 112 is also implemented as a web component with a backend application 113. Micro frontend applications 104a and 104b provide at least two types of applications. In this example, view module 102a can be a process graphic with a frontend user interface 114a. View module 102b can be an emissions report with a frontend user interface 114b shown on a computer display 116. Other example micro frontend modules include an alarm module and a trend visualization module.

[0051] Each of the view modules 102a-b and the common HMI framework 112 and their corresponding backend applications can be hosted on different servers or control units 118a, 118b and 118c, which can be geographically distributed. Therefore, the distributed human-machine interface system 100 can include at least one control unit 118a-c, each of which is configured to run at least one micro frontend 102a, 102b and 112. This further implies that the docker containers for the backend applications 108a, 113 and 108b can be geographically distributed on separate control units 118a-c of a distributed control system or integrated operation management system.

[0052] The common HMI framework 112 with its backend application 113 provides the process graphics module 102a and the emission reporting module 102b with exchange messages M and states S, and the common HMI framework 112 is implemented as a common workplace module 112. The common workplace module 112 is configured to visualize information from the process graphics module 102a and the emission reporting module 102b on a display 116.

[0053] The process graphics application 102a does not have to be aware of the emissions reporting module 102b, but such applications can be "discovered". In fact, it is expected that the systems share a common information model and industrial scene, and at engineering time, the microservices 102a-b are deployed with information that associates objects and different aspects of data. For example, if a physical valve is represented by an object instance in the HMI system 100 and visualized as a symbol through the process graphic 114a, then the user can interact with the graphic symbol to call up its related information in the emissions reporting application 114b without any hard-coded information, just navigating to it via the workplace unit including the display 116.

[0054] In a typical implementation, the process graphic module 102a provides data indicating the characteristics of the object that the user interacts with in the process graphic 114a to the workplace module 112. The workplace module is configured to find the micro-frontend application related to the object. In other words, the workplace module 112 is looking for registered applications of the distributed control system or integrated operations management system that have a relationship with a given object, such as using an object directory service, and returns to the caller, the process graphic module 102a, a list of applications and parameters for calling the object.

[0055] The process graphics module 102a has information showing a context menu with navigation information on the screen 116. The emissions reporting application module 102b may be called. Alternatively, the context menu handling may be provided as a function of the workplace or a dedicated service.

[0056] The emissions reporting application 114b may be loaded into the display 116, or if already loaded, the emissions reporting application 114b may change its content based on the object passed by the process graphics module 102b. In other words, the emissions reporting module 102b is configured to change its content related to the object based on changes made to the object in the process graphics module 102a.

[0057] Although the two software modules 102a-b are unaware of each other, have different lifecycles, and also have different technologies, they can interact with each other in the same user workplace with the same user account and a similar look and feel.

[0058] The set of control units 118 a - c configured to implement the distributed human-machine interface system 100 may be part of a system 150 that also includes a computer display 116 .

[0059] Figure 3Two HTML5 clients are shown, each displaying visual content from a collection of modules. For example, a process graphics application 114a and an alarm list application 123 are running on two clients 120, 121. Another application 125 is running on client 125, which may be a trend visualization application. Each application is its own micro-frontend application, coordinated by the HMI framework as described herein.

[0060] On the device side of the industrial system 155, there may be system services 127, controllers 129, input / output devices 131, management systems, field connection interfaces (FCIs) 133, and brownfield system components 135, i.e., legacy software that must coexist with new software and modules.

[0061] Engineers use engineering tools 137 to configure all software and hardware components of the system 155 through a communication interface 139 (eg, Ethernet). The communication protocol of the communication system 139 may be OPC UA, MQTT, web API, or hardwired or wireless.

[0062] The distributed human-machine interface system 100 receives data signals from the communication system 139 and routes them to the relevant micro-front-end modules to change the visual content of the relevant applications 114a, 123, 125.

[0063] In addition, in order to integrate external applications, local applications 141 or cloud-based applications 142, the distributed human-machine interface system 100 may include a viewer application 148 connected to the communication system 139. The application can be an embedded frame in a web page, which allows third-party websites to be embedded in a web application. For example, application 148 is configured to collect visual content from a micro-frontend application and from at least one external application 141, 142, and provide the visual content to a computer display such as at clients 120 and 121. Application 149 can host a network component loaded into viewer 148. Application 149 can be ABB Edgenius. In addition, other applications, such as ABB applications, can be more closely embedded, which use a common HMI framework as a basis. In this case, those applications will not run completely in isolation.

[0064] System Services 127 are used for system diagnostics, authentication, licensing, security features such as audit logging, certificate management, global discovery services, etc., and retrieve information via the OPC UA client interface, as well as expose certain information using the OPC UA server. Diagnostic information is available via the OPC UA server interface and using an interface (e.g., HTTP) that can be easily consumed by a standard coordinator and potentially used to restart a service if it is behaving abnormally. Diagnostic information can be available to operators at a high level, or a more detailed view can be made available to system engineers and support personnel. Diagnostic information also includes performance measurements that can be used for troubleshooting purposes, or to potentially automatically increase or decrease the number of instances of the corresponding service to better respond to different usage patterns.

[0065] Figure 4 A possible deployment of a web-based HMI that outperforms current DCS technology is schematically shown. The top layer 402 is formed by a collection of micro frontends that together form a web application, where a web browser running the HMI frontend interacts with a backend 404 via HTTPS or Secure Web Sockets (WSS) protocol in a first flow F1.

[0066] The HMI backend 404 then uses the OPC UA protocol (e.g., Figure 3 139) to interact with the DCS system 406. The DCS system provides an interface for the controller in flow F3. Typically, the communication between the DCS and the control hardware involves a proprietary protocol or the OPC Classic protocol, or some other communication. According to flow F6, if the HMI backend 404 supports the OPC UA protocol, then the HMI backend 404 can also interact directly with the control hardware 408. Such a scenario assumes that the control device 408 will be connected to the OPC UA communication backbone 139 directly or via the FCI 133.

[0067] The engineering platform 410 is used to configure the DCS system 406 and the HMI backend 404 in flows F4 and F5, respectively. The configuration is deployed from the engineering platform 410 to the HMI backend using the OPC UA protocol 139. The same engineering platform 410 is used to deploy the DCS system 406. Alternatively, the engineering platform 410 may house two separate modules that are synchronized, each module being responsible for the configuration of the HMI backend 404 and the DCS system 406, respectively.

[0068] Figure 5 Provided Figure 4Specific example of a high-level architecture 500 shown. Any service depicted in this figure can be deployed in multiple instances to obtain a highly available system configuration (this can usually be achieved by using a coordinator). The HMI front end runs in an HTML5 browser 502 as a client application used by the user. Similar to "System Management" 506 and "Certificate Management" 508, the engineering platform (engineering) 504 also exposes a web user interface (based on a common HMI framework). Such an architecture allows users to seamlessly switch between runtime activities (i.e., interaction with the HMI backend 510) and engineering 504 activities, which dominate during system debugging or expansion and upgrades.

[0069] "System Management" 506 provides an overview of the nodes, services, users, roles, etc. deployed in the system. Engineers can manage system nodes (e.g., add, remove, backup, restore), view their status, collect diagnostic information, or collect crash dumps in case of failure. "Authentication Service" 512 implements the OpenID Connect authentication protocol, and it is used by all services and front-end applications to authenticate users and obtain security tokens.

[0070] The "Certificate Management" service 508 provides an automated mechanism for distributing certificates in the system to establish secure communications. This solution primarily handles HTTPS certificates used by web servers and TLS certificates used by OPC UA clients and servers. The "HMI Backend" layer 510 is used with Figure 4 In this particular case, it includes four services: workplace 514, alarm 516, trend 518, and process graphics service 520. In this particular case ( Figure 4 Flow F6 in ), all services interact directly with the control layer 521, which consists of controllers of field connection services (FCI). Similarly, HMI services can interact with DCS systems, such as Figure 4 Indicated by flow F2 in .

[0071] Figure 5 Also shown is a history service 522 for archiving long-term and / or short-term event and process data obtained from the control level or DCS system.

[0072] Figure 5 The flows in are for illustration only, and the trend component can establish a direct data subscription to the controller, such as displaying a trim curve. All services in the HMI backend layer 510 utilize the runtime directory service 524 that maintains the object directory, which can be used for service discovery. In some cases, the HMI service can also host its own copy of the object directory.

[0073] There is also a licensing service 525.

[0074] The authorization service 526 is a shared service that provides access tokens to clients so that they can access resources and helps implement role-based user management. The authorization service 526 can typically be implemented as part of the OPC UA Global Discovery Service (GDS).

[0075] All HMI backend services also interact with a licensing service that typically implements specific license execution functionality. This may involve limiting the number of actively connected clients or restricting access to specific functions of the operator workplace based on parameters specified in the license file.

[0076] A computer program product is also provided, comprising a computer readable medium having computer program means stored thereon to implement the distributed human-machine interface system 100 .

[0077] The computer program product may be executed by the set of distributed control units 118a-c.

[0078] The control unit may include a microprocessor, a microcontroller, a programmable digital signal processor, or another programmable device. The control unit may also or alternatively include an application specific integrated circuit, a programmable gate array or programmable array logic, a programmable logic device, or a digital signal processor. In the case where the control unit includes a programmable device (such as the microprocessor, microcontroller, or programmable digital signal processor described above), the processor may also include computer executable code that controls the operation of the programmable device.

[0079] In one or more examples, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted on a computer-readable medium as one or more instructions or codes and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media corresponding to tangible media such as data storage media, or communication media including any media that facilitates the transfer of computer programs from one place to another, for example, according to a communication protocol. In this manner, computer-readable media may generally correspond to (1) tangible, non-transitory computer-readable storage media, or (2) communication media such as signals or carrier waves. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in the present disclosure. A computer program product may include computer-readable media.

[0080] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer.

[0081] Even though the invention has been described above with reference to specific exemplifying embodiments thereof, many different alterations, modifications and the like will become apparent for those skilled in the art.

[0082] Moreover, variations to the disclosed embodiments may be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. A distributed human-machine interface system (100) for distributed control systems or integrated operation management, comprising: A set of view modules (102) is a separately isolated micro-frontend application (104), wherein the set of view modules (102) is each implemented by a network component (106), Each micro frontend application includes a backend application (108) contained in a corresponding separate container (110), the backend application communicating with the view module of the backend application using a network-based protocol, A common HMI framework (112) for all of the view modules, the common HMI framework comprising at least infrastructure functionality and user experience design. 2 . The distributed human-machine interface system of claim 1 , wherein the common HMI framework is configured to provide coordinated control of all of the view modules in a workplace.

3. According to the distributed human-machine interface system according to any one of claims 1 and 2, the backend application uses HTTPS REST API or network socket to communicate with the view module of the backend application.

4. A distributed human-machine interface system according to any one of the preceding claims, wherein the network component is HTML5.

5. A distributed human-machine interface system according to any of the preceding claims, wherein the common HMI framework is configured to exchange messages (M) and states (S) across different micro-frontends.

6. A distributed human-machine interface system according to any one of the preceding claims, comprising at least one control unit (118a-c), wherein the at least one control unit is configured to run at least one micro front-end respectively.

7. A distributed human-machine interface system according to any one of the preceding claims, wherein the micro-frontend application includes at least a process graphics module (102a) and a workplace module, wherein the process graphics module is configured to provide data indicating characteristics of an object with which a user interacts, and the workplace module is configured to search for a micro-frontend application related to the object.

8. A distributed human-machine interface system according to any one of the preceding claims, wherein the micro-frontend application provides at least two types of applications.

9. The distributed human-machine interface system according to claim 8 comprises a viewer application (149) configured to collect visual content from the micro-frontend application and from at least one external application, and to provide the visual content to a computer display.

10. A distributed human-machine interface system according to any one of the preceding claims, wherein the docker containers are geographically distributed on separate control units of the distributed control system.

11. A distributed human-machine interface system according to any one of the preceding claims, wherein the HMI framework is configured to provide token-based authentication for user login.

12. A distributed human-machine interface system according to any one of the preceding claims, wherein the micro-frontend application includes at least one of a process graphics module, an alarm module and a trend view module.

13. A collection of control units configured to implement a distributed human-machine interface system according to any one of the preceding claims.

14. A system comprising a computer display and an assembly of the control unit according to claim 13.

15. A computer program product comprising a computer readable medium having stored thereon computer program means for implementing the human-machine interface system according to any one of the preceding claims.

Images