Method and engineering system for generating user interface for industrial automation device

Abstract

The invention relates to a method and an engineering system for generating a user interface for a handling and monitoring device of an industrial automation installation. In this case, viable graphical objects of the user interface (Screen) are identified using a generative syntax and on the basis of meta-information relating to the data object, such as an identifier (Tags) of the data object, and then a plurality of viable user interfaces (Screen) are generated from the identified graphical objects, the allocation of the state variables of the graphical object to the data object is then effected for a plurality or all of the generated user interfaces (Screen), then a plurality of the user interfaces (Screen) generated in the third step are selected, the selection being effected on the basis of the number and / or quality of all the allocations of the respective user interfaces (Screen), and the allocation of the state variables of the graphical object to the data object is effected on the basis of the number and / or quality of all the allocations of the respective user interfaces (Screen). Wherein for each assignment of a graphical object to a data object, a degree of matching of the respective state variable with the assigned data object is evaluated, and finally a previously selected user interface (Screen) is proposed or provided for use. The proposed generative syntax for use in defining feasible user interfaces in an engineering system for operating and monitoring devices thus achieves a high degree of flexibility and significantly reduces the workload of relevant personnel responsible for generating user interfaces.

Description

Technical Field

[0001] The present invention relates to a method for generating a user interface for controlling and monitoring equipment of an industrial automation device, according to the preamble of claim 1, and also to an engineering system for generating a user interface for controlling and monitoring equipment of an industrial automation device, according to the preamble of claim 8. Background Technology

[0002] Industrial automation equipment is typically controlled via operating and monitoring devices with graphical user interfaces, also known as screens. Common operating and monitoring devices include HMI devices or HMI panels (HMI = Human-Machine Interface), but SCADA systems (SCADA = Monitoring and Data Acquisition) and various other similar operating and monitoring devices can also be used. What all these devices have in common is that they have one or more different user interfaces or screen pages, i.e., screens, where multiple elements of the industrial automation equipment are represented by graphical objects on each screen and therefore on each user interface. These graphical objects may represent, for example, the state of the corresponding element (e.g., machine, process element, or the like), or multiple operating elements, such as switches or adjustment knobs, that affect the corresponding element.

[0003] The components of industrial automation devices are typically controlled by industrial controllers, such as programmable logic controllers (PLCs) or similar devices, where the industrial controller is connected to the sensors and actuators of the corresponding components.

[0004] Within industrial controllers, sensors and actuators of components are mapped to data objects, which can be used in the control program as variables or other data structures. In operating and monitoring devices, graphical objects each have state variables, where, for example, the values ​​of state variables can be displayed through graphical components of the corresponding graphical object, or where graphically represented operating elements (such as switches, buttons, sliders, or similar elements) affect the data values ​​of the corresponding state variables of the graphical object.

[0005] The state variables of graphical objects are directly or indirectly linked to the data objects of components in the industrial controller in terms of data technology, thereby enabling interaction between graphical objects, data objects, and the components of the industrial automation device.

[0006] The published document WO 2021 / 104608 A1 (authors Hanniman et al.) "VERFAHREN ZUM ERZEUGENEINES ENGINEERING-VORSCHLAGS FÜR EINE VORRICHTUNG ODER ANLAGE (Method for generating engineering recommendations for devices or equipment)" discloses a method for generating recommendations in automated systems engineering using multiple graphs.

[0007] The publicly available document US 2021 / 397171 A1 (authors Sayyarrodsari et al.) "INDUSTRIALAUTOMATION HMI PROGRAM FILE GENERATION FROM COMPUTER-AIDED DESIGN" proposes generating graphical representations for HMI systems from digital design drawings.

[0008] Patent document US 11 513 508 B2 (authors Sun et al.) entitled "METHOD, COMPONENT, ANDELECTRONIC DEVICE FOR CREATING HUMAN MACHINE INTERFACE" proposes a method to generate a system user interface by automatically combining the HMI resources of various system components into a user interface.

[0009] SCADA systems or other control and monitoring devices are typically developed based on the specific machines or production lines they monitor and control. Engineers are usually provided with a list of numerous variables (a list of labels), also known as data points or tags, upon which they create visualizations and graphical user interfaces (screens).

[0010] The following refers to all variables, addresses, and structures of industrial controllers (PLCs) or other devices that need to be linked to HMI or SCADA systems as “tags.” Tags typically have names or name structures, and the components of a name or multi-level name structure are derived from its meaning or function. Therefore, a tag is the name of an address or data structure assigned to a device or programmable logic controller or similar device. Depending on the device / SPS manufacturer, it may also be referred to as a “variable” or “symbol.”

[0011] Given the lack of documentation and standards, screen creation (planning) is often a difficult task.

[0012] Currently, screens are typically designed manually in graphical editors. This method works well for individual screens but is difficult to scale to large systems. Since the design process usually starts with a blank screen, the work is often repetitive. When reuse is necessary, it's usually done by copying and pasting, which is difficult for long-term maintenance and management. Summary of the Invention

[0013] Therefore, the object of the present invention is to simplify and at least partially automate the creation of graphical user interfaces for industrial automation devices.

[0014] This objective is achieved by the method according to claim 1 and the system according to claim 8.

[0015] This paper proposes a method for generating user interfaces for operating and monitoring equipment in industrial automation systems. The method involves placing graphical objects representing elements for operating and / or monitoring the automation system on the user interface, and linking data objects associated with at least one industrial controller to state variables of the graphical objects. In a first step, a generative grammar is used to identify feasible graphical objects for the user interface based on metadata related to the data objects, particularly identifiers of the data objects. In a second step, multiple feasible user interfaces are generated from the identified graphical objects. In a third step, the assignment of state variables of the graphical objects to data objects is implemented for the generated multiple or all user interfaces. In a fourth step, multiple user interfaces generated in the third step are selected, based on the quantity and / or quality of all assignments for the corresponding user interfaces, wherein for each assignment from graphical objects to data objects, the degree of matching between the corresponding state variables and the assigned data objects is evaluated. In a fifth step, the user interface selected in the fourth step is proposed or provided for use. The proposed generative grammar, in its application to defining feasible user interfaces in engineering systems for operating and monitoring equipment, thus achieves high flexibility and significantly reduces the workload of personnel responsible for generating user interfaces.

[0016] Furthermore, this objective is achieved through an engineering system for generating user interfaces for operating and monitoring equipment in industrial automation devices. This engineering system is designed to invoke data objects from a programming system for industrial controllers; to identify feasible graphical objects for user interfaces using a generative grammar based on metadata related to the data objects, particularly the identifiers of the data objects; to assign state variables of graphical objects to data objects for multiple or all generated user interfaces; to select multiple user interfaces generated in a third step, proposing to achieve this selection based on the quantity and / or quality of all assignments to the respective user interfaces; to evaluate the degree of matching between the corresponding state variables and the assigned data objects for each assignment from graphical objects to data objects; and to propose or use the selected user interfaces. Using this engineering system, the advantages discussed in conjunction with the methods described above can be achieved.

[0017] Advantageous embodiments of the method according to the invention are given in the dependent claims. The features and advantages described therein also apply accordingly to the apparatus according to the invention. The advantageous embodiments can be applied individually or in a reasonable combination.

[0018] Advantageously, the rules of the generative grammar are not, or are not merely, manually generated, but derived from the user interface of existing automated devices, particularly through a learning system, where the existing assignments of graphical objects and their state variables to existing components are processed into corresponding rules or rule sets for the grammar. In this process, the generative grammar is preferably stored in a formal description language, or defined using multiple mutually referencing markup language objects. This enables the grammar to be used universally, even across manufacturers.

[0019] To assign state variables to data objects, feasible state variables in generative grammar are defined using expressions with placeholders, where data objects are mapped to these expressions, thus enabling a better allocation of data objects to state variables. Similarly, to identify feasible graphical objects, "fuzzy" identifiers in generative grammar can be used to retrieve metadata, or the retrieval can be based not only on matching but also on similarity.

[0020] Especially in cases of frequent use, in the third step, the assignment is implemented using a neural network trained for assignment, and the neural network outputs the degree of assignment used in the fourth step for each implementation, thereby obtaining better results.

[0021] In principle, weighting factors can be provided for some or each graphical object or its state variables, thereby enabling priority to be placed on important elements during the automatic execution of the method, and enabling the selection and suggestion of more suitable user interfaces (screens) for use.

[0022] Advantageously, in the first step, multiple different graphical objects are found for the data object, and in the second step, only one of the found graphical objects is used for the corresponding data object on each of the generated different user interfaces. This creates different screen options for subsequent selection. Attached Figure Description

[0023] Embodiments of the method according to the present invention will now be described with reference to the accompanying drawings. These embodiments also serve to illustrate the apparatus according to the present invention.

[0024] The diagram shows: Figure 1 A schematic diagram illustrating the flow of this method, and Figure 2 A simplified example of the rules for generative grammar is shown. Detailed Implementation

[0025] The method proposed here for automatically generating user interfaces (hereinafter also referred to as screens) is based on three mechanisms: 1. Syntax-based user interface or screen generation. 2. A matching algorithm that links (matches) variables in the syntax with tags from a tag list, i.e., identifiers of data objects from the control layer. 3. An evaluation mechanism to select the screen (user interface) that best matches the list of tags.

[0026] Basic process in Figure 1 As shown in the diagram. The language of feasible user interfaces / screens is retrieved according to grammatical rules. When a rule generates a variable that is not yet assigned to a data object or its identifier (Tag), the matching mechanism attempts to match that variable with a tag in the tag list.

[0027] When the effective screen is reached, i.e., the rules are no longer applied, the resulting screen is evaluated and the evaluation value is recorded. This evaluation can be based on the degree of matching between labels, preferably those containing appropriate abbreviations and the names of graphical objects or their state variables found in the syntax. Weighting factors can also be specified for rules or objects in the syntax.

[0028] The method continues until a certain number of graphical objects or screens to be displayed are generated or other termination criteria are met.

[0029] Ultimately, the engineer is shown one or more screens with the best score, and that screen is recommended for use. The specific arrangement of the graphical objects, i.e., the layout, can be implemented automatically and does not play a decisive role in the selection process.

[0030] Alternatively, feedback from technical personnel can be requested during the search, or, as previously mentioned, a selection can be made from the list of screens with the highest ratings if multiple suggested options exist.

[0031] The principles of generative grammar are known in many examples, see, for example, https: / / de.wikipedia.org / wiki / Generative_Grammatik.

[0032] This syntax describes the "language" of possible HMI systems and underlying automation devices.

[0033] For example: Screen := PickerLines | ProductionLines | Tanks Picker := DeltaPicker | Gantry | RobotArm This means that the user interface / screen consists of either a picking line (for robots in a warehousing system), a production line, or a storage tank. This can be implemented at each level. The picker can be a Delta picker (parallel arm robot), a gantry (coordinate robot), or a robotic arm.

[0034] Tanks := Tank

[0035] Tanks := Tank | Tanks

[0036] Therefore, a storage tank consists of at least one storage tank or any number of storage tanks.

[0037] Tank := tank_picture fill_indicator ( fill_percentage)

[0038] The storage tank, or the graphical object to be generated for it, consists of a background image of the tank and a fill indicator, which is quantified by a state variable. The fill_percentage parameter is used.

[0039] The example shown here is just one possible way to define the syntax. It can also be defined using templates written in a markup language such as XML and referencing each other.

[0040] As described in the syntax description, the generated screen, or the more specific graphical objects it contains, includes state variables (or simply variables). These state variables must be bound to identifiers of data objects belonging to tags or components of industrial automation devices. In the example above, this would be, for example, a register or variable of the industrial controller that stores the value of the fill level sensor. This variable could be named `tank_fill3`. Several methods can be used to match tags with unbound variables.

[0041] One option is to use regular expressions, which describe which tags might match the variable.

[0042] Example: fill_percentage -> ([Tt] ank)?\d?.? ([Ffill]).

[0043] This means the following tags may be related to Matches fill_percentage: Tank1fill, Fill_indicator, tank_fill3 In addition to regular expressions, other mechanisms can be used, such as classifiers based on neural networks or support vector machines.

[0044] Figure 2 Another simplified example of generative syntax is shown. Here, it is also assumed that a list containing data objects and their identifiers—i.e., a list of labels—has been retrieved from the engineering configuration system. More information, such as application-related information, is often available and useful as additional metadata.

[0045] exist Figure 2 In the example, the system can automatically determine the industry (chemistry) based on the frequency of tags in the tag list, as heuristically presented in the first line, and thus determine the order in which the rules of the grammar are tested (i.e., rules for the identified industry are tested first). The following three lines specify the retrieval pattern for feasible state variables. Subsequent lines describe the retrieval pattern or sequence; this can be adjusted by the engineer.

[0046] Based on the screen or user interface generated through syntax and the labels that match its state variables, the evaluation mechanism ultimately calculates a score. This allows the selection of the screen that best matches a specified list of labels.

[0047] The basic mechanism simply counts the labels that match a variable (state variable) in a given screen. However, more sophisticated mechanisms can be applied where the matches are not "Boolean," but rather where different variables have different importance levels or weighting factors, or where variables are subject to restrictions to prevent them from matching labels that have been assigned to other variables, for example.

[0048] The syntax for defining feasible screens provides a high degree of flexibility in the description methods. Traditional methods rely on a limited number of templates that cannot account for the wide diversity of industrial processes and machines.

Claims

1. A method for generating a user interface of a control and monitoring device for an industrial automation plant, wherein placing graphical objects for operating and / or monitoring elements of the automation plant on the user interface (Screen), wherein data objects of at least one industrial controller associated with the elements are linked to state variables of the graphical objects, characterized in that in a first step, feasible graphical objects for the user interface (Screen) are identified using a generative grammar based on meta-information related to the data objects, in particular based on tags of the data objects, in a second step, a plurality of feasible user interfaces (Screen) is generated from the identified graphical objects by retrieving a language description of the user interface using grammar rules, in a third step, for the generated user interfaces (Screen), the assignment of the state variables of the graphical objects to the data objects is realized using regular expressions or classifiers, respectively, in a fourth step, a plurality of the user interfaces (Screen) generated in the third step is selected, wherein the selection is realized based on the number and / or quality of all assignments of the respective user interface (Screen), wherein for each assignment of a graphical object to a data object, the degree of matching of the respective state variable to the assigned data object is evaluated, and in a fifth step, the user interface (Screen) selected in the fourth step is proposed or provided for use.

2. The method of claim 1, wherein, Prior to the first step, the rules of the generative grammar are derived from existing user interfaces (Screen) of automation plants, in particular by a learning system, wherein the graphical objects and the existing assignments of state variables of the graphical objects to existing elements are processed into respective rules of the grammar.

3. The method according to any of the preceding claims, characterized in that, The generative grammar is stored in a formal description language or defined using a plurality of mutually referencing markup language objects.

4. The method according to any of the preceding claims, characterized in that, In the third step, for the assignment of the state variables to the data objects, feasible state variables in the generative grammar are defined by expressions with placeholders, wherein the data objects are mapped to the expressions with placeholders.

5. The method according to any of the preceding claims, characterized in that, In the third step, the assignment is realized using a neural network trained for the assignment, and the neural network outputs for each realized assignment a degree of the assignment used in the fourth step.

6. The method according to any of the preceding claims, characterized in that, For some or each of the graphical objects or state variables of the graphical objects, a weighting factor is provided.

7. The method according to any of the preceding claims, characterized in that, In the first step, a plurality of different graphical objects is found for a data object, wherein in the second step, only one of the found graphical objects is used for the involved data object on the generated different user interfaces (Screen), respectively.

8. An engineering system for generating a user interface (Screen) of a control and monitoring device for an industrial automation plant, wherein it is proposed to place on the user interface (Screen) graphical objects for operating and / or monitoring elements of the automation device and to link to the graphical objects state variables of data objects of at least one industrial controller which are assigned to the elements, characterized in that the engineering system is designed to identify possible graphical objects for the user interface (Screen) using a generative grammar on the basis of meta-information relating to the data objects, in particular on the basis of tags of the data objects, the engineering system is designed to generate possible user interfaces (Screens) from the identified graphical objects by searching a language description of the user interface using grammar rules, the engineering system is designed to implement for each generated user interface (Screen) the assignment of the state variables of the graphical objects to the data objects using regular expressions or classifiers, the engineering system is designed to select a number of the user interfaces (Screens) generated in the third step, wherein it is proposed to implement the selection on the basis of the number and / or quality of all assignments of the respective user interface (Screen), wherein it is further proposed to evaluate for each assignment of a graphical object to a data object the degree of matching of the respective state variable to the assigned data object, and the engineering system is designed to propose or use the selected user interfaces (Screens).

Images