Method and system for automatic generation of plug-ins for engineering systems

Abstract

This invention provides a method and system for automatically generating plug-in programs for an engineering system. The method includes receiving a first user action from a user via a processing unit (202). The first user action is associated with at least one of a plurality of engineering objects in the engineering system. The method further includes predicting at least a second user action by the processing unit (202) through a user action detection process based on a knowledge graph relating to the first user action and the engineering system. The method further includes generating a sequence of user actions by the processing unit (202) through a sequence identification process based on the received first user action and the predicted at least one second user action. The method further includes generating a plug-in program by the processing unit (202) through a plug-in generation process based on the generated sequence of user actions. The method further includes deploying at least one generated plug-in program in the engineering system via the processing unit (202).

Description

[0001] This invention relates to the field of industrial automation, and more particularly to a method and system for automatically generating plug-in programs for applications in engineering systems.

[0002] The application is configured to perform multiple functions within the engineering system. These functions include, but are not limited to, project creation, device activation, device configuration, and project deletion. Users are able to input function-specific commands into the application to cause it to perform those functions. Users can input commands via input devices such as touchscreens, human-machine interfaces, computer mice, or voice-based input.

[0003] In one example, the application must execute a sequence of functions to achieve a specific user goal. This specific user goal could be an objective or condition that must be met within the system to ensure its optimal operation. In one example, this sequence of functions is referred to as a workflow. In this case, the user must input a sequence of commands to cause the application to execute the sequence of functions. Where the sequence of functions must be executed repeatedly, the user must repeatedly input the sequence of commands. In other words, the user must perform redundant and repetitive actions to achieve a specific goal.

[0004] In conventional systems, plug-ins are used to remedy the aforementioned problems. Plug-ins are supplementary programs added to applications. They add custom commands and features to the application. For example, a plug-in can be activated by entering a single command, and furthermore, the plug-in causes the application to execute a sequence of functions. This saves the user from the hassle of having to enter a sequence of commands to make the application execute a sequence of functions. Instead, it allows the user to type a single command and thus cause the application to execute a sequence of functions.

[0005] In existing systems, plug-ins are manually scripted by coders. Coders must manually identify sets of reusable functional sequences or workflows used by users and then script the plug-ins to execute those identified workflows. Therefore, the round-trip time for creating and deploying plug-ins within the application is typically significant. Furthermore, the workflows include information that is confidential to the company using the application. Therefore, allowing coders to access such information often results in violations of the engineering system's data security policies. Moreover, application users lack the necessary knowledge and skills to generate such plug-ins.

[0006] Furthermore, in the field of factory automation, each user's preferred workflow differs from that of other users among a group of users. Therefore, coders must manually script multiple plug-ins, ensuring that each plug-in is customized for each of the multiple users. Manually generating such plug-ins for each of the multiple users is an expensive, time-consuming, and potentially impossible endeavor.

[0007] In view of the above, there is a need for an efficient and cost-effective method and system for automatically generating plug-in programs for each of multiple users of an application running in an engineering system. Therefore, the object of the present invention is to provide an efficient and cost-effective method and system for automatically generating plug-in programs for each of multiple users of an application running in an engineering system.

[0008] The objective of this invention is achieved through a method for automatically generating plug-in programs in engineering systems.

[0009] As mentioned in this paper, an engineering system comprises multiple components and processes. These components and processes are interconnected and designed to achieve specific technical objectives. In one example, an engineering system is configured to achieve technical objectives across multiple disciplines such as mechanical, electrical, software, and power generation. For instance, engineering systems can be deployed in a variety of fields, including but not limited to manufacturing, telecommunications, transportation, and energy. As used herein, the term "engineering system" is intended to encompass a wide range of technical architectures and configurations, from discrete hardware components to complex software-driven platform variations, all designed to perform specific functions or tasks within a broad engineering application context.

[0010] Applications within an engineering system refer to software components specifically designed to perform multiple predefined tasks or functions to enable the engineering system to achieve specific technical goals. In one example, the application is implemented as a standalone software application. In other examples, the application includes integrated modules within a larger software framework. Examples of applications include, but are not limited to, control algorithms, simulation tools, data processing applications, and user interfaces. As described herein, applications within an engineering system embody customized software solutions that are crucial for the seamless integration and optimal performance of the engineering system.

[0011] An engineering system comprises multiple engineering objects. An "engineering object" refers to a digital entity that plays a crucial role in the representation and functionality of the engineering system. Multiple engineering objects are represented as graphical and software objects within the application and engineering system, typically appearing as icons or templates within the application or graphical user interface of the engineering system. In another example, engineering objects include project files and project items stored within the engineering system.

[0012] Each engineering object is assigned a distinct identity by a specified name and associated with a set of properties that define the characteristics and behaviors of engineering objects within the engineering system. This set of properties encapsulates the necessary information, allowing users to effectively understand and interact with engineering objects. Users are authorized to interact with multiple engineering objects through a Human-Machine Interface (HMI) or other input devices.

[0013] The engineering system also includes a knowledge graph, which contains information associated with each of the multiple engineering objects. The knowledge graph comprises multiple nodes. Each node in the knowledge graph contains information associated with a specific engineering object among the multiple engineering objects. In one example, each node in the knowledge graph contains information associated with a category of engineering objects among multiple categories of engineering objects. The knowledge graph is stored internally within a database within the engineering system. The knowledge graph seamlessly orchestrates a dynamic network of interconnected nodes, each representing a unique engineering object. Advantageously, the knowledge graph facilitates a comprehensive understanding of the complexities of the engineering system.

[0014] In one example, the knowledge graph is implemented as a graph database such as Neo4j. In another example, the knowledge graph is generated based on a data model such as Apache Jena. The engineering system also includes one or more application programming interfaces (APIs) that facilitate real-time updates within multiple engineering objects of the engineering system. By incorporating these implementation strategies, the knowledge graph becomes not only an information repository but also a dynamic, user-responsive framework that empowers the engineering community to navigate, understand, and evolve engineering systems with unparalleled efficiency.

[0015] As mentioned in this article, a plug-in is a software component designed to perform a series of predefined functions using a single command, thereby eliminating the need for multiple users to manually input command sequences. A plug-in is a supplementary program added to one of the applications or engineering systems. Plug-ins add custom commands and features to one of the applications or engineering systems. For example, a plug-in is activated by entering a single command, and furthermore, the plug-in causes one of the applications or engineering systems to execute a sequence of functions. Where the plug-in is not present, the user may have to enter a sequence of commands to cause the application or engineering system to execute the sequence of functions. Therefore, it is advantageous to spare the user the hassle of having to enter a sequence of commands to cause the application to execute the sequence of functions.

[0016] One of the engineering systems or applications is configured to continuously receive multiple user inputs from the operator or user of the engineering system. Multiple user inputs include one or more commands, one or more text inputs, or one or more voice inputs. Multiple user inputs also include a series of interactions, including but not limited to command input via a graphical user interface, parameter adjustment, initiation of a simulation scenario, or any other instructions provided by the operator or user. The engineering system receives multiple user inputs via an input device such as a human-machine interface. Multiple user inputs, hereinafter referred to as multiple user actions, constitute a command initiated by the operator or user to instruct one of the applications or engineering systems.

[0017] Each of the multiple user actions corresponds to a specific command. Upon receiving each of the multiple user actions, one of the engineering systems or applications is triggered to execute a predefined function within the engineering system. This allows operators or users to configure, direct, or modify the behavior of the application or engineering system by using the multiple user actions, thereby influencing the overall execution and output of the engineering system. The term "user action" within the scope of this patent refers to the dynamic and interactive nature of the human-machine interface associated with the engineering system. This allows operators or users to exert control and influence over the application to customize one or more operations of the application according to the evolving requirements of the engineering process.

[0018] In one example, an identifier is used in the engineering system to identify each of multiple user actions. The identifier for a user action is a data object that includes the name of the user action, the purpose of the user action, and one or more code segments associated with the user action. The one or more code segments associated with the user action include one or more code statements that are executed by either the engineering system or the application when the engineering system receives the user action. In one example, the one or more code segments include portions of the source code of either the application or the engineering system. In another example, the one or more code segments include one or more function calls or procedure calls to invoke one or more functions in the source code of an application that are scripted. In this case, the engineering system executes one or more functions whenever it receives a user action.

[0019] The identifier for a user action also includes a natural language description of one or more functions performed by the engineering system when the user action is received at the engineering system. In other words, the identifier is a crucial data object designed to store information directly linked to one or more user actions received by the engineering system. Furthermore, each identifier is structured to encapsulate further details associated with a specific user action, thus forming a chronological sequence mirroring the receipt of one or more user actions within one of the engineering systems or applications. Therefore, the string of identifiers considers a cohesive representation of user activities within one of the engineering systems or applications. Examples of identifiers include alphanumeric code-based identifiers, timestamp-based identifiers, and security code-based identifiers. In one example, the engineering system assigns an identifier to each user action simultaneously with receiving the user action at the engineering system. In another example, the identifier for each user action is scripted by the coders who have already coded the application. The identifier for a user action can be stored in the engineering system and referenced by the engineering system when it receives a user action.

[0020] Each of the multiple user actions refers to one or more project objects among multiple project objects in the engineering system. In one example, the multiple user actions include, but are not limited to, assigning a user action, renaming a user action, reconfiguring a user action, creating a new user action, and deleting a user action. In this case, assigning and renaming user actions cause the engineering system to assign or modify the name of a project object. Reconfiguring a user action causes the engineering system to adjust one or more configuration settings of a project object. Creating a user action causes the engineering system to create a new project object. Deleting a user action allows users to selectively remove project objects that are no longer relevant or necessary, thus simplifying the engineering system and enhancing its efficiency.

[0021] The engineering system also includes an automation module configured to monitor multiple continuously received user actions. In a preferred embodiment, the method includes determining at least one engineering object to which each of the multiple user actions points. In other words, the automation module is configured to determine at least one engineering object associated with at least one user action. The method also includes determining at least one node of a knowledge graph by a processing unit, wherein the at least one node includes information associated with the determined at least one engineering object. In one example, the automation module is configured to analyze multiple nodes of the knowledge graph by applying a natural language processing algorithm to determine at least one node of the multiple nodes of the knowledge graph. The method also includes updating at least one node by a processing unit. To update at least one node, the automation module is configured to insert information associated with at least one user action into the at least one node. Thus, the knowledge graph includes information associated with the relationships between each of the multiple user actions and each of the multiple engineering objects. Furthermore, each node of the knowledge graph includes information associated with a list of engineering objects and user actions, where a user action performed by one or more users on an engineering object is said user action.

[0022] The method also includes monitoring the order or sequence of each user action received at the engineering system via a processing unit. The automation module is also configured to update the knowledge graph by inserting one or more links between multiple nodes. The inserted links indicate the sequence of multiple user actions received by the automation module. In one example, the automation module receives a first user action pointing to a first engineering object and then receives a second user action pointing to a second engineering object. In this case, the automation module generates a first node associated with the first engineering object and a second node associated with the second engineering object. The automation module also inserts information associated with the first user action into the first node. The automation module also inserts information associated with the second user action into the second node. Furthermore, since the second user action is received after the first user action, the automation module generates a link between the first node and the second node. Therefore, the knowledge graph includes information associated with multiple engineering objects, multiple user actions pointing to the multiple engineering objects, and the sequence of multiple user actions received at the engineering system.

[0023] In other words, the knowledge graph comprises multiple links. Each of these links indicates the order in which user actions and subsequent user actions are received at the engineering system. In one example, the first node includes information associated with user actions received from one or more users of the engineering system. The second node includes information associated with at least one subsequent user action received from one or more users after the initial user action. In this case, at least one link connects the first node to the second node of the knowledge graph. At least one link indicates the order in which user actions and subsequent user actions are received.

[0024] The method also includes receiving a first user action from a user of the engineering system via a processing unit. The first user action refers to at least one of a plurality of engineering objects in the engineering system. In one example, the first user action includes the user clicking a graphical icon associated with at least one engineering object. In another example, the first user action includes the user typing a text-based command, wherein the text-based command includes the name of at least one engineering object. In other words, the first user action includes one or more parts that include information indicating at least one engineering object.

[0025] The method also includes predicting, via a processing unit, at least a second user action that can be received from the user after the first user action. The processing unit is configured to predict at least one second user action by implementing a user action detection process based on a knowledge graph and the first user action.

[0026] In one example, as mentioned in this paper, the user action detection process is a software-based process configured to anticipate and predict upcoming user actions within an engineered system. The user action detection process operates by carefully analyzing multiple user action flows and a knowledge graph encapsulating historical user interactions. By leveraging advanced algorithms and machine learning techniques, the user action detection process identifies patterns and correlations between multiple user actions, thus enabling it to make informed predictions about possible next user actions. Therefore, the predictive capabilities of the user action detection process facilitate proactive system responses and customized user experiences, thereby enhancing overall efficiency and user satisfaction. The user action detection process plays a crucial role in simplifying user interactions and ensuring seamless, anticipated user interfaces.

[0027] To implement the user action detection process, the automation module executes a first set of steps. In the first step of the first set of steps, the automation module is configured to cause the processing unit to analyze a knowledge graph based on the received first user actions. In the second step of the first set of steps, the automation module causes the processing unit to determine at least one node among a plurality of nodes in the knowledge graph. The at least one node includes information associated with at least one engineering object.

[0028] In one example, the first user action includes information associated with the identity of at least one project object. In this case, the automation module is configured to traverse the knowledge graph, searching each node of the knowledge graph for information associated with the at least one project object. The automation module is configured to traverse the knowledge graph until it finds one or more nodes that contain information about the at least one project object. The automation module is configured to traverse the knowledge graph using any of the known graph traversal techniques.

[0029] In the third step of the first plurality of steps, the automation module is further configured to analyze one or more nodes to identify at least one node that includes information associated with a previously received user action that matches the first user action. The previously received user action is received by the engineering system before the first user action is received. In one example, the automation module is configured to extract the name of the first user action from an identifier associated with it. Furthermore, the automation module is configured to analyze one or more nodes by searching for the extracted name of the first user action within one or more nodes. The identified at least one node includes information associated with a previously received user action having a name that matches the extracted name of the first user action.

[0030] In the fourth step of the first plurality of steps, the automation module is also configured to analyze one or more links associated with at least one node. The one or more links connect at least one node to a set of nodes within a plurality of nodes. The set of nodes includes information associated with a set of previously received user actions received by the engineering system after receiving previously received user actions. The set of nodes also includes information associated with one or more engineering objects pointed to by one or more users by the set of previously received user actions. Therefore, the automation module is configured to determine the set of previously received user actions received by the engineering system after receiving previously received user actions. Since the previously received user actions match a first user action, it is possible to roughly predict one or more user actions that the user might type that match the set of previously received user actions.

[0031] In the fifth step of the first plurality of steps, the automation module is configured to analyze one or more nodes to determine whether the one or more nodes include information associated with at least one project object or one or more project objects related to at least one project object. In the sixth step of the first plurality of steps, the automation module is configured to determine at least one second user action from the one or more nodes based on the determination that the one or more nodes include information associated with at least one project object or one or more project objects related to at least one project object. Advantageously, at least one second user action is predicted based on a knowledge graph, which includes contextual information associated with multiple user actions and multiple project objects. Therefore, at least one second user action is accurately predicted based on the interrelationships between multiple user actions and multiple project objects. This eliminates the need for the time-consuming task of analyzing the interrelationships between multiple user actions and multiple project objects and thus participating in the manual coding of the plug-in program.

[0032] In one example, the automation module is configured to determine a first node and a second node from one or more nodes. The first and second nodes include information associated with one or more engineering objects related to at least one engineering object. In this case, the automation module is configured to determine at least one second user action from the first and second nodes. For example, the automation module may determine all user actions in both the first and second nodes as part of at least one second user action.

[0033] The method also includes generating a sequence of user actions by a processing unit implementing a sequence identification algorithm with respect to a first user action and at least one second user action. The sequence identification algorithm can be a machine learning algorithm, such as the KNN classification algorithm. The sequence identification algorithm is configured to cluster the first user action and at least one user action into the sequence of user actions. In one example, the sequence identification algorithm includes a neural network, such as a deep neural network, trained to cluster the first user action and at least one second user action into the sequence of user actions. The neural network is trained by applying a supervised training algorithm to a labeled dataset. The labeled dataset includes a list of user actions applied to multiple engineering objects, and a list of subsequent user actions that can be performed by the user after each user action in the list of user actions.

[0034] In another example, the sequence identification algorithm is implemented as computer code, which, when executed by the processing unit, causes the automation module to analyze a first user action and at least one second user action. The computer code is also configured to generate a sequence of user actions. To implement the sequence identification algorithm, the automation module is configured to compare each of the first user action and at least one second user action with a list of user-defined user actions. User-defined user actions are those defined by the user or operator. The automation module is also configured to use the identified user actions as trigger points to ascertain the sequence of user actions. For example, the automation module is configured to identify a first type of user action as an initiation point and a second type of user action as an end point. The automation module is also configured to effectively delineate the defined range between the initiation point and the end point as a sequence of user actions. In other words, the automation module is configured to classify one or more user actions received between the identified initiation point and the identified end point as part of a sequence of user actions. Based on one or more user preferences, the user can generate and modify the list of user-defined user actions. Therefore, advantageously, the sequence of user actions generated by the automation module can be highly customized by the user.

[0035] For example, the first user action could be a user action such as "user selects a project object". The first user action is identified as the initiation point. At least one second user action could be a user action such as "user deselects a project object". At least one second user action is identified as the end point. Therefore, one or more user actions performed by the operator or user between selecting and deselecting a project object are determined as a sequence of user actions.

[0036] The automation module is also configured to generate workflow templates based on sequences of generated user actions. The workflow template includes the sequence of generated user actions and one or more parameters associated with each user action in the sequence. The one or more parameters associated with each user action include numerical variables and text data typed by the user via the user action. For example, one or more parameters include information about the intensity, magnitude, or level and timing associated with the user action. The workflow template is stored as a data object within the engineering system. In one example, the workflow template indicates a specific path within a knowledge graph. The specific path connects groups of nodes within the knowledge graph, where the groups of nodes include information associated with user actions in the set of generated user actions. Where the engineering system receives a stream of user actions, the automation module is configured to compare each user action in the stream of user actions with each user action in the workflow template.

[0037] The automation module is also configured to determine if a specific user action in a flow of user actions matches at least one user action in a workflow template. Where a specific user action matches at least one user action, the automation module is configured to record one or more subsequently received user actions as a string representing the user action. Therefore, the automation module is also configured to determine multiple strings representing user actions.

[0038] The automation module is also configured to compare each of multiple user actions with a generated sequence of user actions. The automation module also counts the number of times that the majority of user actions for each of multiple strings match user actions in the workflow template. In other words, the automation module counts the number of times it traverses a specific path associated with the workflow template.

[0039] The automation module is also configured to compare the counted number of times with a threshold number. If the count exceeds the threshold, the automation module is configured to perform a plugin generation process on the generated sequence of user actions. If the count does not exceed the threshold, the automation module prevents plugin generation. Advantageously, plugins are created only when the user repeatedly uses the sequence of user actions. Therefore, the generated plugins are useful to the user.

[0040] The automation module is also configured to optimize the workflow template and the generated sequence of user actions by applying a user action optimization algorithm to the workflow template and multiple user actions. In one example, the user action optimization algorithm is a machine learning algorithm configured to identify common user actions within each of a plurality of strings representing user actions. The automation module 112 is also configured to optimize the workflow template based on the identified common user actions. In one example, the automation module 112 is configured to optimize the workflow template and the generated sequence of user actions by applying either a known term frequency-inverse document frequency algorithm to the plurality of strings representing user actions. In another example, the automation module is also configured to optimize the generated sequence of user actions by applying a K-means classification algorithm to the plurality of strings representing user actions. In one example, the automation module 112 is configured to optimize the workflow template by pruning all user actions from the generated sequence of user actions except for the identified common user actions. Advantageously, the generated sequence of user actions includes only the common user actions identified from the plurality of strings representing user actions.

[0041] The automation module also causes the processing unit to generate a plugin program 108N by implementing a plugin generation process based on a sequence of generated user actions. To implement the plugin generation process, the processing unit is configured to execute a second set of steps.

[0042] In the first step of the second multiple steps, the processing unit is configured to receive one or more user actions from the user within a specific time interval. In the second step of the second multiple steps, the processing unit compares each of the received one or more user actions with a specific user action in a sequence of generated user actions. In the third step of the second multiple steps, the processing unit is configured to determine a count of the number of times the received one or more user actions match the sequence of generated user actions. In the fourth step of the second multiple steps, the processing unit determines that the determined count is greater than a threshold. If the determined count is greater than the threshold, the processing unit extracts a string of identifiers from the sequence of generated user actions. The string of identifiers is a data object containing information associated with each user action in the sequence of generated user actions. In the fifth step of the second multiple steps, the processing unit extracts one or more code segments from the extracted string of identifiers. In the sixth step of the second multiple steps, the processing unit generates a plug-in program by implementing a string concatenation process with respect to one or more code segments. The string concatenation process can be any known algorithm for concatenating strings.

[0043] Advantageously, the automation module automatically generates sequences of user actions that are frequently used by the user or operator.

[0044] In the seventh step of the second series of steps, the processing unit is configured to extract a natural language description of each user action in the generated sequence of user actions from the string of the identifier. In the eighth step, the processing unit is configured to generate a natural language description of the generated plug-in program by performing a string concatenation process using the extracted natural language descriptions of each user action in the generated sequence of user actions. Advantageously, the plug-in program is generated automatically without manual intervention. Furthermore, the plug-in program is generated based on one or more sequences specifically determined for the user.

[0045] The method also includes displaying information associated with the generated plug-in program to a user or operator via a processing unit. The displayed information associated with at least one generated plug-in program includes a natural language description of the plug-in program. The displayed information also includes a natural language description of each user action in a sequence of generated user actions. The displayed information also includes information associated with one or more code statements extracted from a string of identifiers. The displayed information also includes one or more user commands that can be used to invoke the at least one generated plug-in program. In one example, the one or more user commands may be text-based commands, voice-based commands, gesture-based commands, or mouse-click-based commands. Advantageously, the automation module learns from continuously received user actions from the user and provides suggestions to the user to optimize the user's efforts.

[0046] The method further includes comparing a generated sequence of user actions with a set of sequences of user actions via a processing unit. The set of sequences of user actions includes user actions stored in a database for reference. Each sequence in the set of sequences of user actions is linked to a set of plug-ins stored in the database. A user or operator can assign multiple mappings between the set of plug-ins and each sequence in the set of sequences of user actions. The method further includes determining whether a sequence in the set of sequences matches a generated sequence of user actions via the processing unit. If at least one sequence in the set of sequences matches a generated sequence of user actions, the processing unit is configured to determine a plug-in linked to the generated sequence of user actions from the set of plug-ins. The method further includes displaying the determined plug-in to the user via the processing unit. If the sequence does not match one or more determined sequences of user actions, the processing unit is configured to deploy a generated plug-in to the engineering system.

[0047] The object of the present invention is also achieved by an engineering system for automatically generating plug-in programs for an engineering system. The engineering system includes a processing unit and a memory coupled to the processing unit. The memory includes an automation module stored in the form of machine-readable instructions executable by a processor. The automation module is configured to perform the method described above.

[0048] The object of the present invention is also achieved through an industrial environment. The industrial environment includes an engineering system, an industrial plant, comprising one or more physical components, and multiple human-machine interfaces communicatively coupled to the engineering system and the industrial plant. The engineering system is configured to perform the method steps described above.

[0049] The object of the present invention is also achieved by a computer program product having machine-readable instructions stored therein, which, when executed by one or more processors, cause one or more processors to perform the method steps described above.

[0050] The above-described and other features of the invention will now be explained with reference to the accompanying drawings. The illustrated embodiments are intended to be illustrative and not to limit the scope of the invention.

[0051] The invention is further described below with reference to the embodiments illustrated in the accompanying drawings, wherein: Figure 1 This is a block diagram of an industrial environment capable of automatically generating plug-in programs for engineering systems according to an embodiment of the present invention; Figure 2 This is a block diagram of an engineering system in which embodiments of the present invention can be implemented, such as... Figure 1 Those shown; Figure 3 This is a block diagram of an automation module that can implement embodiments of the present invention, such as... Figure 2 Those shown; Figure 4 This is a flowchart illustrating an exemplary method for automatically generating a plug-in program according to an embodiment of the present invention; Figure 5 This is an exemplary illustration of the operation of an automation module configured for the automatic generation of plug-in programs according to an embodiment of the present invention; and Figure 6 This is an exemplary illustration of the operation of a user action optimization process according to an embodiment of the present invention; Various embodiments are described with reference to the accompanying drawings, wherein the same reference numerals are used to refer to the same elements throughout the drawings. In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding of one or more embodiments. It may be apparent that such embodiments can be practiced without these specific details.

[0052] Figure 1 This is a block diagram of an industrial environment 100 capable of automatically generating plug-in programs for an engineering system 102 according to an embodiment of the present invention. Figure 1 In this context, the industrial environment 100 includes an engineering system 102 and multiple human-machine interfaces 120A-N. As used herein, an "industrial environment" refers to an environment that includes configurable physical and logical computing resources, such as networks, servers, storage devices, applications, services, etc., and a data processing environment distributed across platforms such as cloud computing platforms. The industrial environment 100 provides on-demand network access to a shared pool of configurable physical and logical computing resources. The engineering system 102 is communicatively connected via a network connection 104, such as a local area network (LAN), wide area network (WAN), Wi-Fi, the Internet, or any short-range or wide-area communication. The engineering system 102 is also connected to the multiple human-machine interfaces 120A-N via the network connection 104.

[0053] As mentioned herein, engineering system 102 comprises multiple components and processes. These components and processes are interconnected and designed to achieve specific technical objectives. In one example, engineering system 102 is configured to achieve technical objectives across multiple disciplines such as mechanical, electrical, software, and power generation. For instance, engineering system 102 can be deployed in various fields, including but not limited to manufacturing, telecommunications, transportation, and energy. As used herein, the term "engineering system" is intended to encompass a wide range of technical architectures and configurations, from discrete hardware components to complex software-driven platform variations, all designed to perform specific functions or tasks within a broad context of engineering applications.

[0054] Applications within engineering system 102 refer to software components specifically designed to perform multiple predefined tasks or functions, enabling engineering system 102 to achieve specific technical objectives. In one example, the application is implemented as a standalone software application. In other examples, the application includes integrated modules within a larger software framework. Examples of applications include, but are not limited to, control algorithms, simulation tools, data processing applications, and user interfaces. As described herein, applications within engineering system 102 embody innovative and customized software solutions crucial for seamless integration and optimal performance of the engineering system.

[0055] Engineering system 102 includes multiple engineering objects. An "engineering object" refers to a digital entity that plays a significant role in the representation and functionality of engineering system 102. These engineering objects are represented as graphical and software objects within applications and engineering system 102, typically appearing as icons or templates within engineering applications or graphical user interfaces. In another example, the multiple engineering objects include project files and project items stored within engineering system 102.

[0056] Each engineered object is assigned a distinct identity by a specified name and associated with a set of properties that define the characteristics and behaviors of engineered objects within the engineered system 102. This set of properties encapsulates the necessary information, allowing users to effectively understand and interact with engineered objects. Users are authorized to interact with multiple engineered objects through a Human-Machine Interface (HMI) or other input devices such as multiple HMIs 120A-N.

[0057] Engineering system 102 also includes a knowledge graph containing information associated with each of a plurality of engineering objects. The knowledge graph comprises multiple nodes. Each node of the knowledge graph includes information associated with a specific engineering object among the plurality of engineering objects. In one example, each node of the knowledge graph includes information associated with an engineering object type among a plurality of types of engineering objects. The knowledge graph is stored internally within a database within the engineering system. The knowledge graph seamlessly orchestrates a dynamic network of interconnected nodes, each node representing a unique engineering object. Advantageously, the knowledge graph facilitates a comprehensive understanding of the complexities of engineering system 102. Furthermore, the knowledge graph includes information associated with one or more interrelationships between the plurality of engineering objects. The knowledge graph can be manually generated and fed into engineering system 102 by one or more users. In another example, the knowledge graph can be received from a database or an external server.

[0058] In one example, the knowledge graph is implemented as a graph database such as Neo4j. In another example, the knowledge graph is generated based on a data model such as Apache Jena. Engineering system 102 also includes one or more application programming interfaces (APIs) that facilitate real-time updates within multiple engineering objects of engineering system 102. By incorporating these implementation strategies, the knowledge graph not only becomes an information repository but also a dynamic, user-responsive framework that empowers multiple users to navigate, understand, and evolve engineering system 102 with unparalleled efficiency.

[0059] Engineering system 102 also includes an application toolbar 106, which includes multiple plug-ins 108A-N. The application toolbar 106 is an interactive graphical user interface that acts as a central hub enabling multiple users to seamlessly access and manage the various functionalities of engineering system 102. The application toolbar 106 includes multiple graphical icons and hyperlinks, which, when activated by a user, cause engineering system 102 to invoke at least one of the multiple plug-ins 108A-N. Furthermore, the application toolbar 106 is configured to accept keyboard shortcuts and gesture-based interactions, thus granting users alternative methods to access one or more functionalities of engineering system 102. This approach caters to diverse user groups and facilitates a more streamlined workflow. In one example, the application toolbar 106 is a programmable object programmed using web technologies such as HTML5, CSS3, and JavaScript frameworks.

[0060] As mentioned herein, each of the multiple plug-in programs refers to a software component designed to perform a series of predefined functions using a single command, thereby eliminating the need for multiple users to manually input command sequences. Each plug-in program is a supplementary program added to one of the application or engineering systems 102. The plug-in program adds custom commands and features to one of the application or engineering systems 102. For example, each plug-in program is activated by inputting a single command, and furthermore, the plug-in program causes one of the application or engineering systems 102 to execute a sequence of functions. Where the plug-in program is not present, the user may have to input a sequence of commands to cause the application or engineering system 102 to execute the sequence of functions. Therefore, advantageously, the user is spared the ordeal of having to input a sequence of commands to cause the application or engineering system 102 to execute the sequence of functions.

[0061] Although, Figure 1 The illustration shows an engineering system 102 connected to an industrial plant 106, but those skilled in the art will appreciate that the engineering system 102 can be connected to several industrial plants located in different geographical locations via a network connection 104.

[0062] The multiple human-machine interfaces 120A-N can be desktop computers, laptop computers, tablet computers, smartphones, and the like. Each of the multiple human-machine interfaces 120A-N is equipped with engineering tools 122A-N for controlling the engineering system 102. The multiple human-machine interfaces 120A-N can access the engineering system 102 to automatically generate engineering programs. The multiple human-machine interfaces 120A-N can access cloud applications. Throughout this specification, the terms "human-machine interface," "client device," and "user device" are used interchangeably. The multiple human-machine interfaces 120A-N are also configured to receive multiple user actions from multiple users. The multiple user actions include user input, user commands, user gestures, programming instructions, and user passwords.

[0063] In one example, engineering system 102 can be implemented as a standalone server deployed at a control station, or it can be a remote server on a cloud computing platform. In a preferred embodiment, engineering system 102 can be a cloud-based engineering system. Engineering system 102 may include a digitization platform 110 (such as a cloud computing platform), an automation module 112, a server 114 including hardware resources and an operating system (OS), a network interface 116, and a database 118. Network interface 116 enables communication between engineering system 102, industrial plant 106, and multiple human-machine interfaces 120A-N.

[0064] Server 114 may include one or more servers on which an OS is installed. Server 114 may include one or more processors, one or more storage devices such as memory units for storing, for example, data and machine-readable instructions, applications and application programming interfaces (APIs), and other peripheral devices required to provide computing (e.g., cloud computing) functionality. In one example, digitization platform 110 may be implemented in server 114. Digitization platform 110 uses the hardware resources of server 114 and the OS to enable functionality such as data reception, data processing, data rendering, data communication, etc., and delivers the aforementioned services using the application programming interfaces deployed therein. Digitization platform 110 may include a combination of dedicated hardware and software built on top of the hardware and OS. In an exemplary embodiment, digitization platform 110 may correspond to an integrated development environment (IDE) including a program editor and compiler that allow users of multiple human-machine interfaces 120A-N to generate engineering programs. Digitization platform 110 may also include an automation module 112 configured to automatically generate at least one plug-in program. Figure 3 The details of the automation module 112 are explained in the text.

[0065] Database 118 stores information related to industrial plant 106, multiple plug-in programs 108A-N, and multiple human-machine interfaces 120A-N. For example, database 118 may be a Structured Query Language (SQL) data store or a NoSQL data store. In an exemplary embodiment, database 118 may be configured as a cloud-based database implemented in industrial environment 100, where computing resources are delivered as a service on digital platform 110. According to another embodiment of the invention, database 118 is a location on a file system that can be directly accessed by automation module 112.

[0066] Figure 2 This is a block diagram of an engineering system 102 in which embodiments of the present invention can be implemented, such as... Figure 1 Those shown. In Figure 2 In the process, the engineering system 102 includes a processing unit 202, an accessible memory 204, a storage unit 206, a communication interface 208, an input / output unit 210, a network interface 212, and a bus 214.

[0067] As used herein, processing unit 202 means any type of computing circuitry, such as, but not limited to, a microprocessor unit, a microcontroller, a complex instruction set computing microprocessor unit, a reduced instruction set computing microprocessor unit, a very long instruction word microprocessor unit, an explicit parallel instruction computing microprocessor unit, a graphics processing unit, a digital signal processing unit, a neural processing unit (NPU), or any other type of processing circuitry. Processing unit 202 may also include embedded controllers, such as general-purpose or programmable logic devices or arrays, application-specific integrated circuits (ASICs), single-chip computers, and the like.

[0068] Memory 204 may be a non-transitory volatile memory and a non-volatile memory. Memory 204 may be coupled to communicate with processing unit 202, such as a computer-readable storage medium. Processing unit 202 may execute machine-readable instructions and / or source code stored in memory 204. Various machine-readable instructions may be stored in and accessed from memory 204. Memory 204 may include any suitable element for storing data and machine-readable instructions, such as read-only memory, random access memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, hard disk drive, removable media drive for disposing of optical discs, digital video discs, floppy disks, magnetic tape cassettes, memory cards, and the like. In this embodiment, memory 204 includes an integrated development environment (IDE) 216. IDE 216 includes an automation module 112, which is stored in the form of machine-readable instructions on any of the aforementioned storage media and can communicate with and be executed by processor(s) 202(s).

[0069] In one example, as mentioned herein, engineering system 102 includes multiple components and processes. These components and processes are interconnected and designed to achieve specific technical objectives. In one example, engineering system 102 is configured to achieve technical objectives in multiple professional fields such as mechanical, electrical, software, and power generation. For example, engineering system 102 can be deployed in various fields, including but not limited to manufacturing, telecommunications, transportation, and energy. As used herein, the term "engineering system" is intended to encompass a wide range of technical architectures and configurations, from discrete hardware components to complex software-driven platform variations, all designed to perform specific functions or tasks within the broader context of engineering applications.

[0070] One of the engineering systems 102 or applications is configured to continuously receive multiple user inputs from an operator or user of the engineering system 102. The multiple user inputs include one or more commands, one or more text inputs, or one or more voice inputs. The multiple user inputs also include a series of interactions, including but not limited to command input via a graphical user interface, parameter adjustment, initiation of a simulation scenario, or any other instructions provided by the operator or user. The engineering system 102 receives the multiple user inputs via an input device such as multiple human-machine interfaces 120A-N. The multiple user inputs, hereinafter referred to as multiple user actions, constitute a command initiated by the operator or user to instruct one of the applications or engineering systems 102.

[0071] Each of the multiple user actions corresponds to a specific command. Upon receiving each of the multiple user actions, one of the engineering system 102 or applications is triggered to execute a predefined function within the engineering system 102. This allows the operator or user to configure, direct, or modify the behavior of the application or engineering system by using the multiple user actions, thereby influencing the overall execution and output of the engineering system. Within the scope of this patent, the term "user action" refers to the dynamic and interactive nature of the human-machine interface associated with the engineering system. This allows the operator or user to exert control and influence over the application to customize one or more operations of the application according to the requirements of the evolving engineering process.

[0072] In one example, an identifier is used in engineering system 102 to identify each of multiple user actions. The identifier for a user action is a data object that includes the name of the user action, the purpose of the user action, and one or more code segments associated with the user action. The one or more code segments associated with the user action include one or more code statements that are executed by engineering system 102 or one of the applications when the engineering system receives the user action. In one example, the one or more code segments include portions of the source code of one of the applications or engineering systems 102. In another example, the one or more code segments include one or more function calls or procedure calls to invoke one or more functions that are scripted into the source code of the application. In this case, whenever engineering system 102 receives a user action, engineering system 102 invokes one or more functions.

[0073] The identifier for a user action also includes a natural language description of one or more functions performed by engineering system 102 when the user action is received in engineering system 102. In other words, the identifier is a crucial data object designed to store information directly linked to one or more user actions received by engineering system 102. Furthermore, each identifier is structured to encapsulate further details associated with a specific user action, thus forming a chronological sequence mirroring the receipt of one or more user actions in engineering system 102 or one of the applications. Therefore, the string of identifiers considers a uniform representation of user activities in engineering system 102 or one of the applications. Examples of identifiers include alphanumeric code-based identifiers, timestamp-based identifiers, and security code-based identifiers. In one example, an identifier for each user action is assigned by engineering system 102 simultaneously with the receipt of the user action at engineering system 102. In another example, the identifier for each user action is scripted by the coders who have already coded the application. The identifier for the user action can be stored in engineering system 102 and referenced by engineering system 102 when it receives a user action.

[0074] Each of the multiple user actions refers to one or more project objects among multiple project objects in the engineering system 102. In one example, the multiple user actions include, but are not limited to, assigning a user action, renaming a user action, reconfiguring a user action, creating a new user action, and deleting a user action. In this case, assigning a user action and renaming a user action cause the engineering system 102 to assign or modify the name of a project object. Reconfiguring a user action causes the engineering system 102 to adjust one or more configuration settings of a project object. Creating a user action causes the engineering system 102 to create a new project object. Deleting a user action allows users to selectively remove project objects that are no longer relevant or necessary, thus simplifying the engineering system and enhancing its efficiency.

[0075] When executed by processing unit 202, automation module 112 causes processing unit 202 to determine the engineered object to which at least one of a plurality of user actions points. In other words, automation module 112 is configured to determine the engineered object associated with at least one user action. Automation module 112 also causes processing unit 202 to determine at least one node of the knowledge graph. The at least one node includes information associated with the determined engineered object. In one example, automation module 112 is configured to analyze a plurality of nodes of the knowledge graph by applying a natural language processing algorithm to determine at least one node of the plurality of nodes of the knowledge graph. It should be noted that each of the plurality of nodes is associated with a specific engineered object and includes the identity of the specific engineered object. Automation module 112 also causes processing unit 202 to update at least one node. To update at least one node, automation module 112 is configured to insert information associated with at least one user action into at least one node. Therefore, the knowledge graph includes information associated with the relationships between each of the plurality of user actions and each of the plurality of engineered objects. In addition, each node of the knowledge graph includes information associated with a list of project objects and user actions, which are user actions performed on or directed to project objects.

[0076] Automation module 112 also causes processing unit 202 to monitor the order or sequence of each user action received at engineering system 102. Automation module 112 is also configured to update the knowledge graph by inserting one or more links between multiple nodes. The inserted one or more links indicate the sequence of multiple user actions received by automation module 112. In one example, automation module 112 receives a first set of user actions pointing to a first engineering object, and then receives a second set of user actions pointing to a second engineering object. In this case, automation module 112 generates a first node associated with the first engineering object and a second node associated with the second engineering object. Automation module 112 also inserts information associated with the first set of user actions into the first node. Automation module 112 also inserts information associated with the second set of user actions into the second node. Furthermore, since the second set of user actions is received before the first set of user actions, automation module 112 generates a link between the first node and the second node. Therefore, the knowledge graph includes information associated with multiple engineering objects, multiple user actions pointing to multiple engineering objects, and the sequence of multiple user actions received at engineering system 102.

[0077] In other words, the knowledge graph comprises multiple links. At least one of these links connects a first node to a second node in the knowledge graph. The first node includes information associated with user actions received from one or more users of the engineering system. The second node includes information associated with at least one subsequent user action received from one or more users after the initial user action.

[0078] Automation module 112 also causes processing unit 202 to receive a first user action from a user of engineering system 102. The first user action refers to at least one of a plurality of engineering objects in engineering system 102. In one example, the first user action includes the user clicking a graphical icon associated with at least one engineering object. In another example, the first user action includes the user typing a text-based command, wherein the text-based command includes the name of at least one engineering object. In other words, the first user action includes one or more parts that include information indicating at least one engineering object.

[0079] The automation module 112 also causes the processing unit 202 to predict at least a second user action that can be received from the user after the first user action. The processing unit 202 is configured to predict at least one second user action by implementing a user action detection process based on the knowledge graph and the first user action.

[0080] In one example, as mentioned in this paper, the user action detection process is a software-based process configured to anticipate and predict upcoming user actions within the engineered system 102. The user action detection process operates by carefully analyzing the flow of multiple user actions and a knowledge graph encapsulating historical user interactions. By leveraging advanced algorithms and machine learning techniques, the user action detection process identifies patterns and correlations between multiple user actions, thus enabling it to make informed predictions about possible next user actions. Therefore, the predictive capabilities of the user action detection process facilitate proactive system responses and customized user experiences, thereby enhancing overall efficiency and user satisfaction. The user action detection process plays a crucial role in simplifying user interactions and ensuring a seamless, anticipated user interface.

[0081] To implement the user action detection process, the automation module 112 causes the processing unit 202 to execute a first plurality of steps. In the first step of the first plurality of steps, the automation module 112 is configured to cause the processing unit 202 to analyze a knowledge graph based on the received first user action. In the second step of the first plurality of steps, the automation module 112 causes the processing unit 202 to determine at least one node among a plurality of nodes in the knowledge graph. The at least one node includes information associated with at least one engineering object.

[0082] In one example, the first user action includes information associated with the identity of at least one project object. In this case, the automation module 112 is configured to traverse the knowledge graph while searching each node of the knowledge graph for information associated with the at least one project object. The automation module 112 is configured to traverse the knowledge graph until the processing unit 202 determines one or more nodes that include information about the at least one project object. The automation module 112 is configured to traverse the knowledge graph using any of the known graph traversal techniques.

[0083] In the third step of the first plurality of steps, the automation module 112 is further configured to analyze one or more nodes to identify at least one node that includes information associated with a previously received user action that matches the first user action. The previously received user action was received by the engineering system 102 before the first user action was received. In one example, the automation module 112 is configured to extract the name of the first user action from an identifier associated with the first user action. Furthermore, the automation module 112 is configured to analyze one or more nodes by searching for the extracted name of the first user action within one or more nodes. The identified at least one node includes information associated with a previously received user action. The previously received user action has a name that matches the extracted name of the first user action.

[0084] In the fourth step of the first plurality of steps, the automation module 112 is further configured to analyze one or more links associated with at least one node. The one or more links connect at least one of the plurality of nodes to a set of nodes. The set of nodes includes information associated with a set of previously received user actions received by the engineering system after receiving previously received user actions. The set of nodes also includes information associated with one or more engineering objects pointed to by one or more users by the set of previously received user actions. Therefore, the automation module 112 is configured to determine the set of previously received user actions received by the engineering system 102 after receiving previously received user actions. Since the previously received user actions match a first user action, it is possible to predict one or more user actions that the user might type that match the set of previously received user actions.

[0085] In the fifth step of the first plurality of steps, the automation module 112 is configured to analyze one or more nodes to determine whether the one or more nodes include information associated with at least one engineering object or one or more engineering objects related to at least one engineering object. In the sixth step of the first plurality of steps, the automation module 112 is configured to determine at least one second user action from the one or more nodes based on determining that the one or more nodes include information associated with at least one engineering object or one or more engineering objects related to at least one engineering object. Advantageously, at least one second user action is predicted based on a knowledge graph, which includes contextual information associated with multiple user actions and multiple engineering objects. Therefore, at least one second user action is accurately predicted based on the interrelationships between multiple user actions and multiple engineering objects. Thus, the need for analyzing the interrelationships between multiple user actions and multiple engineering objects and thereby participating in the time-consuming task of manually coding the plug-in program is eliminated. Furthermore, at least one second user action is predicted based on the interrelationships between at least one engineering object and other engineering objects of the engineering system 102. Thus, the predicted at least one second user action is aligned with the interrelationships between at least one engineering object and other engineering objects of the engineering system 102.

[0086] In one example, automation module 112 is configured to determine a first node and a second node from one or more nodes. The first and second nodes include information associated with at least one project object or one or more project objects related to at least one project object. In this case, automation module 112 is configured to determine at least one second user action from the first and second nodes. For example, automation module 112 may determine all user actions mentioned in the first and second nodes as part of at least one second user action.

[0087] Automation module 112 further causes processing unit 202 to generate a sequence of user actions by implementing a sequence identification algorithm with respect to a first user action and at least one second user action. The sequence identification algorithm can be a machine learning algorithm, such as the KNN classification algorithm. The sequence identification algorithm is configured to cluster the first user action and at least one second user action into a sequence of user actions. In one example, the sequence identification algorithm includes a neural network, such as a deep neural network, trained to cluster the first user action and at least one second user action into a sequence of user actions. The neural network is trained by applying a supervised training algorithm to a labeled dataset. The labeled dataset includes a list of user actions applied to multiple engineering objects, and multiple sequences of user actions associated with the list of user actions. Each of the multiple sequences of user actions is associated with a specific user action in the list of user actions. Therefore, the neural network is trained to generate a sequence of user actions from the first user action and at least one second user action.

[0088] In another example, the sequence identification algorithm is implemented as computer code, which, when executed by the processing unit, causes the automation module 112 to analyze a first user action and at least one second user action. The computer code is also configured to generate a sequence of user actions. To implement the sequence identification algorithm, the automation module 112 is configured to compare each of the first user action and at least one second user action with a list of user-defined user actions. User-defined user actions are user actions defined by a user or operator. The automation module 112 is also configured to use the identified user actions as trigger points to determine the sequence of user actions. For example, the automation module 112 is configured to identify a first type of user action as an initiation point and a second type of user action as an end point. The automation module 112 is also configured to effectively identify the defined range between the initiation point and the end point as a sequence of user actions. In other words, the automation module 112 is configured to classify one or more user actions received between the identified initiation point and the identified end point as part of a sequence of user actions. The user can generate and modify the list of user-defined user actions according to one or more user preferences. Therefore, advantageously, the sequence of user actions generated by the automation module can be highly customized by the user.

[0089] For example, the first user action could be a user action such as "user selects a project object". The first user action is identified as the initiation point. At least one second user action could be a user action such as "user deselects a project object". At least one second user action is identified as the end point. Therefore, one or more user actions performed by the operator or user between selecting and deselecting a project object are determined as a sequence of user actions.

[0090] Automation module 112 is also configured to generate a workflow template based on a sequence of generated user actions. The workflow template includes a sequence of user actions and one or more parameters associated with each user action in the sequence. The one or more parameters associated with each user action include numerical variables and text data typed by the user via the user action. For example, one or more parameters include information about the intensity, magnitude, or level and timing associated with the user action. The workflow template is stored as a data object within engineering system 102. In one example, the workflow template indicates a specific path within a knowledge graph. The specific path connects a group of nodes within the knowledge graph, where the group of nodes includes information associated with user actions in the set of generated user actions. Where engineering system 102 receives a stream of user actions, automation module 112 is configured to compare each user action in the stream of user actions with each user action in the workflow template.

[0091] Automation module 112 is also configured to determine if a specific user action in the flow of user actions matches at least one user action in the workflow template. Where a specific user action matches at least one user action, automation module 112 is configured to record one or more subsequently received user actions as strings representing user actions. Therefore, automation module 112 is also configured to determine multiple strings representing user actions.

[0092] Automation module 112 is also configured to compare each of the multiple user actions with a generated sequence of user actions. Automation module 112 also counts the number of times that the majority of user actions for each of the multiple strings match the user actions of the workflow template. In other words, automation module 112 counts the number of times it traverses a specific path associated with the workflow template.

[0093] The automation module 112 is also configured to compare the counted number of times with a threshold number. If the count exceeds the threshold number, the automation module 112 is configured to implement a plugin generation process for the generated sequence of user actions. If the count does not exceed the threshold number, the automation module 112 then prevents the plugin from being generated. Advantageously, the plugin is created only when the user repeatedly uses the sequence of user actions.

[0094] Automation module 112 is further configured to optimize the workflow template and the generated sequence of user actions by applying a user action optimization algorithm to the workflow template and multiple user actions. In one example, the user action optimization algorithm is a machine learning algorithm configured to identify common user actions within each of a plurality of strings representing user actions. Automation module 112 is also configured to optimize the workflow template based on the identified common user actions. In one example, automation module 112 is configured to optimize the workflow template and the generated sequence of user actions by applying any of the known term frequency-inverse document frequency algorithms to the plurality of strings representing user actions. Automation module 112 is also configured to optimize the generated sequence of user actions by applying a K-means classification algorithm to the plurality of strings representing user actions. In one example, automation module 112 is configured to optimize the workflow template by pruning all user actions from the generated sequence of user actions except for the identified common user actions. Advantageously, the generated sequence of user actions includes only the common user actions identified from the plurality of strings representing user actions.

[0095] refer to Figure 5 The diagram illustrates the operation of the automation module 112. The automation module 112 receives user actions 504 from user 502. The automation module 112 also receives multiple user actions from one or more users 508. The automation module 112 generates a workflow template 510 based on the knowledge graph 506 and the user actions 504. The automation module 112 then applies a user action optimization process 512 to the generated workflow template 510.

[0096] refer to Figure 6 The user action optimization process 512 uses multiple strings of user actions 602 to generate an optimized workflow template 604.

[0097] Reference Back Figure 2 The automation module 112 further causes the processing unit 202 to generate a plugin program 108N by implementing a plugin generation process based on a sequence of generated user actions. To implement the plugin generation process, the processing unit 202 is configured to execute a second set of steps.

[0098] In the first step of the second plurality of steps, processing unit 202 is configured to receive one or more user actions from the user at a specific time interval, which occurs after receiving the first user action. In the second step of the second plurality of steps, processing unit 202 compares each of the received one or more user actions with a specific user action in a sequence of generated user actions. In the third step of the second plurality of steps, processing unit 202 is configured to determine a count of the number of times the received one or more user actions match the sequence of generated user actions. In the fourth step of the second plurality of steps, processing unit 202 determines that the determined count is greater than a threshold number. If the determined count is greater than the threshold number, processing unit 202 extracts a string of identifiers from the sequence of generated user actions. The string of identifiers is a data object including information associated with each user action in the sequence of generated user actions. In the fifth step of the second plurality of steps, processing unit 202 extracts one or more code segments from the extracted string of identifiers. In the sixth step of the second plurality of steps, processing unit 202 generates plug-in program 108A by performing a string concatenation process with respect to one or more code segments. The string concatenation process can be any known algorithm used to concatenate strings. In one example, one or more code segments are concatenated according to the order of user actions in a sequence of user actions.

[0099] Advantageously, the automation module automatically identifies sequences of user actions frequently used by the user or operator. Furthermore, these sequences of frequently used user actions are used to generate plug-in programs. Therefore, the generated plug-in programs are configured to execute sequences of frequently used and useful user actions.

[0100] In the seventh step of the second series of steps, processing unit 202 is configured to extract a natural language description of each user action in the generated sequence of user actions from the string of the identifier. In the eighth step, processing unit 202 is configured to generate a natural language description for the generated plug-in by performing a string concatenation process using the extracted natural language descriptions of each user action in the generated sequence of user actions. Advantageously, plug-in 108A is generated automatically without manual intervention. Furthermore, plug-in 108A is generated based on one or more sequences specifically determined for the user. Additionally, the generated plug-in includes a natural language description of the technical goals or objectives of the generated plug-in. Therefore, this enables the user to make an informed decision about whether to use the generated plug-in.

[0101] The automation module 112 also causes the processing unit 202 to display information associated with the generated plug-in program to the user or operator. The displayed information associated with at least one generated plug-in program includes a natural language description of the plug-in program. The displayed information also includes a natural language description of each user action in the sequence of generated user actions. The displayed information also includes information associated with one or more code statements extracted from a string of identifiers. The displayed information also includes one or more user commands that can be used to invoke the at least one generated plug-in program. In one example, the one or more user commands may be text-based commands, voice-based commands, gesture-based commands, or mouse-click-based commands. Advantageously, the automation module learns from continuously received user actions from the user and provides suggestions to the user to optimize the user's efforts.

[0102] The automation module 112 further causes the processing unit 202 to compare the generated sequence of user actions with a set of sequences of user actions. The set of sequences of user actions includes user actions stored for reference in a database. Each sequence in the set of sequences of user actions is linked to a set of plug-ins stored in the database. Users or operators can assign multiple mappings between the set of plug-ins and each sequence in the set of sequences of user actions. The automation module 112 further causes the processing unit 202 to determine whether a sequence in the set of sequences matches a sequence of generated user actions. If at least one sequence in the set of sequences matches a sequence of generated user actions, the processing unit 202 is configured to determine a plug-in linked to the sequence of generated user actions from the set of plug-ins. The automation module 112 is also configured to cause the processing unit 202 to display the determined plug-in to the user. If the sequence does not match one or more determined sequences of user actions, the processing unit 202 is configured to deploy at least one generated plug-in 108A to the application toolbar 106 of the engineering system 102.

[0103] Communication interface 208 is configured to establish communication sessions between multiple human-machine interfaces 120A-N, engineering system 102, spraying equipment 24 and imaging equipment 126.

[0104] Input / output unit 210 may include an input device capable of receiving one or more input signals, such as user commands for processing project files, a keypad, a touch-sensitive display, a camera (such as a camera that receives gesture-based input), etc. Furthermore, input / output unit 210 may be a display unit for displaying a graphical user interface (GUI), which visualizes a behavioral model associated with the modified project program and also displays status information associated with each set of actions performed on the GUI. The set of actions may include performing predefined tests, downloading, compiling, and deploying the graphical program. Bus 214 serves as the interconnect between processor 202, memory 204, and input / output unit 210.

[0105] Network interface 212 can be configured to handle network connectivity, bandwidth and network traffic between engineering system 102, multiple human-machine interfaces 120A-N and industrial plant 106.

[0106] Those skilled in the art will understand that Figure 2 The hardware depicted may vary for a particular implementation. For example, other peripheral devices, such as optical disc drives and such, local area network (LAN), wide area network (WAN), wireless (e.g., Wi-Fi) adapters, graphics adapters, disk controllers, and input / output (I / O) adapters may be used in addition to or in lieu of the hardware depicted. The examples depicted are provided for illustrative purposes only and are not intended to imply any architectural limitations with respect to this disclosure.

[0107] Those skilled in the art will recognize that, for simplicity and clarity, the complete structure and operation of all data processing systems suitable for use with this disclosure are not depicted or described herein. Instead, only the parts of the engineering system 102 that are unique to or necessary for understanding this disclosure are depicted and described. The remaining construction and operation of the engineering system 102 may conform to any of the various current implementations and practices known in the art.

[0108] Figure 3 This is a block diagram of the automation module 112 that can implement embodiments of the present invention. The automation module 112 is such as... Figure 2 Those shown. In Figure 3 In the process, the automation module 112 includes a request handler module 302, a knowledge graph module 304, an analysis module 306, a modifier module 308, an engineering object database 310, a verification module 312, and a deployment module 314. Combined with... Figure 1 and Figure 2 explain Figure 3 .

[0109] The request handler module 302 is configured to receive multiple user actions from a user or operator. For example, multiple user actions may be received via a network from one or more users outside the industrial environment 100. In an alternative embodiment, multiple user actions may be received via a network from one or more human-machine interfaces 120A-N.

[0110] The knowledge graph module 304 is configured to update the knowledge graph based on the receipt of multiple user actions.

[0111] Analysis module 306 is configured to apply a sequence identification process to one or more user actions to generate a sequence of user actions.

[0112] Modifier module 308 is configured to modify the generated sequence of user actions by applying a user action optimization algorithm to the generated sequence of user actions.

[0113] The project object database 310 is configured to generate a project object library that includes multiple plug-in programs 108A-N.

[0114] The verification module 312 is configured to verify at least one generated plug-in program.

[0115] Deployment module 314 is configured to deploy the generated plug-in program in the project system.

[0116] Figure 4 This is a flowchart illustrating an exemplary method 400 for automatically generating plug-in programs in an engineering system 102 according to an embodiment of the present invention. (In conjunction with...) Figures 1 to 3 explain Figure 4 .

[0117] At step 402, a first user action is received from the user by processing unit 202, the first user action being associated with at least one of a plurality of project objects in the project system. At step 404, processing unit 202 performs a user action detection process to predict at least a second user action using a knowledge graph about the first user action and its association with the project system. At step 406, a sequence of user actions is generated by performing a sequence identification process based on the received first user action and the predicted at least one second user action. At step 408, a plug-in is generated by performing a plug-in generation process based on the generated sequence of user actions. At step 410, processing unit 202 deploys the generated at least one plug-in in the project system.

[0118] This invention can take the form of a computer program product comprising program modules accessible from a computer-usable or computer-readable medium storing program code for use by or in connection with one or more computers, processors, or instruction execution systems. For the purposes of this specification, a computer-usable or computer-readable medium can be any device that can contain, store, transmit, propagate, or deliver a program for use by or in connection with an instruction execution system, device, or apparatus. The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or device or apparatus), or a propagation medium in itself, since signal carriers are not included in the definition of a physical computer-readable medium, which includes semiconductor or solid-state memory, magnetic tape, removable computer floppy disks, random access memory (RAM), read-only memory (ROM), rigid disks, and optical disks, such as optical disc read-only memory (CD-ROM), optical disc read / write, and DVD. As is known to those skilled in the art, both the processor and the program code used to implement each aspect of this technology can be centralized or distributed (or a combination thereof).

[0119] While the invention has been described in detail with reference to certain embodiments, it should be understood that the invention is not limited to those embodiments. As described herein, in view of this disclosure, many modifications and variations will appear to those skilled in the art without departing from the scope of the various embodiments of the invention. Therefore, the scope of the invention is indicated by the following claims rather than by the foregoing description. All changes, modifications, and variations falling within the meaning and scope of equivalents of the claims will be considered to be within its scope. All advantageous embodiments claimed in the method claims are also applicable to the system / device claims.

Claims

1. A computer-implemented method for automatically generating plug-in programs for an engineering system, the method comprising: The processing unit (202) receives a first user action from the user, wherein the first user action is associated with at least one of a plurality of engineering objects in the engineering system, and wherein the first user action is performed by the user using an input device. The processing unit (202) predicts at least a second user action by implementing a user action detection process through a knowledge graph about the first user action and associated with the engineering system, wherein the user action detection process includes: The knowledge graph is analyzed to determine one or more nodes that link to previously received user actions. Predict at least the second user action based on one or more identified nodes; The processing unit (202) generates a sequence of user actions by performing a sequence identification process based on the received first user action and at least one predicted second user action, wherein the sequence identification process includes applying a machine learning algorithm to: Analyze the first user action and the second user action, and Based on the analysis of the first user action and at least the second user action, the sequence of actions performed by the user is identified; The processing unit (202) generates a plugin program by implementing a plugin generation process based on a sequence of generated user actions, wherein the plugin generation process includes: Extract one or more code segments from the first user action and at least the second user action, and Concatenate the one or more code segments to generate the plug-in program; and At least one plug-in program generated by the processing unit (202) is deployed in the engineering system.

2. The method of claim 1, wherein the generated sequence of user actions is encapsulated in a workflow template.

3. The method according to any one of the preceding claims, wherein generating the plug-in program further comprises: The generated sequence of user actions and the workflow template are optimized by implementing a user action optimization algorithm based on the workflow template and the multiple user actions. as well as The plugin program is generated by implementing the plugin generation process using an optimized sequence of user actions.

4. The method according to any one of the preceding claims, a. The knowledge graph mentioned therein includes multiple nodes. b. Each of the plurality of nodes includes information associated with at least one of the plurality of engineering objects in the engineering system, and c. The information associated with at least one of the plurality of engineering objects includes information associated with one or more user actions performed on the engineering object by one or more users.

5. The method according to any one of the preceding claims, a. The knowledge graph mentioned therein includes multiple links. b. Wherein at least one of the plurality of links connects the first node to the second node among the plurality of nodes. c. Wherein the first node includes information associated with user actions received from one or more users of the engineering system, and d. The second node includes information associated with at least one subsequent user action received from the one or more users after the user action is received.

6. The method according to any one of the preceding claims, wherein predicting the at least one second user action by implementing a user action detection process with respect to the knowledge graph comprises: The processing unit (202) determines at least one node of the knowledge graph by analyzing the knowledge graph and the first user action, wherein the at least one node includes information associated with at least one of the plurality of engineering objects; The processing unit (202) analyzes one or more links associated with the at least one node to determine one or more nodes linked to the at least one node, wherein the one or more nodes include information associated with a set of user actions; as well as The processing unit (202) predicts the at least one second user action from the set of user actions by analyzing the one or more nodes.

7. The method according to any one of the preceding claims, wherein generating the plug-in program by means of the processing unit (202) through the plug-in generation process with respect to the determined sequence of user actions comprises: The processing unit (202) receives one or more user actions from the user within a specific time interval; The processing unit (202) compares the received one or more user actions with at least one user action in the sequence of generated user actions; The processing unit (202) determines that at least one of the received one or more user actions matches at least one user action in the sequence of generated user actions; as well as The processing unit (202) determines that the number of times at least one of the received user actions matches at least one user action in the sequence of generated user actions is greater than a threshold number.

8. The method according to any one of the preceding claims, wherein generating the plug-in program further comprises: When it is determined that the count is greater than the threshold number, an identifier string is extracted from the sequence of generated user actions, wherein the identifier string is a data object that includes information associated with each user action in the sequence of generated user actions; The processing unit (202) extracts one or more code segments from the extracted identifier string; as well as The processing unit (202) generates the plugin program by performing a string concatenation process on one or more extracted code segments.

9. The method according to any one of the preceding claims further comprises: The processing unit (202) extracts a natural language description of each user action from the string of the extracted identifier; as well as The processing unit (202) generates the natural language description of the generated plug-in program by performing the string concatenation process through the extracted natural language description of each user action in the sequence of generated user actions.

10. The method according to any one of the preceding claims, wherein deploying at least one generated plug-in program in the engineering system comprises: The processing unit (202) compares the generated sequence of user actions with the set of sequences of user actions associated with the set of plug-in programs, wherein the set of plug-in programs is stored in a database. The processing unit (202) determines whether the generated sequence of user actions matches a sequence in the set of sequences of user actions. The processing unit (202) determines from the set of plug-ins the plug-ins associated with the sequence of generated user actions; When the set of generated user actions matches a sequence in the set of user action sequences, the determined plug-in program is displayed to the user through the processing unit (202); as well as When the plugin does not match at least one generated plugin, the at least one generated plugin is deployed to the engineering system by the processing unit (202).

11. An engineering system (102) for automatically generating plug-in programs, wherein the engineering system (102) comprises: Processing unit (202); as well as A memory (204) coupled to the processing unit (202), wherein the memory includes an automation module (112) stored in the form of machine-readable instructions executable by the one or more processors, wherein the automation module (112) is capable of performing the method according to any one of claims 1-10.

12. An industrial environment (100), comprising: The engineering system (102) according to claim 11; Industrial plant (106), comprising one or more physical components; as well as Multiple human-machine interfaces (120A-N) are communicatively coupled to the engineering system (102) via a network (104), wherein the engineering system (102) is specifically configured to perform the method according to any one of claims 1 to 10.

13. A computer program product having machine-readable instructions stored therein, which, when executed by a processing unit (202), cause the processor to perform the method according to any one of claims 1-10.

Images