Method and system for extracting compatibility rules from a black box configuration software

Abstract

In order to extract compatibility rules from black box configuration software, a candidate generator (CG) generates configuration candidates (CC), wherein each configuration candidate identifies components of an industrial automation system and their connections. An interface receives a validity label for each configuration candidate from a black-box configuration software (BBS). An observed configurations database (OC) stores each configuration candidate and its validity label. An inductive logic programming engine ( ILPE ) processes the content of the observed configurations database (OC) and outputs a rule base (RB) storing compatibility rules. This solution provides an alternative to -1- manual exploration of the search space, which would be infeasible due to the infinite number of possible configurations, -2- performing a random brute-force search, which would be inefficient, and -3- interviewing expert users of the black-box configuration software, which would be expensive and time-consuming. Embodiments provide a novel neural rule extraction system and include a candidate discriminator that can be seen as a soft validator, computing how likely each configuration is valid. In summary, a solution for recovering a large number of implicit rules from legacy software is provided, which is vital for maintaining knowledge with a retiring workforce.

Description

[0001] Description

[0002] Method and system for extracting compatibility rules from a black box configuration software

[0003] Independent of the grammatical term usage, individuals with male, female or other gender identities are included within the term.

[0004] Obtaining high-quality digital representations real-world systems, i.e., so-called "digital twins", is a major chal- lenge in industry. One of the main problems is integrating information from legacy software which contains a substantial amount of implicit domain knowledge. For example, device con- figuration software usually maintains a large rule base in- ternally against which the input is checked. In case of con- figuration software for building up factory automation solu- tions, simple examples of such rules would include verifying whether the maximum load of a controller is not exceeded, matching voltage requirements of interconnected components, and verifying whether a controller has enough compute capaci- ty to serve all connected digital outputs.

[0005] Oftentimes, however, such software is only available in com- piled (binary) form with no access to the actual source code or rule base. This leads to black-box configuration software that is still used by domain experts, but impossible, or at least very costly, to interpret or debug. In the following, the term "black-box configuration software" is to be under- stood in this way.

[0006] It is an object of the present invention to identify a prob- lem in the prior art and to find a technical solution for this.

[0007] The objectives of the invention are solved by the independent claims. Further advantageous arrangements and embodiments of the invention are set forth in the respective dependent claims.

[0008] According to the method for extracting compatibility rules from a black box configuration software, the following opera- tions are performed by modules, wherein the modules are hard- ware modules and / or software modules executed by one or more processors: generating, by a candidate generator, configuration can- didates, wherein each configuration candidate identifies components of an industrial automation system and their connections, receiving, from a black-box configuration software, for each configuration candidate a valid response or an in- valid response and creating a corresponding validity la- bel, storing, by an observed configurations database, each configuration candidate and its validity label, and outputting, by an inductive logic programming engine processing the content of the observed configurations database, a rule base storing compatibility rules.

[0009] The system for extracting compatibility rules from a black box configuration software comprises the following modules, wherein the modules are hardware modules and / or software mod- ules executed by one or more processors: a candidate generator, configured for generating config- uration candidates, wherein each configuration candidate identifies components of an industrial automation system and their connections, an interface, configured for receiving, from a black-box configuration software, for each configuration candidate a valid response or an invalid response and creating a corresponding validity label, an observed configurations database, configured for storing each configuration candidate and its validity label, and an inductive logic programming engine, configured for processing the content of the observed configurations database and outputting a rule base storing compatibil- ity rules.

[0010] The compatibility rules are interpretable logical rules de- fining what makes a configuration valid, for example by spec- ifying a reguired minimum compute capacity for a controller to serve all connected digital outputs.

[0011] The interface is for example a network interface and / or an application programming interface that is adapted to query the black-box configuration software.

[0012] The following advantages and explanations are not necessarily the result of the object of the independent claims. Rather, they may be advantages and explanations that only apply to certain embodiments or variants.

[0013] In connection with the invention, unless otherwise stated in the description, the terms "training", "generating", "comput- er-aided", "calculating", "determining", "reasoning", "re- training" and the like relate preferably to actions and / or processes and / or processing steps that change and / or generate data and / or convert the data into other data, the data in particular being or being able to be represented as physical quantities, for example as electrical impulses.

[0014] The term "computer" should be interpreted as broadly as pos- sible, in particular to cover all electronic devices with da- ta processing properties. Computers can thus, for example, be personal computers, servers, clients, programmable logic con- trollers (PLCs), handheld computer systems, pocket PC devic- es, mobile radio devices, smartphones, devices, or any other communication devices that can process data with computer support, processors, and other electronic devices for data processing. Computers can in particular comprise one or more processors and memory units. In connection with the invention, a "memory", "memory unit" or "memory module" and the like can mean, for example, a vol- atile memory in the form of random-access memory (RAM) or a permanent memory such as a hard disk or a Disk.

[0015] The method and system, or at least some of their embodiments, provide an alternative to manual exploration of the search space, which would be infeasible due to the infinite number of possible configurations.

[0016] The method and system, or at least some of their embodiments, provide an alternative to performing a random brute-force search, which would be inefficient.

[0017] The method and system, or at least some of their embodiments, provide an alternative to interviewing expert users of the black-box configuration software, which would be expensive and time-consuming.

[0018] The method and system, or at least some of their embodiments, provide a novel neural rule extraction system that allows to retrieve rules from black-box configuration software more ef- ficiently than brute-force.

[0019] The method and system, or at least some of their embodiments, include a candidate discriminator that can be seen as a soft validator, computing how likely the configuration is valid.

[0020] The method and system, or at least some of their embodiments, provide a solution for recovering a large number of implicit rules from legacy software, which is vital for maintaining knowledge with a retiring workforce. Black-box software is a major pain point in the digitalization of assets.

[0021] The method and system, or at least some of their embodiments, relieve users from having to manually create configurations or brute-force search spaces in order to extract knowledge from black-box configuration software.

[0022] In an embodiment of the method, the candidate generator re- ceives as one of its inputs a domain vocabulary.

[0023] In another embodiment of the method, the candidate generator receives as one of its inputs a reward that is calculated by a candidate discriminator. The reward in particular encour- ages that the configuration candidates generated by the can- didate generator are diverse and substantially different from the configuration candidates already stored in the observed configurations database.

[0024] In an embodiment of the method, the reward is high if the configuration candidate currently generated by the candidate generator leads to high uncertainty in the candidate discrim- inator. The reward is in particular calculated as the entropy of a predicted probability distribution of the candidate dis- criminator.

[0025] In another embodiment of the method, the candidate discrimi- nator has been trained using binary cross entropy to act as a differentiable approximation of the rule base.

[0026] In an embodiment of the method, the candidate discriminator has been trained to learn which configurations are similar in a latent constraint space.

[0027] In another embodiment of the method, the candidate generator is a policy-based reinforcement learning agent that builds up each configuration candidate in an iterative fashion, in par- ticular formulated as a Markov Decision Process, starting from an empty configuration and adding components and connec- tions until a terminal state is reached.

[0028] In an embodiment of the method, the candidate generator is implemented as a Graph Convolutional Policy Network. The computer program product comprises instructions which, when the program is executed by a computer, cause the comput- er to carry out the method.

[0029] The provisioning device stores and / or provides the computer program product.

[0030] The foregoing and other aspects of the present invention are best understood from the following detailed description when read in connection with the accompanying drawings. For the purpose of illustrating the invention, the drawings show em- bodiments that are presently preferred. However, the inven- tion is not limited to the specific instrumentalities dis- closed. The embodiments may be combined with each other. Fur- thermore, the embodiments may be combined with any of the features described above. Unless stated otherwise, identical reference signs denote the same features or functionally identical elements between drawings. Included in the drawings are the following Figures:

[0031] Fig. 1 shows a first embodiment,

[0032] Fig. 2 shows another embodiment,

[0033] Fig. 3 shows an example of a manual configuration workflow,

[0034] Fig. 4 shows an overall system architecture,

[0035] Fig. 5 shows an example of a latent constraint space, and

[0036] Fig. 6 shows a flowchart of a possible exemplary embodiment of a method for extracting compatibility rules from a black box configuration software.

[0037] In the following description, various aspects of the present invention and embodiments thereof will be described. However, it will be understood by those skilled in the art that embod- iments may be practiced with only some or all aspects there- of. For purposes of explanation, specific numbers and config- urations are set forth in order to provide a thorough under- standing. However, it will also be apparent to those skilled in the art that the embodiments may be practiced without these specific details.

[0038] The described modules can each be hardware modules or soft- ware modules. For example, a software module can be a soft- ware library; an individual procedure, subroutine, or func- tion; or, depending on the programming paradigm, any other portion of software code that implements the function of the software module. A combination of hardware modules and soft- ware modules can occur, in particular, if some of the effects according to the invention are preferably exclusively imple- mented by special hardware (e.g., a processor in the form of an ASIC or FPGA) and some other part by software.

[0039] Fig. 1 shows one sample structure for computer-implementation of the invention which comprises:

[0040] (101) computer system

[0041] (102) processor

[0042] (103) memory

[0043] (104) computer program (product)

[0044] (105) user interface

[0045] In this embodiment of the invention the computer program 104 comprises program instructions for carrying out the inven- tion. The computer program 104 is stored in the memory 103 which renders, among others, the memory 103 and / or its relat- ed computer system 101 a provisioning device for the computer program 104. The computer system 101 may carry out the inven- tion by executing the program instructions of the computer program 104 by the processor 102. Results of invention may be presented on the user interface 105. Alternatively, they may be stored in the memory 103 or on another suitable means for storing data. Fig. 2 shows another sample structure for computer- implementation of the invention which comprises:

[0046] (201) provisioning device

[0047] (202) computer program (product)

[0048] (203) computer network / Internet

[0049] (204) computer system

[0050] (205) mobile device / smartphone

[0051] In this embodiment the provisioning device 201 stores a com- puter program 202 which comprises program instructions for carrying out the invention. The provisioning device 201 pro- vides the computer program 202 via a computer net- work / Internet 203. By way of example, a computer system 204 or a mobile device / smartphone 205 may load the computer pro- gram 202 and carry out the invention by executing the program instructions of the computer program 202.

[0052] In a variation of this embodiment, the provisioning device 201 is a computer-readable storage medium, for example a SD card, that stores the computer program 202 and is connected directly to the computer system 204 or the mobile de- vice / smartphone 205 in order for it to load the computer pro- gram 202 and carry out the invention by executing the program instructions of the computer program 202.

[0053] Preferably, the embodiments shown in Figs. 3 to 6 can be im- plemented with a structure as shown in Fig. 1 or Fig. 2.

[0054] The embodiments described in the following provide solutions that extract information from a black-box configuration soft- ware in the form of interpretable logical rules. The basic idea is to employ a reinforcement learning agent to generate inputs (configuration candidates) into the black-box configu- ration software, use a "valid" / "invalid" response from the black-box configuration software to obtain sets of positive and negative examples, and to employ an inductive logic pro- gramming (ILP) engine to turn those into a rule (which is likely present in the black-box configuration software's im- plicit rule base).

[0055] The embodiments described in the following employ reinforce- ment learning to generate configuration candidates efficient- ly, as the configuration search space is exponentially grow- ing, making manual exploration infeasible. This problem has similarities with the problem of active learning, where a query strategy samples candidates from a pool of unlabeled data such that the informativeness of these samples is maxi- mal. The embodiments described in the following deal with a pool of unlabeled data that is potentially infinite, i.e., every possible configuration.

[0056] Fig. 3 shows an example of a manual configuration workflow. A human expert HE interacts with a black-box configuration software BBS, creating two device configurations, a valid configuration VC and an invalid configuration IC. The black- box configuration software BBS then responds with a valid re- sponse VR or an invalid response IR, which are collected as observed facts OF.

[0057] Each configuration can be expressed in terms of logical pred- icates involving the configuration components, e.g., "con- nected(a,b)" or types such as "s7(a)". After obtaining a set of valid configurations VC and invalid configurations IC as observed facts OF, a rule learning system (here, an inductive logic programming engine ILPE) can extract (induce) rules R from the observations.

[0058] Some of the technical features of the embodiments described in the following are: a) a Rule Extraction System, and b) an end-to-end training procedure.

[0059] Each of these features is described in more detail below. Rule Extraction System

[0060] Component Architecture

[0061] Fig. 4 shows an overall architecture of a compatibility rule extraction system CRES according to a current embodiment, re- ceiving as input a domain vocabulary DV as well as outputs of a black-box configuration software BBS. The compatibility rule extraction system CRES outputs a rule base RB.

[0062] The compatibility rule extraction system CRES stores the in- puts received from the black-box configuration software BBS (these inputs being validity labels for configuration candi- dates CC) along with the corresponding configuration candi- dates CC in an observed configurations database OC.

[0063] Candidate Discriminator

[0064] Within the compatibility rule extraction system CRES, a can- didate discriminator CD encourages that configuration candi- dates CC generated by a candidate generator CG are diverse (and substantially different from already observed ones), since classifying already observed configuration candidates is simple and therefore the reward for the candidate genera- tor CG would be small. Given a set of tuples (Xi,yi) with ob- served configuration candidate and its validity yiG{0,1}, the candidate discriminator CD function D\X -*Y is trained using binary cross entropy k

[0065] LD=^ -yilog(DOi))- (1- ydiog(i- D(xt)). i

[0066] Thus, the candidate discriminator CD acts as a differentiable approximation of the rule base RB. The underlying idea here is that the candidate discriminator CD learns which configu- rations are similar in a latent constraint space, therefore assuming a lower dimensional manifold for the original data generation, not every possible combination of components may need to be tried.

[0067] Fig. 5 shows an example of the latent constraint space men- tioned above. In Fig. 5, a classifier is given by a hyper- plane, which is a decision boundary DB that discriminates be- tween valid configurations and invalid configurations. The validity of configurations that are close to the decision boundary DB (i.e., the hyperplane) is harder to predict and lead to larger entropy.

[0068] Candidate Generator

[0069] In the current embodiment, the candidate generator CG is a policy-based reinforcement learning agent that builds up a configuration candidate CC in an iterative fashion, formulat- ed as a Markov Decision Process. Starting from the initial state space So, which is the empty configuration, it applies actions, namely addition of components and connections, until a final stop action or terminal state STis reached. The agent receives a reward based on the terminal state STwhich is given by: which is the entropy of the predicted probability distribu- tion of the candidate discriminator CD. In other words, the reward of the candidate generator CG is high if the produced final state candidate leads to high uncertainty in the candi- date discriminator CD. Therefore, novel configuration candi- dates CC that are associated with a high uncertainty are en- couraged. Based on the heuristic depicted in Fig. 5, the can- didate generator CG attempts to generate configurations that are located close to the decision boundary DB shown in Fig.

[0070] 5.

[0071] You, Jiaxuan, et al., Graph convolutional policy network for goal-directed molecular graph generation, Advances in neural information processing systems 31, 2018, discloses a Graph Convolutional Policy Network (GCPN), which is a neural archi- tecture that can be used to implement the candidate generator CG. The entire contents of that document are incorporated herein by reference.

[0072] End-to-end Training Procedure

[0073] According to this embodiment of a training procedure for the compatibility rule extraction system CRES, initially the ob- served configurations database OC is empty, and the candidate generator CG and the candidate discriminator CD are randomly initialized neural networks.

[0074] Then the training iteratively goes through the following:

[0075] 1.The candidate generator CG samples a new configuration candidate CC from its policy

[0076] 2.The candidate discriminator CD gives a reward to the candidate generator CG based on the configuration candi- date CC

[0077] 3.The candidate generator CG updates its policy

[0078] 4.The black-box configuration software BBS produces a va- lidity label for the configuration candidate CC

[0079] 5.The configuration candidate CC and its validity label are stored in the observed configuration database OC

[0080] 6.Every n iterations, the candidate discriminator CD is trained based on the examples in the observed configura- tions database OC.

[0081] The training procedure ends based on some convergence crite- rion, e.g., when the candidate generator CG and candidate discriminator CD weight updates vanish.

[0082] After training is done, the candidate generator CG can still be used to sample more candidates.

[0083] Once enough observed configurations are obtained, an induc- tive logic programming engine ILPE produces a rule base. Quinlan, J. Ross, Learning logical definitions from rela- tions, Machine learning 5, 1990, 239-266, discloses algo- rithms that are suitable for implementing the inductive logic programming engine ILPE. The entire contents of that document are incorporated herein by reference.

[0084] Fig. 6 shows a flowchart of a possible exemplary embodiment of a method for extracting compatibility rules from a black box configuration software.

[0085] In a first operation 1, a candidate generator generates con- figuration candidates, wherein each configuration candidate identifies components of an industrial automation system and their connections.

[0086] In a second operation 2, for each configuration candidate a valid response or an invalid response is received from a black-box configuration software, and a corresponding validi- ty label is created.

[0087] In a third operation 3, an observed configurations database stores each configuration candidate and its validity label.

[0088] In a fourth operation 4, an inductive logic programming en- gine processes the content of the observed configurations da- tabase and outputs a rule base storing compatibility rules.

[0089] Of course, at least some of these operation can be executed in parallel and / or recurrently. For example, the third opera- tion of storing can be executed after each configuration can- didate has been generated and labeled. In other words, the first three operations can be performed iteratively in a loop, wherein a new configuration candidate is generated, la- beled and stored in each iteration. Preferably, the fourth operation is executed only after sufficient information has been gathered in the observed configurations database. For example, the method can be executed by one or more pro- cessors. Examples of processors include a microcontroller or a microprocessor, an Application Specific Integrated Circuit (ASIC), or a neuromorphic microchip, in particular a neuro- morphic processor unit. The processor can be part of any kind of computer, including mobile computing devices such as tab- let computers, smartphones or laptops, or part of a server in a control room or cloud.

[0090] The above-described method may be implemented via a computer program product including one or more computer-readable stor- age media having stored thereon instructions executable by one or more processors of a computing system. Execution of the instructions causes the computing system to perform oper- ations corresponding with the acts of the method described above.

[0091] The instructions for implementing processes or methods de- scribed herein may be provided on non-transitory computer- readable storage media or memories, such as a cache, buffer, RAM, FLASH, removable media, hard drive, or other computer readable storage media. Computer readable storage media in- clude various types of volatile and non-volatile storage me- dia. The functions, acts, or tasks illustrated in the figures or described herein may be executed in response to one or more sets of instructions stored in or on computer readable storage media. The functions, acts or tasks may be independ- ent of the particular type of instruction set, storage media, processor or processing strategy and may be performed by software, hardware, integrated circuits, firmware, micro code, and the like, operating alone or in combination. Like- wise, processing strategies may include multiprocessing, mul- titasking, parallel processing, and the like.

[0092] The invention has been described in detail with reference to embodiments thereof and examples. Variations and modifica- tions may, however, be effected within the scope of the in- vention covered by the claims. The phrase "at least one of A, B and C" as an alternative expression may provide that one or more of A, B and C may be used.

Claims

Patent claims1. A computer implemented method for extracting compatibility rules from a black box configuration software, wherein the following operations are performed by modules, and wherein the modules are hardware modules and / or software modules exe- cuted by one or more processors: generating (1), by a candidate generator (CG), configu- ration candidates (CC), wherein each configuration can- didate (CC) identifies components of an industrial auto- mation system and their connections, receiving (2), from a black-box configuration software (BBS), for each configuration candidate (CC) a valid re- sponse (VR) or an invalid response (IR) and creating a corresponding validity label, storing (3), by an observed configurations database (OC), each configuration candidate (CC) and its validity label, and outputting (4), by an inductive logic programming engine (ILPE) processing the content of the observed configura- tions database (OC), a rule base (RB) storing compati- bility rules.

2. The method of claim 1, wherein the candidate generator (CG) receives as one of its inputs a domain vocabulary (DV).

3. The method according to any of the preceding claims, wherein the candidate generator (CG) receives as one of its inputs a reward that is calculated by a candidate discriminator (CD), and wherein the reward in particular encourages that the configuration candidates (CC) generated by the candidate generator (CG) are diverse and substantially different from the configuration candidates (CC) already stored in the observed configurations database (OC).

4. The method according to claim 3,wherein the reward is high if the configuration candi- date (CC) currently generated by the candidate generator (CG) leads to high uncertainty in the candidate discrim- inator (CD), and wherein the reward is in particular calculated as the entropy of a predicted probability distribution of the candidate discriminator (CD).

5. The method according to any of the preceding claims 3 and4, wherein the candidate discriminator (CD) has been trained using binary cross entropy to act as a differen- tiable approximation of the rule base (RB).

6. The method according to any of the preceding claims 3-5, wherein the candidate discriminator (CD) has been trained to learn which configurations are similar in a latent constraint space.

7. The method according to any of the preceding claims, wherein the candidate generator (CG) is a policy-based reinforcement learning agent that builds up each config- uration candidate (CC) in an iterative fashion, in par- ticular formulated as a Markov Decision Process, start- ing from an empty configuration and adding components and connections until a terminal state is reached.

8. The method according to any of the preceding claims, wherein the candidate generator (CG) is implemented as a Graph Convolutional Policy Network.

9. A system (CRES) for extracting compatibility rules from a black box configuration software, comprising: a candidate generator (CG), configured for generating configuration candidates (CC), wherein each configura- tion candidate (CC) identifies components of an indus- trial automation system and their connections,an interface, configured for receiving, from a black-box configuration software (BBS), for each configuration candidate (CC) a valid response (VR) or an invalid re- sponse (IR) and creating a corresponding validity label, an observed configurations database (OC), configured for storing each configuration candidate (CC) and its valid- ity label, and an inductive logic programming engine (ILPE), configured for processing the content of the observed configura- tions database (OC) and outputting a rule base (RB) storing compatibility rules.

10. A computer program product comprising instructions which, when the program is executed by a computer, cause the comput- er to carry out a method according to one of the method claims.

11. A provisioning device for the computer program product according to the preceding claim, wherein the provisioning device stores and / or provides the computer program product.

Images