Intelligent anomaly handling in process control and other systems

Abstract

Methods for identifying and processing anomalies in AI and ML models, and corresponding systems and computer-readable mediums. A method (500) includes receiving (502) evidence data (204), the evidence data (204) including model metric data (210), historic application domain data (212), and causal chain graphs (214). The method includes identifying (504) an anomaly (302) in an artificial intelligence (AI) model or a machine learning (ML) model (152). The method includes identifying (506) an application domain context (174) corresponding to the identified anomaly (302) and assembling (508) a causal chain (300) corresponding to the identified anomaly (302) and the identified application domain context (174). The method includes outputting (510) the causal chain (300), including a responsive action (306).

Description

INTELLIGENT ANOMALY HANDLING IN PROCESS CONTROL AND OTHER SYSTEMSTECHNICAL FIELD

[0001] The present disclosure is directed, in general, to employing artificial intelligence and machine learning techniques for detecting anomalies in process control systems.BACKGROUND OF THE DISCLOSURE

[0002] Increasingly, artificial intelligence (Al) and machine learning (ML) capabilities are being leveraged across various industries, with one key area of application being anomaly detection. Examples for such anomaly detection is quality assurance in manufacturing, building management, energy management, cybersecurity, and other process control systems. Current techniques, however, are not easily usable by those who are not data science experts and do not identify causes or solutions to identified anomalies. Improved systems are desirable.SUMMARY OF THE DISCLOSURE

[0003] Various disclosed embodiments include methods for identifying and processing anomalies in Al and ML models, and corresponding systems and computer- readable mediums. A method includes receiving evidence data, the evidence data including model metric data, historic application domain data, and causal chain graphs. The method includes identifying an anomaly in an Al model or an ML model. The method includes identifying an application domain context corresponding to the identified anomaly and assembling a causal chain corresponding to the identified anomaly and the identified application domain context. The method includes outputting the causal chain, including a responsive action.

[0004] Various embodiments also include performing the responsive action by the computer system. In various embodiments, the anomaly is identified based on the model metric data. In various embodiments, the anomaly is identified by comparing model metric data to one or more thresholds to determine if the model metric data exceeds the one or more thresholds. In various embodiments, the application domain context is identified based on the historic application domain data.

[0005] In various embodiments, the causal chain includes the identified anomaly, a root cause corresponding to the anomaly, and the responsive action. In various embodiments, the output is produced to a presentation layer of the computer system for display to a user. In various embodiments, the output is stored in a non-transitory medium. In various embodiments, the method is performed by an explainability layer implemented by the computer system.

[0006] Various embodiments include a computer system having a processor and an accessible memory, configured to perform processes as described herein. Various embodiments include a non-transitory computer-readable medium encoded with executable instructions that, when executed by one or more computers, cause the one or more computers to perform processes as described herein.

[0007] The foregoing has outlined rather broadly the features and technical advantages of the present disclosure so that those skilled in the art may better understand the detailed description that follows. Additional features and advantages of the disclosurewill be described hereinafter that form the subject of the claims. Those skilled in the art will appreciate that they may readily use the conception and the specific embodiment disclosed as a basis for modifying or designing other structures for carrying out the same purposes of the present disclosure. Those skilled in the art will also realize that such equivalent constructions do not depart from the spirit and scope of the disclosure in its broadest form.

[0008] Before undertaking the DETAILED DESCRIPTION below, it may be advantageous to set forth definitions of certain words or phrases used throughout this patent document: the terms “include” and “comprise,” as well as derivatives thereof, mean inclusion without limitation; the term “or” is inclusive, meaning and / or; the phrases “associated with” and “associated therewith,” as well as derivatives thereof, may mean to include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to or with, have, have a property of, or the like; and the term “controller” means any device, system or part thereof that controls at least one operation, whether such a device is implemented in hardware, firmware, software or some combination of at least two of the same. It should be noted that the functionality associated with any particular controller may be centralized or distributed, whether locally or remotely. Definitions for certain words and phrases are provided throughout this patent document, and those of ordinary skill in the art will understand that such definitions apply in many, if not most, instances to prior as well as future uses of such defined words and phrases. While some terms may include a wide variety of embodiments, the appended claims may expressly limit these terms to specific embodiments.BRIEF DESCRIPTION OF THE DRAWINGS

[0009] For a more complete understanding of the present disclosure, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, wherein like numbers designate like objects, and in which:

[0010] FIG. 1 illustrates a block diagram of a computer system in which an embodiment can be implemented;

[0011] FIG. 2 illustrates an example of layers of an AI / ML system implemented in one or more computer systems in accordance with disclosed embodiments;

[0012] FIG. 3 illustrates an example of a causal chain in accordance with disclosed embodiments;

[0013] FIG. 4 illustrates an example of an explainability layer in accordance with disclosed embodiments;

[0014] FIG. 5 depicts a flowchart of a process in accordance with disclosed embodiments;

[0015] FIG. 6 illustrates an example of a metric threshold table in accordance with disclosed embodiments;

[0016] FIG. 7 illustrates an example of an application domain context table in accordance with disclosed embodiments; and

[0017] FIGS. 8-13 illustrate exemplary outputs of an explainability layer or computer system in accordance with disclosed embodiments.DETAILED DESCRIPTION

[0018] FIGS. 1 through 13, discussed below, and the various embodiments used to describe the principles of the present disclosure in this patent document are by way of illustration only and should not he construed in any way to limit the scope of the disclosure. Those skilled in the art will understand that the principles of the present disclosure may be implemented in any suitably arranged device. The numerous innovative teachings of the present application will be described with reference to exemplary non-limiting embodiments.

[0019] As described above, Al and ML tools can be useful in identifying anomalies in the operation or output of many process control systems and other systems. The anomalies detected by Al and machine learning models are typically quantified and visualized using various methods for more effective interpretation. These may include metrics displayed as time series charts, which allow for easy identification of trends and patterns over a given period. Alternatively, significant anomalies can be isolated and represented as individual data points, providing a focus on singular events of interest.

[0020] However, such representations often require a data science background to be fully understood, creating a barrier for operators who need to respond to these detected anomalies but lack the necessary expertise. Most current operators are trained in the application domain (e.g., manufacturing, building management, energy management, cyber security), not in the Al domain. Hence, displayed Al model metrics are ineffective for current operators, and effective analysis requires companies to hire data scientists. This option is not viable due to limited availability of data scientists in the job market and their higher salary.

[0021] Another problem with current systems is that detected anomalies, displayed as Al model metrics, do not reveal their root causes. The detected anomaly is only a symptom. Yet another problem is that detected anomalies, displayed as Al model metrics, do not reveal how to fix them (preventive or corrective action, called here “responsive action”).

[0022] Disclosed embodiments overcome the issues of detected anomalies displayed with opaque Al-model metrics, detected anomalies that do not identify their root causes, and detected anomalies that to not identify their responsive actions, among others.

[0023] Disclosed embodiments include more intuitive and user-friendly visualizations of Al-detected anomalies that can be readily interpreted across different expertise levels.

[0024] In conventional cases, data scientists are responsible for detecting and dealing with anomalies. These include the case when the ML / Al model is offline and when the ML / Al model is online. When the model is offline, a data scientist can analyze the model metrics during model training, model testing, and in operation, and identify deviations if such exists. When the model is online, the model metrics are reported constantly, and data scientists have access to such model metrics. If a deviation occurs, the model metrics will be available for the data scientists, who can identify the deviation. In both of these cases, the data scientists need to talk to an operator from the application domain to identify the root cause for the deviation and a responsive action. After an anomaly was detected, it can take up to several days today to identify root cause and to initiate a responsive action.

[0025] For purposes of illustration of an exemplary hardware environment, FIG. 1 illustrates a block diagram of a computer system in which an embodiment can be implemented, for example as a computer system particularly configured by software or otherwise to perform the processes as described herein, and in particular as each one of a plurality of interconnected and communicating systems as described herein. The computer system depicted includes a processor 102 connected to a level two cache / bridge 104, which is connected in turn to a local system bus 106. Local system bus 106 may be, for example, a peripheral component interconnect (PCI) architecture bus. Also connected to local system bus in the depicted example are a main memory 108 and a graphics adapter 110. The graphics adapter 110 may be connected to display 111.

[0026] Other peripherals, such as local area network (LAN) / Wide Area Network / Wireless (e.g. WiFi) adapter 112, may also be connected to local system bus 106.Expansion bus interface 114 connects local system bus 106 to input / output (I / O) bus 116. I / O bus 116 is connected to keyboard / mouse adapter 118, disk controller 120, and I / O adapter 122. Disk controller 120 can be connected to a storage 126, which can be any suitable machine usable or machine readable storage medium, including but not limited to nonvolatile, hard-coded type mediums such as read only memories (ROMs) or erasable, electrically programmable read only memories (EEPROMs), magnetic tape storage, and user-recordable type mediums such as floppy disks, hard disk drives and compact disk read only memories (CD-ROMs) or digital versatile disks (DVDs), and other known optical, electrical, or magnetic storage devices.

[0027] Storage 126 can store any data necessary or useful for performing the processes described herein, including executable code 150, AI / ML model 152, model metric data 154, historic application domain data 156, causal chain graph(s) 158, causal chain(s) 160, anomalies 162, root causes 164, responsive actions 166, functional components 168, evidence data 170, tables 172, application domain contexts 174, and other data 176.

[0028] Also connected to I / O bus 116 in the example shown is audio adapter 124, to which speakers (not shown) may be connected for playing sounds. Keyboard / mouse adapter 118 provides a connection for a pointing device (not shown), such as a mouse, trackball, trackpointer, touchscreen, etc.

[0029] Those of ordinary skill in the art will appreciate that the hardware depicted in Figure 1 may vary for particular implementations. For example, other peripheral devices, such as an optical disk drive and the like, also may be used in addition or in place of the hardware depicted. The depicted example is provided for the purpose of explanation only and is not meant to imply architectural limitations with respect to the present disclosure.

[0030] A computer system in accordance with an embodiment of the present disclosure includes an operating system employing a graphical user interface. The operating system permits multiple display windows to be presented in the graphical user interface simultaneously, with each display window providing an interface to a different application or to a different instance of the same application. A cursor in the graphicaluser interface may be manipulated by a user through the pointing device. The position of the cursor may be changed and / or an event, such as clicking a mouse button, generated to actuate a desired response.

[0031] One of various commercial operating systems, such as a version of Microsoft Windows™, a product of Microsoft Corporation located in Redmond, Wash, may be employed if suitably modified. The operating system is modified or created in accordance with the present disclosure as described.

[0032] LAN / WAN / Wireless adapter 112 can be connected to a network 130 (not a part of computer system 100), which can be any public or private computer system network or combination of networks, as known to those of skill in the art, including the Internet. Computer system 100 can communicate over network 130 with server system 140, which is also not part of computer system 100, but can be implemented, for example, as a separate computer system 100.

[0033] Disclosed embodiments overcome the technical shortcomings of current approaches by implementing an “explainability layer” that is able to identify and assemble elements of the causal chain for a detected anomaly. The explainability layer generates a causal chain that the presentation layer displays to an operator.

[0034] FIG. 2 illustrates an example of layers of an AI / ML system 200 implemented in one or more computer systems 100, to illustrate an explainability architecture as disclosed herein. AI / ML system 200 includes an evidence layer 208 which can include such elements as model metric data 210, historic application domain data 212, and a causal chain graph 214. Each of the model metric data 210, historic application domain data 212, and a causal chain graph 214, and other elements of evidence layer 208, can communicate with explainability layer 204, described in more detail herein. Explainability layer 204 can then sent its output to presentation layer 202 to present the output 208, such as the detected anomaly with causal chain and responsive action.

[0035] Causal chain graph 214 can be implemented as a knowledge graphs or other suitable knowledge repository or data structure. For example, a causal chain graph 214 can be implemented as a directed graph that associates specific anomalies or classes of anomalies with potential causes or other conditions that may produce an anomaly or bean indicator of a potential issue that would identify a cause of an anomaly. Causal chain graph 214 can be a weighted graph where specific associations between anomalies and potential causes are weighted based on, for example, the probability that a specific potential cause is the actual cause of an anomaly. Causal chain graph 214 can include responsive actions that are associated with specific anomalies or classes of anomalies and specific causes, where the responsive actions are those determined to address the respective cause(s) to eliminate or prevent the anomaly.

[0036] FIG. 3 illustrates an example of a causal chain 300 in accordance with disclosed embodiments. A causal chain 300 includes an anomaly 302 (also called a symptom), a root cause 304, and a responsive action 306 that addresses the root cause and removes the anomaly 302. An assembled causal chain 300 can contain one or multiple root causes 304 for an anomaly, and for each root cause 304, it can contain one or multiple responsive actions 306. Each root cause and responsive action is associated with a confidence level.

[0037] FIG. 4 illustrates an example of an explainability layer 204 in accordance with disclosed embodiments. Explainability layer 400 includes a number of functional components, implemented by executable instructions as part of AI / ML system 200 implemented in one or more computer systems 100, which operate based on information in evidence layer 206 of FIG. 2. Functional component 402 assembles causal chains. Functional component 404 identifies domain application contexts. Together, functional components 402 and 404 can identify an anomaly in an Al application, ML model, or similar. This can include, for example, identifying outlier data, data points, or events in the model metric data. In particular, this can include identifying whether a model metric exceeds a threshold for a given amount if time. In this context, “exceeds” a threshold should be understood to mean a value that is greater than an upper threshold or less than a lower threshold.

[0038] Functional component 406 identifies root causes of the anomaly, which can be based on the identified anomaly, historic application data, model metric data, and a causal chain graph. Functional component 408 identifies responsive actions. Functional component 410 builds a causal chain model.

[0039] FIG. 5 illustrates a flowchart of a process 500 in accordance with disclosed embodiments that may be performed, for example, by one or more computer systems 100 as disclosed herein (below, simply the “system”) implementing an explainability layer 204 in an ML / Al system 200. In process 500, the system assembles a causal chain from a detected anomaly and from historic application domain data.

[0040] At 502, the system receives evidence data. “Receiving,” as used herein, can include loading from storage, receiving from another device or process, receiving via an interaction from a user, or otherwise. In particular, in this context, this can include the explainability layer 204 receiving evidence data from the evidence layer 208.

[0041] Receiving evidence data can include receiving model metric data 210, receiving historic application domain data 212, and receiving causal chain graphs 214. In particular, receiving the evidence data need not be performed necessarily as a preliminary step to the remainder of the process 500, but can be performed concurrently with or as needed for each of the other actions of process 500.

[0042] At 504, the system identifies an anomaly in an Al or ML model. This can be performed, for example, based on the model metric data 210.

[0043] In one exemplary process for identifying an anomaly, the system processes the model metric data 210 frequently, such as every second, every 10 seconds, every 60 seconds, every 3600 seconds. The system compares the model metrics with thresholds (lower and / or upper threshold) and determines if a model metric exceeds one of the thresholds. If the component has identified a model metric exceeds a threshold, the component can determine whether the model metric has or will exceed the model threshold for a defined minimum time duration. If a model metric has or will exceed the threshold for at least a defined minimum time, the model metric is identified as an anomaly.

[0044] FIG. 6 illustrates an example of a metric threshold table 600 that identifies specific model metrics 602 along with corresponding lower thresholds 604 (where appropriate), upper thresholds 606 (where appropriate), and minimum time durations 608.

[0045] The output of 504 is the identified anomaly.

[0046] At 506, the system identifies application domain context(s) corresponding to the identified anomaly. This can be performed using the historic application domain data.

[0047] In one exemplary process for identifying application domain context(s), the system can identify a model ID (or equivalent identifier of an AI / ML model), and read the application domain context for the identified model ID. This can be performed using an application domain context table, which can be part of the model metric data and / or the historic application domain data.

[0048] FIG. 7 illustrates an example of an application domain context table 700 that identifies specific AI / ML models 702 along with corresponding context for each model 702, such as production line 704 (where appropriate), station 706, and material dependencies 708, and process dependencies 710. Of course, the nature and implementation of specific AI / ML models will dictate the corresponding contexts of their implementations.

[0049] The output of 506 is the identified application domain context(s).

[0050] At 508, the system assembles one or more causal chains corresponding to the identified anomaly and the identified application domain context(s). This can be performed using historic application domain data 212 and a causal chain graph 214.

[0051] In one exemplary process for assembling causal chains, the system can initiates a new causal chain instance CC.1. The system adds the identified anomaly to the causal chain instance CC.l and adds the identified application domain context to the causal chain instance CC. L The system identifies root causes for the identified anomaly with a confidence level, above a defined threshold, in a causal chain graph. The number of root causes can be limited.

[0052] In this exemplary process, the system adds the identified root cases to the causal chain instance CC. l, and searches for response actions for identified root cause with a confidence level, above a defined threshold, in a causal chain graph. The number of responses actions per root cause can be limited.

[0053] In this exemplary process, the system adds the identified response actions per root cause to the causal chain CC. 1.

[0054] The output of 508 is the assembled causal chain(s), such as causal chain 300 including the anomaly 302, the root cause 304, and the responsive action 306.

[0055] At 510, the system produces an output that includes the causal chain or its components. The output can be produced, for example, to a presentation layer that can display the detected anomaly with its causal chain and responsive action. The output can be stored in a tangible or non-transitory medium, and the output can be transmitted to another device process. In some cases, where the responsive action can be automated, the system can also automatically perform the responsive action to address the root cause, at 510 or thereafter.

[0056] FIG. 8 illustrates an exemplary output of an explainability layer (as presented by a presentation layer) of an overview of reported incidents, in accordance with disclosed embodiments.

[0057] FIG. 9 illustrates an exemplary output of an explainability layer (as presented by a presentation layer) of symptoms and other features in accordance with disclosed embodiments.

[0058] FIG. 10 illustrates an exemplary output of an explainability layer (as presented by a presentation layer) of application domain context (location, in this case) and other features in accordance with disclosed embodiments.

[0059] FIG. 11 illustrates an exemplary output of an explainability layer (as presented by a presentation layer) of root causes and other features in accordance with disclosed embodiments.

[0060] FIG. 12 illustrates an exemplary output of an explainability layer (as presented by a presentation layer) of immediate responsive actions and other features in accordance with disclosed embodiments.

[0061] FIG. 13 illustrates an exemplary output of an explainability layer (as presented by a presentation layer) of responsive actions and other features in accordance with disclosed embodiments.

[0062] Using techniques and processes disclosed herein, a computer system can display anomalies, root cause(s). and responsive action(s) that are understandable for operators that do not have a data science background. This enables an operator to select and initiate a responsive action within minutes, not days.

[0063] In various embodiments, the identified responsive action is machine readable and can be machine executed.

[0064] A system as disclosed herein can offer or display abnormal-situation-specific content and controls for an operators.

[0065] Of course, those of skill in the art will recognize that, unless specifically indicated or required by the sequence of operations, certain steps in the processes described above may be omitted, performed concurrently or sequentially, or performed in a different order.

[0066] Those skilled in the art will recognize that, for simplicity and clarity, the full structure and operation of all computer systems suitable for use with the present disclosure is not being depicted or described herein. Instead, only so much of a computer system as is unique to the present disclosure or necessary for an understanding of the present disclosure is depicted and described. The remainder of the construction and operation of computer system 100 may conform to any of the various current implementations and practices known in the art.

[0067] It is important to note that while the disclosure includes a description in the context of a fully functional system, those skilled in the art will appreciate that at least portions of the mechanism of the present disclosure are capable of being distributed in the form of instructions contained within a machine-usable, computer-usable, or computer-readable medium in any of a variety of forms, and that the present disclosure applies equally regardless of the particular type of instruction or signal bearing medium or storage medium utilized to actually carry out the distribution. Examples of machineusable / readable or computer usable / readable mediums include: nonvolatile, hard-coded type mediums such as read only memories (ROMs) or erasable, electrically programmable read only memories (EEPROMs), and user-recordable type mediums such as floppy disks, hard disk drives and compact disk read only memories (CD- ROMs) or digital versatile disks (DVDs).

[0068] Although an exemplary embodiment of the present disclosure has been described in detail, those skilled in the art will understand that various changes, substitutions, variations, and improvements disclosed herein may be made without departing from the spirit and scope of the disclosure in its broadest form.

[0069] None of the description in the present application should be read as implying that any particular element, step, or function is an essential element which must be included in the claim scope: the scope of patented subject matter is defined only by the allowed claims. Moreover, none of these claims are intended to invoke 35 USC §112(f) unless the exact words "means for" are followed by a participle. The use of terms such as (but not limited to) “mechanism,” “module,” “device,” “unit,” “component,” “element,” “member,” “apparatus,” “machine,” “system,” “processor,” or “controller,” within a claim is understood and intended to refer to structures known to those skilled in the relevant art, as further modified or enhanced by the features of the claims themselves, and is not intended to invoke 35 U.S.C. §112(f).

Claims

WHAT IS CLAIMED IS:

1. A method (500) performed by a computer system (100) and comprising: receiving (502) evidence data (204), the evidence data (204) including model metric data (210), historic application domain data (212), and causal chain graphs (214); identifying (504), by the computer system (100), an anomaly (302) in a model (152), wherein the model (152) is an artificial intelligence (Al) model or a machine learning (ML) model; identifying (506), by the computer system (100), an application domain context (174) corresponding to the identified anomaly (302); assembling (508), by the computer system (100), a causal chain (300) corresponding to the identified anomaly (302) and the identified application domain context (174); and outputting (510) the causal chain (300), including a responsive action (306), by the computer system (100).

2. The method of claim 1, further comprising performing (510) the responsive action by the computer system (100).

3. The method of claim 1, wherein the anomaly (302) is identified based on the model metric data (210).

4. The method of claim 1 , wherein the anomaly (302) is identified by comparing model metric data (210) to one or more thresholds to determine if the model metric data (210) exceeds the one or more thresholds.

5. The method of claim 1, wherein the application domain context (174) is identified based on the historic application domain data (212).

6. The method of claim 1, wherein the causal chain (300) includes the identified anomaly (302), a root cause (304) corresponding to the anomaly (302), and the responsive action (306).

7. The method of claim 1, wherein the output is produced to a presentation layer (202) of the computer system (100) for display to a user.

8. The method of claim 1 , wherein the output is stored in a non-transitory medium (126).

9. The method of claim 1, wherein the method (500) is performed by an explainability layer (204) implemented by the computer system (100).

10. A computer system (100) comprising: a processor (102); and an accessible memory (108), the computer system (100) particularly configured to perform a method (500) as in any of claim 1-9.

11. A non-transitory computer-readable medium (126) encoded with executable instructions that, when executed by one or more computers (100), cause the one or more computers (100) to perform a method (500) as in any of claim 1-9.

Images