Anomaly and drift detection using domain-indexed constraint repositories
By using the knowledge graph in the constraint repository for domain-aware exceptions and drift detection, the problem that the prior art cannot explain and automatically detect exceptions and drifts in non-time series data is solved, and automatic interpretation and verification of the original data is realized. Meets the detection needs of user domain information.
Patent Information
- Application Number
- CN202080067694.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-09-27
- Filing Date
- 2020-09-22
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-09-22
AI Technical Summary
Existing exception and drift detection technologies cannot interpret the original data, and cannot automatically detect exceptions and drifts in non-time series data, regardless of user domain information.
Domain-aware knowledge graphs in the constraint repository are used for domain-aware interpretation of exceptions and drift detection, and exceptions in the data are detected by generating natural language interpretations corresponding to exceptions, and based on hierarchical relationship aggregation constraints of multiple domains.
Automatic detection and interpretation of multivariate raw data is realized, and abnormal and drift detection is provided for user context-sensitive abnormality and drift detection, which can effectively verify data quality without encoding or transforming the data.
Smart Images

Figure CN114430827B_ABST
Abstract
Description
Technical Field
[0001] The present application relates generally to information technology and, more particularly, to anomaly and drift detection. Background Art
[0002] Anomaly detection involves identifying irregularities in data and typically relies on statistical or machine learning-based methods applied to numerical or encoded data. Typically, these methods cannot be applied to raw data.
[0003] Anomaly detection is used in a variety of applications such as intrusion detection, fraud detection, fault detection, system health monitoring, event detection in sensor networks, and detecting ecosystem disturbances. Summary of the invention
[0004] In one embodiment of the present invention, a technique for domain-aware interpretable anomaly and drift detection for multivariate raw data using a constraint repository is provided. An exemplary computer-implemented method includes the following steps: obtaining (i) a data set and (ii) information indicating a domain of the data set; detecting one or more anomalies in the data set based at least in part on a domain-indexed constraint repository, the domain-indexed constraint repository comprising a knowledge graph having a plurality of nodes, wherein each node comprises (i) an attribute associated with at least one of a plurality of domains and (ii) one or more constraints corresponding to the attribute; generating an explanation corresponding to each of the one or more anomalies; and outputting (i) an indication of the one or more anomalies and (ii) a corresponding natural language explanation.
[0005] Another exemplary computer-implemented method includes the following steps: obtaining a set of data files including training data, wherein each data file in the set corresponds to a domain of a plurality of domains; for each given data file in the set: (i) inferring at least one constraint based on at least a portion of the training data for the given data file, and (ii) storing the inferred at least one constraint at a node in a constraint repository of a domain index, wherein the node represents a domain corresponding to the given data file, and wherein the domain index constraint repository includes a set of nodes defining a hierarchical relationship of a plurality of domains; aggregating the inferred constraints stored in the domain index constraint repository based on the hierarchical relationship of the plurality of domains; retrieving one or more constraints from the constraint repository of the domain index based on the domain of a new data file; and detecting one or more anomalies in the new data file based on whether one or more portions of the new data file violate the retrieved one or more constraints; outputting (i) an indication of the one or more anomalies and (ii) a natural language explanation describing how the portion of the new data file violates the retrieved one or more constraints.
[0006] Another embodiment of the present invention or its elements may be implemented in the form of a computer program product that tangibly embodies computer-readable instructions that, when implemented, cause a computer to perform a plurality of method steps as described herein. In addition, another embodiment of the present invention or its elements may be implemented in the form of a system comprising a memory and at least one processor that is coupled to the memory and configured to perform the method steps described herein. In addition, another embodiment of the present invention or its elements may be implemented in the form of an apparatus or its elements for performing the method steps described herein; the apparatus may include a hardware module or a combination of hardware and software modules, wherein the software modules are stored in a tangible computer-readable storage medium (or multiple such media).
[0007] These and other objects, features and advantages of the present invention will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] Figure 1 is a diagram showing a system architecture according to an exemplary embodiment of the present invention;
[0009] Figure 2 is a diagram showing another system architecture according to an exemplary embodiment of the present invention;
[0010] Figure 3 is a flow chart of a process for generating a constraint repository according to an exemplary embodiment;
[0011] Figure 4 is a diagram illustrating an example of a subtree of a constraint repository according to an exemplary embodiment;
[0012] Figure 5 is a flow chart of a process for inferring constraints according to an exemplary embodiment;
[0013] Figure 6 is a flow chart illustrating a technique according to an embodiment of the present invention;
[0014] Figure 7 is a system diagram of an exemplary computer system upon which at least one embodiment of the present invention may be implemented;
[0015] Figure 8 Describes a cloud computing environment according to an embodiment of the present invention; and
[0016] Fig. 9 Abstract model layers according to an embodiment of the present invention are depicted. DETAILED DESCRIPTION
[0017] Existing anomaly and drift detection techniques do not provide explanations when anomalies are detected as being far from, for example, an expected value region and their distance from the actual value. There is no intuitive explanation of how the expected value is calculated. Existing techniques also tend to require transformation or encoding of the input data and / or fail to consider the user's domain.
[0018] Many use cases of anomaly and drift detection can benefit from providing intuitive natural language explanations along with the detected anomalies. Additionally, existing anomaly and drift detection techniques focus on time series datasets for anomaly detection and therefore do not consider abnormal outliers and / or drifting data points in non-temporal datasets.
[0019] As described herein, embodiments of the present invention include providing interpretable anomaly detection of data sets. For example, an intelligent data engineering platform can be provided that automatically detects anomalies related to a user's context (or domain) and also provides explanations for these detected anomalies. In addition, according to at least one embodiment, the quality of new or existing data about a base data set is calculated. The calculated quality can then be used, for example, to validate data imported from various data sources. In one or more example embodiments, automatic drift detection is applied to live payload data of a machine learning model.
[0020] Reference now Figure 1 , which shows an example of a system architecture according to an exemplary embodiment of the present invention. Figure 1 The examples shown in can be applied to automatic drift detection in payload data for machine learning models, for example. Figure 1 The system architecture depicted in FIG. 1 includes a constraint repository 104 and a drift analysis module 108. In some examples, the constraint repository 104 includes a knowledge graph (KG) corresponding to multiple domains, wherein each node in the KG can be assigned one or more constraints. For example, one or more constraints can be learned based on training data 102.
[0021] The drift analysis module 108 includes an anomaly detection module 110 and a clustering module 112. Payload data 114 (e.g., live or real-time payload data to be fed into a deployed machine learning model) and (one or more) constraints 106 related to the payload data 114 are provided to the anomaly detection module 110. The anomaly detection module 110 applies the constraints 106 to the payload data 114 to detect one or more anomalies, which are then clustered by the clustering module 112. Based on the clustering, the clustering module 112 determines the data segments that drift. In addition, the drift analysis module 108 generates an explanation of how the drifted segments are calculated. The drift analysis module 108 then outputs the drifted segments with corresponding explanations 116 to, for example, a user. For example, the drift analysis module 108 can cluster the detected anomalies into multiple clusters and refer to a given cluster as a drift if the size of the cluster is greater than a threshold. For example, clustering can be performed based on the types of anomalies and their parameters, such as the distance from the expected value.
[0022] Reference now Figure 2 , which shows another example of a system architecture according to an exemplary embodiment of the present invention. Figure 2 The examples shown may be applied, for example, to detect anomalies based on data quality. Figure 2 The system architecture depicted in 200 includes a constraint repository 204 and a quality analysis module 210. The constraint repository 204 may include, for example, a KG corresponding to a plurality of domains, wherein each node in the KG is assigned one or more constraints based on, for example, a base data set 202.
[0023] The quality analysis module 210 calculates the quality of the new data 214 (e.g., data imported into the database) relative to the base data set 202 using the constraints 206 from the constraint repository 204. The quality analysis module 210 automatically detects anomalies 216 and generates an explanation 218 (e.g., a natural language explanation) based on the calculated quality of the new data 214, which can then be output to, for example, a user. In at least one embodiment, the anomalies 216 and the explanations 218 can be used to automatically verify or validate the new data 214.
[0024] Reference now Figure 3 , which is a process flow diagram for populating a KG of a constraint repository 310 according to an exemplary embodiment. The constraint repository 310 includes a KG having a domain structure. More specifically, the domain structure includes a domain hierarchy that aligns or groups underlying data according to the semantics of the domain hierarchy. For example, a group of domains are covered under a parent domain (e.g., the parent domain of "Company" covers IT companies, banks, etc.).
[0025] At step 304, a set of constraints is inferred from one or more data sets 302 (e.g., data files). The set of constraints may be inferred using inference rules, which may be based on data types (e.g., text, numeric values, etc.) or based on attribute types (e.g., birthdays, salaries, PIN codes, etc.). The process continues to step 306, where a domain-based constraint merging process is performed. The domain-based constraint merging process may include determining statistics of constraints from different data sets in the data sets 302 under a particular domain, and merging constraints according to merging rules based on, for example, data types or attribute types. In some example embodiments, the merging operation may be performed on a per-request basis, such as when a user provides user input 312, which, for example, includes a request for a specific constraint on a specific attribute at a particular domain. Step 308 includes performing a domain-based constraint index, which indexes the constraints to nodes of the KG in the constraint repository 310. Thus, constraints including overall constraint information for a particular domain or domains under a hierarchy may be retrieved.
[0026] According to one or more example embodiments, the KG is stored in a content repository (e.g., content repository 310) using the following types of vertices:
[0027] ● Domain Vertex (DV): Each domain vertex represents a specific domain (e.g., IT company, bank, etc.). ● File Information Vertex (FIV): Each dataset (or file) is represented by a unique identifier (e.g., FileID), and each FIV represents a node relative to each file (indexed by FileID). FIV includes file-level metadata information, such as the number of rows, columns, etc.).
[0028] ● File-Level Constraint Vertex (FLCV): Each attribute of a given file can involve single-attribute or multi-attribute constraints. Each constraint condition represents a separate FLCV in the KG. FLCV attributes include constraint information statistics.
[0029] =Domain Level Constraint Vertex (DLCV): Each DLCV includes aggregate statistics for a specific constraint at a given domain. The aggregation includes all information from files located in the subdomain hierarchy of the given domain. At a given domain, a DLVC is generated on a per-request basis. A DLCV may maintain an "updated" flag indicating whether the constraint statistics have been updated.
[0030] In addition, a KG can include the following types of edge relations:
[0031] - Subdomain: defines the relationship between domains in the immediate hierarchy (eg, the domain vertex "IT Company" is a subdomain of "Company").
[0032] l attachedTo: defines the relationship between the FIV and the DV.
[0033] l fileConstaintsOf: defines the relationship between FIV and FLCV.
[0034] l domainConstrainOf: defines the relationship between DLCV and DV.
[0035] Given a data file and a domain as input, constraints can be added to the knowledge graph by inferring constraints from the data file (e.g., constraints for the "age" column in the data file can include inferred statistics, such as minimum, maximum, and median). Next, a FIV is then created for the dataset, and a FLCV is created for each constraint linked to a FIV vertex. The FIV vertex is linked to the corresponding DV (e.g., based on the input domain). For each FLCV in the FIV, a test is performed to check if there is a DLCV vertex in the DV. If the updated flag is set to true, the value of the flag is changed to false, and the process is repeated for the parent DV. If the updated flag is set to false, the constraint is added.
[0036] By providing the constraint attribute and domain as input, the constraints stored in the KG can be retrieved based on the following process. DLCV processing is performed, which includes performing a test to check whether there is a DLCV relative to the input constraint attribute at the DV relative to the input domain. If the DLCV exists and the updated flag is set to true, the vertex information statistics are returned. Otherwise, FLCV processing is performed, which includes: for each FIV linked to the DIV corresponding to the input domain, a first list of FLCVs is created by checking each FLCV against the input constraint attribute and checking the information of all FLCVs in the list. Then subdomain processing is performed, which includes generating a second list of DIVs for subdomains of the input domain and calling the constraint access stream for all DIVs in the second list. Information is collated for each call in the second list.
[0037] The first list information and the second list information are merged using a merge rule (e.g., specific to a data type or attribute). The merged information is stored at a DLCV vertex corresponding to the input domain, and an updated flag is set to true. The process then returns the merged information.
[0038] Reference now Figure 4, which illustrates an example of a subtree 400 of a constraint repository according to an exemplary embodiment. For example, the subtree 400 can be part of a KG in a constraint repository. The subtree 400 includes a plurality of levels 402-1 to 402-5 (collectively referred to as levels 402). The root node at layer 402-1 indicates that the subtree 400 represents an "institution". Each successive layer 402 increases the layer abstraction relative to the previous node. For example, layer 402-2 includes a "company" node, and layer 402-3 includes nodes corresponding to company types (i.e., IT companies and banks).
[0039] According to one or more example embodiments, a data set may be mapped to a node of a KG in a constraint repository. If no suitable KG node is found, a new node is created under the most specific abstract node. For example, referring again to Figure 4 , if a new data set (e.g., a data file or document) includes information of an IT company named "Company D", a new node under the "IT Company" node will be created for Company D, since there is no existing node for such a company. Constraints are learned from each new data set (e.g., from a set of training data sets or based on a new data set). The constraints are then added to the corresponding nodes in the constraint repository. Constraints on intermediate nodes can be determined by aggregating constraints on all subtrees.
[0040] According to at least one example embodiment, a new data set (or document) is validated with reference to a constraint repository. For example, the new data set is mapped to the most specific possible node in the constraint repository. Figure 4 , assuming that the new data set includes information of an IT company named "Company E", since there is no "Company E" in the subtree 400, the data set is mapped to the "IT Company" node. Then, the constraints in the mapped node are used to verify the anomalies of the records in the test data set.
[0041] In some example embodiments, the constraint repository stores a domain graph (e.g., using ), where constraints are associated with attributes. This allows, for example, searching for attributes of a given domain, retrieving constraints for an attribute, and performing CRUD (copy, read, update, and delete) operations on domains and constraints.
[0042] In addition, constraints in the constraint repository can be merged across datasets and across different domains. For example, an age constraint of 28 < age < 42 with weights associated with subranges (e.g., 29-30, 31-34, 35-39, and 40-42) can be merged to form constraints such as 29 < age < 34 and 34 < age < 42 for each 'age' node in the constraint repository.
[0043] Figure 55 is a flow chart 500 for inferring constraints from a data set according to an exemplary embodiment. In this example, constraints 502 are divided into single attribute constraints 504 and multi-attribute constraints 506. Single attribute constraints 504 include three different types 508, namely, text type 510, numeric type 512, and categorical type 514, which can be identified based on the uniqueness of the data corresponding to the attribute, for example.
[0044] Text types 510 can be unpatterned 516 (e.g., comments, names in documents, etc.), in which case anomalies can be detected by, for example, maintaining unigram and bigram frequencies of the text in a Trie. For patterned types 518 (e.g., phone numbers, dates, social security numbers, etc.), anomalies can be detected based on, for example, whether the text follows a regular expression. For numeric type 512 attributes, anomalies can be detected based on statistical properties such as distribution or range (minimum, maximum), for example. Categorical types 514 can be in the form of text 520 (e.g., gender) or numeric values 522 (e.g., country names, quarter numbers, etc.), and anomalies can be based on probability values. As an example, if for a record in a dataset with categorical type 514, P(value) < 10 -4 , the record is detected as abnormal.
[0045] The multi-attribute 506 constraint includes a combination of single attributes. For example, Figure 4 In the example shown, different types of multi-attribute 506 constraints include numeric-numeric 524 (e.g., age-salary); categorical-numeric 526 (e.g., marital status, age); categorical-categorical 528 (e.g., country, state); and categorical text 530 (e.g., name, gender).
[0046] Some non-limiting examples of how multi-attribute 506 constraints can be used to detect anomalies are as follows: ● Numerical-Category: For each found range of the numerical attribute, if P(value) < 10-4 for the categorical attribute, an anomaly is detected.
[0047] • Value-to-value: For each discovered range of an independent numerical attribute, a linear, polynomial or exponential function is learned with a confidence value; if the value of a dependent numerical attribute is away from the learned function beyond the bounds determined by the confidence, it is detected as an anomaly.
[0048] • Classification-Classification: Classification attribute pairs with confidence (x->y) below a threshold are marked as anomalies. • Classification-Numerical: For each value of the categorical attribute, a separate distribution of the numerical attribute will be learned, and samples outside the distribution will be detected as anomalies.
[0049] As an example, assume that the following constraints are inferred from a base dataset with attributes of marital status and age.
[0050] ●Confidence (0 < age <= 18) = 100%
[0051] Confidence (100 > age > 18) = exp (-x / 50-18)
[0052] The constraint can be applied to the target dataset to determine the confidence values for the records in the target dataset. For example, the record "Single, 5" would result in a 100% confidence level, while the record "Married, 14" would result in a 0% confidence level. According to one or more example embodiments, both of these examples can be detected as anomalies, and an explanation can be generated detailing why the records were marked as anomalies. Anomalies can also be detected in the case where records are updated / inserted. For example, the record "Single, 140" that was added to the base dataset can be detected as an anomaly because the confidence value is also 0%.
[0053] Figure 6 600 includes obtaining (i) a data set and (ii) information indicating a domain of the data set. Step 602 includes obtaining one or more constraints from a domain-indexed constraint repository based at least in part on (i) the data set and (ii) the information, wherein the domain-indexed constraint repository includes a knowledge graph having a plurality of nodes, wherein each node includes (i) an attribute associated with at least one of a plurality of domains and (ii) one or more constraints corresponding to the attribute. Step 604 includes detecting one or more anomalies in the data set based at least in part on whether one or more portions of the data set violate the obtained one or more constraints. Step 606 includes generating an explanation corresponding to each of the one or more anomalies, the explanation describing the attribute corresponding to the one or more constraints violated. Step 608 includes outputting (i) an indication of the one or more anomalies and (ii) a corresponding explanation.
[0054] Each of the one or more subtrees of the knowledge graph may correspond to a given one of a plurality of domains, and each level of the given one of the subtrees may increase the level of abstraction of the given domain relative to a previous level in the given subtree. The detection may include mapping a data set to one or more of a plurality of nodes based on information indicating the domain of the data set; and applying constraints of the mapped nodes to the data set to detect one or more anomalies. The one or more constraints may include at least one of: a single attribute constraint; and a multi-attribute constraint. The single attribute constraint may correspond to one of the following: (i) a text attribute; (ii) a numerical attribute; and (iii) a categorical attribute, and wherein the multi-attribute constraint corresponds to two or more of the following: (i) a text attribute; (ii) a numerical attribute; and (iii) a categorical attribute. Each constraint may include a model based on one or more of (i) a type of a single attribute constraint and (ii) a type of a multi-attribute constraint, and wherein the detection includes applying the model to at least a portion of the data set. The data set may include real-time data of at least one machine learning model, and Figure 6 The process may include: using machine learning clustering techniques to detect drift in one or more segments of real-time data; and outputting the segments of the real-time data corresponding to the drift. Generating an explanation corresponding to each of the one or more anomalies may include generating and outputting an explanation for each of the segments of the real-time data corresponding to the drift. The explanation corresponding to each of the one or more anomalies may include a natural language explanation describing how the portion violates one or more constraints retrieved. The detection may include determining a value indicating the quality of the data set based on a comparison of the data set with a base data set. Optionally, Figure 6 The process depicted in may include validating the data set in response to a value indicating that a quality of the data set satisfies a threshold. The data set may include raw data that has neither been encoded nor transformed.
[0055] Another example embodiment includes a method comprising: obtaining a set of data files including training data, wherein each data file in the set corresponds to a domain of a plurality of domains; for each given data file in the set: (i) inferring at least one constraint based on at least a portion of the training data for the given data file, and (ii) storing the inferred at least one constraint at a node in a constraint repository of a domain index, wherein the node represents a domain corresponding to the given data file, and wherein the constraint repository of the domain index includes a set of nodes defining a hierarchical relationship of a plurality of domains; aggregating the inferred constraints stored in the constraint repository of the domain index based on the hierarchical relationship of the plurality of domains; retrieving one or more constraints from the constraint repository of the domain index based on the domain of a new data file; and detecting one or more anomalies in the new data file based on whether one or more portions of the new data file violate the retrieved one or more constraints; outputting (i) an indication of the one or more anomalies and (ii) a natural language interpretation describing how the portion of the new data file violates the retrieved one or more constraints.
[0056] As stated in this article, Figure 6 The technology depicted in the can also include providing a system, wherein the system includes different software modules, each different software module is contained on a tangible computer-readable recordable storage medium. For example, all modules (or any subset thereof) can be on the same medium, or each can be on a different medium. The module can include any or all components shown in the figure and / or described herein. In an embodiment of the present invention, the module can be run on a hardware processor, for example. Then, the different software modules of the system executed on the hardware processor as described above can be used to perform the method steps. In addition, the computer program product can include a tangible computer-readable recordable storage medium, which has a code suitable for being executed to perform at least one method step described herein, including providing different software modules to the system.
[0057] in addition, Figure 6 The techniques described in the can be implemented via a computer program product, which can include a computer usable program code stored in a computer readable storage medium in a data processing system, and wherein the computer usable program code is downloaded from a remote data processing system over a network. In addition, in an embodiment of the present invention, the computer program product can include a computer usable program code stored in a computer readable storage medium in a server data processing system, and wherein the computer usable program code is downloaded to the remote data processing system over a network for use in the computer readable storage medium with the remote system.
[0058] Embodiments of the present invention or elements thereof may be implemented in the form of an apparatus including a memory and at least one processor coupled to the memory and configured to perform the exemplary method steps.
[0059] Additionally, embodiments of the present invention may utilize software running on a computer or workstation. Figure 7 , such an embodiment may employ, for example, a processor 702, a memory 704, and an input / output interface formed, for example, by a display 706 and a keyboard 708. The term "processor" as used herein is intended to include any processing device, such as a processing device including a CPU (central processing unit) and / or other forms of processing circuitry. In addition, the term "processor" may refer to more than one individual processor. The term "memory" is intended to include memory associated with a processor or CPU, such as RAM (random access memory), ROM (read-only memory), fixed memory devices (e.g., hard disk drives), removable memory devices (e.g., disks), flash memory, etc. In addition, the phrase "input / output interface" as used herein is intended to include, for example, a mechanism for inputting data to a processing unit (e.g., a mouse) and a mechanism for providing results associated with a processing unit (e.g., a printer). The processor 702, the memory 704, and an input / output interface such as a display 706 and a keyboard 708 may be interconnected, for example, via a bus 710 as part of a data processing unit 712. For example, appropriate interconnections via bus 710 may also be provided to a network interface 714 (e.g., a network card, which may be provided to interface with a computer network), and to a media interface 716 (e.g., a disk or CD-ROM drive, which may be provided to interface with media 718).
[0060] Therefore, computer software including instructions or codes for executing the methods of the present invention as described herein can be stored in an associated memory device (e.g., ROM, fixed or removable memory), and when ready for use, partially or completely loaded (e.g., loaded into RAM) and implemented by the CPU. Such software may include, but is not limited to, firmware, resident software, microcode, etc.
[0061] A data processing system suitable for storing and / or executing program code will include at least one processor 702 coupled directly or indirectly to memory elements 704 through a system bus 710. The memory elements may include local memory employed during actual implementation of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during implementation.
[0062] Input / output or I / O devices (including but not limited to keyboard 708, display 706, pointing device, etc.) can be coupled to the system either directly (such as via bus 710) or through intervening I / O controllers (omitted for clarity).
[0063] Network adapters such as network interface 714 may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modem and Ethernet cards are just a few of the currently available types of network adapters.
[0064] As used herein (including the claims), a "server" includes a physical data processing system (e.g., a Figure 7 System 712 is shown.) It will be appreciated that such a physical server may or may not include a display and keyboard.
[0065] The present invention may be a system, method and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or multiple media) having computer-readable program instructions thereon, and the computer-readable program instructions are used to cause a processor to perform an embodiment of the present invention.
[0066] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device such as a punch card or a raised structure in a groove on which instructions are recorded, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be interpreted as a temporary signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.
[0067] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium in the corresponding computing / processing device.
[0068] The computer-readable program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data of an integrated circuit, or source code or object code written in any combination of one or more programming languages (including object-oriented programming languages, such as Smalltalk, C++, etc.) and procedural programming languages (such as "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider via the Internet). In some embodiments, in order to perform embodiments of the present invention, an electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer-readable program instructions to personalize the electronic circuit by utilizing the state information of the computer-readable program instructions.
[0069] Embodiments of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present invention. It will be understood that each frame of the flowchart and / or block diagram and the combination of frames in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0070] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can guide the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable storage medium having the instructions stored therein includes an article of manufacture, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes of the flowchart and / or block diagram.
[0071] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram.
[0072] Flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention.In this regard, each frame in flow chart or block diagram can represent the module, segment or part of instruction, and it comprises one or more executable instructions for realizing the specified logical function.In some alternative embodiments, the function noted in the frame may not occur in the order noted in the figure.For example, two frames shown continuously can actually be performed substantially simultaneously, or these frames can sometimes be performed in reverse order, depending on the function involved.It will also be noted that the combination of each frame of block diagram and / or flow chart illustration and the frame in block diagram and / or flow chart illustration can be realized by the dedicated hardware-based system that performs the specified function or action or performs the combination of special hardware and computer instruction.
[0073] It should be noted that any of the methods described herein may include the additional step of providing a system including different software modules implemented on a computer-readable storage medium; the modules may include, for example, any or all of the components described in detail herein. The method steps may then be performed using the different software modules and / or submodules of the system as described above executed on the hardware processor 702. Furthermore, a computer program product may include a computer-readable storage medium having code adapted to be implemented to perform at least one of the method steps described herein, including providing the system with different software modules.
[0074] In any case, it should be understood that the components shown herein may be implemented in various forms of hardware, software, or a combination thereof, such as application specific integrated circuits (ASICS), functional circuits, appropriately programmed digital computers with associated memory, etc. Given the teachings of the present invention provided herein, a person of ordinary skill in the relevant art will be able to envision other implementations of the components of the present invention.
[0075] In addition, it is understood in advance that although the present disclosure includes detailed descriptions about cloud computing, the implementation of the teachings recorded herein is not limited to a cloud computing environment. Instead, embodiments of the present invention can be implemented in conjunction with any other type of computing environment now known or later developed.
[0076] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be quickly provisioned and released with minimal management effort or interaction with the provider of the service. The cloud model may include at least five characteristics, at least three service models, and at least four deployment models.
[0077] Features are as follows:
[0078] On-demand self-service: Cloud consumers can unilaterally and automatically provision computing capabilities, such as server time and network storage, as needed without requiring manual interaction with the service provider.
[0079] Wide Area Network Access: Capabilities are available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).
[0080] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. There is a sense of location independence, as consumers typically do not control or know the exact location of the provided resources, but are able to specify the location at a higher level of abstraction (e.g., country, state, or data center).
[0081] Rapid elasticity: In some cases, the ability to scale out quickly and in quickly can be provisioned quickly and elastically. To the consumer, the capacity available for provisioning often appears to be unlimited and can be purchased in any quantity at any time.
[0082] Metered Services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the utilized services.
[0083] The service model is as follows:
[0084] Software as a Service (SaaS): The capability provided to the consumer is to use the provider's applications running on the cloud infrastructure. The applications are accessible from a variety of client devices through a thin client interface such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure including the network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.
[0085] Platform as a Service (PaaS): The capability provided to consumers is to deploy consumer-created or acquired applications onto cloud infrastructure, where the applications are created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but has control over the deployed applications and possible configuration of the application hosting environment.
[0086] Infrastructure as a Service (IaaS): The capabilities provided to consumers are processing, storage, networking, and other basic computing resources on which consumers can deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but have control over the operating system, storage, deployed applications, and possibly limited control over selected networking components (e.g., host firewalls).
[0087] The deployment model is as follows:
[0088] Private Cloud: The cloud infrastructure is operated only for the organization. It can be managed by the organization or a third party and can exist inside or outside the building.
[0089] Community cloud: The cloud infrastructure is shared by several organizations and supports a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). It can be managed by the organization or a third party and can exist on-premises or off-premises.
[0090] Public cloud: Cloud infrastructure is available to the general public or large industrial groups and is owned by an organization that sells cloud services.
[0091] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community or public) that remain a unique entity but are bound together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).
[0092] The cloud computing environment is service-oriented, with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is an infrastructure consisting of a network of interconnected nodes.
[0093] Reference now Figure 8 , depicts an illustrative cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 with which a local computing device used by a cloud consumer can communicate, such as a personal digital assistant (PDA) or cellular phone 54A, a desktop computer 54B, a laptop computer 54C, and / or an automobile computer system 54N. The nodes 10 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as a private cloud, community cloud, public cloud, or hybrid cloud, or a combination thereof, as described above. This allows the cloud computing environment 50 to provide infrastructure, platform, and / or software as a service for which the cloud consumer does not need to maintain resources on a local computing device. It should be understood that Figure 8 The types of computing devices 54A-N shown in are intended to be illustrative only, and computing node 10 and cloud computing environment 50 may communicate with any type of computerized device over any type of network and / or network addressable connection (eg, using a web browser).
[0094] Reference now Fig. 9 , showing the cloud computing environment 50 ( Figure 8 ) provides a set of functional abstraction layers. It should be understood in advance that Fig. 9 The components, layers, and functions shown in are intended to be illustrative only, and embodiments of the present invention are not limited thereto. As depicted, the following layers and corresponding functions are provided:
[0095] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: host 61; server 62 based on RISC (Reduced Instruction Set Computer) architecture; server 63; blade server 64; storage device 65; and network and network components 66. In some embodiments, software components include network application server software 67 and database software 68.
[0096] Virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 71; virtual storage 72; virtual networks 73, including virtual private networks; virtual applications and operating systems 74; and virtual clients 75. In one example, management layer 80 can provide the functions described below. Resource provisioning 81 provides dynamic procurement of computing resources and other resources for performing tasks within a cloud computing environment. Metering and pricing 82 provides cost tracking when utilizing resources in a cloud computing environment, as well as billing or invoicing for the consumption of these resources.
[0097] In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides cloud computing resource allocation and management so that the required service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides pre-scheduling and procurement of cloud computing resources, where future demand is anticipated according to the SLA.
[0098] Workload layer 90 provides examples of functionality that can take advantage of a cloud computing environment. Examples of workloads and functionality that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analytics processing 94; transaction processing 95; and automatic domain-aware interpretable anomaly and drift detection on multivariate raw data using a constraint repository 96 according to one or more embodiments of the present invention.
[0099] The terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the present invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the terms "include" and / or "comprise" when used in this specification indicate the presence of the features, steps, operations, elements and / or components, but do not exclude the presence or addition of another feature, step, operation, element, component and / or combination thereof.
[0100] At least one embodiment of the present invention may provide beneficial effects, such as automatically detecting anomalies and drifts related to the user's context, and providing corresponding explanations. The detection may be performed using the data set in its most natural form, without any encoding or transformation.
[0101] The description of various embodiments of the present invention has been given for the purpose of illustration, but it is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements existing in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A computer-implemented method, the method comprising: obtaining a data set and information indicating a domain of the data set; obtaining one or more constraints from a domain-indexed constraint repository based at least in part on the data set and the information, wherein the domain-indexed constraint repository comprises a knowledge graph having a plurality of nodes, wherein each node comprises an attribute associated with at least one domain of a plurality of domains and one or more constraints corresponding to the attribute; detecting one or more anomalies in the data set based at least in part on whether one or more portions of the data set violate the obtained one or more constraints; generating an explanation corresponding to each of the one or more exceptions, the explanation describing properties corresponding to the one or more constraints that were violated; and An indication of the one or more anomalies and a corresponding explanation is output.
2. The computer-implemented method of claim 1 , wherein: Each of the one or more subtrees of the knowledge graph corresponds to a given one of the multiple domains, and wherein each level of a given one of the one or more subtrees increases the level of abstraction of the given one of the domains relative to a previous level in the given one of the subtrees.
3. The computer-implemented method of claim 1 , wherein: The detection includes: mapping the data set to one or more nodes of the plurality of nodes based on the information indicating a domain of the data set; and Constraints of the mapped nodes are applied to the data set to detect the one or more anomalies.
4. The computer-implemented method of claim 1 , wherein: The one or more constraints include at least one of the following: Single attribute constraints; and Multi-attribute constraints.
5. The computer-implemented method of claim 4, wherein: The single-attribute constraint corresponds to one of the following: a textual attribute, a numerical attribute, and a categorical attribute, and wherein the multi-attribute constraint corresponds to two or more of the following: a textual attribute, a numerical attribute, and a categorical attribute.
6. The computer-implemented method of claim 5, wherein: Each constraint comprises a model based on one or more of the type of the single-attribute constraint and the type of the multi-attribute constraint, and wherein the detecting comprises applying the model to at least a portion of the data set.
7. The computer-implemented method of claim 1 , wherein: The data set includes real-time data for at least one machine learning model, and wherein the method comprises: using machine learning clustering techniques to detect drift in one or more segments of the real-time data; and A segment of the real-time data corresponding to the drift is output.
8. The computer-implemented method of claim 7, wherein: The generating includes generating and outputting an interpretation of each of the segments of the real-time data corresponding to the drift.
9. The computer-implemented method of claim 1 , wherein: The explanation corresponding to each of the one or more exceptions includes a natural language explanation describing how the portion violates the retrieved one or more constraints.
10. The computer-implemented method of claim 1, wherein: The detection includes: A value indicative of a quality of the data set is determined based on a comparison of the data set to a base data set.
11. The computer-implemented method of claim 10, comprising: The data set is validated in response to the value indicative of the quality of the data set satisfying a threshold.
12. The computer-implemented method of claim 1, wherein: The data set includes raw data that has neither been encoded nor transformed.
13. A computer program product comprising a computer-readable storage medium, the computer-readable storage medium comprising program instructions executable by a computing device to cause the computing device to: obtaining a data set and information indicating a domain of the data set; obtaining one or more constraints from a constraint repository of a domain index based at least in part on the data set and the information, wherein: The domain-indexed constraint repository includes a knowledge graph having a plurality of nodes, wherein each node includes an attribute associated with at least one domain of the plurality of domains and one or more constraints corresponding to the attribute; detecting one or more anomalies in the data set based at least in part on whether one or more portions of the data set violate the obtained one or more constraints; generating an explanation corresponding to each of the one or more exceptions, the explanation describing properties corresponding to the one or more constraints that were violated; and An indication of the one or more anomalies and a corresponding explanation is output.
14. The computer program product of claim 13, wherein: Each of the one or more subtrees of the knowledge graph corresponds to a given one of the multiple domains, and wherein each level of a given one of the one or more subtrees increases the level of abstraction of the given one of the domains relative to a previous level in the given one of the subtrees.
15. The computer program product of claim 13, wherein the detecting comprises: mapping the data set to one or more nodes of the plurality of nodes based on the information indicating a domain of the data set; as well as The constraints of the mapped nodes are applied to the data set to detect the one or more anomalies.
16. The computer program product of claim 13, wherein: The one or more constraints include at least one of the following: Single attribute constraints; and Multi-attribute constraints.
17. The computer program product of claim 16, wherein: The single-attribute constraint corresponds to one of the following: a textual attribute, a numerical attribute, and a categorical attribute, and wherein the multi-attribute constraint corresponds to two or more of the following: a textual attribute, a numerical attribute, and a categorical attribute.
18. The computer program product of claim 17, wherein: Each constraint comprises a model based on one or more of the type of the single-attribute constraint and the type of the multi-attribute constraint, and wherein the detecting comprises applying the model to at least a portion of the data set.
19. A system comprising: Memory; as well as at least one processor operably coupled to the memory and configured to: obtaining a data set and information indicating a domain of the data set; obtaining one or more constraints from a domain-indexed constraint repository based at least in part on the data set and the information, wherein the domain-indexed constraint repository comprises a knowledge graph having a plurality of nodes, wherein each node comprises an attribute associated with at least one domain of a plurality of domains and one or more constraints corresponding to the attribute; detecting one or more anomalies in the data set based at least in part on whether one or more portions of the data set violate the obtained one or more constraints; generating an explanation corresponding to each of the one or more exceptions, the explanation describing properties corresponding to the one or more constraints that were violated; and An indication of the one or more anomalies and a corresponding explanation is output.
20. A computer-implemented method, the method comprising: obtaining a set of data files comprising training data, wherein each data file in the set corresponds to a domain in a plurality of domains; for each given data file in the set: inferring at least one constraint based on at least a portion of the training data for the given data file, and storing the inferred at least one constraint at a node in a constraint repository of a domain index, wherein the node represents the domain corresponding to the given data file, and wherein the constraint repository of the domain index comprises a set of nodes defining a hierarchical relationship of the plurality of domains; aggregating the inferred constraints stored in a constraint repository of the domain index based on the hierarchical relationship of the plurality of domains; retrieving one or more constraints from a constraint repository of the domain index based on a domain of the new data file; detecting one or more anomalies in the new data file based on whether one or more portions of the new data file violate the retrieved one or more constraints; and An indication of the one or more anomalies is output along with a natural language explanation describing how the portion of the new data file violates the retrieved one or more constraints.
