Best interpretable decision tree using integer programming techniques

Abstract

Aspects of the present invention include optimal interpretable decision trees using integer linear programming techniques. A non-limiting example computer-implemented method includes receiving multiple data inputs from a process using a processor, and selecting a data subset from the multiple data inputs by solving a linear programming to obtain a solution using the processor. The method uses the processor to build and optimize an optimal decision tree based on the data subset, and uses the processor to alert a user when a prediction of the optimal decision tree is greater than a threshold.

Description

Background Art

[0001] The present invention relates generally to predictive modeling and, more particularly, to optimal interpretable decision trees using integer linear programming techniques. Background Art

[0003] Predictive models are useful in process control environments. For example, blast furnaces, refineries, and supply chain management all rely on process control. Deep neural networks (DNNs) and optimal decision trees (ODTs) can be used for process control. DNNs are artificial neural networks with multiple layers between input and output. DNNs find the correct mathematical transformation to transform input into output. ODTs are decision trees that offer the best compromise between training accuracy and model simplicity.

[0004] Decision tree based learners need to be simple and accurate.Therefore, there is a need in the art to solve the above problems. Summary of the Invention

[0005] Viewed from another aspect, the present invention provides a computer program product for predictive modeling, the computer program product comprising a computer-readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit to perform a method for performing the steps of the present invention.

[0006] Viewed from another aspect, the invention provides a computer program stored on a computer readable medium and loadable into the internal memory of a digital computer, comprising software code portions for performing the steps of the invention when said program is run on a computer.

[0007] Viewed from another aspect, the present invention provides a computer program product for creating and optimizing decision trees, the computer program product comprising a computer-readable storage medium having program instructions embodied thereon, the program instructions executable by a computer to cause the computer to perform a method comprising: receiving a plurality of data inputs from a process; selecting a data subset from the plurality of data inputs to obtain a solution by solving a linear program; constructing and optimizing an optimal decision tree ("ODT") based on the data subset; and alerting a user when a prediction of the optimal decision tree is greater than a threshold.

[0008] Embodiments of the present invention relate to optimal interpretable decision trees using integer linear programming techniques. A non-limiting example computer-implemented method includes receiving, using a processor, a plurality of data inputs from a process, and selecting, using the processor, a data subset from the plurality of data inputs to obtain a solution by solving a linear programming problem. The method constructs and optimizes, using the processor, an optimal decision tree based on the data subset, and alerts a user, using the processor, when a prediction of the optimal decision tree exceeds a threshold.

[0009] Other embodiments of the present invention implement the features of the above-described methods in computer systems and computer program products.

[0010] Additional technical features and benefits are achieved through the technology of the present invention. Embodiments and aspects of the present invention are described in detail herein and are considered to be part of the claimed subject matter. For a better understanding, reference is made to the specific embodiments and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The details of the exclusive rights described herein are particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other features and advantages of the embodiments of the present invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0012] Figure 1 A flowchart of an optimal interpretable decision tree implementation process according to an embodiment of the present invention is shown;

[0013] Figure 2 A flowchart of a data set selection process according to an embodiment of the present invention is described;

[0014] Figure 3 shows data set selection according to an embodiment of the present invention;

[0015] Figure 4 A cloud computing environment according to an embodiment of the present invention is shown;

[0016] Figure 5 shows an abstract model layer according to an embodiment of the present invention; and

[0017] Figure 6 Details of an exemplary computing system are shown that can implement aspects of the invention.

[0018] The schematic diagrams depicted herein are illustrative. For example, the actions may be performed in a different order, or actions may be added, deleted, or modified without departing from the scope of the present invention. Furthermore, the term "coupled" and its variations describe the presence of a communication path between two elements and do not imply a direct connection between the elements without intermediate elements / connections therebetween. All such variations are considered part of this specification. DETAILED DESCRIPTION

[0019] One or more embodiments of the present invention provide a robust decision tree-based learner designed to learn provably optimal two or three deep trees using integer programming techniques. This provides interpretability and easy optimization. By using mixed integer linear programming, a scalable global solution to the ODT can be determined. Embodiments of the present invention use data selection of the training data in advance without loss of generalization by selecting a subset of data points that captures as much information as possible of the entire data set, and then establishing a mixed integer programming ("MIP") formulation of the ODT only on this subset of data. In particular, a subset selection process based on linear programming ("LP") is implemented. Through the implementation of embodiments of the present invention, practical applications such as continuous process fault prediction are achieved.

[0020] Mathematical optimization techniques have been a staple of machine learning ("ML") problems, but MIPs were rarely considered previously, primarily due to their poor tractability. However, advances in MIP theory, optimization solvers, and combined with hardware improvements have led to astonishing speedups of a factor of 8 billion in MIPs compared to 25 years ago. Consequently, ML began to encompass MIP-based models, and many methods have been proposed to leverage mixed integer programming to solve classical ML tasks that were traditionally solved with heuristics. Among attempts to study the interplay between MIPs and ML, constructing ODTs has been one of the most popular, primarily due to the combinatorial structure of decision trees and their interpretability. Many related MIP formulations have been proposed for training ODTs for different purposes.

[0021] Existing attempts at solutions have faced challenges with interpretability and ease of optimization. For example, deep neural networks are essentially black boxes, offering little or no insight into their internal workings. Highly nonlinear models are difficult to optimize. In other words, finding local minima is challenging. Past solutions involving deep decision trees are difficult and time-consuming to optimize. They can also suffer from interpretability issues.

[0022] One or more embodiments of the present invention address one or more of the aforementioned shortcomings of the prior art by providing a technical solution comprising a simple decision tree that initially performs data selection for the training data. A linear programming-based data selection method is used to select a subset of the data. This subset of data is used in the MIP formulation of the ODT. This provides performance improvements in terms of: interpretability (the degree to which the reasoning behind the decision can be understood by the user); tractability (the ability to return an output within a reasonable timeframe corresponding to the size of the data); training accuracy; and generalization or test accuracy.

[0023] Now turn Figure 1, generally shows a flow chart of an optimal interpretable decision tree implementation process according to one or more embodiments of the present invention. In block 110, data selection is performed. The LP-based data selection process is implemented as a geometric approximation of a set of points. The process assumes an optimal classification tree with very high training accuracy, and some data points with label y in I0 are all correctly classified and assigned to the same leaf node. In block 110, the process discards I from the training set. o conv({x i │i∈I o ) with the same label y in the convex hull of o is a subset of the training set, and x i is the removed training sample, because in I o Correctly classifying data points in will automatically also correctly classify points within the convex hull. In some embodiments of the present invention, the number of data points within the convex hull is maximized.

[0024] Since there is no prior information about the optimal classification tree, the process here uses a decision tree trained from some other heuristic method as a guide and then performs data selection within each leaf node. Then, among those data points within each leaf node, the process performs data selection separately. Finally, the process combines all the data subsets selected from each leaf node and trains a classification tree on this dataset using MIP-ODT.

[0025] Typically, for data selection, the process selects a data subset I based on the following two criteria: o :(1)conv({x i │i∈I o ) as many points as possible or maximize the number of points within the convex hull of I0; and (2) the number of selected sampling points |I0| is as small as possible. For example, the remaining sample points used for training |I0| are less than 10% of the original dataset, while the number of samples within the convex hull of I0 is greater than 90%. Figure 3 An example is given.

[0026] The basic binary LP for data selection is as follows: In each cluster Among them, there are:

[0027]

[0028] Here f, g are two parameter vectors with non-negative components; a j Indicates whether data point j is selected; b j Indicates whether j can be represented as a convex combination of the selected data points; λ ji represents the convex combination coefficient of point i when point j is represented as a convex combination of some other points.

[0029] When bj = 1, the first two constraints approximately j It is represented as a convex combination of other points i, where λ ji >0, and when b j = 0, these two constraints hold, because in this case, for all i, λ ji = 0. The third inequality 0≤λ ji ≤a i Meaning that the process can use the selected data points, which are the ones with a i = 1 to represent other data points. The final constraint a j +b j ≤ 1 indicates that any selected data point cannot be represented as a convex combination of other selected data points. Depending on the choice of f, g and the convex combination constraint, there will generally be many different variations of the above equation.

[0030] When f = 0, g = 1, it follows that the process can project a and obtain a simpler binary LP:

[0031]

[0032] The next result shows that this binary LP can be equivalently solved by solving the following LP:

[0033]

[0034] For any optimal solution (b,λ) of the above equation, there exists λ such that (b,λ) is an optimal solution to the previous equation, and vice versa.

[0035] For some pre-given threshold parameters β1,β2∈(0,1), discussed later Figure 2 The data selection process (block 110) is provided in .

[0036] After data selection, the decision tree is constructed and optimized as follows, block 120. Without loss of generality, the process assumes that the classes of the dataset are [Y]:={1,2,...,Y}, where Y is the number of classes. Each data point i∈[n] is represented by: (x i ,y i ), where n represents the number of data points, x i is a d-dimensional vector, y i ∈{1,...,Y}. This process will is represented as the index set of all numerical features, and F c represents the index set of all categorical features. In this formula, the process considers only numerical features (F q= [n]). Develop a formula for a balanced binary tree of depth D, even though the trained ODT may actually be unbalanced. Here, the branch nodes of the tree are represented as B: = [2 D -1], and the leaf node is represented by L:={2 D ,...,2 D+1 -1}. This process also uses the symbol A R (l) and A L (l) to denote the set of ancestors of leaf node l whose left (right) branch has been followed on the path from the root node to leaf node l.

[0037] When using MIP to train decision trees, in most cases it is preferable to train multivariate decision trees rather than classic univariate decision trees because for univariate decision trees, it requires additional binary variables to force up to one feature that can be used to split at each branch node. As a generalization of univariate decision trees, multivariate decision trees are more difficult to train using common heuristic algorithms. The only difference between these two types of decision trees is whether the branching hyperplane must be parallel to the axis. Since multivariate decision trees are easier to train with MIP models, the process is only described for optimal multivariate decision trees, even single variable decision trees can be formulated by adding one or more constraints for each branch node to the formula. The MIP-ODT formula is as follows:

[0038]

[0039]

[0040]

[0041]

[0042]

[0043]

[0044]

[0045]

[0046]

[0047]

[0048]

[0049] With variables

[0050] A balanced binary tree with depth D is characterized by only two elements: the branching hyperplane at each branch node, and the label assigned to each leaf node. In our MIP-ODT, the branching hyperplane at branch node b is denoted as <h b ,x i >=g b The branching rule is given by the following formula: Data i goes to <h b ,x i >≤g b Left branch, otherwise go to the right. Assume that there is u l ∈[Y]., the label assigned to the leaf node l is denoted as u l ,.

[0051] Binary variables and Each data point i must enter one of the leaf nodes, so for each i∈[n] the process is performed ∑ l∈L e il =1, which is the constraint (1k). represents the classification class of data i, so the process has the constraint Introducing an additional variable w il To represent these bilinear terms, for w ul =u l e il , the process performs McCormick relaxation. These are constraints (1c)-(1e). Since here e il is forced to binary, so in The McCormick relaxation is exact in the sense that (1c)-(1e) hold for some additional variable w.

[0052] In order to express ci and The relationship between the two, the process has the first constraint (1b): when ci = 0, it means that the data i is correctly classified, there must be When c i =1, Free to take any value in [1,Y].

[0053] Constraints (1f), (1g) and (1i) are about formulating the branching rule at each node b∈B: if data i enters the left branch at node b, then If it goes to the right, then In MIP, by Split into a pair of complementary variables and And it is common to express this relationship in a formula by forcing one of the two variables to 0 each time by the Big-M method. Here complementarity means and They cannot all be positive at the same time.

[0054] This is not exactly the same as our branching rule: when data i goes to the right branch, instead it should strictly satisfy The only special case is when hour.

[0055] Penalty soft margin variable m i,b Motivated by Support Vector Machines (“SVM”): This procedure expects the branching hyperplane to be not too close to all data points. This gives the objective function:

[0056]

[0057] According to Occam’s Razor Principle, the process also takes model simplicity (sparseness) into account by adding an additional penalty term:

[0058] ∑ b ||h b ||0

[0059] Add the number of misclassifications to the target return:

[0060]

[0061] By ||h b ||1 Linearization and ||h b ||0, and the goal is:

[0062]

[0063] Here α1 and α2 are some positive parameters.

[0064] A series of cutting planes can be used to further enhance the above MIP-ODT formulation. Cutting planes include:

[0065]

[0066] as well as

[0067]

[0068] Here, the process represents And assume

[0069] After the ODT is created and optimized, a check is made to determine if the prediction generated by the ODT is less than a threshold, block 130. If so, the asset in the process is under control, block 150. If not, the process alerts the user and recommends a control action based on the analysis of the ODT, block 140. By adjusting the value of the feature vector input x i to implement the control action, which results in the predicted output y i changes.

[0070] For example, you can refer to Figure 1 The described process is used for continuous process fault prediction, a type of supervised anomaly prediction. It is designed to alert the user when a process fault is detected. It is based on historical process data for notification and serves as the ground truth for training AI-based models. Various rule types can be implemented, such as thresholds, trends, or multiples. For example, a threshold can trigger an alarm when the tank volume falls below a minimum. A trend can trigger an alarm, for example, when throughput decreases or when power drops. Multiple triggers can be triggered when there is a loss of flow without a setpoint, or when, for example, the temperature falls below a threshold and the predicted rate rises above a second threshold.

[0071] With process fault prediction, the goal of generating an ODT is to predict process faults. Sequences of process sensor and labeled disturbances are transformed into a classification problem. Fault events are extracted using a look-ahead window to create training targets, enabling learning of a model of expected events. Summary functions are used to extract features from one or more look-ahead windows.

[0072] Figure 2 The flowchart of the data selection process according to an embodiment of the present invention is described. Given β1,β2∈(0,1), solve

[0073]

[0074] And obtain the optimal solution (b, λ), block 210 .

[0075] Indicates I N :={i∈N:b i =1},λ=T(λ), K N :=N\(I N ∪J N ). In block 220, if |I N │ / │N│≥1-β1, then choose N / I N As a training set, box 230. At box 240, if |J N │≥β2│N│≥, then choose J N As a training set, block 250. Otherwise, in block 260, for K N, perform hyperplane data selection, and pick the first β2│N│-│J N │Point, and J N as the selected training set.

[0076] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be quickly provisioned and released with minimal management effort or interaction with the service provider. The cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0077] Features are as follows:

[0078] On-demand self-service: Cloud consumers can unilaterally and automatically provision computing capabilities, such as server time and network storage, as needed, without requiring human interaction with the service provider.

[0079] Broad network access: Capabilities are available over the network and accessed through standard mechanisms that facilitate the use of heterogeneous thin-client or thick-client platforms (e.g., mobile phones, laptops, and PDAs).

[0080] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically assigned and reassigned as needed. There is a sense of location independence, as consumers typically do not have control or knowledge of the exact location of the provided resources, but may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0081] Rapid elasticity: The ability to quickly and elastically provision capacity, in some cases automatically scaling down quickly and releasing quickly to scale up quickly. To the consumer, the capacity available for provisioning typically appears unlimited and can be purchased in any quantity at any time.

[0082] Metered Services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the utilized services.

[0083] The service model is as follows:

[0084] Software as a Service (SaaS): The ability provided to consumers is to use the provider's applications running on a cloud infrastructure. Applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0085] Platform as a Service (PaaS): The capability provided to consumers is to deploy applications created or acquired using programming languages ​​and tools supported by the provider onto cloud infrastructure. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but do have control over the deployed applications and the configuration of the application hosting environment.

[0086] Infrastructure as a Service (IaaS): The capabilities provided to consumers are processing, storage, networking, and other basic computing resources on which consumers can deploy and run arbitrary software, including operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but rather have control over the operating system, storage, deployed applications, and potentially limited control over selected networking components (e.g., host firewalls).

[0087] The deployment model is as follows:

[0088] Private cloud: Cloud infrastructure is operated solely for an organization. It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0089] Community cloud: Cloud infrastructure is shared by several organizations and supports a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0090] Public cloud: Cloud infrastructure is made available to the public or large industry groups and is owned by the organization that sells cloud services.

[0091] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain unique entities but are bound together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0092] Cloud computing environments are service-oriented and focus on statelessness, low coupling, modularity, and semantic interoperability. The core of cloud computing is the infrastructure that consists of a network of interconnected nodes.

[0093] Now refer to Figure 4, an illustrative cloud computing environment 50 is shown. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 with which local computing devices used by cloud consumers can communicate, and the local computing devices are, for example, mobile and / or wearable electronic devices 54A, desktop computers 54B, laptop computers 54C and / or automobile computer systems 54N. The nodes 10 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 50 to provide infrastructure, platforms, and / or software as a service for which cloud consumers do not need to maintain resources on local computing devices. It should be understood that Figure 4 The types of computing devices 54A-N shown in are intended for illustration only, and computing node 10 and cloud computing environment 50 may communicate with any type of computing device over any type of network and / or network-addressable connection (eg, using a web browser).

[0094] Now refer to Figure 5 , showing the cloud computing environment 50 ( Figure 4 ) provides a set of functional abstraction layers. It should be understood in advance that Figure 5 The components, layers, and functions shown in are intended to be illustrative only, and embodiments of the present invention are not limited thereto. As depicted, the following layers and corresponding functions are provided:

[0095] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: host 61; server 62 based on RISC (Reduced Instruction Set Computer) architecture; server 63; blade server 64; storage device 65; and network and network components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0096] Virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 71 ; virtual storage 72 ; virtual networks 73 , including virtual private networks; virtual applications and operating systems 74 ; and virtual clients 75 .

[0097] In one example, the management layer 80 may provide the functionality described below. Resource provisioning 81 provides dynamic procurement of computing and other resources for performing tasks within the cloud computing environment. Metering and pricing 82 provides cost tracking when utilizing resources within the cloud computing environment and bills or invoices the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides cloud computing resource allocation and management so that required service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides pre-arrangement and procurement of cloud computing resources in anticipation of future demand according to the SLA.

[0098] The workload layer 90 provides examples of functionality that can take advantage of a cloud computing environment. Examples of workloads and functionality that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analytics processing 94; transaction processing 95; and ODT building and optimization processing 96.

[0099] Figure 6 Details are described of an exemplary computing system capable of implementing aspects of the invention. Figure 6 A high-level block diagram computer system 600 that can be used to implement one or more aspects of the present invention is shown. The computer system 600 can act as a media device and implement all of the present invention, or it can cooperate with other computers and cloud-based systems to implement the present invention. More specifically, the computer system 600 can be used to implement some hardware components of embodiments of the present invention. Although an exemplary computer system 600 is shown, the computer system 600 includes a communication path 655 that connects the computer system 600 to additional systems (not shown) and can include one or more wide area networks (WANs) and / or local area networks (LANs), such as the Internet, an intranet, and / or a wireless communication network. The computer system 600 and the additional systems communicate via the communication path 655, for example, to transfer data between them.

[0100] Computer system 600 includes one or more processors, such as processor 605. Processor 605 is connected to a communication infrastructure 660 (e.g., a communication bus, a crossbar, or a network). Computer system 600 may include a display interface 615 that forwards graphics, text, and other data from communication infrastructure 660 (or from a frame buffer, not shown) for display on display unit 625. Computer system 600 also includes main memory 610, preferably random access memory (RAM), and may also include secondary memory 665. Secondary memory 665 may include, for example, a hard drive 620 and / or a removable storage drive 630, which may represent, for example, a floppy disk drive, a magnetic tape drive, or an optical disk drive. Removable storage drive 630 reads from and / or writes to a removable storage unit 640 in a manner well known to those skilled in the art. Removable storage unit 640 may represent, for example, a floppy disk, a compact disk, a magnetic tape, or an optical disk, which is read from and written to by removable storage drive 630. As will be appreciated, removable storage unit 640 contains computer-readable media having computer software and / or data stored therein.

[0101] In alternative embodiments, the secondary memory 665 may include other similar devices for allowing computer programs or other instructions to be loaded into the computer system. Such devices may include, for example, a removable storage unit 645 and an interface 635. Examples of such devices may include program packages and package interfaces (such as those found in video game devices), removable memory chips (such as EPROM or PROM) and associated sockets, and other removable storage units 645 and interfaces 635 that allow software and data to be transferred from the removable storage unit 645 to the computer system 600. In addition, the camera 670 communicates with the processor 605, the main memory 610, and other peripheral devices and storage devices via the communication interface 660.

[0102] The computer system 600 may also include a communication interface 650. The communication interface 650 allows software and data to be transferred between the computer system and external devices. Examples of the communication interface 650 may include a modem, a network interface (e.g., an Ethernet card), a communication port, or a PCM-CIA slot and card, etc. The software and data transmitted via the communication interface 650 are in the form of signals, which may be, for example, electronic, electromagnetic, optical, or other signals capable of being received by the communication interface 650. These signals are provided to the communication interface 650 via a communication path (i.e., a channel) 655. The communication path 655 transmits the signals and may be implemented using wires or cables, optical fibers, telephone lines, cellular phone links, RF links, and / or other communication channels.

[0103] In this specification, the terms "computer program medium," "computer-usable medium," and "computer-readable medium" are used to generally refer to media such as main memory 610 and secondary memory 665, removable storage drive 630, and a hard disk installed in hard disk drive 620. Computer programs (also known as computer control logic) are stored in main memory 610 and / or secondary memory 665. Computer programs may also be received via communication interface 650. When executed, such computer programs enable the computer system to perform the features of the present invention as discussed herein. In particular, when executed, the computer programs enable processor 605 to perform the features of the computer system. Thus, such computer programs represent controllers of the computer system.

[0104] Many functional units described in this specification are labeled as modules. Embodiments of the present invention are applicable to various module implementations. For example, a module can be implemented as a hardware circuit including custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module can also be implemented in programmable hardware devices, such as field programmable gate arrays, programmable array logic, programmable logic devices, and the like.

[0105] Modules may also be implemented in software executed by various types of processors. The executable code identifying a module may, for example, comprise one or more physical or logical blocks of computer instructions, which may, for example, be organized as objects, procedures, or functions. However, the executable code identifying a module need not be physically located together, but may comprise different instructions stored in different locations that, when logically combined together, constitute the module and achieve the module's stated purpose.

[0106] The present invention may be a system, method and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or multiple media) having computer-readable program instructions thereon, the computer-readable program instructions being used to cause a processor to perform various aspects of the present invention.

[0107] Computer-readable storage media can be a tangible device that can retain and store instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanical encoding device such as a punch card, or a protruding structure in a groove with instructions recorded thereon, and any suitable combination thereof. As used herein, computer-readable storage media should not be interpreted as a temporary signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse passing through an optical fiber cable), or an electrical signal emitted by a wire.

[0108] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network), or downloaded to an external computer or external storage device. The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium within the corresponding computing / processing device.

[0109] The computer-readable program instructions for performing the operation of the present invention can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, the configuration data of integrated circuit or source code or object code written in any combination of one or more programming languages, these programming languages ​​include object-oriented programming languages ​​(such as Smalltalk, C++ etc.) and process programming languages ​​(such as " C " programming languages ​​or similar programming languages). The computer-readable program instructions can be performed completely on the user's computer, partly on the user's computer, performed as an independent software package, partly on the user's computer, partly on a remote computer or fully on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer by any type of network (including local area network (LAN) or wide area network (WAN)), or can be connected to an external computer (for example, using an internet service provider through the internet). In certain embodiments, the electronic circuit comprising for example programmable logic circuit, field programmable gate array (FPGA) or programmable logic array (PLA) can make the electronic circuit personalized to perform computer-readable program instructions by utilizing the state information of computer-readable program instructions, so as to perform various aspects of the present invention.

[0110] The present invention will be described below with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0111] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the functions / actions specified in the flowchart and / or block diagram or multiple blocks. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to operate in a specific manner. Thus, the computer-readable storage medium having the instructions stored therein includes an article of manufacture containing instructions that implement aspects of the functions / actions specified in the flowchart and / or block diagram or multiple blocks.

[0112] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable apparatus, or other device to produce computer-implemented processing, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in the flowchart and / or block diagram or multiple boxes.

[0113] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each frame in the flow chart or block diagram can represent a module, segment or part of an instruction, which includes one or more executable instructions for realizing the specified logical function. In some alternative embodiments, the function noted in the frame may not occur in the order noted in the figure. For example, the two frames shown in succession can actually be performed substantially simultaneously, or these frames can sometimes be performed in reverse order, depending on the function involved. It will also be noted that the combination of each frame of the block diagram and / or flow chart illustration and the frame in the block diagram and / or flow chart illustration can be realized by a dedicated hardware-based system that performs a specified function or action or performs a combination of special-purpose hardware and computer instructions.

[0114] The description of various embodiments of the present invention has been provided for the purpose of illustration, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, practical applications, or improvements over existing technologies in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments described herein.

Claims

1. A computer-implemented method for predictive modeling, the method comprising: receiving a plurality of data inputs from a process as a training set using a processor; Using the processor to select a data subset I from the plurality of data inputs by solving a linear programming O To obtain a solution, where the data subset I O The data inputs with label y are correctly classified and assigned to the same leaf node, and wherein the data subset I O The convex hull conv({x i │i∈I O ) with the label y are discarded from the training set, where the x i is the data input that is discarded; Using the processor based on the data subset I O To construct the optimal decision tree ODT; and When the prediction of the optimal decision tree is greater than a threshold, the processor is used to alert a user. 2 . The computer-implemented method of claim 1 , further comprising recommending, using the processor, a control action when the prediction of the optimal decision tree is greater than the threshold.

3. A computer-implemented method according to claim 1 or 2, wherein selecting a data subset from the plurality of data inputs using the processor comprises performing a hyperplane data selection using the processor to select the data subset. 4 . The computer-implemented method of claim 1 , further comprising performing, using the processor, data selection within each leaf node of the ODT to select the data subset.

5. The computer-implemented method of claim 1 or 2, further comprising using the processor to enhance the ODT using a set of cutting planes.

6. The computer-implemented method of claim 1 or 2, wherein selecting, using the processor, a subset of data from the plurality of data inputs comprises selecting the subset of data such that points within a convex hull of the subset of data are maximized.

7. The computer-implemented method of claim 1 or 2, wherein the ODT comprises a multivariate decision tree.

8. A system for predictive modeling, the system comprising: Memory; a processor communicatively coupled to the memory, the processor operable to execute instructions stored in the memory, the instructions causing the processor to: receiving a plurality of data inputs from a process as a training set; A solution is obtained by selecting a data subset I0 from the plurality of data inputs by solving a linear programming, wherein, The data subset I O The data input with label y in is correctly classified and assigned to the same leaf node, and Among them, the convex hull conv({x i All data inputs with the label y within │i∈I0) are discarded from the training set, where the x i is the data input that is discarded; Constructing an optimal decision tree ODT based on the data subset; and The user is alerted when the prediction of the optimal decision tree is greater than a threshold. 9 . The system of claim 8 , wherein the instructions further cause the processor to recommend a control action when the prediction of the optimal decision tree is greater than the threshold.

10. The system of claim 8 or 9, wherein selecting a subset of data from the plurality of data inputs comprises performing hyperplane data selection to select the subset of data.

11. The system of claim 8 or 9, wherein the instructions further cause the processor to perform data selection within each leaf node of the ODT to select the data subset.

12. The system of claim 8 or 9, wherein the instructions further cause the processor to enforce the ODT using a set of cutting planes.

13. The system of claim 8 or 9, wherein selecting a data subset from the plurality of data inputs comprises selecting the data subset such that points within a convex hull of the data subset are maximized.

14. The system of claim 8 or 9, wherein the ODT comprises a multivariate decision tree.

15. A computer program product for predictive modeling, the computer program product comprising: Instructions for execution by a processing circuit for performing the method according to any one of claims 1 to 7.

16. A computer-readable medium having stored thereon computer program instructions loadable into an internal memory of a computer, wherein when the computer program instructions are executed on the computer, the computer program instructions are used to execute the method according to any one of claims 1 to 7.

Images