Systems, computer programs, and methods for entity linking in logical neural networks

Abstract

Systems, computer program products, and methods are provided for entity linking in a logical neural network (LNN). A set of features is generated for one or more entity-mention pairs in an annotated dataset. The generated set of features is evaluated against an entity linking LNN rule template that includes one or more logically connected rules and corresponding connector weights structured in a tree structure. An artificial neural network is utilized along with a corresponding machine learning algorithm to learn the connector weights. Connector weights associated with the logically connected rules are selectively updated, and a learned model is generated using the learned thresholds and learned weights for the logically connected rules.

Description

[Technical Field]

[0001] The present embodiments relate to computer systems, computer program products, and computer-implemented methods that use artificial intelligence (AI) and machine learning to disambiguate mentions in text by linking them to entities in a knowledge graph. More particularly, the present embodiments are directed to entity linking in boolean neural networks that use interpretable rules and learn corresponding connective weights and rules. [Background technology]

[0002] Entity linking is the task of disambiguating textual mentions by linking them to canonical entities provided by a knowledge graph. Common approaches target long texts consisting of multiple sentences, extracting features that measure some degree or similarity between the mentions and one or more candidate entities, and a non-learning heuristic disambiguation step to link the mentions to the actual entities. The challenge in entity linking targets short texts, such as single sentences or questions, and limited context around the mentions. Platforms that support short texts include conversational systems such as chatbots. The embodiments shown and described herein are directed to artificial intelligence (AI) platforms and entity linking to mitigate challenges associated with short texts and their corresponding platforms. Summary of the Invention

[0003] Embodiments disclosed herein include computer systems, computer program products, and computer-implemented methods for disambiguating mentions in text by linking them to entities in a logical neural network using interpretable rules. These embodiments are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features or concepts of the claimed subject matter, nor is it intended to be used in any way to limit the scope of the claimed subject matter.

[0004] In one aspect, a computer system includes a processor operatively coupled to a memory and an artificial intelligence (AI) platform operatively coupled to the processor. The AI ​​platform comprises a feature manager, an evaluator, and a machine learning (ML) manager configured with functionality for supporting entity linking in a logical neural network (LNN). The feature manager is configured to generate a set of features for one or more entity-mention pairs in an annotated dataset. The evaluator operatively coupled to the feature manager is configured to evaluate the generated set of features against an entity linking LNN rule template including one or more logically connected rules structured in a hierarchical structure and corresponding connector weights. The ML manager operatively coupled to the evaluator is configured to learn the connector weights using an artificial neural network and a corresponding ML algorithm. The ML manager is further configured to selectively update the connector weights associated with the logically connected rules. A trained model is generated using the learned thresholds and learned connector weights for the logically connected rules.

[0005] In another aspect, a computer program product includes a computer-readable storage medium having program code embodied thereon. The program code is executable by a processing unit and has functionality for generating a set of features for one or more entity-mention pairs in an annotated dataset. The generated set of features is evaluated against an entity-linking LNN rule template that includes one or more hierarchically structured logically connected rules and corresponding connector weights. The program code supports functionality for learning the connector weights utilizing an artificial neural network and corresponding machine learning algorithm. The connector weights associated with the logically connected rules are selectively updated, and a trained model is generated using the learned thresholds and learned connector weights for the logically connected rules.

[0006] In yet another aspect, a method is provided in which a feature set is generated for one or more entity-mention pairs in an annotated dataset. The generated feature set is evaluated against an entity-linking LNN rule template that includes one or more hierarchically structured logically connected rules and corresponding connector weights. An artificial neural network is utilized with a corresponding machine learning algorithm to learn the connector weights. The connector weights associated with the logically connected rules are selectively updated, and a trained model is generated using the learned thresholds and learned connector weights for the logically connected rules.

[0007] These and other features and advantages will become apparent from the following detailed description of the presently preferred embodiments taken in conjunction with the accompanying drawings.

[0008] The drawings referenced in this specification form a part of this specification. Features shown in the drawings are intended to be illustrative of only some embodiments and not all embodiments, unless expressly stated otherwise. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram illustrating a computer system including tools for supporting a neurosymbolic solution to entity linking applied to short text situations in an example embodiment. [Figure 2] FIG. 2 is a block diagram illustrating the tools and their associated APIs shown in FIG. 1. [Figure 3A] 1 is a flow chart showing the thresholding operation and the process for learning weights in the entity linking algorithm. [Figure 3B] 1 is a flow chart showing the thresholding operation and the process for learning weights in the entity linking algorithm. [Figure 3C] 1 is a flow chart showing the thresholding operation and the process for learning weights in the entity linking algorithm. [Figure 4] 1 is a flow chart illustrating a process for learning new rules, including appropriate weights for logical connectives, using an LNN. [Figure 5] FIG. 1 is a block diagram illustrating an exemplary LNN reconstruction for the EL algorithm. [Figure 6] FIG. 6 is a block diagram illustrating an example of a computer system / server of a cloud-based support system for implementing the systems and processes described above with respect to FIGS. 1-5. [Figure 7] FIG. 1 is a block diagram illustrating a cloud computing environment. [Figure 8] FIG. 1 is a block diagram illustrating a series of functional abstraction model layers provided by a cloud computing environment. DETAILED DESCRIPTION OF THE INVENTION

[0010] It will be readily understood that the components of the present embodiments, as generally described and illustrated in the Figures herein, could be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the present apparatus, system, method, and computer program product embodiments as illustrated in the Figures is not intended to limit the scope of the claimed embodiments, but is merely representative of selected embodiments.

[0011] Throughout this specification, references to "selected embodiments," "one embodiment," or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment. Thus, the appearances of the phrases "selected embodiments," "in one embodiment," or "in an embodiment" in various places throughout this specification are not necessarily referring to the same embodiment.

[0012] The example embodiments will be best understood by referring to the drawings, in which like parts are designated by like numerals throughout. The following description is intended to be merely exemplary and merely sets forth certain selected embodiments of devices, systems, and processes consistent with embodiments claimed herein.

[0013] Artificial intelligence (AI) relates to the field of computer science that focuses on computers and computer behavior as they relate to humans. AI refers to intelligence when a machine can make informed decisions that maximize the chances of success in a particular topic. More specifically, AI can learn from data sets to solve problems and provide appropriate recommendations. For example, in the field of artificial intelligence computer systems, natural language (NL) systems (such as the IBM Watson® artificial intelligence computer system or other natural language question-answering systems) process NL based on knowledge acquired by the system.

[0014] In the field of AI computer systems, natural language processing (NLP) systems process natural language based on acquired knowledge. NLP is a field of AI that serves as a translation platform between computer language and human language. More specifically, NLP enables computers to analyze and understand human language. Natural language understanding (NLU) is a category of NLP that focuses on parsing and transforming input according to natural language principles. Examples of such NLP systems are the IBM Watson® artificial intelligence computer system and other natural language question-answering systems.

[0015] Machine learning (ML), a subset of AI, utilizes algorithms to learn from data and make predictions based on this data. ML is the application of AI through the creation of models, such as artificial neural networks, that can exhibit learning behavior by performing tasks for which they were not explicitly programmed. There are many different types of ML, including learning problems such as supervised, unsupervised, and reinforcement learning; hybrid learning problems such as semi-supervised, self-supervised, and multi-instance learning; statistical inference such as inductive, deductive, and transductive learning; and learning techniques such as multi-task learning, active learning, online learning, transfer learning, and ensemble learning.

[0016] At the heart of AI and related reasoning is the concept of similarity. Structures, including static and dynamic structures, dictate a determined output or behavior for specific, limited inputs. More specifically, the determined output or behavior is based on well-defined or inherent relationships within the structure. This configuration may be sufficient for selected environments and conditions. However, it is understood that dynamic structures inherently undergo change, and correspondingly, outputs or behaviors may undergo change. Existing solutions for efficiently identifying objects, understanding NL, and processing content in response to this identification and understanding and changes to the structure are extremely difficult to implement at a practical level.

[0017] Artificial neural networks (ANNs) are models of how the nervous system operates. The basic unit is called a neuron and is typically structured into layers. ANNs function by simulating a large number of interconnected processing units that resemble abstract versions of neurons. ANNs typically have three parts: an input layer with units representing input fields, one or more hidden layers, and an output layer with one or more units representing target fields. These units are connected with various connection strengths or weights. Input data is presented to the first layer, and values ​​are propagated from each neuron to neurons in the next layer. Essentially, each layer of a neural network contains one or more operators or functions operatively coupled to the output and input. The output of evaluating the activation function for each neuron provided with input is referred to herein as activation. Complex neural networks are designed to emulate how the human brain functions, so computers can be trained to support poorly defined abstractions and problems for which training data is available. ANNs are often used in image recognition, speech, and computer vision applications.

[0018] Natural language processing (NLP) is a field of AI and linguistics that studies the problems inherent in processing and manipulating natural language, with the goal of improving computers' ability to understand human language. NLP focuses on extracting meaning from unstructured data.

[0019] Entity linking (EL) is considered here as the task of disambiguating (e.g., removing uncertainty) textual mentions by linking them to canonical entities provided by a knowledge graph (KG). A text or textual data T consists of a set M = {m1, m2, ...} of mentions, where each mention m i is contained in the text data T. A knowledge graph (KG) consists of a set of entities ε, and in this specification, each individual entity in the set of entities is called e ij Entity linking is called i is a many-to-one function that links e ∈ M to entities in KG. More precisely, linking ij ∈C i Targeting C i is a mention m i is the subset of candidates ε associated with

[0020] A logic neural network (LNN) is a neurosymbolic framework designed to simultaneously provide key properties of both neural networks (NNs) and symbolic logic (knowledge and reasoning). More specifically, an LNN functions to simultaneously provide the learning and symbolic properties of knowledge and reasoning. An LNN creates a direct correspondence between artificial neurons and logic elements using the observation that the weights of logic neurons are constrained to function as logical AND or OR gates. The LNN shown and described employs rules expressed in first-order logic (FOL), a symbolic inference in which each sentence or statement is decomposed into a subject and a predicate. Each rule is a disambiguation model that captures specific properties of linking. Given a rule template, the predicates appearing in the rule and the rule's parameters, in the form of a thresholding operation on the predicate weights, are subject to learning based on a labeled dataset. Thus, an LNN learns the rule's parameters and enables and implements parameter tuning.

[0021] Structurally, an LNN is a graph composed of the syntax trees of all represented expressions, connected to each other through an additional neuron for each proposition. Specifically, there is one neuron for each logical operation that appears in each expression, plus one neuron for each unique proposition that appears in every expression. All neurons return a pair of values ​​in the range [0,1] that represent lower and upper bounds on the truth value of their corresponding subexpressions and propositions.

[0022] LNN uses FOL semantics to enforce constraints when learning operators. Examples of such operators include, but are not limited to, logical AND, denoted herein as LNN-∧, and logical OR, denoted herein as LNN-∨. Logical AND (LNN-∧) is represented as follows: max(0,min(1,β-w1(1-x)-w2(1-y))) The following constraints apply: β-1(1-α)(w1+w2)≧α Constraint 1 β-αw1≦1-α Constraint 2 β-αw2≦1-α Constraint 3 w1,w2≧0 where β, w1, w2 are learnable parameters, x, y∈[0,1] are inputs, and α∈[1 / 2,1] is a hyperparameter.Similar to logical AND, logical OR is defined in terms of logical AND as follows: LNN-∨(x,y)=1-LNN-∧(1-x,1-y) Traditionally, Boolean logic only returns 1 or true when both inputs are 1. LNNs relax Boolean conjunctions (e.g., logical AND) by using α as a surrogate for 1 and 1-α as a surrogate for 0. Constraint 1 enforces that the output of logical AND is greater than α when both inputs are greater than α. Similarly, Constraints 2 and 3 constrain the behavior of logical AND when one input is low and the other is high. More specifically, Constraint 2 enforces that the output of logical AND is less than 1-α when y=1 and x≦1-α. This formulation allows for unconstrained learning when x, y∈[1-α,α]. Control over the extent of learning may be obtained by varying α. In example embodiments, constraints (e.g., Constraint 1, Constraint 2, and Constraint 3) may be relaxed.

[0023] A feature is herein considered an attribute that measures the degree of similarity between a text mention and a candidate entity. In an example embodiment, the features are generated using a catalog of feature functions, including non-embedding and embedding-based functions. As shown and described herein, mention m i and candidate entity e ij A set of exemplary non-embedding based feature functions is provided to measure the similarity between the mentions m i The name of the candidate entity e ijA set of general-purpose similarity functions, such as, but not limited to, Jaccard, Jaro Winkler, Levenshtein, and Partial Ratio, for computing the similarity between the names of candidate entities e ij Description and mention of m i In the example embodiment, the context feature Ctx is evaluated as follows:

[0024]

number

[0025] As shown and explained in Figures 1-5, the Entity Linking (EL) algorithm, consisting of a disjunctive set of rules, is reconstructed into an LNN representation for training. Entity Linking is a restricted form of First-Order Logic (FOL) rules that contains a set of Boolean predicates connected by logical operators in the form of logical AND (∧) and logical OR (∨). The Boolean predicates are k >θ, and f kwhere ∈F is one of the feature functions and θ is the learned thresholding operation. Two example entity linking rules are shown below: R1(m i ,e ij )←jacc(m i ,e ij )>θ1∧Ctx(m i ,e ij )>θ2 R2(m i ,e ij )←lev(m i ,e ij )>θ3∧Prom(m i ,e ij )>θ4 Based on these examples, the predicate jacc(m i ,e ij )>θ1 and the predicate Ctx(m i ,e ij ) > θ2 are both true, the first example rule R1(m i ,e ij ) evaluates to true, and the predicate lev(m i ,e ij )>θ3 and the predicate Prom(m i ,e ij ) > θ4 are both true, the second example rule R2(m i ,e ij ) evaluates to true. In example embodiments, rules such as the exemplary first and second rules may be disjuncted together to form larger EL algorithms. An example of such an expansion is shown below: Links(m i ,e ij )←R1(m i ,e ij )∨R2(m i ,e ij ) Here, if either the first or second rule evaluates to true, then Links(m i ,e ij) evaluates to true. In the example embodiment, the Links predicate represents a disjunction between at least two rules and serves to store high-quality links between mentions and candidate entities that pass the conditions of at least one rule.

[0026] The EL algorithm also serves as a scoring mechanism. Examples of exemplary first and second rule-based scoring functions are shown below:

[0027]

number

[0028] Referring to FIG. 1, a block diagram (100) is provided illustrating a computer system including tools for supporting a neurosymbolic solution to entity linking applied to short text contexts in an example embodiment. Generally, entity linking extracts features that measure some degree of similarity between a text mention and any one of multiple candidate entities. In an example embodiment, the short text targets a single sentence or question. A challenge associated with effective techniques in short text environments is the limited context surrounding the mention. The system and associated tools, as described herein, combine logic rules and learning, and use gradient-based techniques to facilitate combining multiple types of EL features with interpretability and learning. As shown in the figure, a server (110) is provided in communication with multiple computing devices (180), (182), (184), (186), (188), and (190) via a network connection (105). The server (110) comprises a processing unit (112) operatively coupled to a memory (114) via a bus (116). A tool in the form of an artificial intelligence (AI) platform (150) is shown local to the server (110) and operatively coupled to the processing unit (112) and the memory (114). As shown, the AI ​​platform (150) includes tools in the form of a feature manager (152), an evaluator (154), a machine learning (ML) manager (156), and a rule manager (158). Together, these tools provide functional support for entity linking via a network (105) from one or more computing devices (180), (182), (184), (186), (188), and (190).The computing devices (180), (182), (184), (186), (188), and (190) communicate with each other and with other devices or components via one or more wired and / or wireless data communication links, each of which may include one or more wires, routers, switches, transmitters, receivers, etc. In this networked arrangement, the server (110) and network connection (105) enable the generation of features and application of the generated features to an EL algorithm consisting of a disjunctive set of rules reconstructed into an LNN representation for training. Other embodiments of the server (110) may be used with components, systems, subsystems, or devices, or combinations thereof, other than those shown herein.

[0029] The tools including the AI ​​platform (150), or tools incorporated into the AI ​​platform (150), including, in one embodiment, the feature manager (152), evaluator (154), ML manager (156), and rule manager (158), may be configured to receive input from a variety of sources, including, but not limited to, input from the network (105) and an operably coupled knowledge base (160). As illustrated herein, the knowledge base (160) is referred to herein as a dataset. 0,0 (164 0,0 ), dataset 0,1 (164 0,1 ), ..., Dataset 0,N (164 0,N ) of annotated datasets. The number of datasets in the first library (1620) is for illustrative purposes and should not be considered limiting. Similarly, in example embodiments, knowledge base (160) may include one or more additional libraries, each containing one or more datasets. As such, the number of libraries shown and described herein should not be considered limiting.

[0030] Various computing devices (180), (182), (184), (186), (188), and (190) in communication with the network (105) represent access points for the AI ​​platform (150) and corresponding tools (e.g., managers and evaluators, including the feature manager (152), the evaluator (154), the ML manager (156), and the rule manager (158). Some of the computing devices may include devices for use by the AI ​​platform (150), and in one embodiment, the tools (152), (154), (156), and (158), to generate trained models including learned thresholding operations and weights for logical connectives and to support dynamically generating templates for application of the trained models. In various embodiments, the network 105 may include local network connections and remote connections so that the AI ​​platform 150 and the embedded tools 152, 154, 156, and 158 may operate within environments of any size, including local and global (e.g., the Internet). Thus, the server 110 and the AI ​​platform 150 serve as a front-end system, and the knowledge base 160, and one or more of the libraries and datasets, serve as a back-end system.

[0031] Data annotation is the process of adding metadata to a dataset, effectively labeling relevant datasets and allowing ML algorithms to leverage corresponding existing data classifications. As described in more detail below, the server (110) and AI platform (150) may store libraries (e.g., library (1620)) and corresponding datasets (e.g., dataset 0,1 (164 0,1 ))). In an example embodiment, the annotated data is a set of entity-mention pairs (m i ,e ij), each of these pairs including a corresponding label. Similarly, in embodiments, the annotated dataset may be transmitted via the network (105) from one or more of the operatively coupled machines or systems. The AI ​​platform (150) utilizes a feature manager (152) to generate a set of features for one or more of the entity-mention pairs in the annotated dataset. In example embodiments, for a subset of the labeled entity-mention pairs, mention m i and candidate entity e ij Features are generated using a catalog of feature functions, including non-embedding and embedding-based functions, to measure (e.g., calculate) the similarity between m and m, each of which includes a corresponding similarity predicate. Examples of such features are mentions m i The name of the candidate entity e ij Name features for calculating the similarity between the names of candidate entities e ij Mention m in the description i Context features for assessing the aggregate similarity of contexts, e ij Domain set and mention m i The type features as overlaps of type similarities of, as well as the candidate entities in the target knowledge graph, e ij Candidate entity e as the number of entities that link to ij These include, but are not limited to, entity salience features for measuring the salience of a mention. Thus, a first aspect is directed to a similarity assessment of candidate entity-mention pairs, which assessment produces a quantified characteristic.

[0032] The evaluator (154), shown herein operatively coupled to the feature manager, subjects the generated features of the entity-mention pairs to entity linking (EL) logic neural network (LNN) rule templates. More specifically, the evaluator (154) reconstructs an entity linking algorithm consisting of a disjunctive set of rules into an LNN representation. An exemplary LNN rule template (e.g., LNN representation) is shown and described in FIG. 5. In example embodiments, one or more LNN rule templates are provided in a knowledge base or otherwise communicated to the evaluator (154) via the network (105). By way of example, the knowledge base (160) may be a library (e.g., a template library) as used herein. 1,0 (164 1,0 ), template 1,1 (164 1,1 ), ..., template 1,M (164 1,M ) along with a second library of LNN rule templates (1621), shown as

[0046] . The number of rule templates in the second library (1621) is for illustrative purposes and should not be considered limiting. Similarly, in example embodiments, the knowledge base (160) may include one or more additional libraries, each containing one or more LNN rule templates. As shown by way of example in FIG. 5, an LNN rule template may be constructed as an inverted binary tree structure containing one or more logically connected rules and corresponding connective weights. This example rule template is relatively rudimentary. In example embodiments, the LNN rule template may be expanded using additional layers in the binary tree and expanded rules. Thus, as shown herein, generated features are evaluated against the selected or identified LNN rule template.

[0033] The LNN rule template may be constructed as an inverted binary tree, with a subset of features or feature functions represented by the leaf nodes of the binary tree. Each feature is then scaled by a corresponding threshold θ, also referred to herein as the thresholding operation. i The internal nodes of the binary tree represent logical AND or OR operations. An edge is provided between each internal node and the thresholding operation and between each internal node and the root node. In an example embodiment, the binary tree may include multiple layers of internal nodes, with edges extending between adjacent layers of nodes. Each edge has a corresponding weight, referred to herein as a rule weight. Each of the thresholding operations and rule weights, collectively referred to herein as a connective weight, is subject to learning. As illustrated herein, the ML manager (156) operably coupled to the evaluator (154) is configured to utilize an ANN and corresponding ML algorithm to learn the thresholding operations and connective weights. With respect to the thresholding operation, the ML manager (156) learns a threshold appropriate for each calculated feature associated with the corresponding similarity predicate. The evaluator (154) interfaces with the ML manager (156) and filters one or more of the features based on the learned threshold. More specifically, this filtering is performed by allowing the evaluator (154) to determine whether to incorporate a feature into the LNN rule template, remove the feature, or assign a non-zero score to the feature.

[0034] A connector weight is identified and associated with each rule template. As shown by way of example in this specification, the template 1,0 (164 1,0 ) is used herein as a weight 1,0 (166 1,0 ), weight 1,1 (166 1,1 ), ..., weight 1,M (166 1,M ) and a set of weights for the combinators called templates (e.g., templates 1,1 (164 1,1 ) and templates1,M (164 1,M Each of the weights has a corresponding connective weight. The number and nature of the weights are based on the corresponding template. Similarly, in the example embodiment, the knowledge base (160) is an ANN, as used herein as an example. 2,0 (164 2,0 ), ANN 2,1 (164 2,1 ), ..., ANN 2,P (164 2,P ) in which reside ANNs. The number of ANNs shown herein is for illustrative purposes and should not be considered limiting. In embodiments, the ANNs may each include a corresponding or embedded ML algorithm. The thresholding operations and connecter weights, individually or collectively, are parameters that are subject to learning and selective updating by the ML manager (156). Details of this learning are shown and described below in FIG. 4. After learning and updating are complete, a trained model is generated that includes the learned thresholding operations and logical connecter weights.

[0035] As shown and described herein, rule templates including corresponding rules may be provided, and thresholding operations and connective weights may be trained to generate a learning model. In an example embodiment, given a set of features and an EL-annotated dataset, new rules including appropriate weights for logical connectives may be trained. A rule manager (158), shown operatively coupled to the evaluator (154) herein, is provided to support such functionality. More specifically, the rule manager (158) trains one or more of the connected rules, dynamically generates a binary tree template, and trains the logical rules associated with the template. After training, the rule manager (158) evaluates selected rules against the labeled dataset and selectively assigns the selected rules to corresponding nodes in the binary tree. The rule manager (158) selectively assigns a conjunction (e.g., logical AND) or disjunction (e.g., logical OR) operator to each internal node of the binary tree. The details of the Rule Manager's (158) functionality with respect to learning rules and assigning operators to nodes are shown and explained in FIG.

[0036] While the AI ​​platform (150) is shown embodied within or integrated with the server (110), it may be implemented within a separate computing system (e.g., 190) connected to the server (110) via a network (105). Similarly, the tools (152), (154), (156), and (158) are shown local to the server (110), but may be collectively or individually distributed via the network (105). The feature manager (152), evaluator (154), ML manager (156), and rule manager (158), wherever embodied, are utilized to support and enable LNN EL.

[0037] The types of information handling systems that can utilize the server (110) range from small handheld devices, such as handheld computers / cell phones (180), to large mainframe systems, such as mainframe computers (182). Examples of handheld computers (180) include personal digital assistants (PDAs) and personal entertainment devices, such as MP4 players, portable televisions, and compact disc players. Other examples of information handling systems include pen or tablet computers (184), laptop or notebook computers (186), personal computer systems (188), and servers (190). As shown, various information handling systems can be networked together using a computer network (105). Types of computer networks (105) used to interconnect various information handling systems include local area networks (LANs), wireless local area networks (WLANs), the Internet, public switched telephone networks (PSTNs), other wireless networks, and any other network topology used to interconnect information handling systems. Many information handling systems include a non-volatile data store, such as a hard drive or non-volatile memory, or both. Some information handling systems may use separate non-volatile data stores (e.g., a server (190) may use a non-volatile data store (190)). A ), and the mainframe computer (182) uses a non-volatile data store (182 A )). Non-volatile data store (182 A ) can be a component external to the various information handling systems or can be internal to one of the information handling systems.

[0038] An information processing system may take many forms, some of which are shown in Figure 1. For example, an information processing system may take the form of a desktop, server, portable, laptop, notebook, or other form factor computer or data processing system. In addition, an information processing system may select other form factors, such as a personal digital assistant (PAD), gaming device, ATM machine, mobile phone device, communications device, or other device that includes a processor and memory.

[0039] An application program interface (API) is understood in the art as intermediate software between two or more applications. With respect to the embodiment shown and described in FIG. 1 , one or more APIs may be utilized to support one or more of the AI ​​platform tools, including the feature manager (152), evaluator (154), ML manager (156), and rule manager (158), and their associated functionality. Referring to FIG. 2 , a block diagram (200) illustrating the AI ​​platform tools and their associated APIs is provided. As shown, multiple tools are incorporated within the AI ​​platform (205), including a feature manager (252) associated with API0 (212), an evaluator (254) associated with API1 (222), an ML manager (256) associated with API2 (232), and a rule manager (258) associated with API3 (242). Each of the APIs may be implemented in one or more languages ​​and interface specifications.

[0040] API 0 (212) provides support for generating feature sets for entity-mention pairs. API 1 (222) provides support for evaluating the generated features against EL LNN rule templates. API 2 (232) provides support for learned thresholding operations and connective weights within rule templates. API 3 (242) provides support for learning EL rules and selectively assigning learned rules to templates.

[0041] As shown in the figure, each of APIs 212, 222, 232, and 242 is operatively coupled to an API orchestrator 260 (otherwise referred to as an orchestration layer), which is understood in the art to act as an abstraction layer for transparently connecting separate APIs. In one embodiment, the functionality of separate APIs may be combined or combined. As such, the configuration of APIs shown herein should not be considered limiting. Accordingly, the functionality of the tools may be embodied or supported by their respective APIs, as shown herein.

[0042] Referring to Figures 3A-3C, a flow chart (300) illustrating a process for training thresholding operations and weights in an entity linking algorithm is provided. As shown, the entity linking (EL) algorithm comprises rules in the form of Boolean predicates connected by logical AND and OR operators (302). To facilitate and enable training of the thresholding operations and weights in the EL algorithm, Boolean-valued logic rules are mapped to an LNN format (304), which constructs logical ORs and ANDs in the LNN format and considers continuous real values ​​within [0, 1]. In an example embodiment, the LNN format may be an inverted tree structure with features assigned to leaf nodes, and entity linking rules are represented by internal nodes and a root node. Each LNN operator generates a value within [0, 1] based on the value of the input, the input weight, and the input bias β, where both the weight and bias are learnable parameters. An internal node of an LNN format, also referred to herein as an LNN rule template, consists of an external node operatively connected to the internal node via a corresponding link, where the external node represents a feature or feature node, and the internal node represents one of a logical AND, logical OR, or thresholding operation.

[0043] Feature weight thresholds and rule weights in the LNN format (e.g., LNN rule templates) are initialized (306). In example embodiments herein, feature weights and rule weights are collectively referred to as weights. Following the initialization in step (306), a subset S (e.g., triplets) of labeled mention-entity pairs in the labeled dataset L is selected or received (308). In example embodiments, the selection in step (308) is a random selection of mention-entity pairs. Each triplet is (m i ,e i ,y i ) and m i indicates a mention, and e i indicates the entity, and yi indicates a match or mismatch, where in a non-limiting example embodiment, 1 is a match and 0 is a mismatch. Total is assigned 310 to the number of selected triplets in the subset, and a corresponding triplet count variable S is initialized 312. The number of features in the inverted tree structure is known or determined, and the number of features is assigned to a variable F Total (314) F = 1 to F Total For each feature, mention m i and candidate entity e i In this specification, the feature function (feature F A similarity measure, also referred to as a feature measure, is calculated (316). Examples of feature measures include, but are not limited to, name, context, type, and entity salience, as previously discussed. As shown, in the example embodiment, a set of features, which are similarity predicates, are calculated for each entity-mention pair, and the set of features is calculated for each mention m i Let candidate entity e i Leverage one or more string similarity functions to compare

[0044] After the features are computed, each entity-mention pair is evaluated against an EL Logic Neural Network (LNN) rule template, which includes one or more logically connected rules and corresponding connective weights, and is structured in a binary tree, also referred to herein as a hierarchical structure. The binary tree is structured such that a root node is operatively connected to two or more internal nodes, which are operatively connected to leaf nodes at the last level of the binary tree. As shown herein, triplets are evaluated by the rule R to be trained. This evaluation is performed by evaluating the triplet triplet. S , and is processed through the tree in a bottom-up manner, starting with, for example, the leaf nodes that represent features. Each node in the tree is referred to herein as a vertex v, and each vertex may be a root node, an internal node, or a leaf node. The number of vertices in the tree is determined by a variable v Total v=1~vTotal For each vertex of, v It is determined whether , is a thresholding operation (320). Each feature is represented by a leaf node, and each feature has a corresponding or associated thresholding operation. After a positive response to the determination in step (320), the corresponding thresholding operation is calculated as follows: f i [1+exp(θ v -f i )] -1 and sending the result of the calculation to the next level upstream in the inverted tree structure (322). In an example embodiment, the evaluation in step (322) involves filtering the feature based on the corresponding learned threshold θ. As an example, the feature value f i If is 0.1, then [1+exp(θ v -f i )] -1 Depending on the value of θ, a value between 1 and 0.29 can be obtained. For example, v If f is 0.9, the thresholding operation evaluates to 0.3. Based on this value, i When multiplied by , this shrinks the output to a value close to zero, effectively removing the feature from consideration. Thus, filtering the features in step (322) effectively removes features or selectively incorporates features into the LNN rule template by assigning them a non-zero score.

[0045] If the response in step (320) is negative, v It is determined whether ∑ i = ...

[0046]

number

[0047]

number

[0048] After step (334), a triplet count variable S is incremented (336), and a determination is made (338) as to whether each of the triplets in the subset has been evaluated. After a negative response to this determination, the process returns to step (314) to evaluate the next triplet in the subset, and an affirmative response ends the first aspect of rule evaluation. More specifically, after a positive response to the determination in step (338), the subset S Total performing backpropagation (340), which includes computing gradients from all losses in Total Propagate the gradient of the parameter θ in the rule R v , β v , and

number

[0049]

number

[0050] As shown in Figures 3A-3C, an LNN is used to learn appropriate weights for logical connectives, given a set of rule templates, a set of features, and a labeled EL dataset. Referring to Figure 4, a flow chart (400) is provided illustrating a process for using an LNN to learn new rules, including appropriate weights for logical connectives. As previously mentioned, mention m i and candidate entity e ijA set of exemplary non-embedding-based feature functions is provided to measure the similarity between entities. The exemplary set includes name features, context features, type features, and entity salience features. Herein, a variable F is utilized to denote such feature partitioning (402). The input is in the form of a labeled dataset L (e.g., entity-mention pairs) and a feature partition F (404). The number of binary trees constructed with the number of leaves defined by |F| is evaluated by C(|F|-1), where C denotes the Catalan number (406). In the steps described below, it is assumed that a node contains one operation, along with the optional assignment of a logical AND or OR operator to the node. The following pseudocode illustrates the process of selecting and assigning a logical operator to the internal nodes of the binary tree: forall a binary tree T with |F| leaves forall Select LNN operation of internal nodes in T do R ← EL rule containing T (with selected operator) Evaluate R (against a validation set, e.g., a labeled dataset) if R is the best rule that has appeared so far then

number

number

[0051] Referring to FIG. 5, a block diagram (500) illustrating an exemplary LNN reconstruction of the EL algorithm is provided. As shown in this example, the reconstruction is an inverted tree structure including features and corresponding thresholds, logical operators, and associated weights. In this example, five features are shown. In example embodiments, a different number of features may be present in the reconstruction, and therefore the number shown and described herein should not be considered limiting. The five features, referred to herein as f0 (510), f1 (512), f2 (514), f3 (516), and f4 (518), are represented as individual leaf nodes in the inverted tree structure. Each of the features is shown with its corresponding threshold. More specifically, feature f0 (510) is shown operatively connected to a corresponding threshold operation θ0 (520), feature f1 (512) is shown operatively connected to a corresponding threshold operation θ1 (522), feature f2 (514) is shown operatively connected to a corresponding threshold operation θ2 (524), feature f3 (516) is shown operatively connected to a corresponding threshold operation θ3 (526), ​​and feature f4 (518) is shown operatively connected to a corresponding threshold operation θ4 (528). Each of the threshold operations is subject to training and is directly associated with one or more feature functions.

[0052] As further shown in the figure, the internal nodes of the inverted tree are 0,0 (530) and internal nodes 0,1 A first set of internal nodes, shown as (550), are operatively connected to the selection of features and their corresponding thresholds. 0,0 (530) is operatively connected to features f0 (510), f1 (512), and f2 (514), and has an internal node 0,1 (550) is operatively connected to features f3 (516) and f4 (518). Edges are shown operatively connecting the leaf nodes and their corresponding thresholds to a first set of interior nodes (530) and (550). Specifically, edges 0,0(532) maps the feature f0 (510) and the corresponding threshold θ0 (520) to the node 0,0 Operatively connected to (530) and edge 0,1 (534) maps the feature f1 (512) and the corresponding threshold θ1 (522) to the node 0,0 Operatively connected to (530) and edge 0,2 (536) maps the feature f2 (514) and the corresponding threshold θ2 (524) to the node 0,0 (530) and operatively connected to the edge 1,0 (552) maps feature f3 (516) and the corresponding threshold θ4 (526) to node 0,1 Connect to (550) and Edge 1,1 (554) maps the feature f5 (518) and the corresponding threshold θ5 (528) to the node 0,1 Connect to (550). Edge 0,0 (532), Edge 0,1 (534), Edge 0,2 (536), Edge 1,0 (552), and Edge 1,1 Each of the edges comprising (554) has a separate corresponding weight, which, like the threshold, is subject to training. In the example embodiment, these weights are called feature weights fw, and the edges 0,0 (532) has feature weight fw0, and edge 0,1 (534) has feature weight fw1, and edge 0,2 (536) has feature weight fw2, and edge 1,0 (552) has feature weight fw3, and edge 1,1 (554) has feature weight fw4. The second internal node (node 1,0 (560)) is an internal node 0,0 (530) and internal nodes 0,1 (550) is shown operatively coupled to the edge 2,0 (562) and Edge 2,1 The two edges containing (564) are connected to the second internal node (node 1,0 (560)). These edges (i.e., edges 2,0 (562) and Edge 2,1Each of the edges (564) has a corresponding weight, referred to herein as the rule weight rw. 2,0 (562) has rule weight rw0 and edge 2,1 (564) has rule weight rw1. Like feature weights and thresholds, rule weights are subject to learning.

[0053] In this example, each internal node 0,0 (530) and internal nodes 0,1 (550) represents the logical AND (∧) operation of the LNN, which in this example is the root node (node 1,0 The second internal node, also called (560), represents the logical OR (∨). For example, the internal node 0,0 The rule R1 associated with (530) is: R1:(f0>θ0)∧(f1>θ1)∧(f2>θ2) Here, R1 evaluates to true if f0>θ0 is true, f1>θ1 is true, and f2>θ2 is true. Similarly, for example, the internal node 0,1 The second rule (Rule R2) associated with (550) is: R2:(f3>θ3)∧(f4>θ4) Here, R2 evaluates to true if f3 > θ3 is true and f4 > θ4 is true. 1,0 (560)) is the root node of the inverted tree structure, and as shown here, the internal nodes 0,0 (530) and internal nodes 0,1 (550) combines the Boolean logic of the root node (node 1,0 Rule R3 in (560) is as follows: R1 ∨ R2 Here, if either the first rule R1 or the second rule R2 evaluates to true, then R3 evaluates to true.

[0054] Aspects of the tools 152, 154, 156, and 158, and their associated functionality, may be embodied in a computer system / server at a single location, or in embodiments, may be configured within a cloud-based system sharing computing resources. Referring to Figure 6, a block diagram 600 is provided illustrating an example of a computer system / server 602 (hereinafter referred to as host 602) in communication with a cloud-based support system to implement the systems and processes described above with respect to Figures 1-5. The host 602 is operational with numerous other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, or configurations, or combinations thereof, that may be suitable for use with the host (602) include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and file systems that include any of these systems, devices, and the like (e.g., distributed storage environments and distributed cloud computing environments).

[0055] The host (602) may be described in the general context of instructions executable by a computer system, such as program modules being executed by the computer system. Typically, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The host (602) may be executed in a distributed cloud computing environment (610) where tasks are performed by remote processing devices linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media, including memory storage devices.

[0056] As shown in FIG. 6, the host (602) is depicted in the form of a general-purpose computing device. The components of the host (602) may include, but are not limited to, one or more processors or processing units (604), a system memory (606), and a bus (608) that couples various system components, including the system memory (606), to the processor (604). The bus (608) represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. Examples of such architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a MicroChannel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnects (PCI) bus. The host (602) typically includes a variety of computer system-readable media. Such media can be any available media that is accessible by the host (602) and includes both volatile and non-volatile media, removable and non-removable media.

[0057] The memory (606) may include computer system-readable media in the form of volatile memory, such as random access memory (RAM) (630) and / or cache memory (632). By way of example only, a storage system (634) may be provided for reading from and writing to non-removable, non-volatile magnetic media (not shown, typically referred to as a "hard drive"). Although not shown, a magnetic disk drive may be provided for reading from and writing to removable, non-volatile magnetic disks (e.g., "floppy disks"), and an optical disk drive may be provided for reading from and writing to removable, non-volatile optical disks, such as CD-ROMs, DVD-ROMs, or other optical media. In such examples, each may be connected to the bus (608) by one or more data media interfaces.

[0058] For example, a program / utility (640) including a set of (at least one) program modules (642) may be stored in memory (606), including, but not limited to, an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data, or a combination thereof, may comprise an implementation of a network environment. The program modules (642) typically perform the functions and / or methods of an embodiment of entity linking within a logical neural network. For example, the set of program modules (642) may include modules configured as tools (152), (154), (156), and (158) described in FIG. 1.

[0059] The host (602) may communicate with one or more external devices (614), such as a keyboard, pointing device, sensory input device, sensory output device, a display (624), one or more devices that allow a user to interact with the host (602), or any device (e.g., a network card, modem, etc.) that allows the host (602) to communicate with one or more other computing devices, or a combination thereof. Such communication may occur through an input / output (I / O) interface 622. Additionally, the host (602) may communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), or a public network (e.g., the Internet), or a combination thereof, via a network adapter (620). As shown, the network adapter (620) communicates with the other components of the host (602) via a bus (608). In one embodiment, multiple nodes of a distributed file system (not shown) communicate with a host (602) through an I / O interface (622) or through a network adapter (620). Although not shown, it should be understood that other hardware and / or software components may be used with the host (602), including, but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archive storage systems.

[0060] In this document, the terms "computer program medium," "computer usable medium," and "computer readable medium" are used generally to refer to media such as main memory (606), including RAM (630), cache (632), and storage systems (634), such as removable storage drives and hard disks installed in hard disk drives.

[0061] Computer programs (also called computer control logic) are stored in the memory (606). The computer programs may be received via a communications interface, such as a network adapter (620). When executed, such computer programs enable the computer system to perform the features of the present embodiments described herein. In particular, when executed, the computer programs enable the processing unit (604) to perform the functions of the computer system. Thus, such computer programs represent the controller of the computer system.

[0062] In one embodiment, the host (602) is a node in a cloud computing environment. As known in the art, cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) and for rapidly provisioning and releasing these resources with minimal administrative effort or service provider interaction. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models. Examples of such characteristics are:

[0063] On-demand self-service: Cloud customers can automatically provision server time, network storage, and other computing power as needed, without requiring unilateral, human interaction with the service provider.

[0064] Wide network access: Cloud capabilities are available over the network and can be accessed using standard mechanisms, facilitating usage by heterogeneous thin- or thick-client platforms (e.g., mobile phones, laptops, and PDAs).

[0065] Resource Pool: The provider's computing resources are pooled and offered to multiple consumers using a multi-tenant model. Various physical and virtual resources are dynamically allocated and reallocated according to demand. There is a sense of location independence; consumers typically have no control or knowledge regarding the exact location of the resources offered, although higher layers of abstraction may allow for the location to be specified (e.g., country, state, or data center).

[0066] Rapid Elasticity: Cloud capacity can be quickly and elastically provisioned, in some cases automatically, to scale out quickly, and quickly released to scale in quickly. Capacity available for provisioning often appears to consumers as unlimited, available for purchase in any quantity at any time.

[0067] Metered Services: Cloud systems leverage metering capabilities to automatically control and optimize resource usage at whatever layer of abstraction is appropriate for the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the services used.

[0068] The service model is as follows:

[0069] SaaS (Software as a Service): The consumer is provided with the ability to use the provider's applications running on a cloud infrastructure. These applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or individual application features, except for the possibility of limited user-specific application configuration settings.

[0070] PaaS (Platform as a Service): The ability offered to a consumer is to deploy applications they create or acquire, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, the configuration of the application hosting environment.

[0071] Infrastructure as a Service (IaaS): The capability provided to a customer is the provisioning of processing, storage, network, and other basic computing resources, upon which the customer can deploy and run any software, which may include operating systems and applications. The customer does not manage or control the underlying cloud infrastructure, but rather controls the operating systems, storage, deployed applications, and in some cases, limited control over selected network components (e.g., host firewalls).

[0072] The deployment model is as follows:

[0073] Private Cloud: This cloud infrastructure is operated solely for the organization, can be managed by the organization or a third party, and can reside on-premise or off-premise.

[0074] Community Cloud: This cloud infrastructure is shared by multiple organizations to support a specific community with shared interests (e.g., mission, security requirements, policy, and compliance considerations). It can be managed by these organizations or a third party and can reside on-premises or off-premises.

[0075] Public Cloud: This cloud infrastructure is available for use by the general public or large industry organizations and is owned by an organization that sells cloud services.

[0076] Hybrid cloud: This cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain distinct but are joined together by standardized or proprietary technologies that allow for data and application portability (e.g., cloud bursting to balance load between clouds).

[0077] A cloud computing environment is a service-oriented environment that emphasizes statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that consists of a network of interconnected nodes.

[0078] Referring now to FIG. 7, an exemplary cloud computing network (700) is shown. As shown, the cloud computing network (700) includes a cloud computing environment (750) that includes one or more cloud computing nodes (710) with which local computing devices used by cloud subscribers may communicate. Examples of these local computing devices include, but are not limited to, a personal digital assistant (PDA) or mobile phone (754A), a desktop computer (754B), a laptop computer (754C), or an automotive computer system (754N), or combinations thereof. Individual nodes within the nodes (710) may further communicate with each other. The nodes (710) may be physically or virtually grouped together in one or more networks (not shown), such as a private cloud, community cloud, public cloud, or hybrid cloud, as previously described herein, or combinations thereof. This allows the cloud computing environment (700) to provide an infrastructure, platform, and / or SaaS that eliminates the need for cloud users to maintain resources on local computing devices. The types of computing devices (754A-N) shown in Figure 7 are intended as examples only, and it is understood that the cloud computing environment (750) can communicate with any type of computer-controlled device via any type of network and / or network-addressable connection (e.g., a connection using a web browser).

[0079] Referring now to FIG. 8, a set of functional abstraction layers (800) provided by the cloud computing network of FIG. 7 is shown. It should be understood that the components, layers, and functions shown in FIG. 8 are intended to be illustrative only and are not limiting. As shown, a hardware and software layer (810), a virtualization layer (820), a management layer (830), and a workload layer (840), along with corresponding functions, are provided. The hardware and software layer (810) includes hardware and software components. Examples of hardware components include mainframes (e.g., IBM® zSeries® systems), RISC (Reduced Instruction Set Computer) architecture-based servers (e.g., IBM pSeries® systems), IBM xSeries® systems, IBM BladeCenter® systems, storage devices, networks, and network components. Examples of software components include network application server software (such as IBM WebSphere® application server software) and database software (such as IBM DB2® database software). (IBM, zSeries, pSeries, xSeries, BladeCenter, WebSphere, and DB2 are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide.)

[0080] The virtualization layer (820) comprises an abstraction layer that can provide virtual entities such as virtual servers, virtual storage, virtual networks including virtual private networks, virtual applications and operating systems, and virtual clients.

[0081] In one example, the management layer (830) may provide functionality for resource provisioning, metering and pricing, a user portal, service layer management, and SLA planning and execution. Resource provisioning dynamically procures computing and other resources used to perform tasks within the cloud computing environment. Metering and pricing tracks costs as resources are utilized within the cloud computing environment and sends bills or invoices for the utilization of those resources. In one example, these resources may include application software licenses. Security verifies the identity of cloud users and tasks and protects data and other resources. The user portal provides users and system administrators with access to the cloud computing environment. Service layer management allocates and manages cloud computing resources to meet required service layers. Service level agreement (SLA) planning and execution proactively prepares and procures cloud computing resources in accordance with SLAs in anticipation of future demand.

[0082] The Workload Layer (840) illustrates examples of functionality available in a cloud computing environment. Examples of workloads and functionality provided by this layer include, but are not limited to, mapping and navigation, software development and lifecycle management, virtual classroom instruction delivery, data analytics processing, transaction processing, and entity linking within logical neural networks.

[0083] The systems and flow charts illustrated herein may be in the form of a computer program device for entity linking in a logical neural network, which device may include embodied program code that is executable by a processing unit to support the described functionality.

[0084] While particular embodiments have been shown and described, it will be apparent to those skilled in the art, based on the contents of this specification, that changes and modifications may be made thereto without departing from the broader aspects of the embodiments. Accordingly, the appended claims encompass within their scope all such changes and modifications that are within the scope of the embodiments. It should further be understood that the embodiments are defined solely by the appended claims. Where a specific number of introduced claim elements is intended, such intention will be expressly recited in the claims; it will be understood by those skilled in the art that, in the absence of such recitation, no such limitation exists. As an aid to understanding with respect to non-limiting examples, the appended claims below include the use of the introductory phrases "at least one" and "one or more" to introduce claim elements. However, the use of such phrases should not be construed as meaning that the introduction of a claim element by the indefinite article "a" or "an" limits any particular claim containing such introduced claim element to embodiments containing only one such element, even if the same claim also contains the introductory phrases "one or more" or "at least one" and an indefinite article such as "a" or "an," and the same applies to the use of definite articles in the claims.

[0085] The present embodiments may be systems, methods, or computer program products, or combinations thereof. In addition, selected aspects of the present embodiments may take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or embodiments combining software or hardware aspects, or both, all of which may be referred to generally herein as "circuits," "modules," or "systems." Furthermore, aspects of the present embodiments may take the form of a computer program product embodied in a computer-readable storage medium containing computer-readable program instructions for causing a processor to execute aspects of the present embodiments. The disclosed systems, methods, or computer program products, or combinations thereof, so embodied, function to improve the functionality and operation of dynamic orchestration of precondition-driven codified infrastructures.

[0086] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes portable floppy disks, hard disks, dynamic or static random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), magnetic storage devices, portable compact disc read-only memory (CD-ROM), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in grooves on which instructions are recorded, and any suitable combination thereof. As used herein, computer-readable storage media should not be construed as being ephemeral signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through fiber optic cable), or electrical signals transmitted over wires.

[0087] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or storage device over a network (e.g., the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof) that may include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage on a computer-readable storage medium within each computing / processing device.

[0088] The computer-readable program instructions for carrying out the operations of the present embodiments may be source or object code written in any combination of one or more programming languages, including assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or object code written in one or more programming languages, including object-oriented programming languages ​​such as Java®, Smalltalk®, C++, and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and on a remote computer, or entirely on the remote computer or on a server or cluster of servers. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, to perform aspects of the present embodiments, electronic circuitry including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions to personalize the electronic circuitry by utilizing state information of the computer-readable program instructions.

[0089] Aspects of the present embodiments are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, are implemented by computer-readable program instructions.

[0090] These computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to create a machine, where the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for performing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may be stored on a computer-readable storage medium and capable of directing a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer-readable storage medium on which the instructions are stored comprises an article of manufacture containing instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0091] Computer-readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device such that the instructions, which execute on the computer, other programmable apparatus, or other device, perform the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams, thereby causing a series of operable steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process.

[0092] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, comprising one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions shown in the blocks may occur in an order different from that shown in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks included in the block diagrams and / or flowchart diagrams, are implemented by a special-purpose hardware-based system that performs the specified function or operation or executes a combination of special-purpose hardware and computer instructions.

[0093] While specific embodiments have been described herein for illustrative purposes, it will be understood that various modifications may be made without departing from the scope of the embodiments. In particular, the annotation of unstructured NL data and the extraction of facts into a structured format may be performed by different computing platforms or across multiple devices. Furthermore, the library may be localized, remotely located, or distributed across multiple systems. Accordingly, the scope of protection of the embodiments is limited only by the appended claims and their equivalents.

Claims

1. a processor operatively coupled to the memory; an artificial intelligence (AI) platform operably coupled to the processor, the AI ​​platform comprising: a feature manager for generating a set of features for one or more entity-mention pairs in the annotated dataset; an evaluator configured to evaluate the generated set of features of the one or more entity-mention pairs against an entity linking (EL) LNN rule template, the template including one or more hierarchically structured logically connected rules and corresponding connective weights; and a machine learning (ML) manager operatively coupled to the evaluator and configured to learn the weights of the connectors using an artificial neural network (ANN) and a corresponding ML algorithm; the ML manager selectively updates the weights of the connectors associated with the logically connected rules; a computer system configured to generate a trained model using the trained thresholds for the logically connected rules and the trained connecter weights.

2. The system of claim 1 , wherein the evaluation further comprises the evaluator for reconstructing an entity linking algorithm consisting of a disjunctive set of rules into an LNN representation.

3. 3. The system of claim 2, wherein the evaluation of the entity-mention pairs further comprises the evaluator for computing one or more features for a subset of labeled entity-mention pairs, each of the features including a corresponding similarity predicate.

4. 4. The system of claim 3, further comprising the ML manager for leveraging the ANN and the ML algorithm to learn appropriate thresholds for each of the computed one or more features associated with the corresponding similarity predicate.

5. 5. The system of claim 4, further comprising the evaluator for filtering the calculated one or more features based on corresponding learned thresholds and selectively incorporating the calculated one or more features into the LNN rule template in response to the filtering, wherein the selective incorporating comprises removing a feature or assigning a non-zero score to the feature.

6. further comprising a rule manager operably coupled to the evaluator, the rule manager comprising: learning one or more of the logically connected rules; dynamically generating said hierarchical template; learning logic rules based on the dynamically generated templates; evaluating the selected rules against the labeled dataset; and selectively assigning the selected rules to corresponding nodes in the hierarchical structure.

7. 7. The system of claim 6, wherein the template is a binary tree and the corresponding node is an interior node, and further comprising the rule manager for selectively assigning conjunctive or disjunctive LNN operators to the interior nodes.

8. A computer program product configured to interface with a computer-readable storage medium having program code embodied therein, said program code being executable by a processor, said program code causing a computer to: generating features for one or more entity-mention pairs in the annotated dataset; evaluating the generated features of the one or more entity-mention pairs against an entity linking (EL) LNN rule template, the template including one or more hierarchically structured logically connected rules and corresponding connector weights; Utilizing an artificial neural network (ANN) and a corresponding ML algorithm to learn the weights of the connectives; selectively updating the weights of the connectors associated with the logically connected rules; generating a trained model using the trained thresholds for the logically connected rules and the trained connective weights.

9. 10. The computer program product of claim 8, further comprising program code configured to: reconstruct an entity linking algorithm consisting of a disjunctive set of rules into an LNN representation;

10. 10. The computer program product of claim 9, wherein the evaluation of the entity-mention pairs further comprises program code configured to compute a set of features for each entity-mention pair, each of the features including a corresponding similarity predicate.

11. Utilizing the ANN and the ML algorithm to learn a suitable threshold for each of the calculated one or more features associated with the corresponding similarity predicate; filtering the calculated one or more features based on a corresponding learned threshold; and selectively incorporating the calculated one or more features into the LNN rule template, wherein the selective incorporating comprises removing a feature or assigning a non-zero score to the feature.

12. learning one or more of the logically connected rules; dynamically generating said hierarchical template; learning logic rules based on the dynamically generated templates; evaluating the selected rules against the labeled dataset; and selectively assigning the selected rules to corresponding nodes in the hierarchical structure.

13. 13. The computer program product of claim 12, wherein the template is a binary tree and the corresponding node is an interior node, further comprising program code configured to selectively assign conjunctive or disjunctive LNN operators to the interior nodes.

14. A computer-based information processing method, comprising: generating features for one or more entity-mention pairs in the annotated dataset; evaluating the generated features of the one or more entity-mention pairs against an entity-linking (EL) logic neural network (LNN) rule template, the template including one or more hierarchically structured logically connected rules and corresponding connective weights; Utilizing an artificial neural network (ANN) and corresponding machine learning (ML) algorithm to learn the weights of the combinatorial operators; selectively updating the weights of the connectors associated with the logically connected rules; generating a trained model using the trained thresholds for the logically connected rules and the trained connective weights.

15. 15. The method of claim 14, wherein evaluating the entity-mention pairs comprises reconstructing an entity linking algorithm consisting of a disjunctive set of rules into an LNN representation.

16. The method of claim 15 , wherein evaluating the entity-mention pairs comprises computing a set of features for each entity-mention pair, each of the features including a corresponding similarity predicate.

17. 17. The method of claim 16, further comprising leveraging the ANN and the ML algorithm to learn appropriate thresholds for each of the computed one or more features associated with the corresponding similarity predicate.

18. 18. The method of claim 17, further comprising: filtering the calculated one or more features based on corresponding learned thresholds; and selectively incorporating the calculated one or more features into the LNN rule template in response to the filtering, wherein the selective incorporating comprises removing a feature or assigning a non-zero score to the feature.

19. learning one or more of the logically connected rules, including dynamically generating templates of the hierarchical structure; learning logic rules based on the dynamically generated templates; evaluating the selected rules against the labeled dataset; 16. The method of claim 15, further comprising selectively assigning the selected rules to corresponding nodes in the hierarchical structure.

20. 20. The method of claim 19, wherein the template is a binary tree and the corresponding node is an interior node, further comprising selectively assigning a conjunctive or disjunctive LNN operator to the interior node.

Images