Knowledge graph optimized prompts for open-domain common sense reasoning decision making with artificial intelligence

Abstract

A computer-implemented method for optimized decision making includes (1) labeling text data extracted from a query and (2) linking the labeled text to knowledge graph entities. The method further includes (3) searching for inference paths from the knowledge graph and (4) eliminating irrelevant knowledge graph inference paths using artificial intelligence trained with a language model consistent with the labeling of the text data. The method may further include (5) using the remaining relevant knowledge graph inference paths to provide an answer prediction.

Description

[Technical Field]

[0001] Related Application Information This application claims priority to U.S. Provisional Application No. 63 / 424,516, filed November 11, 2022, which is incorporated herein by reference in its entirety.

[0002] This application claims priority to U.S. Application No. 18 / 503,517, filed November 7, 2023, which is incorporated herein by reference in its entirety.

[0003] The present invention relates to artificial intelligence, and in particular to using smart graphs for decision making. [Background technology]

[0004] 2. Description of Related Art

[0005] Large-scale pretrained language models (PLMs) learn to implicitly encode basic knowledge about the world by training on very large collections of general text corpora and refining downstream datasets. This has recently become established as a dominant paradigm in natural language processing (NLP). While PLMs have demonstrated excellent performance in many downstream tasks, it has been found that they can face two major challenges in inference-related tasks: 1) PLMs frequently encounter problems when the necessary knowledge is not present in the training corpus or when test instances are not formulated as question-answering. 2) PLMs make predictions based on implicitly encoded knowledge, but this knowledge cannot handle structured inference or provide explanations for selected responses. Summary of the Invention

[0006] According to one aspect of the present invention, a method for optimized decision support is provided. In one embodiment, the computer-implemented method for decision support includes labeling text data extracted from a query and linking the labeled text to knowledge graph entities. The computer-implemented method may further include retrieving knowledge graph inference paths from the knowledge graph entities. The computer-implemented method may further include filtering out irrelevant knowledge graph inference paths using artificial intelligence trained with a language model consistent with the labeling of the text data. Finally, the computer-implemented method may use the remaining relevant knowledge graph to predict an answer to the query.

[0007] According to another aspect of the present invention, there is provided a system for optimized decision support. The system can include a hardware processor and a memory that stores a computer program product. When executed by the hardware processor, the computer program product causes the hardware processor to label text data extracted from a query and link the labeled text to knowledge graph entities. The computer program product can also use the hardware processor to search for knowledge graph inference paths from the knowledge graph entities and eliminate irrelevant knowledge graph inference paths using artificial intelligence trained with a language model consistent with the labeling of the text data. Finally, the system can also use the hardware processor to provide answer predictions for the query using the remaining relevant knowledge graph.

[0008] According to yet another embodiment of the present invention, a computer program product for optimized decision making is provided. The computer program product includes a computer-readable storage medium having computer-readable program code embodied therein. The program instructions are executable by a processor and cause the processor to label text data extracted from a query and link the labeled text to knowledge graph entities. The computer program product can also use a hardware processor to search for knowledge graph inference paths from the knowledge graph entities and remove irrelevant knowledge graph inference paths using artificial intelligence trained with a language model consistent with the labeling of the text data. Finally, the computer program product can use the remaining relevant knowledge graph to provide an answer prediction for the query.

[0009] These and other features and advantages will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings. [Brief explanation of the drawings]

[0010] The present disclosure provides details in the following description of preferred embodiments with reference to the following figures.

[0011] [Figure 1] FIG. 1 illustrates an exemplary environment for knowledge-based open-domain common sense reasoning.

[0012] [Figure 2] 1 illustrates an embodiment of a knowledge-enhanced prompting method that can solve open-domain common sense reasoning problems and answer questions without providing candidate answers.

[0013] [Figure 3] FIG. 2 is a block / flow diagram illustrating an embodiment of a computer-implemented method for decision support using knowledge-based open-domain common sense reasoning, according to an embodiment of the present invention.

[0014] [Figure 4] 1 is a block diagram illustrating a system for decision support using knowledge-based open-domain common sense reasoning according to one embodiment of the present invention.

[0015] [Figure 5] FIG. 1 is an illustration of knowledge graph expansion through iterative inference steps, according to one embodiment of the present disclosure.

[0016] [Figure 6] FIG. 1 illustrates a transformation of knowledge descriptions and clause-based prompt construction according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0017] According to embodiments of the present invention, systems and methods are provided for approaching open-domain common sense reasoning via external knowledge bases. For example, given a natural text question (also called a query), the computer-implemented method, system, and computer program product of the present disclosure can answer the question without providing candidate answers or fine-tuned examples. The computer-implemented method, system, and computer program product provide optimized decision-making using artificial intelligence principles suitable for application in fields such as health care, medicine, personal care, and disease treatment.

[0018] Figure 1 illustrates what happens when a pretrained language model (PLM) 50 is presented with a question whose domain differs from the examples seen during training. In the example shown in Figure 1, the pretrained language model (PLM) artificial intelligence is presented with a multiple-choice question and answer scenario 51 and is able to calculate the likelihood of the entire sentence by filling in the blanks for each possible answer. However, both answers employ learning from others, and a complete job can be done to fit the semantics of the question. The pretrained language model (PLM) generally cannot justify why a particular answer was chosen. Figure 1 illustrates how common sense reasoning predictions have traditionally relied on robust, structured inference to integrate explicit information provided by the question context and external knowledge.

[0019] Figure 1 shows that there are two key issues with using pre-trained language models (PLMs) for common sense reasoning: 1) Without fine-tuning, pre-trained language models (PLMs) may not be able to handle out-of-distribution or domain-specific reasoning questions, and 2) pre-trained language models (PLMs) generally rely on existing candidate answers and generally cannot justify their prediction results.

[0020] The computer-implemented methods, systems, and computer program products focus on open-domain common sense reasoning tasks that involve machine learning models making inferences about the type and nature of typical situations without being presented with candidate answers or fine-tuning examples, e.g., using multiple-choice question and answer scenarios identified by reference numeral 51 in Figure 1 .

[0021] The computer-implemented methods, systems, and computer program products described herein leverage neural language models to iteratively search inference chains on external knowledge bases without the need for task-specific supervision (e.g., without the need for training with multiple-choice candidate answers). The inference chains help identify the most accurate answers to common sense questions and the corresponding knowledge statements to justify the selection of those answers.

[0022] Referring again to FIG. 1 , in some embodiments, methods and systems can use three components to provide knowledge-based, open-domain common sense reasoning: 1) entity extraction and linking, 2) local knowledge graph expansion, and 3) explanation generation and answer prediction. Entity extraction and linking can be provided by a natural language model 54 pre-trained to extract and label text from queries. Any query, e.g., question, can be submitted to the computer-implemented method and system, but in some embodiments, the query may be related to the medical and healthcare fields. The query may be entered into the system by a healthcare professional 15, such as a nurse or doctor, or by a patient in a healthcare environment. The query can be entered by a healthcare professional, such as a doctor, paramedic, nurse, medical professional, health professional, or healthcare service worker.

[0023] For example, the query may be for decision support in the operation of a medical facility, pharmacy, or rehabilitation center. In some examples, the query may be a treatment decision for a patient. In some examples, the query may be entered by a healthcare professional using the computer-implemented methods and systems to obtain information about a patient's medications or treatments and conduct a medical interview. In some examples, the query may be entered by a patient using the system to obtain information about a prescribed medication or treatment, such as a cancer treatment. The computer-implemented methods, systems, and computer program products described herein can be used to assist users in decision-making applications. For example, a healthcare professional can determine a patient's medication or treatment to address an illness. A patient can decide whether to consent to a prescribed medication or treatment.

[0024] In the example shown in Figure 1, the question asked is directed to the composition of painkillers and could serve as an example of decision support / decision making for hospital management to stockpile appropriate medications.

[0025] In the example shown in Figure 1, the question being asked is directed towards the composition of the pain medication and may serve as an example of decision support / decision making for the pharmacist, stocking up on the right medication, and / or checking the suitability of the medication for the patient.

[0026] In the example shown in Figure 1, the question being asked is directed to the composition of an analgesic and could serve as an example of a decision aid / decision making that a doctor or nurse could use in treating a patient.

[0027] FIG. 2 illustrates one embodiment of a knowledge-enhanced prompting method that can solve open-domain common sense inference problems and answer questions without presenting candidate answers. In some examples, the flow illustrated in FIG. 2 can address open-domain common sense inference problems over text data. This is because, in contrast to alternative methods that require direct supervision of the inference process, it provides a novel knowledge-enhanced prompting framework that leverages knowledge implicitly stored in a pre-trained language model (PLM) to iteratively recover inference chains from a curated external knowledge base. Furthermore, each inference path retrieved serves as an explicit justification for answer selection. To achieve this goal, the framework follows the steps illustrated in blocks 101-103.

[0028] Referring first to block 101, the computer-implemented methods, systems, and computer program products described herein can leverage knowledge of natural language words and phrases connected by labeled edges to extract a set of significant entities from a text query that have a surjective mapping to nodes in a knowledge graph. A "knowledge graph," also known as a semantic network, represents a network of real-world entities (i.e., objects, events, situations, or concepts) and shows the relationships between them. This information is typically stored in a graph database and visualized as a graph structure, hence the term knowledge "graph." A knowledge graph consists of three main components: nodes, edges, and labels. A node can be any object, place, or person. Edges define the relationships between nodes. Block 101 of FIG. 2 is described in more detail below with reference to block 1 of the detailed method illustrated in FIG. 3. Furthermore, the functions described in block 101 of FIG. 2 and block 1 of FIG. 3 can be performed by an entity extractor 541, as illustrated in the computing device 500 illustrated in FIG. 4.

[0029] Referring to block 102 of FIG. 2, a computer-implemented method and system may search for inference paths from the knowledge graph within L hops from the knowledge graph to form a local knowledge subgraph with the highest coverage for the query concept. This not only makes the inference path expansion process scalable, but also incorporates the implicit knowledge of the PLM to reduce unnecessary paths. Block 102 of FIG. 2 is described in further detail below with reference to blocks 2 and 3 of the detailed method shown in FIG. 3. Furthermore, the functions described in block 102 of FIG. 2 and blocks 2 and 3 of FIG. 3 may be performed by a knowledge graph expansion generator 542, as shown in the computing device 500 shown in FIG. 4.

[0030] Referring to block 103 of FIG. 2, a computer-implemented method and system can consider all inference paths as supporting knowledge explanations, and the computer-implemented method and system can aim to perform answer prediction. For example, beam search can be used to retain only highly reliable inference paths and convert them into natural language using templates designed in a set of knowledge descriptions during the search phase. In some embodiments, beam search can be a heuristic search algorithm that explores a graph by expanding the most promising nodes in a limited set. Beam search is an optimization of best-first search that reduces memory requirements. Best-first search is a graph search that orders all partial solutions (states) according to some heuristic. However, in beam search, only a predetermined number of best partial solutions are retained as candidates. Block 103 of FIG. 2 is described in more detail below with reference to blocks 4 and 5 of the detailed method shown in FIG. 3. Furthermore, the functions described in block 103 of FIG. 2 and blocks 4 and 5 of FIG. 3 can be performed by a knowledge predictor 543, as shown in the computing device 500 shown in FIG. 4.

[0031] 3 illustrates one embodiment of a computer-implemented method for decision support / decision making using knowledge-based, open-domain common sense reasoning with artificial intelligence. Block 1 may include labeling text data extracted from a query. For example, the query may be a question input to the system, i.e., a question for which a user 15 seeks assistance. For example, the query may be entered into an interface of a computing system 49 that provides input to the system, and the query may be provided in the form of text (e.g., a sentence). As discussed above, the query may be on a subject related to healthcare and / or healthcare facility management.

[0032] 4 is a block diagram illustrating a decision support system using knowledge-based, open-domain common sense reasoning. Queries can be entered into the system through a peripheral interface 560. The peripheral interface 560 can include any number of additional input / output devices, such as one with a keyboard for data entry.

[0033] Referring again to block 1 of FIG. 2, in some embodiments, a text data corpus is selected for labeling the text data. In some embodiments, the text data corpus is created from question-and-answer text and includes a collection of strongly labeled data including multiple-choice answers. In some embodiments, natural language processing is used to label the text data in block 1 of the computer-implemented method using a model trained on the text data corpus. In some examples, the terms selected from the question-and-answer queries include medical topics selected from the group consisting of medications and their compositions, diagnoses and their treatments, medical staff job titles and their responsibilities, medical facility classifications and their inventory contents, and combinations thereof.

[0034] In some embodiments, the functionality described in block 1 of FIG. 3 may be performed by an entity extractor 541, as illustrated in the computing device 500 shown in FIG.

[0035] In some embodiments, a computer-implemented method can solve open-domain common sense inference questions using knowledge from a pre-trained language (PLM) and a structured knowledge graph G. The knowledge graph G=(V,E) is a multi-relational graph, where V is a set of entity nodes, and

number

[0036] In some embodiments, for an open-domain common sense inference question q (i.e., given question q without providing a candidate answer), the goal of this work is to: 1) generate a local knowledge graph

number

number

number

number

[0037] For example, to answer the open-domain common sense question "what do people aim to do at work?", the system first aims to extract all possible knowledge statements from an external knowledge base that can provide logical information to answer the question. Starting with concept extraction and mapping, the concepts in the question can be "people," "work," and "aim." These provide the initial entities, denoted by reference numeral 62 in Figure 5.

[0038] Figure 5 shows the iterative inference steps, i.e., graph expansion steps with first hoop expansion 63 and second hoop expansion 64, for the question "what do people aim to do at work?". Among all the descriptions, the system selects the most accurate one, as indicated by the node with reference number 65. In the example shown in Figure 5, the most accurate description is "people learn to work at the office to finish jobs." From this description, the following answers are extracted:

number

number

[0039] Referring to block 2 of Figure 3, the computer-implemented method may proceed with linking knowledge graph entities. In this step, a knowledge base (e.g., a knowledge graph) is used to extract a set of significant entities from the text question. In some embodiments, linking the text to knowledge graph entities includes a knowledge graph that connects natural language words and phrases with labeled edges.

[0040] In some embodiments, knowledge graph entity links can comprise a knowledge graph that enables a variety of useful context-oriented reasoning tasks on real-world text, thereby providing structured knowledge in open-domain common sense reasoning tasks.

[0041] To infer a given commonsense context using knowledge from both the pre-trained model (PLM) and the knowledge graph G, the first step of our framework is to generate a surjective mapping to the nodes of the knowledge graph.

number

number

number

[0042] Additionally, the functionality described in block 2 of FIG. 3 may be performed by a knowledge graph extension generator 542 as shown in the computing device 500 illustrated in FIG.

[0043] Referring to block 3 of Figure 3, the computer-implemented method may continue by retrieving a set of inference paths from the knowledge graph 55. Referring again to Figure 1, the inference paths are identified by reference numeral 60. The inference paths 60 shown in Figure 1 are the relevant inference paths after removing irrelevant inference paths, all of which are extracted from the knowledge graph.

[0044] Referring to block 3 of Figure 3 and Figure 1, the step of searching for a set of inference paths 60 may refer to inference on a local knowledge graph. A computer-implemented method and system may be directed to searching for inference paths within L hops from G to form a local knowledge subgraph. Referring to Figure 5, examples of L hops on a knowledge graph are shown at 62 and 63.

[0045] In some embodiments, G q Each path in can be seen as a chain of reasoning that helps to find the most accurate answer to question q and its explanation.

number

number

[0046] Additionally, the functionality described in block 3 of FIG. 3 may be performed by a knowledge graph extension generator 542 as shown in the computing device 500 illustrated in FIG.

[0047] Referring to block 4 of FIG. 3, the computer-implemented method can continue by removing irrelevant knowledge graph inference paths using a language model trained to match the labeling of the text data. In some embodiments, this step may be referred to as inference path pruning. In some embodiments, during inference path pruning, the computer-implemented method and system can incorporate implicit knowledge in the pre-trained language model to remove irrelevant paths, making the inference path expansion process scalable. Furthermore, the functionality described in block 4 of FIG. 3 can be performed by a knowledge predictor 543, as shown in the computing device 500 illustrated in FIG. 4.

[0048] In some embodiments, the computer-implemented method and system combines a question q with a node v along with an inference path-transformed knowledge description to transform the local graph expansion problem into an explicit reasoning procedure by directly answering the explained question, and then performs a claus-based prompt.

number

[0049] For example, Figure 6 shows the knowledge description transformation and clause-based prompt construction. The prompt is formatted as follows: What do people aim to do at work?<node>, because<inference path>

[0050] The inference path for the example question "What do people try to do at work?" 67 is shown at reference numeral 60. The computer-implemented method and system utilizes predefined templates to generate triplets

number

[0051] The computer-implemented method and system can translate the inference path triplets into natural language using predefined templates such as those shown in Table 68. For example, "job, antonym, unemployment" can be translated as "job is the antonym of unemployment." The table designated by reference numeral 68 also provides some example integration types and templates, as shown in FIG. 6.

[0052] Returning to removing irrelevant knowledge graph inference paths according to block 4 of Figure 3, to evaluate whether an inference path should be retained or removed, the computer-implemented method and system calculates a common sense score for the inference path 60. For example, a pre-trained learning model (PLM) can be used to score the relevance of each inference path given the context of question q.

[0053] For example, if a logical sentence W consists of N words,

number

number

number

number

number

number

[0054] Computer-implemented methods and systems q As we iteratively expand

number

number

number

[0055] Moving to block 5 of Figure 3, the computer-implemented method can continue to make answer predictions using the remaining relevant graph inference paths, i.e., following the pruning in block 4. Block 5 may be referred to as the knowledge integration and prediction step. In this step, beam search is used to retain only the most reliable reasoning paths and convert them into natural language for final answer prediction. The function described in block 5 of Figure 3 can be performed by a knowledge predictor 543, as shown in the computing device 500 shown in Figure 4.

[0056] In some embodiments, a subgraph G consists of all inference paths in L hops that have a high common sense score. q After obtaining the inference paths 60, all the inference paths 60 can be considered as supporting knowledge explanations. The final step is to make an answer prediction. In some embodiments, a beam search is used to keep only the inference paths with high confidence and to optimize the knowledge description during the search phase.

number

number

number

number

number

number

number

number

[0057] Returning to the example shown in Figure 5, if the question q = "what do people aim to do at work?", the question concept

number

number

number

[0058] It should be noted that the example described in the preceding paragraph is merely an example, and the present disclosure is not limited to only this example. In some embodiments, the answer prediction 19 is selected from the group consisting of inventory management according to drug composition, assignment of office locations by job function according to the use of the medical facility, and assignment of treatment according to diagnostic characteristics. Returning to Figure 1, the predicted answer 19 may be displayed on a display of an interface 19 through which a user 15, such as a healthcare professional, doctor, nurse, or patient, interacts with the system.

[0059] In one embodiment, the computer-implemented method and system described with reference to Figures 1-6 can provide an open-domain common sense reasoning method (KEP) for answering text questions without providing candidate answers or fine-tuned examples. In some embodiments, the KEP iteratively gathers inference chains from an external structured knowledge base, using implicit information stored in a pre-trained language model as guidance. In some embodiments, the KEP can identify the most appropriate answer and automatically generate an explanation supporting that selection.

[0060] 4, an exemplary computing device 500 is shown, according to one embodiment of the present invention. The computing device 500 can be configured to provide decision support. For example, the decision support system may include a hardware processor 510 and a memory 530 that stores the computer program product. The memory 530 may include a data store 540. The data store 540 may include an entity extractor 541, a knowledge graph extension generator 542, and a knowledge predictor 543.

[0061] The contents of data storage device 540, when executed by hardware processor 510, cause hardware processor 510 to label text data extracted from a query, link the labeled text to knowledge graph entities, and retrieve the labeled text from knowledge graph inference paths. Entity extractor 541 can perform the steps of labeling text data extracted from a query. Knowledge graph extender 542 can perform the steps of linking the labeled text to knowledge graph entities and retrieving the labeled text from knowledge graph inference paths. These functions are described in more detail above in block 102 of FIG. 2 and block 2 of FIG. 3. Knowledge graph extender 542 can also remove irrelevant knowledge graph inference paths using a language model trained to match the labeling of the text data. This feature is described in more detail above in block 102 of FIG. 2 and block 5 of FIG. 3. Knowledge predictor 543 can provide answer predictions using the remaining relevant graph inference paths. This feature is described above with reference to block 103 of FIG. 2 and block 5 of FIG. 3.

[0062] Computing device 500 may be embodied as any type of computing device or computer device capable of performing the functions described herein, including, but not limited to, a computer, a server, a rack-based server, a blade server, a workstation, a desktop computer, a laptop computer, a notebook computer, a tablet computer, a mobile computing device, a wearable computing device, a network appliance, a web appliance, a distributed computing system, a processor-based system, and / or a consumer electronics device. Additionally or alternatively, computing device 500 may be embodied as one or more computing sleds, memory sleds, or other racks, sleds, computing chassis, or other components of a physically distributed computing device.

[0063] 4, computing device 500 includes a processor 510, an input / output subsystem 520, a memory 530, a data storage device 540, a communications subsystem 550, and / or other components and devices typically found in a server or similar computing device. Computing device 500, in other embodiments, may include other or additional components, such as those typically found in a server computer (e.g., various input / output devices). Furthermore, in some embodiments, one or more of the illustrated components may be incorporated into or form part of another component. For example, in some embodiments, memory 530, or portions thereof, may be incorporated into processor 510.

[0064] The processor 510 may be embodied as any type of processor capable of performing the functions described herein, and may be embodied as a single processor, multiple processors, a central processing unit (CPU), a graphics processing unit (GPU), a single or multi-core processor, a digital signal processor, a microcontroller, or other processor or processing / control circuitry.

[0065] The memory 530 may be embodied as any type of volatile or non-volatile memory or data storage capable of performing the functions described herein. During operation, the memory 530 may store various data and software used during operation of the computing device 500, such as an operating system, applications, programs, libraries, drivers, etc. The memory 530 is communicatively coupled to the processor 510 via an I / O subsystem 520, which may be embodied as circuits and / or components that facilitate input / output operations with the processor 510, the memory 530, and other components of the computing device 500. For example, the I / O subsystem 520 may be embodied as or include a memory controller hub, an input / output control hub, a platform controller hub, an integrated control circuit, a firmware device, a communication link (such as a point-to-point link, a bus link, a wire, a cable, an optical guide, a trace on a printed circuit board, etc.), and / or other components and subsystems that facilitate input / output operations. In some embodiments, the I / O subsystem 520 may form part of a system-on-chip (SOC) and may be integrated on a single integrated circuit chip along with the processor 510, memory 530, and other components of the computing device 500.

[0066] The data storage device 540 may be embodied as any type of device(s) configured for short-term or long-term data storage, such as, for example, memory devices and circuitry, memory cards, hard disk drives, solid-state drives, or other data storage devices. The data storage device 540 may store program code for the entity extractor 541, the knowledge graph extension generator 542, and the knowledge predictor 543.

[0067] Any or all of these program code blocks may be included in a given computing system. The communications subsystem 550 of the computing device 500 may be embodied as any network interface controller or other communications circuitry, device, or collection thereof that enables communications between the computing device 500 and other remote devices over a network. The communications subsystem 550 may be configured to use any one or more communications technologies (e.g., wired or wireless communications) and associated protocols (e.g., Ethernet, InfiniBand, Bluetooth, Wi-Fi, WiMAX, etc.) to achieve such communications.

[0068] As shown, computing device 500 may also include one or more peripheral devices 560. Peripheral devices 560 may include any number of additional input / output devices, interface devices, and / or other peripheral devices. For example, in some embodiments, peripheral devices 560 may include a display, a touch screen, graphics circuitry, a keyboard, a mouse, a speaker system, a microphone, a network interface, and / or other input / output devices, interface devices, and / or peripheral devices.

[0069] Of course, computing device 500 may include other elements (not shown) or omit certain elements, as would readily occur to one skilled in the art. For example, various other sensors, input devices, and / or output devices may be included in computing device 500, depending on the particular implementation, as would be readily understood by one skilled in the art. For example, various types of wireless and / or wired input and / or output devices may be used. Furthermore, additional processors, controllers, memory, etc. may be utilized in a variety of configurations. These and other variations of processing system 500 will be readily contemplated by one skilled in the art in view of the teachings of the present invention provided herein.

[0070] The embodiments described herein may be entirely hardware, entirely software, or contain both hardware and software elements. In a preferred embodiment, the invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.

[0071] Embodiments may include a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For example, the computer program product may be provided for decision support. The computer program product may be a computer-readable storage medium having computer-readable program code embodied therein, wherein the program instructions executable by a processor cause the processor to label text data extracted from a query and link the labeled text to knowledge graph entities. The computer program product may also use the processor to search through the knowledge graph inference paths and use the processor to remove irrelevant knowledge graph inference paths using a language model trained to match the labeling of the text data. The computer program product may also use the processor to provide answer predictions using the remaining relevant graph inference paths.

[0072] A computer-usable medium or computer-readable medium may include any apparatus that stores, communicates, propagates, or transfers a program for use by or in connection with an instruction execution system, apparatus, or device. The medium may be a magnetic, optical, electronic, electromagnetic, infrared, or semiconductor system (or apparatus or device) or propagation medium. The medium may include computer-readable storage media such as semiconductor or solid-state memory, magnetic tape, removable computer diskettes, random access memory (RAM), read-only memory (ROM), rigid magnetic disks, and optical disks.

[0073] Each computer program can be tangibly stored on a machine-readable storage medium or device (e.g., program memory or magnetic disk) readable by a general-purpose or special-purpose programmable computer to configure and control the operation of the computer when the storage medium or device is read by the computer to perform the procedures described herein. The system of the present invention can also be considered to be embodied in a computer-readable storage medium configured with a computer program, where the configured storage medium causes the computer to operate in a particular, predetermined manner to perform the functions described herein.

[0074] A data processing system suitable for storing and / or executing program code may include at least one processor coupled directly or indirectly to memory elements via a system bus. The memory elements may include local memory employed during the actual execution of the program code, bulk storage, and cache memory that provides temporary storage of at least some program code to reduce the number of times the code is retrieved from bulk storage during execution. Input / output or I / O devices (including but not limited to keyboards, displays, pointing devices, etc.) may be coupled to the system either directly or through intervening I / O controllers.

[0075] Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modems, and Ethernet cards are just a few of the types of network adapters currently available.

[0076] As employed herein, the terms “hardware processor subsystem” or “hardware processor” can refer to a processor, memory, software, or combination thereof that cooperate to perform one or more specific tasks. In useful embodiments, a hardware processor subsystem may include one or more data processing elements (e.g., logic circuits, processing circuits, instruction execution devices, etc.). The data processing elements may be included in a central processing unit, a graphics processing unit, and / or a separate processor- or computing element-based controller (e.g., logic gates, etc.). A hardware processor subsystem may include one or more on-board memories (e.g., cache, dedicated memory array, read-only memory, etc.). In some embodiments, a hardware processor subsystem may include one or more memories (e.g., ROM, RAM, basic input / output system (BIOS), etc.) that may be on-board or off-board or dedicated for use by the hardware processor subsystem.

[0077] In some embodiments, the hardware processor subsystem may include and execute one or more software elements, which may include an operating system and / or one or more applications and / or specific code for achieving a specified result.

[0078] In other embodiments, the hardware processor subsystem may include dedicated, specialized circuitry that performs one or more electronic processing functions to achieve a specified result. Such circuitry may include one or more application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and / or programmable logic arrays (PLAs).

[0079] These and other variations of the hardware processor subsystem are also contemplated in accordance with embodiments of the present invention.

[0080] References herein to "in one embodiment" or "in an embodiment" of the present invention, as well as other variations, mean that a particular feature, structure, characteristic, etc. described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrase "in one embodiment" or "in an embodiment," and any other variations thereof, in various places throughout this specification do not necessarily all refer to the same embodiment. However, it should be understood that features of one or more embodiments may be combined given the teachings of the present invention provided herein.

[0081] For example, in the case of "A / B," the use of any of the following " / ," "and / or," "at least one," such as "A and / or B" or "at least one of A and B" will be understood to be intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), the selection of only the first and third listed alternatives (A and C), the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A, B, and C). This can be expanded as many times as there are listed items.

[0082] The foregoing is understood in all respects to be illustrative and exemplary, but not restrictive, and the scope of the invention disclosed herein is to be determined not from the detailed description, but from the claims which are interpreted in accordance with the full breadth permitted by the patent laws. It will be understood that the embodiments shown and described herein are merely exemplary of the invention, and that those skilled in the art could make various modifications without departing from the scope and spirit of the invention. Various other feature combinations could be implemented by those skilled in the art without departing from the scope and spirit of the invention. Having thus described aspects of the invention with the detail and particularity required by the patent laws, what is desired to be claimed and protected by Letters Patent is set forth in the appended claims.

Claims

1. 1. A computer-implemented method for decision-making, comprising: (1) labeling text data extracted from a query; (2) linking the labeled text to knowledge graph entities; (3) retrieving knowledge graph inference paths from the knowledge graph entities; (4) removing irrelevant knowledge graph inference paths using a language model trained to match the labeling of the text data; and (5) using the remaining associated graph inference paths to provide an answer prediction for said query.

2. 10. The computer-implemented method of claim 1, wherein the response prediction is selected from the group consisting of stocking medications according to composition, allocating office locations by job function according to healthcare facility use, and allocating treatments according to diagnostic characteristics.

3. 2. The computer-implemented method of claim 1, wherein the text data extracted from the query comprises collecting a text data corpus of question and answer text from multiple-choice queries, and using natural language processing to perform labeling of the text data using an artificial intelligence model trained on the text data corpus.

4. 4. The computer-implemented method of claim 3, wherein the terms selected from the question and answer text include medical topics selected from the group consisting of drugs and their compositions, diagnoses and their treatments, medical professional job titles and their responsibilities, medical facility classifications and their inventory contents, and combinations thereof.

5. 10. The computer-implemented method of claim 1, wherein linking the text to the labeled text links the knowledge graph entities includes a knowledge graph connecting natural language words and phrases with labeled edges.

6. The computer-implemented method of claim 1 , wherein providing the answer prediction using the remaining relevant graph inference paths comprises determining a most reliable inference path using a beam search.

7. 7. The computer-implemented method of claim 6, further comprising using natural language processing artificial intelligence to generate a most reliable reasoning path for the answer prediction in natural language.

8. a hardware processor; and a memory storing a computer program product that, when executed by the hardware processor, causes the hardware processor to: Label the text data extracted from the query (1), Linking the labeled text to knowledge graph entities (2); Retrieving knowledge graph inference paths from the knowledge graph entities (3); (4) removing irrelevant knowledge graph inference paths using a language model trained to match the labeling of the text data; The system for decision making uses the remaining relevant graph inference paths to provide (5) a prediction of an answer to the query.

9. 10. The system of claim 8, wherein the response prediction is selected from the group consisting of stocking medications according to composition, allocating office locations by job function according to healthcare facility use, and allocating treatments according to diagnostic characteristics.

10. 10. The system of claim 8, wherein labeling the text data extracted from the query comprises collecting a text data corpus of question and answer text from multiple-choice queries and using natural language processing to perform labeling of the text data using an artificial intelligence model trained on the text data corpus.

11. 11. The system of claim 10, wherein the terms selected from the question and answer text include medical topics selected from the group consisting of medications and their compositions, diagnoses and their treatments, medical professional job titles and their responsibilities, medical facility classifications and their inventory contents, and combinations thereof.

12. 10. The system of claim 8, wherein the linking of the text to the labeled text links to the knowledge graph entities includes a knowledge graph connecting natural language words and phrases with labeled edges.

13. The system of claim 8 , wherein providing the answer prediction using the remaining relevant graph inference paths comprises determining a most reliable inference path using a beam search.

14. 14. The system of claim 13, further comprising using natural language processing artificial intelligence to create a most reliable reasoning path for the answer prediction in natural language.

15. 1. A computer program product for decision making, the computer program product comprising a computer readable storage medium having computer readable program code embodied therein, the program instructions executable by a processor causing the processor to: using said processor to label text data extracted from a query (1); using said processor to link the labeled text to knowledge graph entities (2); Retrieving knowledge graph inference paths from the knowledge graph entities using the processor (3); using the processor to prune irrelevant knowledge graph inference paths using a language model trained to match the labeling of the text data (4); a computer program product for causing the processor to (5) use the remaining associated graph inference paths to provide an answer prediction to the query;

16. 16. The computer program product of claim 15, wherein the response prediction is selected from the group consisting of stocking medications according to composition, allocating office locations by job function according to healthcare facility use, and allocating treatments according to diagnostic characteristics.

17. 16. The computer program product of claim 15, wherein labeling the text data extracted from the query comprises collecting a text data corpus of question and answer text from multiple-choice queries and using natural language processing to perform labeling of the text data using an artificial intelligence model trained on the text data corpus.

18. 20. The computer program product of claim 17, wherein the terms selected from the question and answer text include medical topics selected from the group consisting of medications and their compositions, diagnoses and their treatments, medical professional job titles and their responsibilities, medical facility classifications and their inventory contents, and combinations thereof.

19. 16. The computer program product of claim 15, wherein linking the text to the labeled text links to the knowledge graph entities includes a knowledge graph connecting natural language words and phrases with labeled edges.

20. 16. The computer program product of claim 15, wherein providing the answer prediction using the remaining associated graph inference paths comprises: determining a most reliable inference path using a beam search; and using natural language processing artificial intelligence to create a most reliable inference path for the answer prediction in natural language.

Images