LLM inference based on medical knowledge graph

Abstract

Provided in the present disclosure are an LLM inference method based on a medical knowledge graph, and a related device. A knowledge graph comprises a plurality of nodes, and a plurality of edges for connecting the nodes, wherein the nodes represent entities, and the edges represent relationships between the entities. The method comprises: acquiring target text that is input by a user, and determining whether the target text includes an entity corresponding to a node in a knowledge graph; if the target text includes an entity corresponding to a node in a knowledge graph, finding from the knowledge graph a subgraph related to the entity, wherein the subgraph includes the node corresponding to the entity, and a plurality of neighbor nodes connected to the node by means of a plurality of edges; performing feature extraction on the subgraph to obtain auxiliary information related to the target text; and constructing a prompt on the basis of the auxiliary information and the target text, and inputting the prompt into an LLM, such that the LLM executes logical inference on the basis of the prompt, and outputs an inference result corresponding to the target text.

Description

Inference using an LLM model based on medical knowledge graphs Technical Field

[0001] This disclosure relates to one or more embodiments in the field of medical technology, and more particularly to LLM model reasoning based on medical knowledge graphs. Background Technology

[0002] Large-scale language models (LLMs), also known as pre-trained models, are language models built from deep neural networks containing hundreds of billions of parameters. They utilize self-supervised learning methods, pre-training on vast and diverse public datasets to master numerous linguistic phenomena. However, LLM models are black-box models, often failing to capture and acquire factual knowledge, and frequently exhibiting "hallucinations," fabricating non-existent information. In contrast, knowledge graphs are structured semantic knowledge bases used to describe rich factual concepts and their relationships in the physical world in symbolic form. Their basic building block is the "entity-relationship-entity" triple.

[0003] Therefore, how to accurately utilize the factual knowledge stored in knowledge graphs to enhance the reasoning ability of pre-trained models is an urgent problem to be solved. Summary of the Invention

[0004] In view of this, one or more embodiments of this disclosure provide an LLM model reasoning method and related equipment based on medical knowledge graph.

[0005] Firstly, this disclosure provides an LLM model reasoning method based on a knowledge graph, wherein the knowledge graph contains multiple nodes and edges connecting the nodes, the nodes represent entities, and the edges represent relationships between entities; the method includes: acquiring target text input by a user, and determining whether the target text contains an entity corresponding to a node in the knowledge graph; if so, finding a subgraph related to the entity in the knowledge graph; wherein the subgraph contains a node corresponding to the entity, and multiple neighboring nodes connected to the node via several edges; performing feature extraction on the subgraph to obtain auxiliary information related to the target text; constructing prompt words based on the auxiliary information and the target text, and inputting the prompt words into the LLM model, wherein the LLM model performs logical reasoning based on the prompt words, and outputs a reasoning result corresponding to the target text.

[0006] Secondly, this disclosure provides an LLM model reasoning device based on a knowledge graph. The knowledge graph contains multiple nodes and edges connecting the nodes, where each node represents an entity and each edge represents a relationship between entities. The device includes: a determination unit, configured to acquire target text input by a user and determine whether the target text contains an entity corresponding to a node in the knowledge graph; a subgraph search unit, configured to, if so, search for a subgraph related to the entity in the knowledge graph; wherein the subgraph contains a node corresponding to the entity and multiple neighboring nodes connected to the node via several edges; an auxiliary information acquisition unit, configured to extract features from the subgraph to obtain auxiliary information related to the target text; and a reasoning unit, configured to construct prompt words based on the auxiliary information and the target text, input the prompt words into the LLM model, and have the LLM model perform logical reasoning based on the prompt words to output a reasoning result corresponding to the target text.

[0007] Accordingly, this disclosure also provides a computing device, including: a memory and a processor; the memory stores a computer program / instructions executable by the processor; when the processor executes the computer program / instructions, it performs the knowledge graph-based LLM model reasoning method described in the first aspect above.

[0008] Accordingly, this disclosure also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, performs the knowledge graph-based LLM model reasoning method as described in the first aspect above.

[0009] Accordingly, this disclosure also provides a computer program product comprising a computer program / instruction, which, when executed by a processor, performs the knowledge graph-based LLM model reasoning method as described in the first aspect above.

[0010] In summary, this application first obtains the target text input by the user and determines whether the target text contains an entity corresponding to a node in a knowledge graph. If so, it searches for a subgraph related to that entity in the knowledge graph. This subgraph contains the node corresponding to the entity and multiple neighboring nodes connected to that node via edges. Then, this application can extract features from the found subgraph to obtain auxiliary information related to the target text. Further, this application can construct prompt words based on the auxiliary information and the target text, and input the constructed prompt words into an LLM model. The LLM model then performs logical reasoning based on the prompt words and outputs a reasoning result corresponding to the target text. Thus, after obtaining the target text input by the user, this application can first identify the entities contained in the text that correspond to the nodes in the knowledge graph, then extract the subgraph related to the entity from the knowledge graph, and input the content contained in the subgraph as auxiliary information along with the target text into the LLM model to assist the LLM model in reasoning. This achieves the accurate use of the information contained in the knowledge graph to enhance the reasoning ability of the LLM model, ensuring the correctness and reliability of the final reasoning result output by the LLM model, and meeting the actual usage needs of users. Attached Figure Description

[0011] Figure 1 is a flowchart illustrating an exemplary embodiment of an LLM model reasoning method based on a knowledge graph.

[0012] Figure 2 is a flowchart illustrating another knowledge graph-based LLM model reasoning method provided in an exemplary embodiment;

[0013] Figure 3 is a schematic diagram of a sub-diagram provided in an exemplary embodiment;

[0014] Figure 4 is a schematic diagram of the structure of an LLM model inference device based on a knowledge graph, provided in an exemplary embodiment.

[0015] Figure 5 is a schematic diagram of the structure of a computing device provided in an exemplary embodiment. Detailed Implementation

[0016] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this disclosure as detailed in the appended claims.

[0017] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this disclosure in other embodiments. In some other embodiments, the methods may include more or fewer steps than those described in this disclosure. Furthermore, a single step described in this disclosure may be broken down into multiple steps in other embodiments; and multiple steps described in this disclosure may be combined into a single step in other embodiments.

[0018] It should be noted that "multiple" as used in this application refers to two or more.

[0019] Furthermore, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0020] This disclosure provides a technical solution that can extract a subgraph related to the entities contained in the user-input question from a knowledge graph, and input the content contained in the subgraph as auxiliary information along with the question into an LLM model to assist the LLM model in reasoning.

[0021] In implementation, this application first obtains the target text input by the user and determines whether the target text contains an entity corresponding to a node in the knowledge graph. If so, it searches for a subgraph related to the entity in the knowledge graph. This subgraph contains the node corresponding to the entity and multiple neighboring nodes connected to the node via edges. Then, this application can extract features from the found subgraph to obtain auxiliary information related to the target text. Further, this application can construct prompt words based on the auxiliary information and the target text, and input the constructed prompt words into an LLM model. The LLM model then performs logical reasoning based on the prompt words and outputs a reasoning result corresponding to the target text.

[0022] In the above technical solution, after obtaining the target text input by the user, this application can first determine the entities contained in the text that correspond to the nodes in the knowledge graph, then extract the subgraph related to the entity from the knowledge graph, and input the content contained in the subgraph as auxiliary information along with the target text into the LLM model to assist the LLM model in reasoning. This achieves the accurate use of the information contained in the knowledge graph to enhance the reasoning ability of the LLM model, ensuring the correctness and reliability of the reasoning results finally output by the LLM model, and meeting the actual needs of users.

[0023] Please refer to Figure 1, which is a flowchart illustrating an exemplary embodiment of an LLM model reasoning method based on a knowledge graph. As shown in Figure 1, the method may specifically include the following steps S101-S104.

[0024] Step S101: Obtain the target text input by the user and determine whether the target text contains an entity corresponding to a node in the knowledge graph.

[0025] In one illustrated embodiment, this application may first obtain target text input by the user. This target text may be text related to a target application scenario. For example, the target application scenario may be a medical inquiry scenario, and correspondingly, the target text may be medical inquiry text input by the user, which may include the user's symptoms, condition, medication history, and examination results. For example, the target application scenario may also be a transportation planning scenario, and correspondingly, the target text may be transportation travel consultation text input by the user, which may include the user's departure point, destination, and budget, etc. This disclosure does not specifically limit the scope of the application.

[0026] In one illustrated implementation, taking a medical inquiry scenario as an example, a user can input the target text in a dialog box provided by a client or website related to the medical inquiry service. The server associated with this medical inquiry service can be equipped with the aforementioned LLM model. This LLM model can be integrated with a knowledge graph, which contains structured, verified facts and professional knowledge. This knowledge graph can serve as the knowledge base for the LLM model, compensating for potential knowledge gaps or errors that may arise from the LLM model's reliance on training with large amounts of unstructured text. The knowledge graph contains multiple nodes and edges connecting them. Nodes can represent entities, and edges can represent relationships between entities. Its basic unit is the "entity-relationship-entity" triple. For example, this knowledge graph could be a medical knowledge graph.

[0027] In one illustrated embodiment, the knowledge graph-based LLM model reasoning method provided in this application can be applied to the aforementioned server.

[0028] Furthermore, this application can determine whether the target text contains entities corresponding to nodes in the knowledge graph. Specifically, this application can use entity linking (EL) to unambiguously and correctly link several identified entities in the target text (such as the aforementioned symptoms, conditions, etc.) to entities in the knowledge graph.

[0029] It should be noted that this application does not impose any specific restrictions on the implementation of entity links.

[0030] In one illustrated implementation, a thesaurus can be used to determine whether the target text contains an entity corresponding to a node in the knowledge graph. Specifically, based on a pre-defined thesaurus, any target entity among several entities identified in the target text can be matched with entities represented by multiple nodes in the knowledge graph. The matching result determines whether the target entity is a synonym of any entity represented by a node in the knowledge graph. If so, the target entity is determined to be the entity corresponding to a node in the knowledge graph, and the target entity is linked to the corresponding entity in the knowledge graph.

[0031] For example, the target entity identified in the target text could be "increased urination at night," and the corresponding entity in the knowledge graph could be "frequent urination." For example, the target entity identified in the target text could be "lower back pain," and the corresponding entity in the knowledge graph could be "lower back pain." For example, the target entity identified in the target text could be "high fever," and the corresponding entity in the knowledge graph could be "high fever," and so on. This disclosure does not impose any specific limitations on these entities.

[0032] In one illustrated implementation, embedding retrieval can also be used to determine whether the target text contains entities corresponding to nodes in the knowledge graph. Specifically, the target entities identified in the target text that correspond to any target entity type can first be mapped to embedding vectors. For example, the target entity type can be "symptoms," and the target entity can be "increased nocturia." Further, the vector similarity between the embedding vector of the target entity and the embedding vectors of all entities in the knowledge graph that correspond to the target entity type can be calculated. For example, the vector similarity between the embedding vector of "increased nocturia" and the embedding vectors of all entities in the knowledge graph that correspond to "symptoms" can be calculated.

[0033] In one illustrated embodiment, if the vector similarity between the target entity and the embedding vector of an entity (e.g., "frequent urination") in the knowledge graph is greater than a preset threshold, the target entity can be determined to be the entity corresponding to the node contained in the knowledge graph, and the target entity is linked to the corresponding entity in the knowledge graph.

[0034] Alternatively, in one illustrated embodiment, if the vector similarity between the target entity and the embedding vectors of multiple entities in the knowledge graph is greater than a preset threshold, then the entity with the highest vector similarity among these multiple entities can be determined as the entity in the knowledge graph corresponding to the target entity, and so on. This disclosure does not make any specific limitations in this regard.

[0035] Step S102: If yes, then find the subgraph related to the entity in the knowledge graph; wherein the subgraph contains the node corresponding to the entity, and multiple neighboring nodes connected to the node via several edges.

[0036] In one illustrated embodiment, if the target text contains an entity corresponding to a node in the knowledge graph—that is, if the entity identified from the target text can match a certain entity in the knowledge graph—a subgraph related to that entity can be further searched in the knowledge graph. This subgraph may contain the node corresponding to the entity, and multiple neighboring nodes connected to that node via several edges. In one illustrated embodiment, this subgraph may specifically be an N-hop neighbor subgraph, where the N-hop neighbor subgraph may include the node corresponding to the entity, and multiple neighboring nodes connected to that node via at most N edges.

[0037] In one illustrated embodiment, among all entities identified from the target text, there may be multiple entities that correspond to multiple entities in a knowledge graph. Therefore, multiple subgraphs related to these multiple entities can be found in the knowledge graph. This disclosure does not specifically limit this approach. Furthermore, this application can also concatenate at least some subgraphs from multiple subgraphs based on the connection relationships between entities contained in the multiple subgraphs, etc., and this disclosure does not specifically limit this approach either.

[0038] In one illustrated embodiment, considering that each edge in the subgraph may have constraints, edges in the subgraph that do not meet the conditions can be pruned according to the user's actual situation. For example, the subgraph may include: dysuria--(related disease)-->prostatitis, wherein the edge connecting the entity "dysuria" and the entity "prostatitis" has the constraint "male". If it is determined by analyzing the target text entered by the user that the user is female, this edge can be pruned, that is, the entity "prostatitis" can be deleted from the subgraph, as well as the edge connecting the entity "dysuria" and the entity "prostatitis". This disclosure does not specifically limit this.

[0039] In one of the illustrated embodiments, if the target text input by the user does not contain any entity in the knowledge graph, that is, if all the entities identified in the target text cannot be matched with any entity in the knowledge graph, then this application no longer needs to search for a subgraph in the knowledge graph, but can directly input the target text into the LLM model, and the LLM model will perform logical reasoning based on the target text and output the corresponding reasoning result.

[0040] Step S103: Perform feature extraction on the sub-graph to obtain auxiliary information related to the target text.

[0041] Furthermore, after finding the relevant subgraph from the knowledge graph, feature extraction can be performed on the subgraph to obtain auxiliary information related to the target text.

[0042] It should be noted that this application does not impose any particular limitations on the specific implementation methods of feature extraction.

[0043] On one hand, in one illustrated embodiment, this application can extract multiple paths related to the entities determined in step S101 from the subgraph based on the input target text. In one illustrated embodiment, in order to improve the efficiency of path extraction and avoid interference from too much information, the number of nodes contained in each extracted path can be within a preset range, such as each path containing 3 to 5 hops (i.e., 3 edges or 5 edges), and this disclosure does not specifically limit this.

[0044] In one illustrated embodiment, this application can extract multiple paths from a subgraph using Knowledge Graph Path Reasoning. Knowledge Graph Path Reasoning is a technique for reasoning within a knowledge graph, discovering new knowledge or insights based on the paths between entities and relationships within the graph. As mentioned above, in a knowledge graph, entities are typically nodes, and the relationships between them are the edges connecting the nodes. Path reasoning infers the potential relationships or attributes of entities by analyzing the connection paths between them.

[0045] Furthermore, the extracted paths can be processed into text to obtain prompt text describing the logical relationships between the multiple nodes (or entities) contained in each path.

[0046] In one illustrated implementation, the prompt text can be considered as semantic information contained within the subgraph.

[0047] In one illustrated embodiment, this application does not specifically limit the specific implementation method of text processing. In one illustrated embodiment, the path can be input into a pre-trained model, which will process the path into text and output the corresponding prompt text, etc., and this disclosure does not specifically limit it.

[0048] For example, the extracted path can be: ureteral stones -- (related symptoms) -- (related diseases) -- (related diseases) -- (related symptoms) -- (related symptoms) -- (related diseases ...

[0049] For example, the prompt text obtained after textualizing the above path can be: Ureteral stones may cause back pain, which may be associated with urethritis. One of the common symptoms of urethritis is frequent urination.

[0050] On the other hand, in one illustrated embodiment, this application can also map the nodes and edges contained in the subgraph to corresponding embedding vectors. Specifically, this application can input the subgraph into a pre-trained vector mapping model, which maps the nodes and edges contained in the subgraph to corresponding embedding vectors. In one illustrated embodiment, the vector pre-training model can specifically be a self-supervised link prediction model, which is not specifically limited in this disclosure. Furthermore, it should be noted that this application does not particularly limit the training process of the self-supervised link prediction model. In one illustrated embodiment, the parameters of the self-supervised link prediction model can be continuously optimized during model training to make its output more consistent with the expected embedding vectors, thereby better representing the connection structure in the subgraph.

[0051] In one illustrated embodiment, the knowledge graph can be stored in the form of triples [h, r, t], where h, r, and t can be a node, an edge, and another node connected in the knowledge graph (i.e., entity-relationship-entity), respectively. Correspondingly, the subgraph can include multiple triples. Specifically, this application can input the multiple triples included in the subgraph into a pre-trained vector mapping model, which maps the node, edge, and other node in each triple to corresponding embedding vectors and outputs multiple vector-mapped triples. For example, each vector-mapped triple can be represented as [eh, er, et], where eh, er, and et can be the embedding vector of the node, the embedding vector of the edge, and the embedding vector of the other node, respectively.

[0052] In one illustrated implementation, the embedding vector can be viewed as structural information contained within the subgraph.

[0053] As mentioned above, feature extraction of a subgraph actually includes extracting the semantic information contained in the subgraph (the prompt text obtained by textualizing the path) and extracting the structural information contained in the subgraph (embedding vector), thereby obtaining more comprehensive auxiliary information.

[0054] In one illustrated implementation, the aforementioned prompt text can serve as hard prompting to assist the LLM model in reasoning, while the aforementioned embedding vectors can serve as soft prompting to assist the LLM model in reasoning. Hard prompts typically refer to explicitly defined textual prompts directly input into the model; these prompts are specific questions, instructions, or contextual descriptions received by the model. Soft prompts, on the other hand, are a more abstract concept, involving implicit vectors in the model's parameter space. These vectors are inserted into the model's input sequence to adjust the model's behavior or guide the direction of its generated content. Technically, soft prompts may manifest as an additional, learnable embedding layer whose parameters are continuously optimized during training to adapt to specific tasks. Compared to the textual form of hard prompts, soft prompts are not directly visible, but they operate internally within the model, influencing the model's interpretation of input and output. Thus, this application combines soft and hard prompts, further enhancing the model's reasoning ability and ensuring the correctness and reliability of the reasoning results.

[0055] Step S104: Construct prompt words based on the auxiliary information and the target text, and input the prompt words into the LLM model. The LLM model performs logical reasoning based on the prompt words and outputs the reasoning result corresponding to the target text.

[0056] Furthermore, after extracting features from the subgraph to obtain auxiliary information corresponding to the target text, this application can input the auxiliary information and the target text together into the LLM model to assist the LLM model in performing logical reasoning for the target text.

[0057] In one illustrated embodiment, prompt words can be constructed based on the auxiliary information and the target text input by the user, and the prompt words can be input into the LLM model. The LLM model then performs logical reasoning on the target text based on the prompt words and outputs the reasoning result corresponding to the target text.

[0058] Specifically, the target text input by the user, the prompt text obtained by textualizing the path, and multiple triples after vector mapping (i.e., multiple node and edge embedding vectors) can be fused to construct the corresponding prompt word. For example, the prompt text can be added before the target text input by the user, and then the embedding vectors of multiple nodes and edges can be added as prefix embeddings before the target text with the added prompt text to construct the corresponding prompt word. Alternatively, in one illustrated embodiment, the embedding vectors can be added at any possible position, and this disclosure does not specifically limit this.

[0059] In one illustrated embodiment, the aforementioned LLM model can be an LLM service model obtained by fine-tuning a pre-trained LLM base model based on data related to the target application scenario. For example, the target application scenario can be a medical inquiry scenario, and the relevant data can be medical data, such as medical literature and medical records, or the medical data can be a knowledge graph of the medical industry; this disclosure does not specifically limit this. Correspondingly, the target text input by the user can be a medical inquiry text, and the inference result output by the LLM model can be a medical inquiry result, such as symptom diagnosis results, case analysis results, and medication recommendation results, etc.; this disclosure does not specifically limit this.

[0060] It should be noted that this application does not specifically limit the fine-tuning training process of the LLM model. In one illustrated embodiment, the fine-tuning training process of the LLM model can refer to the method flow of the inference stage described above. First, a large number of users' historical questions can be obtained, and based on these questions and the knowledge graph, the corresponding entities in the knowledge graph are found through entity links. Then, subgraphs of these entities are extracted from the knowledge graph. Further, relevant paths are extracted from the subgraphs and processed into text, and the nodes and edges contained in the subgraphs are mapped into embedding vectors. Finally, the text corresponding to the above paths and the embedding vectors can be added as auxiliary information to the original questions to construct training samples for the LLM model and perform fine-tuning training, etc. This disclosure does not specifically limit this process.

[0061] The knowledge graph-based LLM model reasoning method provided in this application will be described below through specific examples. For example, please refer to Figure 2, which is a flowchart illustrating another knowledge graph-based LLM model reasoning method provided in an exemplary embodiment.

[0062] As shown in Figure 2, users can enter their questions (i.e., the target text mentioned above) in the dialog box provided by the client related to the medical consultation service. For example, it could be "Hello doctor, I am a 58-year-old woman. I have had right lower back pain for 2 days and suddenly developed a high fever. Yesterday, my temperature reached a maximum of 39.9℃. I feel weak and have no energy. I went to the hospital yesterday and a CT scan showed: right ureteral stone with hydronephrosis and exudation around the right kidney...".

[0063] Furthermore, as shown in Figure 2, the server-side related to the medical inquiry service can obtain the user's input question from the client and link entities using a knowledge graph to connect several entities contained in the question to their corresponding entities in the knowledge graph. For example, based on the text content contained in the question, the linked entities in the knowledge graph may include "back pain", "high fever", "ureteral stones", and "upper urinary tract hydrops".

[0064] Furthermore, as shown in Figure 2, the server can extract subgraphs related to the identified entities from the knowledge graph. These subgraphs can be represented as multiple triples, such as [h1, r1, t1], [h2, r2, t2], ..., [hn, rn, tn] as shown in Figure 2.

[0065] For example, please refer to Figure 3, which is a schematic diagram of a subgraph provided in an exemplary embodiment. Based on the question input by the user, the final extracted subgraph may include entities such as "upper urinary tract hydrops", "ureteral stones", "lower back pain", "high fever", "urethritis", "frequent urination", and "acute pyelonephritis". The relationship between the entities can be shown in Figure 2, which will not be elaborated here.

[0066] Furthermore, as shown in Figure 2, the server can perform path extraction and embedding mapping (or embedding prediction) on the extracted subgraphs respectively.

[0067] For example, taking the subgraph shown in Figure 3 as an example, the extracted multiple paths may include: Path 1: Ureteral stones -- (related symptoms) -- (related diseases) -- (related diseases) -- (related symptoms ...

[0068] Further, as shown in Figure 2, the extracted multiple paths can be text-processed separately to obtain prompt text corresponding to each path. For example, the prompt text obtained after text-processing path 1 could be: Ureteral stones may cause lower back pain, which may be associated with urethritis. One common symptom of urethritis is frequent urination. For example, the prompt text obtained after text-processing path 2 could be: Ureteral stones may cause lower back pain, which may be associated with acute pyelonephritis. For example, the prompt text obtained after text-processing path 3 could be: Ureteral stones may cause lower back pain, which may be associated with urethritis. One common symptom of urethritis is high fever, which may be associated with acute pyelonephritis. For example, the prompt text obtained after text-processing path 4 could be: Upper urinary tract hydronephrosis may cause lower back pain, which may be associated with urethritis. One common symptom of urethritis is frequent urination. Wait, this disclosure does not make any specific limitations on this.

[0069] As shown in Figure 2, the server can map multiple nodes and edges contained in multiple triples into embedding vectors, resulting in multiple triples after vector mapping, such as [eh1, er1, et1], [eh2, er2, et2], ..., [ehn, ern, etn] shown in Figure 2. It should be noted that this application does not specifically limit the vector dimension of the mapped embedding vectors. In one illustrated embodiment, eh1, er1, et1, etc., can be a 64-dimensional embedding vector; this disclosure does not specifically limit this.

[0070] In one illustrated embodiment, the present application may further process the vector-mapped triples, such as compressing them, etc.

[0071] For example, multiple triples after vector mapping can be input into a multi-head attention mechanism for further feature extraction or compression to obtain the final embedding vector. This disclosure does not specifically limit this.

[0072] Furthermore, as shown in Figure 2, the prompt text obtained after the path has been textualized can be used as hard prompting, and the above-mentioned embedding vector can be used as soft prompting. Together with the question input by the user, they are input into the LLM model, where the LLM model performs logical reasoning and outputs the corresponding reasoning results.

[0073] For example, based on the user question shown in Figure 2, the final inference result output by the LLM model could be something like, "Hello patient, your right lower back pain may be due to ureteral stones with hydronephrosis, and you may have urethritis...".

[0074] In one illustrated implementation, the server can send the inference results output by the LLM model to the client, and the client can then display the received inference results to the user.

[0075] In summary, after obtaining the target text input by the user, this application can first identify the entities contained in the text that correspond to the nodes in the knowledge graph. Then, it extracts a subgraph related to the entity from the knowledge graph and inputs the content of the subgraph as auxiliary information along with the target text into the LLM model to assist the LLM model in performing logical reasoning. Specifically, this application can use the semantic information contained in the subgraph as hard cues and the structural information contained in the subgraph as soft cues, inputting them along with the target text into the LLM model to assist the LLM model in performing logical reasoning. This achieves accurate and comprehensive utilization of the information contained in the knowledge graph to enhance the reasoning ability of the LLM model, ensuring the correctness and reliability of the final reasoning result output by the LLM model and meeting the actual needs of users.

[0076] Corresponding to the above method flow, embodiments of this disclosure also provide a knowledge graph-based LLM model inference device. The knowledge graph contains multiple nodes and edges connecting the nodes; nodes represent entities, and edges represent relationships between entities. Please refer to Figure 4, which is a schematic diagram of the structure of a knowledge graph-based LLM model inference device provided in an exemplary embodiment.

[0077] As shown in Figure 4, the device 40 includes: a determining unit 401, used to acquire target text input by a user and determine whether the target text contains an entity corresponding to a node in the knowledge graph; a subgraph searching unit 402, used to, if so, search for a subgraph related to the entity in the knowledge graph; wherein the subgraph contains a node corresponding to the entity and multiple neighboring nodes connected to the node via several edges; an auxiliary information acquisition unit 403, used to extract features from the subgraph to obtain auxiliary information related to the target text; and a reasoning unit 404, used to construct prompt words based on the auxiliary information and the target text, input the prompt words into the LLM model, and have the LLM model perform logical reasoning based on the prompt words to output a reasoning result corresponding to the target text.

[0078] In one illustrated embodiment, the LLM model is an LLM service model obtained by fine-tuning a pre-trained LLM base model based on data related to the target application scenario; the target text is text related to the target application scenario.

[0079] In one illustrated embodiment, the target application scenario is a medical inquiry scenario, the target text includes medical inquiry text related to the medical inquiry scenario, and the reasoning result includes medical inquiry results.

[0080] In one illustrated embodiment, the auxiliary information acquisition unit 403 is specifically configured to: extract multiple paths related to the nodes corresponding to the entity in the subgraph, and perform textual processing on the multiple paths to obtain prompt text describing the logical relationships between the multiple nodes contained in each path; and input the subgraph into a pre-trained vector mapping model, which maps the nodes and edges contained in the subgraph into corresponding embedding vectors.

[0081] In one illustrated embodiment, the number of nodes contained in each path is within a preset range.

[0082] In one illustrated embodiment, the subgraph includes multiple triples, each triple including a node, an edge, and another node connected in the subgraph; the auxiliary information acquisition unit 403 is specifically used to: input the multiple triples included in the subgraph into a pre-trained vector mapping model, and have the vector mapping model map the node, edge, and other node in each triple into corresponding embedding vectors, thereby obtaining multiple vector-mapped triples; wherein each vector-mapped triple includes the embedding vector of the node, the embedding vector of the edge, and the embedding vector of the other node.

[0083] In one illustrated embodiment, the inference unit 404 is specifically configured to: add the prompt text before the target text, and add multiple vector-mapped triples before the prompt text to construct the corresponding prompt word.

[0084] In one illustrated embodiment, the subgraph is an N-hop neighbor subgraph, which includes a node corresponding to the entity and a plurality of neighbor nodes connected to the node via at most N edges.

[0085] The specific implementation process of the functions and roles of each unit in the aforementioned device 40 is detailed in the description of the above embodiments and will not be repeated here. It should be understood that the aforementioned device 40 can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor (CPU) of the device loading the corresponding computer program instructions into memory for execution. From a hardware perspective, in addition to the CPU and memory, the device typically includes other hardware such as chips for wireless signal transmission and reception, and / or other hardware such as boards for implementing network communication functions.

[0086] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the units or modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0087] The devices, units, and modules described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, in-vehicle computer, or any combination of these devices.

[0088] Corresponding to the above method embodiments, the embodiments of this disclosure also provide a computing device. Please refer to FIG5, which is a schematic structural diagram of a computing device provided in an exemplary embodiment. As shown in FIG5, the computing device includes a processor 1001 and a memory 1002, and may further include an input device 1004 (e.g., a keyboard) and an output device 1005 (e.g., a display). The processor 1001, memory 1002, input device 1004, and output device 1005 can be connected via a bus or other means. As shown in FIG5, the memory 1002 includes a computer-readable storage medium 1003, which stores a computer program executable by the processor 1001. The processor 1001 may be a CPU, a microprocessor, or an integrated circuit for controlling the execution of the above method embodiments. When the processor 1001 runs the stored computer program, it can execute various steps of the knowledge graph-based LLM model reasoning method in this embodiment of the present disclosure, including: acquiring target text input by the user and determining whether the target text contains an entity corresponding to a node in the knowledge graph; if so, finding a subgraph related to the entity in the knowledge graph; wherein the subgraph contains a node corresponding to the entity and multiple neighboring nodes connected to the node via several edges; performing feature extraction on the subgraph to obtain auxiliary information related to the target text; constructing prompt words based on the auxiliary information and the target text, and inputting the prompt words into the LLM model, whereby the LLM model performs logical reasoning based on the prompt words and outputs a reasoning result corresponding to the target text, etc.

[0089] For a detailed description of each step in the knowledge graph-based LLM model reasoning method described above, please refer to the previous content, which will not be repeated here.

[0090] Corresponding to the above method embodiments, embodiments of this disclosure also provide a computer-readable storage medium storing computer programs. When executed by a processor, these computer programs perform the various steps of the knowledge graph-based LLM model inference method in the embodiments of this disclosure. Please refer to the description of the above embodiments for details, which will not be repeated here.

[0091] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

[0092] In a typical configuration, a terminal device includes one or more CPUs, input / output interfaces, network interfaces, and memory.

[0093] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0094] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data.

[0095] Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0096] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0097] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, embodiments of this disclosure can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this disclosure can take the form of computer program products implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A reasoning method for an LLM model based on knowledge graphs, wherein, The knowledge graph contains multiple nodes and edges connecting the nodes, where each node represents an entity and each edge represents a relationship between entities; the method includes: Obtain the target text input by the user, and determine whether the target text contains an entity corresponding to a node in the knowledge graph; If so, then find the subgraph related to the entity in the knowledge graph; wherein the subgraph contains the node corresponding to the entity, and multiple neighboring nodes connected to the node via several edges; Feature extraction is performed on the sub-graph to obtain auxiliary information related to the target text; Based on the auxiliary information and the target text, a prompt word is constructed and input into the LLM model. The LLM model then performs logical reasoning based on the prompt word and outputs a reasoning result corresponding to the target text.

2. The method according to claim 1, wherein, The LLM model is an LLM service model obtained by fine-tuning a pre-trained LLM base model based on data related to the target application scenario; the target text is text related to the target application scenario.

3. The method according to claim 2, wherein, The target application scenario is a medical inquiry scenario, the target text includes medical inquiry text related to the medical inquiry scenario, and the reasoning result includes medical inquiry results.

4. The method according to claim 1, wherein, The step of extracting features from the sub-graph to obtain auxiliary information related to the target text includes: Multiple paths related to the nodes corresponding to the entity are extracted from the subgraph, and each path is text-processed to obtain prompt text describing the logical relationships between the multiple nodes contained in each path; and, The subgraph is input into a pre-trained vector mapping model, which maps the nodes and edges contained in the subgraph into corresponding embedding vectors.

5. The method according to claim 4, wherein, The number of nodes in each path is within a preset range.

6. The method according to claim 4, wherein, The subgraph comprises multiple triples, each triple including a node, an edge, and another node connected in the subgraph; The step of inputting the subgraph into a pre-trained vector mapping model, whereby the vector mapping model maps the nodes and edges contained in the subgraph into corresponding embedding vectors, includes: The subgraph contains multiple triples which are input into a pre-trained vector mapping model. The vector mapping model maps the node, edge and other node in each triple into corresponding embedding vectors, resulting in multiple vector-mapped triples. Each vector-mapped triple includes the embedding vector of the node, the embedding vector of the edge and the embedding vector of the other node.

7. The method according to claim 6, wherein, The step of constructing prompt words based on the auxiliary information and the target text includes: Add the prompt text before the target text, and add multiple vector-mapped triples before the prompt text to construct the corresponding prompt word.

8. The method according to any one of claims 1-7, wherein, The subgraph is an N-hop neighbor subgraph, which includes a node corresponding to the entity and multiple neighbor nodes connected to the node via at most N edges.

9. A knowledge graph-based LLM model inference device, wherein, The knowledge graph contains multiple nodes and edges connecting the nodes, where each node represents an entity and each edge represents a relationship between entities; the device includes: A determining unit is used to acquire the target text input by the user and determine whether the target text contains an entity corresponding to a node in the knowledge graph; A subgraph lookup unit is used to find a subgraph related to the entity in the knowledge graph if the entity is found; wherein the subgraph contains a node corresponding to the entity and multiple neighboring nodes connected to the entity via several edges. An auxiliary information acquisition unit is used to extract features from the sub-graph to obtain auxiliary information related to the target text; The reasoning unit is used to construct prompt words based on the auxiliary information and the target text, input the prompt words into the LLM model, and have the LLM model perform logical reasoning based on the prompt words to output a reasoning result corresponding to the target text.

10. A computing device, comprising: A memory and a processor; wherein the memory stores a computer program / instructions executable by the processor; and when the processor executes the computer program / instructions, it performs the method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program / instructions stored thereon, wherein, When the computer program / instructions are executed by the processor, they implement the method as described in any one of claims 1-8.

12. A computer program product comprising a computer program / instructions, wherein, When the computer program / instructions are executed by the processor, they implement the method as described in any one of claims 1-8.

Images